Disc Golf Course Review (https://www.dgcoursereview.com/forums/index.php)
-   General Disc Golf Chat (https://www.dgcoursereview.com/forums/forumdisplay.php?f=2)
-   -   Hole Enjoyment Survey - Creating a measure (https://www.dgcoursereview.com/forums/showthread.php?t=135086)

 ChrisWoj 09-30-2019 12:49 PM

Hole Enjoyment Survey - Creating a measure

Good morning to anyone interested. We started a brief discussion on this project in the Par Talk thread, and it was requested by a few that we break it out into its own thread. What I'm attempting to do is create a measure of hole enjoyment. One of the things we're good at in general with measurement of sensation/experience like this is ranking. What we're bad at is estimating appropriate meaning from the differences in ranking.

This URL takes you to the PDF shared via Google Drive:

This is a trial instrument. This instrument isn't a final version. My plan is to trial this instrument at a few local courses. The goal is to identify a set of prompts that function to measure a single construct. Once those have been identified - then the prompts that emerge need to be looked at critically to determine if they are, indeed, looking at the trait we want it to. I will be working with Rasch analysis to convert the results, which are in ordinal form, to something with abstract invariance associated with the intervals. The idea is to create a reasonable facsimile that tells us how much better one hole is than another, in relatively certain terms (key word in there: relatively, I am aware that measuring human sensation is fickle - probabilities are at the core of using this as a measure).

Some things have already been brought up in the other thread:

1. Long term I would like to see this used at multiple courses in regions with more diverse terrain than my home course. That would give me the chance to look at its results alongside the DGCR favorited hole statistics. This helps determine if we have convergent validity - if it functions to rank in a manner similar to the favorite hole scores, we can begin to trust that it is measuring some element of enjoyment.

2. I would also like to compare performance by relevant demographics: how do players at one skill level, or who throw with one dominant throw measure the same holes against others.

3. If what results can be established as a consistent tool, I would like to see it used by players on tour to see how the enjoyment factors line up with some of the numbers others like Steve West have produced on how the holes play.

At the moment, the most important thing is that the survey I'm trialing has construct validity: if, at a glance, an item on the survey doesn't appear to measure in the end some element of enjoyment (or, as one thread refers to it: fun factor) - let me know. This same thought process will be applied after the first analysis is done, but if its clear I chose something really dumb - why include it? So let me know.

Thoughts? Questions? Expletives?

 DavidSauls 09-30-2019 01:04 PM

I'm not sure how you can add it in, but one factor I think of is uniqueness.

Is this hole significantly different, or very similar, to other holes (1) on this course, (2) that I play regularly, or (3) that I have played.

Subjectively, many of my favorite holes stand out because of their unique qualities.

Some of the worst, too, I guess.

 PMantle 10-01-2019 10:40 AM

Quote:
 Originally Posted by ChrisWoj (Post 3502544) The goal is to identify a set of prompts that function to measure a single construct. Once those have been identified - then the prompts that emerge need to be looked at critically to determine if they are, indeed, looking at the trait we want it to. I will be working with Rasch analysis to convert the results, which are in ordinal form, to something with abstract invariance associated with the intervals. Questions?
Yes, what line of work are you in?

 ChrisWoj 10-01-2019 12:01 PM

Quote:
 Originally Posted by PMantle (Post 3502804) Yes, what line of work are you in?
Fair question - at the moment I'm an academic and a grill cook. The former a bit more relevant than the latter, though the latter involves walking in circles a lot - apply that how you may.

In terms of academic background - I'm in a PhD Research & Measurement program - basically researching the nuts and bolts of research - primarily focused on development of best practice. Measurement using Rasch is one of the primary focuses of the department, so I get a lot of it. This is more an opportunity to kind of use that in something I enjoy - in a way I want to develop a way to integrate what I enjoy about my professional life into my favorite hobby.

My own focus is on the use of algorithms for sample matching to minimize confounding factors in cases where experimental research is not possible. An example: identifying the success of educational interventions, where I'm looking at broadening our understanding of the appropriateness of using propensity scores and coarsened exact matching as viable strategies. The Pro Tour is getting close to having enough numbers available to match competitors year over year on how they've performed so that we could really discuss the impact of changes to the courses - though I'd still like additional demos like distance, throw/hand dominance, etc. I may look at that in the future.

Overall I'm still kind of feeling out where my hobby and my expertise are going to fit together. I'm hopeful that this project and really yield something fun and lead to a good little extended line of hobby research.

 SonicGuy 10-01-2019 12:09 PM

Quote:
 Originally Posted by ChrisWoj (Post 3502544) Good morning to anyone interested. What I'm attempting to do is create a measure of hole enjoyment. This is a trial instrument. This instrument isn't a final version. The idea is to create a reasonable facsimile that tells us how much better one hole is than another, in relatively certain terms (key word in there: relatively, I am aware that measuring human sensation is fickle - probabilities are at the core of using this as a measure). I would also like to compare performance by relevant demographics. Thoughts? Questions? Expletives?
https://i.imgflip.com/1gql1x.jpg

 ChrisWoj 10-01-2019 12:21 PM

1 Attachment(s)
Quote:
 Originally Posted by SonicGuy (Post 3502847) https://i.imgflip.com/1gql1x.jpg
:(:(:(:thmbdown:
Sadly this will prove to be a difficult demo to work with, see attached.

 BuzzzChief 10-01-2019 12:24 PM

One thing that I don't see addressed is the "visual appeal" of a hole. I think in most cases, the signature hole at a course is one where there's a dramatic appearance of some kind. A big dropoff, a throw over water from a scenic viewpoint, a unique elevated basket, a rock formation or wood carving, something like that.

For example, DeLa #27 and BRP #4. There are plenty of "top of the world" and tunnel shots out there, but those two are consistently mentioned by people as "favorite" holes. It's because they look cool.

 elmexdela 10-01-2019 12:40 PM

Quote:
 Originally Posted by BuzzzChief (Post 3502860) One thing that I don't see addressed is the "visual appeal" of a hole. I think in most cases, the signature hole at a course is one where there's a dramatic appearance of some kind. A big dropoff, a throw over water from a scenic viewpoint, a unique elevated basket, a rock formation or wood carving, something like that. For example, DeLa #27 and BRP #4. There are plenty of "top of the world" and tunnel shots out there, but those two are consistently mentioned by people as "favorite" holes. It's because they look cool.
and now all of minnesotas copycat brp 4 style holes that now took away from brp uniqueness

 Hampstead 10-01-2019 01:19 PM

https://media1.popsugar-assets.com/f..._H/i/Holes.JPG

 Steve West 10-01-2019 01:53 PM

If I were forced to never again play one hole of my choosing for the rest of my life, I would gleefully pick BRP 444.

Yet, if I wanted to try to improve the enjoyability of the local disc golf scene by picking holes in the Twin Cities to be eliminated from existence, BRP 444 would be one of the last survivors.

Good luck resolving those ideas into one number.

All times are GMT -4. The time now is 12:43 PM.