Scoring spread?

[Doofenshmirtz]

After we expanded our local course by adding 19 more holes, I started keeping stats after some tournaments. I wanted to evaluate scoring spread, comparative hold difficulty, etc. This and other feedback led to a few refinements like shortening one hole and adding some alternate tees, etc. For scoring spread, I simply calculated standard deviation across all tournament participants. But then I was left with the questions of what a good scoring spread was supposed to be and was I even doing it right.

So here is a table that I created after our last tournament.

https://dl.dropboxusercontent.com/u/35613458/2014%20Red%20River%20Heat.pdf

Should I be looking at ratios to determine scoring spread, e.g., birdies/par or birdies/bogeys, (birdies/player)/(bogeys/player)

Is standard deviation a good measure?

I included the table from one of the tournaments for reference. I calculated the average over par that each hole played (a negative number means it averaged under par). Average score. Standard deviation and number of birdies. All the scores are there. Ultimately, I am trying to figure the best way to assess the scoring spread for each hole.

Incidentally, Gold 4, which gave up no birdies, has already been shortened.

 

[grodney]

motherofgod i can't believe my employer blocks dropbox. cripes.

 

[Cgkdisc]

Basic scoring spread should be done using scores of players within say 40 rating points plus/minus the reference rating for the skill level of the course layout played. So if it's a blue level course with reference rating 950, use the scores of players rated from 910 to 990 whose ratings average real close to 950 to determine scoring spread. Scoring average and standard deviation are decent ways but looking at actual mix of score percentages and just how many different scores were thrown on the hole is also useful.

 

[Doofenshmirtz]

Basic scoring spread should be done using scores of players within say 40 rating points plus/minus the reference rating for the skill level of the course layout played.

Unfortunately, that's not likely to happen anytime soon for two reasons: 1) most tournaments at that course are not PDGA sanctioned so there's no practical way to get the player ratings, and 2) most tournaments are fairly small (mostly less than 50 participants, although the example was bigger) so the sample size would always be exceedingly small if it were restricted to players of a certain rating.

This is a two-year-old course complex in an area that is about 100 miles from the nearest pre-existing course (although a few more have been built in the last couple of years), so the local tournament player base is still pretty small.

I had hoped that by just including everyone that played in the biggest tournament, I'd get a better idea, even if still flawed.

 

[DavidSauls]

At the least, you'll have an idea how the holes compare to each other.

 

[DavidSauls]

Are they playing divisions? You might try it with a single division. Or, with, say, the top 20 scores. Won't be as good as ratings, but will be better than the entire field which, presumably, has a huge range of skills.

It's true that you'll be dealing with a small sample. But depending on how many tournaments you have, and how many rounds each one is, you might accumulate enough numbers over time.

 

[Doofenshmirtz]

Are they playing divisions? You might try it with a single division. Or, with, say, the top 20 scores. Won't be as good as ratings, but will be better than the entire field which, presumably, has a huge range of skills.

It's true that you'll be dealing with a small sample. But depending on how many tournaments you have, and how many rounds each one is, you might accumulate enough numbers over time.

True. I didn't think about it that way. They are playing standard-ish divisions, so I guess I could go with just the Open division, or just exclude the Rec's and Ams.

 

[Doofenshmirtz]

Although, having thought about it a very little bit more, I guess I could just take the top 50% of scores.

 

[Cgkdisc]

Again, even if you don't have ratings, hopefully the course was designed for a particular skill level. In which case, you should use players from those division(s) to analyze the info but maybe not including those who scored a bit too high and maybe were playing up.

 

[Steve West]

A specific skill level measured by rating is the best group to target. However, you can also target for other well-defined and consistent groups. For example, you could use the scores themselves as a form of rating and target the course to the group of players that would expect a 57 on this course.

So I did.

[FONT="Courier New"]
Hole     A      B        C       D      E        F     
1	2.76	2.15	3.24	67%	40%	 3.0%
2	3.28	3.30	3.60	48%	36%	-0.6%
3	3.07	2.68	3.46	59%	42%	 2.4%
4	3.05	2.49	3.45	67%	35%	 0.2%
5	3.12	2.12	3.50	73%	24%	 2.9%
6	3.12	2.72	3.50	60%	37%	-0.5%
7	3.53	2.87	3.89	44%	48%	-0.9%
8	3.15	3.02	3.48	57%	28%	 4.6%
9	3.95	4.54	4.28	35%	23%	-3.2%
10	2.88	2.50	3.31	65%	25%	-1.0%
11	2.73	2.35	3.19	61%	31%	-0.9%
12	2.89	2.08	3.35	74%	38%	 0.4%
13	3.78	3.04	4.06	42%	36%	-0.4%
14	2.83	2.43	3.27	62%	27%	 3.7%
15	3.41	3.37	3.71	46%	22%	 0.2%
16	3.02	2.38	3.41	71%	20%	 2.8%
17	3.43	2.35	3.82	49%	29%	-1.6%
18	3.01	2.91	3.37	58%	29%	 4.7%
[/FONT]

Let look at Hole 8.

A. Those players who would expect a 57 on the course would average 3.15 on hole 8.

B. The Scoring Spread Width is a fairly wide 3.02. This means the hole would sort these players slightly better than a hole that gave out three different scores, evenly distributed.

C. The integer portion of Exact Par is the score that these players would get at least 44% of the time. Hole 8 is a par 3 for players who expect to score 57 on this course. The fractional part of Exact Par indicates how close the hole is to being one higher par. (For example, if you really wanted to call another hole a par 4, Hole 7 would be the next choice.)

D. is the Frequency of the most common score. A rule of thumb is that this should be no higher than 67%. So, 57% is good. Useful if you can't compute Scoring Spread Width.

The last two columns are stats based on the entire field.

E. Correlation to other holes shows whether this hole gives high scores to the same players as all the other holes.

F. The King of stats is Standardized Contribution to Scoring Spread Width. This tells you how much a hole helped rank the players. Hole 8 (and 18) did a lot to sort players out by skill. Breaking ties is one way they did that.

We can also see how the scoring distribution varies for players of different skill levels.

This chart shows the percent of players that get each score as a function of total expected score. 3 is the most common score across all skills, but the other scores are allocated neatly by skill. Better players are on the left, so there are more 2s there. As player skill goes down, there are more and more 5s and 6s.

Let's compare Hole 9. Hole 9 has a suspiciously wide scoring spread width of 4.54. By itself, this hole gives out a lot of different scores. But, how well do all these different scores actually measure skill?

Notice that the Standardized Contribution to Scoring Spread Width is a big negative. This means Hole 9 actually increases the number of ties among players. Usually, a tournament would be better off playing skipping 9 and only playing 17 holes.

Let's see if the scoring distributions by skill can show us why.

Doesn't that look a lot messier than hole 8? Sure, only the best players get 2s on hole 9. However, some of the best players are getting 7s! For the better players, a 7 instead of a 2 can wipe out what happens on a lot of other holes. Notice also a significant number of 5s and 6s everywhere. In fact, the average score on this hole is as high for players who expect a 54 as it is for players who expect a 61.

 

[Doofenshmirtz]

Wow. Thanks for the analysis, but I guess I should have made one thing much more clear. This tournament was played on two courses. The purple chart was played on the Purple course, the Gold/Orange chart was played on the Gold course. Both holes 8 are +/- 250' tunnel shots in the woods with P8 having a very tight fairway and G9 having a tight path through a more open fairway that rewards a hyzer flip with a better chance at birdie than a straight shot. Both holes 9 have elevated tee shots, so there are some similarities, but P9 is a 400 foot shot requiring a right turn that occasionaly yields a 2. G9 is a +/- 850', par 5. The tee is 30' higher than the fairway, but nobody got a 2, or is ever going to get a 2, on that hole.

So my apologies for not making that clearer. However, just looking at the numbers, by score, it looks like P9 is the hole that didn't provide much spread across all skill levels and that makes a lot of sense. The birdie window is very narrow on that hole, a three simply requires a lack of big mistakes and a 4 is typically the result of a bad throw or putt. G9 gives a lot of opportunities for mistakes, especially in tempting people to try for max distance when they really don't need it. 350 or 400 off the 30' elevated tee and good placement with a fairway driver then mid will yield a chance for a 4. But lots of people get in trouble right off the tee by throwing as hard as they can.

 

[ToddL]

I made a fancy-pants spreadsheet for myself/my club which does a pretty good job at breaking down the scoring spreads. (all this data is from a local tournament we held last summer)

1. Enter course information (holes, pars).
2. Enter all the individual scores with player divisions and player ratings.
3. Examine results tables.

Table 1:
Examine each hole individually, broken down by division.

Here we see the scores for hole #7. 2 MPO players scored a 2, 9 MPO players scored a 3, etc. On the right side we have Average Score (raw value), To Par score (avg score - par), and Standard Deviation.

Red signifies a low score or a low standard deviation.
Green signifies a high score or a high std dev.


Table 2:
Examine each division, broken down by hole:

This is all the scores for the MPO field. Obviously, with only 18 data points (9 MPO players for 2 rounds) the data can be a little iffy, especially since it was a small, local tournaments where lots of people like me play MPO who should really be playing MA1. (And then there's that guy who kept scoring 6s... He might be an MA2 player.)

Here we can quickly see the easiest and hardest holes. Holes 8 and 15 are very easy, both averaging 2.39. Additionally, those both have the lowest std dev, meaning they're consistently easy (no bogies for the entire division).
Holes 10 and 11 are the hardest holes on the course, both by average score and by to par, both of them averaging par+0.4. Hole 7 is right with them, averaging par+0.39.


Table 3:
Examine a range of ratings, broken down by hole.

Here we're looking at all players between 920-980 (this is an advanced-level, par 58 course with a target audience of ~950 rated players). This is 14 players for 2 rounds (count = 28).
The first thing that pops out is that holes 3 and 14 have >70% of the players scoring the same score. This is generally a bad thing, as it provides no scoring separation. We might want to think about making those holes a touch easier to get a few more birdies.
14 of the holes averaged below par while 4 averaged above par.
Holes 6 and 9 are interesting in that they averaged almost exactly par (par 3), but it had a very wide spread (std dev=.92).
Hole 4 is darn near perfect. 30% birdies, 50% pars, 20% bogies.
Again, we see that holes 7, 10, and 11 are the hardest in terms of average score and to-par score. Hole 14 is hard according to average score, but middle of the pack according to to-par score.

 

[grodney]

ToddL, can you describe Hole 7? Length, elevation, trees, OB, slope around pin? Pictures?

 

[ToddL]

That's the Foothills course in Easley, SC.

7 is 350 ft, straight, very narrow, sloping sidehill, water at bottom of hill. It's a pretty hard hole. Too hard? Maybe, but I like watching people squirm on it. Take your par and move on.

http://www.dgcoursereview.com/course_pics/534/c43c64cd.jpg
http://www.dgcoursereview.com/course_pics/534/92dfa2b7.jpg
http://www.dgcoursereview.com/course_pics/534/f5ceaf94.jpg

 

[grodney]

Thanks. Yeah, a little too long/narrow/punishing for your stated intended skill level (~950). The 2 deuces, while having scores spread among 3/4 (with some 5's) for higher-skilled divisions, was the giveaway. But hey, it's only one hole. Not bad to push the limits of the skill level to the very edge on a hole or two.

 

[Steve West]

Wow. Thanks for the analysis, but I guess I should have made one thing much more clear. This tournament was played on two courses.

"Well that's different....

Never mind."

(At least now we know how to get a scoring spread 4.54 wide - randomly select some of the players to play a completely dissimilar hole.)

 

[grodney]

Thanks. Yeah, a little too long/narrow/punishing for your stated intended skill level (~950). The 2 deuces, while having scores spread among 3/4 (with some 5's) for higher-skilled divisions, was the giveaway. But hey, it's only one hole. Not bad to push the limits of the skill level to the very edge on a hole or two.

p.s. Or maybe you want to view this hole as a very short risk/reward 2-shot hole for this skill level, which might be just fine depending on how it looks/plays in person.

 

[Lewis]

The King of stats is Standardized Contribution to Scoring Spread Width. This tells you how much a hole helped rank the players.

Would you mind sharing how you calculate the king of stats, or is it a professional secret?

 

[jeverett]

Would you mind sharing how you calculate the king of stats, or is it a professional secret?

I would be interested in this formula too.

 

[Steve West]

I'll describe it. Scoring Spread is from information theory. It tells you how much information is contained in a set of scores. The wider the scoring spread, the more you know about the relative performances of the players.

I look at how each hole affects the Scoring Spread of the Total Scores from a round or a tournament. If a hole gave the same score to all players, it would do nothing to the scoring spread of total scores. Also, we know that scoring spread shrinks when random scores are added to the mix. So, a "good" hole will widen the scoring spread of the total scores. This is the only direct measure of whether the hole is doing its job, which is to give us information about the performance of the players.

At first, I computed the Contribution to Scoring Spread of Total Scores by simply comparing the scoring spread of total scores with and without the hole in question. That's fine, but it can generate quirky results, because there are relatively few possible scores. Also, it is hard to compare across tournaments of different sizes.

So now, to standardize it and eliminate the quirkiness, I look at thousands of simulated tournaments, using actual scores from 72 of the players selected at random. The scoring spread of total scores is computed with and without each hole and the difference noted. The average of all the differences for a particular hole is that hole's standardized contribution to scoring spread width.

More to read here.