Estimating Scores

[Ryan P.]

Branching this from The 2023 Texas States thread.

A couple things:
What software or formula(s) are you using to produce your forecasts?
Possibly a mistake in your hole 1 forecast which UDisc shows as a 717 ft Par 4?

Starting to include more OB penalties in course design is similar to when the 3-pt shot was introduced in basketball. Incorporating lots of OB areas changes the game from traditional golf scoring (mostly penalty free at the elite level*) to a new not-yet-labeled golf scoring scheme where some throws cost the player two strokes instead of one. However, unlike 3-pointers which are scored based on skill, there are many OB penalty strokes which are randomly "earned" (fluky) versus earning them strictly from very "unskillful" throws.

Point being that there are two separate games being played on tour depending on the amount of penalty elements - OB, hazard, mandos, 2-meter - actively in play within a course design and their stats should be separated, especially ratings. For those who feel ratings have become inflated, OB penalties padding propagator scores is a key factor.

*People might be surprised to know that the top 200 PGA tour players average just 1 penalty of any kind every 2.5 rounds which is 0.4 per round. The average penalties per player per round ranges from 3 to 6 on our DGPT courses heavy with OB. In a 4-round event, a PGA player might be expected to get 2-3 penalties total. On our DGPT OB courses, our players can get anywhere from 12 to 24 penalties total and that doesn't account for additional hidden strokes from losing distance on some penalties.

If we're going to have courses with OB and other penalties, do them on every hole, like using gutters in bowling, and ideally design their positions well to minimize random penalties. Or use/design traditional courses to expect few or no penalties. Separate each course type's rating stats and tour stats.

TL;DR: I overanalyzed your question and have a short, reasonable answer in the last paragraph.

As a programmer, my brain can't help but think about this. I think to have an accurate algorithm for this, we'd need to first gather a lot of data.

Specifically, we'd first need to graph where players land on what types of holes. This would allow us to see the expected drive length, range of errors, and distribution of errors for each hole. We'd need to do this for at least three types of shots: drives from the tee, putts, and anything else(approaches, second shots on par 5s, pitch outs, etc.). We have enough of this data for putts (UDisc's rough approximations), but we don't have it for drives and the "anything else" category.

Second, we'd need to map out holes a lot more accurately. Tee signs/caddie books are a great start, but they're sometimes not accurate, and I've yet to see one that is as accurate as is needed. They'd need to be drawn to scale and be 3d to some degree. They'd also need to know where the gaps are, where the trees are, etc.

Lastly, we'd need another algorithm to basically determine how a pro is going to play the hole. Take Winthrop's hole 10. Some pros play for the birdie 3 by taking the simple hole out to the left, and some go for the eagle. Determining how many would take the birdie route and how many would take the eagle route could be done by writing an algorithm that compares similar holes and estimates.

That's probably far more than what you're looking for. I think a fairly accurate algorithm could be written based off of the data we have now, plus a little bit more. If someone were to have access to the UDisc score data and perhaps PDGA ratings, they could add a few data pieces to each hole and get a decent predictor of scores. UDisc provides hole length, so we'd need to add (at least) elevation change and density (something like 1 meaning no trees, 10 being the most wooded hole ever), we could have a decent predictor of scores. It wouldn't account for weather or the holes nuances (forced layups) but it'd be a lot better than what we have now (nothing). I could write the code for most of that, but I don't have the time/desire to.

 

[Ryan P.]

Golf uses ten parameters. We'll need more.

(Maybe move the theoretical discussion to a new thread in Course Design.)

What are those ten/can you send a link that covers them?

 

[Steve West]

What are those ten/can you send a link that covers them?

Don't know; saw the number ten in a high-level description of the process. Found out that the details are not easily Googleable. Good luck, and please share if you uncover their secret sauce.

 

[Cgkdisc]

Here's a good explanation of the factors assessed to determine a ball golf course rating during good mid-summer playing conditions:

https://www.mgagolf.org/handicapping/course-rating-top-ten

 

[DavidSauls]

Ah, we're bag to the tail wagging the dog -- ratings dictating course design.

I thought the main rationale for ratings, was to group players by similar skill level for fairer competition. Is there any documentation that the same players don't tendto do better on OB-full and OB-free courses -- with a discrepancy greater than other variations in course design, like length or openness, that might favor or hinder certain players?

Perhaps while we're at it, we can incorporate another style of course into a separate rating system -- one with rough tough enough that missing the fairway has about the same scoring effect as OB.

 

[Cgkdisc]

Ah, we're bag to the tail wagging the dog -- ratings dictating course design.

I thought the main rationale for ratings, was to group players by similar skill level for fairer competition. Is there any documentation that the same players don't tend to do better on OB-full and OB-free courses -- with a discrepancy greater than other variations in course design, like length or openness, that might favor or hinder certain players?

Perhaps while we're at it, we can incorporate another style of course into a separate rating system -- one with rough tough enough that missing the fairway has about the same scoring effect as OB.

Yes, appropriate ratings should sync with course layouts if we want to emulate (ball) golf where their course layout distances sync with player skill levels.

The rating system relies on the propagators earning their ratings on the same range of course types as the ones they are rating. Otherwise, GIGO. For example, we've seen the stats in the past where PP has lost ground on courses with OB. Anecdotal example but until the courses and their stats are handled separately, you can't tell how much it matters. When I've done correlations between player rating and finish rank, the correlation is higher on traditional courses than OB-laden ones like USDGC. That is a specific indication that something is different.

One way to emulate ball golf rough and sand traps is to apply a half stroke instead of whole stroke penalty for going OB. PGA tour stats show that players save a stroke half the time when shooting from the rough or sand traps.

 

[DavidSauls]

Emulating ball golf was never high on my priority list, certainly not since I decided not to play ball golf.

The effect of OB goes far beyond the penalty strokes. At least, well-designed OB. It affects shot selection and routes as well.

So how would you split up the courses? A course with 2 holes with marginal OB on them? A course with 5 holes of serious but avoidable OB? A course with 11 holes that have OB, ranging from OB that only comes into play on really bad shots, to OB close to a basket? Perhaps Winthrop Gold is at one end of the spectrum, and a long OB-free course at the other. Finding the line to separate them would be tricky, measuring how many holes, how likely the OB, and sorting out OB lining fairways from OB close to greens from OB-carries from islands, all of various sizes.

For the record, my favorite courses range from those with lots of OB, to those with little or none. Whatever works for that property. I notice that I score better when I throw better, either way.

 

[Ryan P.]

One way to emulate ball golf rough and sand traps is to apply a half stroke instead of whole stroke penalty for going OB. PGA tour stats show that players save a stroke half the time when shooting from the rough or sand traps.

While that makes sense mathematically, I don't think anyone will adopt it. Partial strokes don't seem like a thing people will want to track.

 

[Ryan P.]

Emulating ball golf was never high on my priority list, certainly not since I decided not to play ball golf.

The effect of OB goes far beyond the penalty strokes. At least, well-designed OB. It affects shot selection and routes as well.

So how would you split up the courses? A course with 2 holes with marginal OB on them? A course with 5 holes of serious but avoidable OB? A course with 11 holes that have OB, ranging from OB that only comes into play on really bad shots, to OB close to a basket? Perhaps Winthrop Gold is at one end of the spectrum, and a long OB-free course at the other. Finding the line to separate them would be tricky, measuring how many holes, how likely the OB, and sorting out OB lining fairways from OB close to greens from OB-carries from islands, all of various sizes.

For the record, my favorite courses range from those with lots of OB, to those with little or none. Whatever works for that property. I notice that I score better when I throw better, either way.

I assume you're asking me. If not, file this post under the "idiotic" end of the spectrum.

Never giving a player a chance to recover (i.e., lining every hole with OB) is frustrating. It's slightly less frustrating than lining every hole with thick natural OB (because you don't have to hack through trees to get your disc and aren't asked to make an impossible shot). However, it's in that same direction.

To me, OB on an ideal course would flow somewhat like this: the first few holes would be forgiving (very little if any OB), but difficult. Any birdies would feel like an accomplishment, but bogeys would be uncommon. After that, difficulty would ebb and flow as the designer decides, maybe throwing in a string of easy holes where you expect to gain some strokes, and then some hard holes where you hope not to lose any. Perhaps on a harder hole you remind the player that there's OB in certain parts of the course, but its easy to avoid if they're careful (such as OB 25' to one side of a basket where you can choose to throw to the other side). Then, somewhere between hole 10 and 15, once you're certain the player is warmed up and knows how they're playing that day, you make them carry OB, but allow them to determine how much OB they want to carry. Think hole 10 at the 2022 OTB open. Within a few holes (I think 15-18) you force them to either carry the OB or layup to somewhere that doesn't allow birdie. The forced carry should require a top-tier drive for a player of that caliber.

What's this do? I think for most people the opening holes allow them to get into a flow/figure out their game that day. Using OB sparingly, you remind them that there's some tough shots coming to whet their appetite/stir their emotions. You then allow them bite off as much as they want to chew on a hole that allows them to carry OB for as far as they want. This allows them to test out how they are playing that day in a real scenario. Regardless, they know it's coming; that forced carry in a few holes. In the holes between the first and second OB carry, they'll have to work out any kinks and get ready for the coming challenge. When they get to that hole, they'll have to decide: do I want to go for it and risk my score, or am I going to wimp out. This will usually lead to three outcomes:

1. They go for it and make it. The thrill of accomplishment leaves them with great memories, and they'll want to come back to experience htat again.
2. They'll go for it and miss it, leaving them with that itch to come back to the course and try again.
3. They won't go for it, leading to them thinking they need to improve. They'll go work on their game with hopes of coming back and feeling the rush of accomplishment.

In any case, the player will leave with a desire to play the course again.

 

[Ryan P.]

to succinctly address your thoughts that OB and non-OB courses are the same to you: maybe that's your preference, or maybe there just aren't that many well designed courses so you don't notice the difference. I don't know which one it is, but I prefer less OB for the reasons I stated above.

 

[Cgkdisc]

Emulating ball golf was never high on my priority list, certainly not since I decided not to play ball golf.

The effect of OB goes far beyond the penalty strokes. At least, well-designed OB. It affects shot selection and routes as well.

So how would you split up the courses? A course with 2 holes with marginal OB on them? A course with 5 holes of serious but avoidable OB? A course with 11 holes that have OB, ranging from OB that only comes into play on really bad shots, to OB close to a basket? Perhaps Winthrop Gold is at one end of the spectrum, and a long OB-free course at the other. Finding the line to separate them would be tricky, measuring how many holes, how likely the OB, and sorting out OB lining fairways from OB close to greens from OB-carries from islands, all of various sizes.

For the record, my favorite courses range from those with lots of OB, to those with little or none. Whatever works for that property. I notice that I score better when I throw better, either way.

I would start by separating the courses into those that average less than 1 penalty stroke per player per round from those with greater than that value. The hardest part so far is to get administrators to recognize there is a difference. Once that is officially recognized, guidelines will develop for evaluating how well a course does what it's designed to do within its OB/No OB category, the same thing we do on DGCR with murkier "guidelines" for determining great or better from average on various factors.

It may not make any difference for recreational player enjoyment like a variety of course factors that appeal to some individuals and not others. However, if the PDGA and DGPT aspire to establish more professional credibility for the tour including better understanding of the internal dynamics of the game(s), how certain players fare within them, establishing course design requirements to attain better performance on several factors, and stats credible for legitimate gambling, they will need to go beyond just how holes should be designed to allow for spectators.

Note that ball golf spectators have mostly been standing/sitting inbounds for years such that shots landing where they're standing is typically not a penalty area. Why is that less common on the disc golf tour? (Certainly, insurability may sometimes play a role.) They could still mark paint lines, use rope or rows of ad panels indicating where spectators can stand behind but most of those lines don't have to define an OB line. It could be a free relief line. In fact, if they don't define an OB line it would avoid the issue of discs contacting objects and people affecting whether a shot gets accidentally deflected OB or back IB.

 

[Ryan P.]

Ah, we're bag to the tail wagging the dog -- ratings dictating course design.

I thought the main rationale for ratings, was to group players by similar skill level for fairer competition. Is there any documentation that the same players don't tendto do better on OB-full and OB-free courses -- with a discrepancy greater than other variations in course design, like length or openness, that might favor or hinder certain players?

Perhaps while we're at it, we can incorporate another style of course into a separate rating system -- one with rough tough enough that missing the fairway has about the same scoring effect as OB.

I don't care how we estimate. Rating was the first thing that came to mind when I thought about a metric for translating player skill from one course to another. If you know of a better one, suggest it.

Also, I wasn't suggesting that we use ratings to influence design to any noteworthy degree. I was suggesting that one small piece of the puzzle for less experienced designers to learn how to design according to skill level might be to use a scoring estimation formula to determine how players of the intended skill level might score on a hole. That would be balanced out by comparing the hole to other similar holes and by applying thoughtful ideas to the hole in order to know how players of a certain type will feel when they play that hole. As designers improved that estimator would become less and less important.

 

[DavidSauls]

@ Ryan P.

My posts were in response to Chuck, not you.

 

[DavidSauls]

I would start by separating the courses into those that average less than 1 penalty stroke per player per round from those with greater than that value. The hardest part so far is to get administrators to recognize there is a difference. Once that is officially recognized, guidelines will develop for evaluating how well a course does what it's designed to do within its OB/No OB category, the same thing we do on DGCR with murkier "guidelines" for determining great or better from average on various factors.

It may not make any difference for recreational player enjoyment like a variety of course factors that appeal to some individuals and not others. However, if the PDGA and DGPT aspire to establish more professional credibility for the tour including better understanding of the internal dynamics of the game(s), how certain players fare within them, establishing course design requirements to attain better performance on several factors, and stats credible for legitimate gambling, they will need to go beyond just how holes should be designed to allow for spectators.

Note that ball golf spectators have mostly been standing/sitting inbounds for years such that shots landing where they're standing is typically not a penalty area. Why is that less common on the disc golf tour? (Certainly, insurability may sometimes play a role.) They could still mark paint lines, use rope or rows of ad panels indicating where spectators can stand behind but most of those lines don't have to define an OB line. It could be a free relief line. In fact, if they don't define an OB line it would avoid the issue of discs contacting objects and people affecting whether a shot gets accidentally deflected OB or back IB.

It's quite a leap from saying the DGPT & other top-tier pro events should give serious thought to the course designs that best enhance their public presentation, and saying that courses incorporating OB into their design constitute a different game, for which we should have a separate player ratings system.

The DGPT isn't disc golf -- just a little corner of it. Nevertheless, I agree that they and the PDGA should certainly be looking at the courses they use, and the hole designs on them. This could include the usage of OB -- as well as other features (lengths, tunnels through woods, slopes, etc.).

I'm not convinced that OB is a demon, even there. Winthrop Gold catches a lot of flack, sometimes rightfully so, but it remains once of the most intensely-followed venues, and the holes spectators most want to see heavily feature OB.

And golf? (Ball) golf? That's golf.

We should worry about what's most fun to play, and most interesting to watch.

 

[Steve West]

I wanted to get a look at just how daunting a task it would be to predict scores before anyone plays a hole. We are not just trying to predict the average score, but the whole distribution. So, I looked at how the distribution of scores varies by length for 1000-rated players. These are clumped into cells to the nearest 10 feet of width, and the nearest 5% of frequency.

Spoiler

For example, based on experience, all holes that are 140 or 150 feet long generated 75% 2s.

On the other hand, holes that are 400 feet long sometimes generated 0% scores of 3, and sometimes 100% 3s, and just about everything else in-between.

Success for estimating scores for a 1000-rated player on a specific length of hole would be picking which of the 2s cells a certain hole belongs to, and which of the 3s cells, and 4s and 5s, etc.

Then, we can move on to other skill levels.

 

[Steve West]

Here's a good explanation of the factors assessed to determine a ball golf course rating during good mid-summer playing conditions:

https://www.mgagolf.org/handicapping/course-rating-top-ten

So they do all that to get a measure of difficulty of the course in total. Although they do it by adding up hole scores, the purpose is not really to estimate hole scores as the finished product. Let alone the distribution of scores.

The point is, estimating scoring distributions for each hole is a lot harder than estimating course difficulty. Even if disc golf scores were as closely related to length as golf score are.

 

[Cgkdisc]

So they do all that to get a measure of difficulty of the course in total. Although they do it by adding up hole scores, the purpose is not really to estimate hole scores as the finished product. Let alone the distribution of scores.

The point is, estimating scoring distributions for each hole is a lot harder than estimating course difficulty. Even if disc golf scores were as closely related to length as golf score are.

From a design standpoint, determining in advance whether a potential hole is in the correct effective length range for a specific skill level to produce birdies in the 10% -50% range is all that's necessary as a foundation to allow players of that skill level to "score." Then, orchestrate the other design elements available to make each hole fun, interesting and challenging (not punitive) for that skill level to where each hole provides a unique experience for variety and balance among the 18 holes.

If you can do this without needing OB or mandos, then kudos. If you want/need OB, then at least place it so only lesser skilled players get penalized. For boundary OB lines that must be included, if they'll likely produce random penalties, then don't make that line or area a penalty. Provide free relief at last point IB or to a drop zone so the experience is more like the randomness of hitting a tree and not an immediate penalty.

 

[FluentDisc]

OB can be a deeply personal topic, so take this with a grain of salt, it's good to talk about! With the exception of Ball Golf Courses and their inherent shortage of obstacles, OB seems to be the default answer for designers that don't/can't put enough time or thought into natural consequences. IMO the primary reason for this is designers that aren't doing this for a profession, and simply don't have the time to spend 3 or 4 days looking closely at the foliage and taking more time to consider different routings or hole layouts. The result is often, lines that are too easy, being artificially made more difficult with OB rather than through natural consequences. Shoehorning more complicated courses into spaces that don't work well for them is a common culprit.

Like it's been stated above, too much OB gets really frustrating for the vast majority of players in the same way that too much brush on the sides of holes does, it eliminates the chance for recovery. There needs to be a fair balance between penalizing a bad throw, and providing the opportunity to recover from that shot. In most cases, preventing the opportunity for the birdie is good enough, but OB putting the player into bogie territory automatically is doubling up on the concept of score division.

OB is also too often used like Mando's, as a way to make a course "safer", but both are misnomers... just because you put it there doesn't mean people wont through into it. OB should still be a tool in our design toolbox, but it, like Mando's, should be a last resort. Thoughtful use of natural consequences is a fairer, and more enjoyable game for the player.

There's another concept for a better way to use OB that Chuck Kennedy outlined on a call recently, the idea of a no-penalty OB zone. Where you don't take any stroke penalty for landing OB, but you are pulled back to a drop zone that gives you no chance at the birdie. When we treat our game like ball golf and the red stake areas where you simply pull out of the zone where it entered, it's too often too easy to recover and so it's almost encouraging people to always go for it rather than consider the risk. If however your drop puts you out of contention on the hole but doesn't add strokes, the player has to think about how to finish the hole out.

Kevin

 

[FluentDisc]

OB design segway aside, OB makes it harder to predict scoring because it's more often than not a random element that really depends on the player. For some players, it can ruin the head game and cause them to tank the rest of their round, for others, it's the match under their ass that causes them to think harder. The only thing it really does is spread scores out, but not in a meaningful way like hitting a primary line vs. a secondary one and the added complexity of the hole comes into play. Think about how wind on a hole with OB affects scores, such as those cringeworthy vids of baskets on mounds on a wide open golf course, with OB right at the bottom of the mound with no physical barrier. That use of OB is can punish great throws and separate score, but is it a true measure of the difficulty of the course when it's all added up? When the hole scores on a par 3 are averaging 4.5, does that mean that the course is really difficult and in the league of a more thoughtfully designed course, or simply punitive with inflated scores. If you lean on the course design docs at the PDGA and discgolfcoursedesigners, distances and pars, foliage and density, elevation, etc. and then add a touch of experience in understanding the different ways a hole will get played are the only real solid way of estimating scores / setting par IMO.