Hey hey ho ho round ratings have got to GO!

[txmxer]

So your definition of a top player is one that wins a lot but not with any reference to who they beat.

I think of a top player as someone who demonstrates a high skill level. The player ratings do this perfectly. They unfortunately are not as dynamic and changing as 'top player' lists so don't get people excited. You'll find most discussions of who top players are tends to centre around people claiming that someone else should be the top player because they demonstrate more skill. Unfortunately the player ratings are too good to provide this sort of argument.

I don't know much about the PGA ranking system, but I do recall Tiger being ranked #1 for a long time after his fall from grace.

Basically there is a LOT of lag in that system for a player that is as dominant as he was. I think tennis has similar rankings lag.

Less than ideal IMO.

But, I can see the point that the ratings imply skill level and possibly overlook some intangibles.

Nothing is perfect.

 

[DG_player]

So your definition of a top player is one that wins a lot but not with any reference to who they beat.

That's not the case at all.

Just to clarify the system: the ranking points are based off of the quality of the field. So if you win a tournament with most of the top players in the world present, you get a lot of ranking points. If you win a junior tour event you'll get much much less.

 

[DG_player]

I don't know much about the PGA ranking system, but I do recall Tiger being ranked #1 for a long time after his fall from grace.

Basically there is a LOT of lag in that system for a player that is as dominant as he was. I think tennis has similar rankings lag.

Less than ideal IMO.

But, I can see the point that the ratings imply skill level and possibly overlook some intangibles.

Nothing is perfect.

There's a two year lag, where your event points are reduced incrementally. So if you win a major it's worth 100, after a year it would be worth around 50, and near the end of two years it would be worth like 1. It works pretty good right now, the best player playing the best seems to be consistently ranked #1. Tiger was just so unbelievably dominant before his injuries and personal problems, his results, even after their value being cut in half, still ranked him higher than anyone else at the time.

I mean, Paul McBeth could have stroke and lose the use of his throwing arm, he'd probably remain the top rated player until such time as his scores rolled out of the PDGA system (however long that takes?).

 

[txmxer]

There's a two year lag, where your event points are reduced incrementally. So if you win a major it's worth 100, after a year it would be worth around 50, and near the end of two years it would be worth like 1. It works pretty good right now, the best player playing the best seems to be consistently ranked #1. Tiger was just so unbelievably dominant before his injuries and personal problems, his results, even after their value being cut in half, still ranked him higher than anyone else at the time.

I mean, Paul McBeth could have stroke and lose the use of his throwing arm, he'd probably remain the top rated player until such time as his scores rolled out of the PDGA system (however long that takes?).

No expert, but the round ratings reflect the competition played against—thus the system seems that it does consider that only not as explicitly.

Just my understanding—I could be completely wrong.

 

[hiflyer]

if paul is gonna say "ratings don't matter!" I think it's pretty fair to point out that paul has financially benefitted a number of times by (a sponsor) specifically promoting his rating. terry could've asked a bit differently but that's the gist of it, and instead of a good/reasonable explanation we get this ¯\_(ツ)_/¯

if he has addressed it in further detail elsewhere, let me know and I'll give it a listen

this was 6 years ago and of course people are allowed to change their minds. but still kinda funny

https://www.innovadiscs.com/team-news/paul-mcbeth-sets-a-new-standard-of-excellence-1050-rating/

Feeling a bit of a shifting attitude toward Paul lately.

HEY!! Paul!!!
Don't let money change ya!!!

 

[araytx]

Actually, it's bothish. If I understand it.

The criterion is established as the SSA of the course. The SSA depends on propagator scores and prior ratings, so that's kind of normative. (I think.) That SSA alone establishes the points per throw (PPT). Round ratings are 1000 plus (SSA - score) times PPT.

So, maybe criterion-referenced with a normative criterion?

If ratings were purely normative, they would take into account the dispersion of all propagator scores. They don't. That's the biggest gap between the pragmatic ratings formula and a true statistical best estimate of player skill.

But, it seems to work OK.

So far.

We should keep an eye on it, though.

Steve, I think the SSA is NOT, however, an independent variable in this case. If I understand it correctly, the formula finds "how the course is playing" that day/that round/ that whatever, calculates an SSA or something similar, then utilizes that SSA for further calculations. That would make SSA in this scenario a dependent variable, would it not, Steve?

 

[teemkey]

Steve, I think the SSA is NOT, however, an independent variable in this case. If I understand it correctly, the formula finds "how the course is playing" that day/that round/ that whatever, calculates an SSA or something similar, then utilizes that SSA for further calculations. That would make SSA in this scenario a dependent variable, would it not, Steve?

Technically SSA is a summary statistic, not a variable.

 

[Steve West]

...the formula finds "how the course is playing" that day/that round/ that whatever, calculates an SSA ..

That's what I said. It does that based on a secret formula applied to the propagators' scores and prior ratings.

 

[adamgservo]

This is a hard concept to explain, but the example discussed here--a field of 800 rated players shooting the same score as a field of 1000-rated player--does not expose a flaw in the round rating system. Here's my best shot at a clear explanation:

Round ratings, like all statistics, are an estimate rather than a clearly observable and defined phenomena. Your round rating on any particular day isn't an exact figure, it's just the best estimate of how good your round was, based on the data. As more and more data comes in, the estimate will get more and more accurate until it's essentially perfect.

If a statically impossible event occurs--and an entire field of rec players averaging the exact same scores as McBeth, Mcmahon, Wysocki, Heimburg, and Dickerson is certainly impossible--the statistic based on that event will indeed produce a wildly inaccurate estimate. But this doesn't invalidate the field of statistics.

For example, imagine you wanted to create a statistic that measured the number of eyes that a person has but you only used 18th century pirates as your sample population. Your statistic would estimate that people have an average of 1.0000 eyes per person, which is of course a little low. The problem isn't the statistic though, the problem was a weird and unrepresentative data set. As more and more data comes in, the estimate will better and better,

Did that make sense? No? We really need a statistics teacher in here.

Right. In other words, statistics never converge to individual truths, but statistics always converge to general truths.

Since I sit around and write code all day anyway, I took 5 minutes to run a statistics simulation. Take an ideal coin giving 50% heads and 50% tails. Flip it 10 times. How many heads do you get? There is no right answer. Now do the same experiment, and flip the coin 1 million times. How many heads do you get? 500k +/- some tiny amount. I ran the experiment and did 100 simulations of 10 coin flips and 100 simulations of 1 million coin flips. What was the max percent variance from the expected result of 5 heads and 500k heads on the 100 simulations? For the 10 flips, I saw 80% variation. Meaning 9 heads. (9-5)/5*100. For 1 million flips, I saw a max of 0.33% variation. Basically meaning it was exactly 500k each time.

What is the point of the simulation? If a single 1000 rated player and a single 800 rated player play the same course, same conditions but get rated differently: sure, they could get the same score and have completely different ratings if rated separately. What if ten 1000 rated players and ten 800 rated players now play? Mathematically sure you can break the system by pretending they can get the same scores, but it is not statistically possible. Then think of how many players are getting rated every weekend in tournaments. Each one building a better statistical foundation. The rating system is fine.

 

[araytx]

Technically SSA is a summary statistic, not a variable.

Well, duh. Isn't that the same thing in a statistical analysis? (dependent variable, that is) :wall:

Right. In other words, statistics never converge to individual truths, but statistics always converge to general truths.

Since I sit around and write code all day anyway, I took 5 minutes to run a statistics simulation. Take an ideal coin giving 50% heads and 50% tails. Flip it 10 times. How many heads do you get? There is no right answer. Now do the same experiment, and flip the coin 1 million times. How many heads do you get? 500k +/- some tiny amount. I ran the experiment and did 100 simulations of 10 coin flips and 100 simulations of 1 million coin flips. What was the max percent variance from the expected result of 5 heads and 500k heads on the 100 simulations? For the 10 flips, I saw 80% variation. Meaning 9 heads. (9-5)/5*100. For 1 million flips, I saw a max of 0.33% variation. Basically meaning it was exactly 500k each time.

What is the point of the simulation? If a single 1000 rated player and a single 800 rated player play the same course, same conditions but get rated differently: sure, they could get the same score and have completely different ratings if rated separately. What if ten 1000 rated players and ten 800 rated players now play? Mathematically sure you can break the system by pretending they can get the same scores, but it is not statistically possible. Then think of how many players are getting rated every weekend in tournaments. Each one building a better statistical foundation. The rating system is fine.

Exactly what I've been trying to tell these 'let's run a scenario' people. By the definitions is just not possible. It's like the old joke:

"...but there is no..."

 

[Jugular]

Well, duh. Isn't that the same thing in a statistical analysis? (dependent variable, that is) :wall:



Exactly what I've been trying to tell these 'let's run a scenario' people. By the definitions is just not possible. It's like the old joke:

"...but there is no..."

Sorry to be a pedant but the statistics suggest it is possible, it's just incredibly unlikely, perhaps a 1 in 10 trillion+ tournament event. Therefore, in a practical sense impossible but "by the definitions" not impossible. Whether or not you believe that the model represents the real world such that these twenty players could play such a round is another question. In theory twenty such players could conspire to score the same for a round, and because of that I suspect it's actually more likely than the model would suggest since the model is based (we must assume) and expects true competitive rounds.
That very unlikely scenario could still end up rated (though the 1000 rated players will probably have their round ratings dropped from their record since it'll be 2 std devs outside of their expected level of play).

 

[araytx]

Sorry to be a pedant but the statistics suggest it is possible, it's just incredibly unlikely, perhaps a 1 in 10 trillion+ tournament event. Therefore, in a practical sense impossible but "by the definitions" not impossible. Whether or not you believe that the model represents the real world such that these twenty players could play such a round is another question. In theory twenty such players could conspire to score the same for a round, and because of that I suspect it's actually more likely than the model would suggest since the model is based (we must assume) and expects true competitive rounds.
That very unlikely scenario could still end up rated (though the 1000 rated players will probably have their round ratings dropped from their record since it'll be 2 std devs outside of their expected level of play).

No. Impossible. What evidence do you have that it is possible? You're saying a group of "better players" will play worse than a group of "worse players." That doesn't make sense. If they "played worse," then they weren't "better players" to begin with.

It's Like a DNA paternity test. If you are 99.999% "NOT EXCLUDED from being the father," that means that "YOU ARE the father." It's a factoid. And no need to apologize. None at all. I take contrary discussion as a positive thing.

 

[Jugular]

No. Impossible. What evidence do you have that it is possible? You're saying a group of "better players" will play worse than a group of "worse players." That doesn't make sense. If they "played worse," then they weren't "better players" to begin with.

It's Like a DNA paternity test. If you are 99.999% "NOT EXCLUDED from being the father," that means that "YOU ARE the father." It's a factoid. And no need to apologize. None at all. I take contrary discussion as a positive thing.

The evidence that with enough money + cajoling + threatening (I assure you that I don't have the power, wealth or desire to make this happen) I could convince twenty people (10 of them rated 800 and 10 rated 1000) to "play" a round in a tournament I've set up and get an identical score. It would be utterly futile and tell us nothing at all about the rating system but it is possible.

Even without this, someone could design a course on which it is probable for competitive players to score the exact same score. Say 18 x 10' holes as an example. Again, stupid, but possible, i.e. not impossible.

Further, in a real world tournament, on a proper course with competitive players, I do not expect to ever see this occur in my lifetime nor that it would occur before the heat death of the universe, but it's not impossible.

Also your understanding of 99.999% is wrong. Regardless of all the real world possibilities for error that percentage is leaving open the possibility of an alternative result. In a practical sense you should feel supremely confident that you are the father but it is not a certainty. By DEFINITION.

I suggest you don't say 'by definition' and ignore the definition.

 

[txmxer]

I see what is being suggested--mathematically it is possible. Historically, nothing of the sort has happened.

An individual could have a horrible day with an anomalous score, but groups have shown to be highly consistent for 20+ years and round after round (based on what Chuck said in the Nick and Matt interview).

 

[Monocacy]

No. Impossible. What evidence do you have that it is possible? You're saying a group of "better players" will play worse than a group of "worse players." That doesn't make sense. If they "played worse," then they weren't "better players" to begin with.

It's Like a DNA paternity test. If you are 99.999% "NOT EXCLUDED from being the father," that means that "YOU ARE the father." It's a factoid. And no need to apologize. None at all. I take contrary discussion as a positive thing.

Extremely unlikely is not the same as impossible. For example, it is possible (although stupendously unlikely) that all of the air in a room randomly packs itself into the corners of the room and you suffocate.

Am I worried about that? No, but it is possible, and the probability can be estimated (spoiler alert: something like 1/(10^10^27) for an average-sized room).

In practical terms you are correct; to a first approximation that which is sufficiently unlikely can be treated as if it was impossible.

 

[teemkey]

Well, duh. Isn't that the same thing in a statistical analysis? (dependent variable, that is) :wall:

No it's not. Once you have the player's scores, SSA will only vary if you mess up the calculations.

 

[Jugular]

Just for fun I figured we could consider a set of Bernoulli trials.
Assume:
All holes played by each player are independent (a big assumption)
Each player's score is independent of all others (another big assumption)
There is an 18 hole course where the following are true (probably an ok assumption, but I bet there's good data somewhere to be used instead):
- 800 rated players score par 20% of the time (and mostly worse the rest)
- 1000 rated players score par 20% of the time (and mostly better the rest)
The tournament consists of one round.
We only count the 'same score' as all players scoring par every hole (another big assumption)

Then using a Binomial distribution with 360 (20 x 18) trials and probability 20% the probability of all trials being successful is 2.35 x 10^(-252) or 1 in 4.26 x 10^251. For a reference size the number of particles in the observable universe has been estimated as 3.28 x 10^80. So if every particle in the observable universe had it's own observable universe and all the particles in those universes had their own observable universe then each of those particles run 12 billion of these tournaments, we should expect that one of them would have this result. So, definitely possible.

 

[txmxer]

 

[adamgservo]

Just for fun I figured we could consider a set of Bernoulli trials.
Assume:
All holes played by each player are independent (a big assumption)
Each player's score is independent of all others (another big assumption)
There is an 18 hole course where the following are true (probably an ok assumption, but I bet there's good data somewhere to be used instead):
- 800 rated players score par 20% of the time (and mostly worse the rest)
- 1000 rated players score par 20% of the time (and mostly better the rest)
The tournament consists of one round.
We only count the 'same score' as all players scoring par every hole (another big assumption)

Then using a Binomial distribution with 360 (20 x 18) trials and probability 20% the probability of all trials being successful is 2.35 x 10^(-252) or 1 in 4.26 x 10^251. For a reference size the number of particles in the observable universe has been estimated as 3.28 x 10^80. So if every particle in the observable universe had it's own observable universe and all the particles in those universes had their own observable universe then each of those particles run 12 billion of these tournaments, we should expect that one of them would have this result. So, definitely possible.

Ha ha. In other words, we don't have to worry about a group of 800 rated players getting the same score as a group of 1000 rated players.

 

[DiscFifty]

Ha ha. In other words, we don't have to worry about a group of 800 rated players getting the same score as a group of 1000 rated players.

Of course not, my point about it being mathematically possible was to point out the flaw/bug/anomaly whatever you want to call it that exists in the ratings system, that as long as divisions are rated separately, identical scores can yield different round ratings, and will always favor the division with higher rated players. As far as inflation, bubbling up, whatever you want to call it, it's pretty easy to grasp the concept that as long as the higher rated players are always rated among themselves (DGPT events for example) combined with a ratings system that knows no limits, well...the rich will get richer.