[Request] Let's Settle The Ratings Mystery

[araytx]

"Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes." - Goodhart's Law.

For the same reasons that the stock market is a garbage indicator of economic health right now, any system you can imagine in order to critique would be prone to the exact same criticisms you're applying to the PDGA rating system. When you start targeting the system with explicit factors designed purely for its manipulation: it ceases to be a good system.

Any system with enough completely ridiculous parameters and inputs applied is going to crumble.

Exactly. You can't trick the math, and anything done intentionally to try and "manipulate" what "the ratings would be" is an experience in assumptions for which the current PDGA Player Ratings system was not designed; and accordingly, does not apply.

Off topic:

So this proves the competition during the Climo era was better than the competition during the McBeth era. opcorn:

No sir, I don't see it.

You two guys, quit trying to hijack D50's thread. If you want to play in your own playpen there's a button called "Post a New Thread" available to you.

 

[ChrisWoj]

The new frontier for the sport will be breaking out ratings by course type to better see these effects and decide how they should be used to improve the elite tours.

I'm also gonna go off topic:

I think that's just a shift in the same world. The new frontier, to me, comes from the breadth of the data available. The ability to develop a metric measuring player skill based on putting, approach, and driving elements and weighted according to the amount that each contributes to the variance in a round score.... that's where I see an actual "new frontier" for disc golf metrics. It comes from the expanded numbers available to us.

And yes I'm aware that's realistically limited to the pro fields reliably on UDisc putting in more discrete information, so it isn't really something that the PDGA is going to be able to implement on such a broad scale as the rating (which is why I see the rating having continued importance moving forward, getting what it does out of just a few easy to record factors is really solid).

 

[ChrisWoj]

You two guys, quit trying to hijack D50's thread. If you want to play in your own playpen there's a button called "Post a New Thread" available to you.

The universe is constantly revealing to me its mysteries...

 

[Jay Dub]

Payback is fun.

 

[araytx]

The universe is constantly revealing to me its mysteries...

Oh, my gosh. I haven't laughed this hard, Woj, in a while. Thanks for that.

 

[Jay Dub]

No sir, I don't see it.

Numbers don't lie

 

[ChrisWoj]

Numbers don't lie

The same numbers can also be interpreted as meaningless with regard to your point. There's a lot left to the imagination in the few numbers I mentioned.

 

[DiscFifty]

Thanks for all the comments. I learned alot from all the posts. Good stuff. :clap::thmbup:

 

[DiscFifty]

Was looking at one of my fav player's (Simon Lizotte) stats and I ended up scanning over his ratings history and I saw alot of rounds not included for his ratings. But this event has me puzzled....

https://www.pdga.com/tour/event/44224

He shot a 975 rated round at an A tier event (DGLO) in round 2 which was not included. Rounds 1 (1032) and 3 (1024) were included. His rating at the time was 1045, certainly the 975 was not too low to be included. Why wouldn't round 2 be included in his ratings?

 

[Discette]

Was looking at one of my fav player's (Simon Lizotte) stats and I ended up scanning over his ratings history and I saw alot of rounds not included for his ratings. But this event has me puzzled....

https://www.pdga.com/tour/event/44224

He shot a 975 rated round at an A tier event (DGLO) in round 2 which was not included. Rounds 1 (1032) and 3 (1024) were included. His rating at the time was 1045, certainly the 975 was not too low to be included. Why wouldn't round 2 be included in his ratings?

The round was more than 2.5 standard deviations below his current rating. This basic nformation about ratings was found by reading the PDGA FAQ's on ratings.




https://www.pdga.com/faq#t133n207051




[qoute]All rated rounds you have played and have been reported to the PDGA within 12 months of your most recently rated round will be Included in your rating calculation. However, if any one of those ratings is either more than 100 points below your average rating or more than 2.5 standard deviations below your rating – whichever number is smaller – that round will not be included in your current rating update and indicated with a No.[/quote]

 

[DiscFifty]

The round was more than 2.5 standard deviations below his current rating.

Thank you! Now...any idea what a standard deviation is?

 

[brutalbrutus]

Thank you! Now...any idea what a standard deviation is?

Google probably knows.

 

[Jugular]

Thank you! Now...any idea what a standard deviation is?

It's the square root of the average of the distance of each point in the distribution from the mean.

Most people assume a Normal distribution (or bell curve) when talking about standard deviations and I would assume that's the case here, but it could easily be something else.

2.5 standard deviations below the mean in a Normal distribution should mean all but the 0.31% worst rounds should be included in their rating. i.e. if they played 10,000 rounds the 31 worst rounds would be discarded.

Exactly how they're calculating the standard deviations for each player I'm not sure, I would suspect that by now if there were a fixed ratio or rating point difference it would have been deduced by now. In which case it may be based on fitting a distribution to their range of rounds. i.e. if a player is less consistent then a round would have to be worse to be discarded than a round from a more consistent player.