Why Ratings are awful

[iacas]

Detailed information on how the ratings work is out there, you just have to have collected the data over the years. The PDGA is correct in protecting their assets here. If some web sites hadn't copied the PDGA course directory database to rebrand, the PDGA might have not had to become so tight lipped about their other IP.

Once the formula's cracked, there's really no reason to keep the IP private. They haven't patented the formula, have they?

And of course, this site now has a better course database than the PDGA, which still doesn't list (for example) Oil City.

I say publish the formula. Or come up with a new formula that makes more sense to people and allows for relatively easy handicapping.

 

[bikinjack]

It works okay. But the fact that there is secret sauce in the mix brings it a bit below fine.

The formula should be public. Nothing professional about making it obscure.

Chuck will say that it is a members benefit, but from my point of view the benefit of having it completely transparent completely outweighs any "loss" I would suffer from non-members being able to calculate their rating.
The latter would be completely fine by me - and I suspect PDGA would also be better off, as that would invariably introduce the concept to more players.

I couldn't agree more.

I also can't comprehend why ratings aren't updated every time a tournament report is sent in and the results made official. Why is there a wait for ratings updates?

 

[krupicka]

Ratings updates are mostly automatic, but there is a manual process due to the large number of errors made by TDs entering data. Catching these requires man power.

TDs also have 30 days to turn in their reports.

(I'd like to see updates happen more frequently myself. Just explaining some of the issues).

 

[bikinjack]

Ratings updates are mostly automatic, but there is a manual process due to the large number of errors made by TDs entering data. Catching these requires man power.

TDs also have 30 days to turn in their reports.

(I'd like to see updates happen more frequently myself. Just explaining some of the issues).

None of this has anything to do with making the ratings closer to real time. As soon as that man power does its required thing, ratings are updated for every player on that report. Done. Why wait to batch it all together with every other TD report in the entire world?

 

[Lewis]

None of this has anything to do with making the ratings closer to real time. As soon as that man power does its required thing, ratings are updated for every player on that report. Done. Why wait to batch it all together with every other TD report in the entire world?

Imagine Player A and Player B both play events on July 10th. Player A's TD gets his report in immediately, and his rating and statistics are updated by July 17th. Player B's TD drags his feet and doesn't get his report in until August 7th, so his rating and statistics are not yet updated. Meanwhile, both players compete against each other on July 24th, in the same tournament, and they are both propagators. Does this cause problems for the round ratings on July 24th? Would it only cause problems if Player B's rating going into July 24th would have significantly changed by his round ratings from July 10th? I don't know, but it's possible Chuck only uses data going back to the last date with all TD reports complete, to avoid dealing with these questions.

Or maybe there is just the simple problem of the manpower it takes to manage the data. It would take a lot more manpower to update 12 times a year than 4, and much more to update every day. Maybe they just don't have the resources to do any more than they do. How much more would you be willing to pay in annual or event fees if they would hire people to do update the player ratings as a full-time job?

 

[wake911]

None of this has anything to do with making the ratings closer to real time. As soon as that man power does its required thing, ratings are updated for every player on that report. Done. Why wait to batch it all together with every other TD report in the entire world?

likely because the PDGA is a much smaller organization than we think it is, and telling us that it will be official within a week, would put them under a time crunch they can't guarentee to meet. By setting a once a month update, they allow themselves to hit it, thus keeping us from complaining "you promised every week, why isn't my round there"

I think the TD's should have to update quicker. Still rating on my best rounds ever to show up as unofficial, and is pushing 2 weeks out.

 

[Lewis]

Chuck will say that it is a members benefit, but from my point of view the benefit of having it completely transparent completely outweighs any "loss" I would suffer from non-members being able to calculate their rating.
The latter would be completely fine by me - and I suspect PDGA would also be better off, as that would invariably introduce the concept to more players.

They already publish the round ratings for every event on their website, so non-members can look up all their round ratings and calculate what their rating would be on their own. They have also already shared enough information about how the ratings system works that it pretty much has been reverse-engineered. The only difference between someone with the reverse-engineered formula and the PDGA themselves is the event database and appropriate software to run the calculations. Who wants to spend their personal time and money building all that, when they can spend a few $ to join the PDGA and have the service done for them?

 

[bikinjack]

Imagine Player A and Player B both play events on July 10th. Player A's TD gets his report in immediately, and his rating and statistics are updated by July 17th. Player B's TD drags his feet and doesn't get his report in until August 7th, so his rating and statistics are not yet updated. Meanwhile, both players compete against each other on July 24th, in the same tournament, and they are both propagators. Does this cause problems for the round ratings on July 24th? Would it only cause problems if Player B's rating going into July 24th would have significantly changed by his round ratings from July 10th? I don't know, but it's possible Chuck only uses data going back to the last date with all TD reports complete, to avoid dealing with these questions.

Or maybe there is just the simple problem of the manpower it takes to manage the data. It would take a lot more manpower to update 12 times a year than 4, and much more to update every day. Maybe they just don't have the resources to do any more than they do. How much more would you be willing to pay in annual or event fees if they would hire people to do update the player ratings as a full-time job?

The exact scenario you describe here happens now. How many times have you read about TDs not getting their report in quickly and the rounds not counting toward the rating update. As for the manpower thing, is it not more or less a matter of proof reading the report, investigating any errors, and then plugging the numbers into a database? That shouldn't change the amount of time it takes to process an individual report nor should it change the amount if time it takes to process the second or third or forty-sixth report.

 

[Dave242]

They already publish the round ratings for every event on their website, so non-members can look up all their round ratings and calculate what their rating would be on their own. They have also already shared enough information about how the ratings system works that it pretty much has been reverse-engineered. The only difference between someone with the reverse-engineered formula and the PDGA themselves is the event database and appropriate software to run the calculations. Who wants to spend their personal time and money building all that, when they can spend a few $ to join the PDGA and have the service done for them?

Well....there is the DGCR Player Ratings which give you an estimation of your skill level simply accomplished by entering rounds you have played.....and a closer estimation by entering many rounds on a wide variety of courses.

To get a good idea of what your PDGA Player Rating probably is you no longer have to take the time or expense to play lots of PDGA-sanctioned events, nor do you have to pay annual membership dues.

Does this hurt the PDGA? I would say "no" - it is free advertising for the concept of Player Ratings......and you still have to go the PDGA route to get a real/legit/official Player Rating.

 

[ScottyLove]

Yup... still awaiting a delinquent TD report from the first weekend in June!

Newbie TD didn't understand that a report was required and thought updating the scores was the end. Oh yeah... doesn't have the money to pay PDGA. Meanwhile I'm still tied for 2nd in Missouri for Adv. Masters points when I should be in 2nd all by myself... and that's what's really important, right?

 

[mattc]

Talk about a sport with diversity---3% of tournament players can't add a scorecard full of 2s, 3s, and 4s correctly....meanwhile calculus debates are breaking out.

nice point

 

[DonCasper]

I haven't read the whole thread, but I've yet to see a good argument for "why ratings are awful" ... lots about why they're not perfect ... lots about why they're good ... not much about why they're "awful" ...

I'm sure the people that have read the whole thread can point me right to the post I didn't read.:|

My biggest beef with the ratings is how unapplicable they are to the real world. I can go bolfing with a bunch of my friends and they'll give me 10 strokes because of my handicap. If we go disc golfing, I get no strokes even though I usually shoot 3 strokes worse on any course because I don't have any justification beyond averaging the last few rounds.

Add to the fact that you can only get the roughest of ideas of what your rating is without playing in a tournament, and I'd say that ratings are pretty terrible way to handle this.

I understand that a lot of factors can change on a course over its lifetime (or even in a given week), but I don't think that excuses a lot of the flaws of the current system.

 

[mashnut]

My biggest beef with the ratings is how unapplicable they are to the real world. I can go bolfing with a bunch of my friends and they'll give me 10 strokes because of my handicap. If we go disc golfing, I get no strokes even though I usually shoot 3 strokes worse on any course because I don't have any justification beyond averaging the last few rounds.

Add to the fact that you can only get the roughest of ideas of what your rating is without playing in a tournament, and I'd say that ratings are pretty terrible way to handle this.

I understand that a lot of factors can change on a course over its lifetime (or even in a given week), but I don't think that excuses a lot of the flaws of the current system.

I'm a little confused, don't you have to join a club to get a golf handicap? Also, once you do have a disc golf rating it's not hard to use that to do some basic handicapping. Or you could keep track of your round scores and use the many freely available handicapping spreadsheets that people have posted here and on other disc golf sites.

The ratings system is not designed to handicap casual rounds, that's not one of the things it was actually designed to do. It's intended to measure skill levels in a competitive setting in a way that's portable among the very different types of courses our sport is played on.

 

[DonCasper]

I'm a little confused, don't you have to join a club to get a golf handicap? Also, once you do have a disc golf rating it's not hard to use that to do some basic handicapping. Or you could keep track of your round scores and use the many freely available handicapping spreadsheets that people have posted here and on other disc golf sites.

The ratings system is not designed to handicap casual rounds, that's not one of the things it was actually designed to do. It's intended to measure skill levels in a competitive setting in a way that's portable among the very different types of courses our sport is played on.

You do have to join a club, but that isn't any different than joining the PDGA. There is a huge benefit from having a handicap from an official source when it comes to betting, even if the bet is just who buys drinks at the end of the round.

I think the fact that the system wasn't designed to handicap casual rounds is a major flaw. Another major flaw is that it can't be derived from a casual round. Both of these are things the bolf handicap does well.

I understand the massive variety of courses, the huge effect weather can have on a round, and the transient nature of some layouts- but I don't think these problems are insurmountable. There is an immense variety in ball golf courses as well, from windy scottish-style links courses with lots of elevation and no trees, to short community courses with wide tree-lined fairways.

Beyond that, the pin location is moved daily on a ball golf course, which can change the difficulty immensely for some holes. Despite all of these challenges, ball golf still manages to have a handicappable rating (and a course rating with slope).

 

[emook]

does anyone know when the next update is? pdga didn't list it like they normally do.

 

[DavidSauls]

I think the fact that the system wasn't designed to handicap casual rounds is a major flaw. Another major flaw is that it can't be derived from a casual round. Both of these are things the bolf handicap does well.

Of all the flaws in the system, not doing what they were not intended to do wouldn't seem to me to be one.

I don't know how much demand there is for handicaps. I know that none of it comes from me. But there are a bunch of ways to create them, if that's what you want. You can do it from the ratings, if you have them. I think Disc Golf United has a handicapping system. If you and your betting opponent play the same course regularly, you can create them yourself.

If you don't play tournaments, you can get ratings from league play in many places. Otherwise, you can derive them by comparing your casual rounds to tournaments on the same course. I think the PDGA app will create them for you, too.

 

[DavidSauls]

I understand the massive variety of courses, the huge effect weather can have on a round, and the transient nature of some layouts- but I don't think these problems are insurmountable. There is an immense variety in ball golf courses as well, from windy scottish-style links courses with lots of elevation and no trees, to short community courses with wide tree-lined fairways.

One of the most insurmountable, or at least difficult to surmount, is the lack of a universally agreed upon and applied definition of "par". See 8,349 other threads for detail, but I don't see any way the PDGA can get five thousand courses in line with their par designations.

 

[Dave242]

And you can get a Player Ratings right here from DGCR when you enter your rounds in the scorebook.If you have say 20 rounds recorded on 5+ courses, your DGCR Player Rating will be very close to what your PDGA Player Rating would be.

 

[DonCasper]

And you can get a Player Ratings right here from DGCR when you enter your rounds in the scorebook.If you have say 20 rounds recorded on 5+ courses, your DGCR Player Rating will be very close to what your PDGA Player Rating would be.

One more reason I need to start entering my rounds here.

But on a fundamental level I think changing the way ratings work is important for the growth of the sport. Not to throw out buzzwords, but I think having a value-less number score as your rating rather than having a value in strokes makes the sport less approachable.

Quick question: How much better can you expect a player that is 880 rated to play than one that is 860 rated to play (on an average course)? 20 better?

How much better can you expect a player that has a 25 stroke handicap to play than one with a 22 stroke handicap to play (on an average course)? 3 strokes better, on average. And if a course is harder or easier than average you can figure out how much that would affect them as well.

 

[megamerican]

does anyone know when the next update is? pdga didn't list it like they normally do.

I can't remember the exact date but it was pushed back from mid September to late September so the ratings will be as up to date as possible for the USDGC.