#21  
Old 05-21-2018, 01:15 PM
wellsbranch250's Avatar
wellsbranch250 wellsbranch250 is offline
Birdie Member
 
Join Date: Jul 2016
Location: Huntsville Alabama
Years Playing: 5.8
Courses Played: 412
Throwing Style: RHBH
Posts: 461
Niced 752 Times in 274 Posts
Default

I was totally surprised by Blackfalls. The one renegade bad review was offset by a bunch of driveby 5s. I really want to play that course. I've wishlisted it, it looks frig'n awesome.

I should be able to compose an excel graphic showing a comparison between current ratings and the adjusted formula method. its just a matter of doing a screenshot of my excel document. There were a few major changes upwards. Maple hill, the Higebridge courses and the IDGC were all heavily boosted using the weighted formula method.

Quote:
Originally Posted by Three Putt View Post
I wasn't so much concerned where on the list they fall but what it does to the rating. I think the assumption was that if you minimize that one weird review of Black Falls it would bump up the rating, but in this case where you minimize both the weird review and the drive by reviews it actually drops the rating from 4.53 to 4.43. The Woodshed has a nominal bump from 4.28 to 4.30. I don't think that is the kind of change a lot of people were expecting.
Sponsored Links
Reply With Quote
  #22  
Old 05-21-2018, 01:26 PM
BogeyNoMore's Avatar
BogeyNoMore BogeyNoMore is offline
* Ace Member *
 
Join Date: Jul 2009
Location: Walled Lake, MI
Years Playing: 15.5
Courses Played: 314
Throwing Style: RHBH
Posts: 9,887
Niced 2,271 Times in 1,122 Posts
Default

While I see the merit in some sort of way to weight reviews to lessen the impact on the odd "agenda driven rating," I have to ask Tim:

How feasible is it to even have this discussion?

If I understand correctly, to implement such system, course ratings would necessarily be changing, not only based on reviews for that particular course, but based on the continuously evolving status of every person that ever reviewed that course, determined by how DGCR members are voting on the collective body of reviews as a whole.
Reply With Quote
  #23  
Old 05-21-2018, 01:31 PM
timg's Avatar
timg timg is online now
*Administrator*
 
Join Date: May 2007
Location: Cortland, NY
Years Playing: 17.5
Courses Played: 257
Throwing Style: RHBH
Posts: 9,961
Niced 893 Times in 295 Posts
Default

The only part of this idea I'd consider is to weight reviews lower where the reviewer has < x reviews primarily to lessen the impact of drive-by 5's and revenge reviews. I've wanted to do something about that for a long time and it would be interesting to see how it pans out with this test data set and see if the rating # ends up more in line with expectations.


Last edited by timg; 05-21-2018 at 01:33 PM.
Reply With Quote
  #24  
Old 05-21-2018, 02:27 PM
1978 1978 is offline
* Ace Member *
 
Join Date: Mar 2009
Location: Charlotte, Nc
Years Playing: 11.4
Courses Played: 286
Throwing Style: RHBH
Posts: 3,920
Niced 208 Times in 87 Posts
Default

Quote:
Originally Posted by wellsbranch250 View Post
whoops, i set the the tyler start east overlay to 0% fill. or perhaps Langley is just that good
Its good but its not "that" good. Lol.
Reply With Quote
  #25  
Old 05-21-2018, 03:35 PM
Johnny Betts Johnny Betts is offline
Par Member
 
Join Date: Feb 2010
Years Playing: 19.5
Courses Played: 35
Posts: 158
Niced 16 Times in 5 Posts
Default

As a disc golf course owner I fully understand the consequences of putting my course on this most excellent site. Big thanks to Tim and the moderators for allowing us to respond to reviews.

I’ve had people tell me directly to my face that they did not enjoy the course or thought it was quirky. That is fine and is expected from time to time.What I have a hard time dealing with is theft, vandalism and personal vendettas.

Reply With Quote
  #26  
Old 05-21-2018, 11:24 PM
bjreagh's Avatar
bjreagh bjreagh is offline
Eagle Member
 
Join Date: Jul 2008
Location: Murfreesboro,TN
Years Playing: 23.3
Courses Played: 322
Throwing Style: RHBH
Posts: 729
Niced 47 Times in 18 Posts
Default

One thing I have always liked about this site is that every level of disc golfer has a voice (from noob all the way to the pros and leaders in the industry).

I think DGCR and the rating/review system does a great job of doing its primary goal. That being said, a lot of the ratings issues seem to revolve around the prestige and free adverstising/publicity about getting your course in the top 10. Are private course owners complaining that their course is not showing up on the homepage really a good reason to change things? I appreciate their work and passion and have played many private courses, but there are other ways to gain an interest in your course, and DGCR does a pretty good job of it already. Besides, when you click on "more top rated" courses (which is my main point below), their are already special filtered lists for "private" and "pay-to-play". You know what is not there?--- PUBLIC.

So I like the op idea of having a few more top rated options. Why not add 1) weighted average [I do like that concept] 2) TR's only 3) non-TR's only 4) non-private 5) free to play, etc? And maybe make the link under the main top 10 a little more apparent. I know a lot of this can be done simply by sorting the course database, but a neat little button click is much less work!

Niced: (1)
Reply With Quote
  #27  
Old 05-22-2018, 01:45 AM
wellsbranch250's Avatar
wellsbranch250 wellsbranch250 is offline
Birdie Member
 
Join Date: Jul 2016
Location: Huntsville Alabama
Years Playing: 5.8
Courses Played: 412
Throwing Style: RHBH
Posts: 461
Niced 752 Times in 274 Posts
Default

below is an excel chart showing the data set refined for the 44 courses I evaluated. There would be a lot of changes compared to the top 25 that uses equal weighted numerics.

The formula for this graphic is as follow; 1 to 4 reviews, weight 1. 5 or more non TR, weight 5. TR, weight 10.

The chart has similar results to my first graphic. Maple Hill, Highbridges, Spearfish, etc. are more heavily favored by those that have reviewed a lot of courses. Where as courses like Smugglers, Phantom and Sabatttus have a higher than normal concentration of drive-by 5s without a vendetta review yet. exception, phantom has one vendetta review, but its offset by 63 drive by 5s!

data key
w ave = weighted ave
w rank = weighted rank
c ave = current ave (unweighted)
c rank = current rank (unweighted)
change = change in rank between data sets
for each reviewer category there are three columns
they are, weight value , # of reviews, and average score for that category

this is just data. I not advocating for or against any changes. However, if it is at all possible, i would like to see a critics top 25 list. No other site does anything remotely like that. I have found in general that those that take reviewing seriously have a more of a knack for suggesting courses I enjoy. perhaps that's just me. There seems to be a lot gang homer-ism in the course rankings on other competing sites and its completely unreliable in helping me target top courses. I recently played a 4.5 out of 5 rated course from another site that was nine holes, no tees and open.

Attached Images
File Type: jpg weighted excel.jpg (154.9 KB, 106 views)
Reply With Quote
  #28  
Old 05-22-2018, 10:45 AM
timg's Avatar
timg timg is online now
*Administrator*
 
Join Date: May 2007
Location: Cortland, NY
Years Playing: 17.5
Courses Played: 257
Throwing Style: RHBH
Posts: 9,961
Niced 893 Times in 295 Posts
Default

What do you mean by a critic's top 25? Like a TR only list?


Also, what sort of changes do you see if you only drop weight for people that have less than say... 5 reviews?
Reply With Quote
  #29  
Old 05-22-2018, 10:57 AM
Stardoggy's Avatar
Stardoggy Stardoggy is offline
**SuIciDe AnKles**
 
Join Date: Jul 2012
Location: Plymouth, WI
Years Playing: 8.4
Courses Played: 528
Throwing Style: RHBH
Posts: 6,261
Niced 743 Times in 319 Posts
Default

Quote:
Originally Posted by timg View Post
I've never seen that graphic before... I usually look at the threads in the suggestions area so if it was in this forum the first time around, it's probably why I missed it.


I think it's interesting although timing out reviews wouldn't really work unless I had a mechanism that said "x course was improved on this date". Some courses just stay the same so those old reviews are still pretty valid.


The idea of reducing weight for people that have less than 5 reviews seems the like the best balance of avoiding homers / vendetta reviews. Increasing the weights of TR reviews I feel could be gamed to some extent. And someone that's bronze maybe just lives in an area where there aren't many courses so they don't have an opportunity to write more reviews. I don't think they should be penalized for where they live or their lack of ability/desire to travel.
I mostly agree with this, except that you're somehow penalizing those reviewers that don't have access to many courses, or aren't willing to travel. Sometimes situation dictates what is possible.

You could get super complicated with it and start introducing multipliers based on how many states (or countries) a person has played.

To me, the most interesting part of it is how ratings would change as reviewer status changed.
Reply With Quote
 

  #30  
Old 05-22-2018, 11:56 AM
wellsbranch250's Avatar
wellsbranch250 wellsbranch250 is offline
Birdie Member
 
Join Date: Jul 2016
Location: Huntsville Alabama
Years Playing: 5.8
Courses Played: 412
Throwing Style: RHBH
Posts: 461
Niced 752 Times in 274 Posts
Default

Yes, a critics top 25 could be some sort of TR list, or hybrid of that. I think a min of 5 TR reviews would be enough to catch most top courses. Of the 44 I've selected to run data on, only 3 courses have less than 5 TR reviews. Spearfish 1, Muddy run 4 and Base Camp 4.

On a re-run of the numbers and only devaluing the "less than 5 reviews" I get similar numbers with some exceptions. Sabattus isn't negatively impacted as much. Riverbend is moves way up now and Claiborne now moves much lower.


Quote:
Originally Posted by timg View Post
What do you mean by a critic's top 25? Like a TR only list?


Also, what sort of changes do you see if you only drop weight for people that have less than say... 5 reviews?
Attached Images
File Type: png weighted3.png (104.9 KB, 87 views)
Reply With Quote
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
need a heavy weighted z zone 176+ michaelmove The Marketplace 0 03-09-2012 01:07 AM
reviews Huff General Disc Golf Chat 22 04-27-2009 10:33 PM
Reviews Innovadude General Disc Golf Chat 12 09-09-2008 07:39 PM


All times are GMT -4. The time now is 01:53 PM.


Powered by vBulletin® Version 3.8.10
Copyright ©2000 - 2019, vBulletin Solutions, Inc.