Throw Simulator

[Steve West]

A thread for those who use my Throw Simulator, which can be downloaded here:

http://stevewestdiscgolf.com/ThrowSimulator.aspx (http://stevewestdiscgolf.com/ThrowSimulator.aspx)

Note: Thanks to Dave242's keen observation, a bug was found and fixed. If you downloaded the simulator before 4/12/2010, you should download the fixed version.

The old version asked you to Update/Don't Update. If you see that, get the new version.

The bug was that the simulator wasn't using the input distribution of players.

[Dave242]

It is working much better now - thanks!

If you don't mind a suggestion and a comment:

Suggestion: It would be cool to add scoring distribution stats (average score & % of 1's, 2's, 3's, 4's etc scored). IMO, this would make it an even more useful design tool. I suppose to do this properly/usefully for most course design, OB would not really be OB complete with a penalty stroke added - rather just some other factor that shows increased difficulty recovering from being off line.

Comment: It seems to me that the spray pattern is too wide. For example, I set up a 150' hole with a 100' wide fairway (50' on each side). For Advanced, it shows 8-10% going OB. I have never measured this like I believe I read that you, CK and others did, but it just seems 2-4 times higher than I would expect. This is one example, but it seems consistent with several other scenarios I generated.

Like I said in the other thread, I love this tool. I had conceptualized something like this but never felt like expending the time and brain power (if I even have enough to begin with) to do something even close to this. Thanks for creating this and making it available!

[Cgkdisc]

Join the DGCD and you can get the Excel tool our designers use that has the features you just asked for with scoring spreads both calculated and forecast plus much more.

[MotoDj]

i looked for a minute or so, confused me, i'll have to wait till i can do some investigating to figure out how its going to make me throw 400ft... other wise, i guess i'm confused what its supposed to simulate

i was hoping for an animated motodj throwing into a tree

[WillACarpenter]

I only played with this for a few minutes...but I feel this thing is optimistic when using the lower ranked players.

Apparently the ladies can pretty accurately park 500' holes.

maybe I really suck at this game...

\/\/

[Dave242]

I only played with this for a few minutes...but I feel this thing is optimistic when using the lower ranked players.

Apparently the ladies can pretty accurately park 500' holes.

maybe I really suck at this game...

\/\/

I started thinking the same thing, but then I read this instructions/explanation:
The graph shows where the first (red), second (yellow), third (pink) and fourth throws would have landed

[Steve West]

It is working much better now - thanks!

If you don't mind a suggestion and a comment:

Suggestion: It would be cool to add scoring distribution stats (average score & % of 1's, 2's, 3's, 4's etc scored). IMO, this would make it an even more useful design tool. I suppose to do this properly/usefully for most course design, OB would not really be OB complete with a penalty stroke added - rather just some other factor that shows increased difficulty recovering from being off line.

Comment: It seems to me that the spray pattern is too wide. For example, I set up a 150' hole with a 100' wide fairway (50' on each side). For Advanced, it shows 8-10% going OB. I have never measured this like I believe I read that you, CK and others did, but it just seems 2-4 times higher than I would expect. This is one example, but it seems consistent with several other scenarios I generated.

Like I said in the other thread, I love this tool. I had conceptualized something like this but never felt like expending the time and brain power (if I even have enough to begin with) to do something even close to this. Thanks for creating this and making it available!

I actually built one that calculated scores. The toughest part was getting the model to decide which of the overlapping out of bounds lines applied (for relocating the lie).

But, I have very little data about short throws around the hole. This model has people throwing back and forth around the hole forever. The reason is that the accuracy in degrees was measured to be worse on shorter throws. That trend probably doesn't extend all the way down to putts, but the model doesn't know that.

So, it's best at tee shots and second shots on long open holes. A 150 foot hole might be getting into thin-data territory.

Another factor it does not take into account is player strategy. For example, if there really were OB to the left of the basket, players might aim more right. Or, players might shorten and tighten their throws on a tunnel hole. You can fudge this a little by guessing where the aiming point would be, and model for that.

Also, as the data was gathered at an NT event, the skills of the lower-rated players are mostly just extrapolation.

The primary purpose of the tool is to help design safe courses, not to guess scores. If the model shows 8% going into an area - when it actually should be 2% - that still tells you that a lot of people will land there in a year. If that area is a street or neighbor's yard, you should move that hole.

[billnchristy]

Comment: It seems to me that the spray pattern is too wide. For example, I set up a 150' hole with a 100' wide fairway (50' on each side). For Advanced, it shows 8-10% going OB. I have never measured this like I believe I read that you, CK and others did, but it just seems 2-4 times higher than I would expect. This is one example, but it seems consistent with several other scenarios I generated.


I think you are also missing the idea that it is calculating 500-1500 throws depending on the number.

So if you stood at the tee of this hypothetical 150' hole with 10 ADV players and they threw 150 throws I would actually think that quite a few would go out.

**though maybe not 150 throws...so you might be on to something in tightening up a bit**

[Dave242]

But, I have very little data about short throws around the hole. This model has people throwing back and forth around the hole forever. The reason is that the accuracy in degrees was measured to be worse on shorter throws. That trend probably doesn't extend all the way down to putts, but the model doesn't know that.


That throwing back and forth around the hole sounds a lot like my game on some days! You could do something like CR-Par does. Once a player gets within close range, you add x.x throws to hole out.

But, I hear you.....different tools have different purposes and doing score-predicting might be like hammering with a wrench. The tool Chuck mentioned does scoring predicting (I have input some courses I know into it and it does pretty well), but it does not have the fun and interesting graphical aspect.

[Dave242]

I think you are also missing the idea that it is calculating 500-1500 throws depending on the number.

So if you stood at the tee of this hypothetical 150' hole with 10 ADV players and they threw 150 throws I would actually think
that quite a few would go out.

**though maybe not 150 throws...so you might be on to something in tightening up a bit**

Actually, I am not missing that. Here is a simulation of this actual hole (http://maps.google.com/maps?hl=en&q=42.211093,+-87.835355&ie=UTF8&hq=&hnear=%2B42%C2%B0+12'+39.93%22,+-87%C2%B0+50'+7.28%22&gl=us&ei=us3FS5TKNIn8NdT89PIN&ved=0CAgQ8gEwAA&ll=42.211103,-87.835492&spn=0.001184,0.002411&t=h&z=19)(marker is near the basket,
tee is north, OB Street down right of fairway and OB path to left). Picture (http://www.dgcoursereview.com/course_pics/1179/c5bf71aa.jpg) from Tee.

http://www.dgcoursereview.com/forums/attachment.php?attachmentid=4654&stc=1&d=1271255072

I have thrown this hole probably 100 throws and gone OB once or twice (1-2%).
I am a Blue level player, and the simulator is showing me going OB 36-40% of the
time....and OB again on my recovery shot 2%! It shows Open level players going
OB 25-27% of the time.

On top of that, the tee pad is made of wood chips! :doh: :D

[billnchristy]

Its probably a matter of scale with the hole length...no clue but you are right...that is too high a percentage.

[Dave242]

This model I believe is based on actual measured throwing data from MN players. I heard MN has some crazy mosquitoes. Maybe they were driving the players absolutely batty that day (or maybe the data collectors). :)

[Cgkdisc]

Maybe half of the players were from Minnesota since it was an NT and big A-tier for Ams that drew nationally and regionally.

[grodney]

On a 300ft hole with no OB, it is predicting 57% of Open players can't get within 50 feet. (Thus 43% will be within 50 feet.)

And 10% not within 100 feet.

[Cgkdisc]

That seems wider than you would think. Consider that the throwing data was gathered on a wide open hole that was 700 feet long. So players were attempting to crush their drive not necessarily worried about accuracy. Ideally, it would be better if each player could drive toward a target that was just a little shorter than their individual distance range where they would know they could reach it if they threw accurately. I think the accuracy factor in the throw simulator would definitely be narrower.

[Dave242]

There is a long article about the data here (how the bear was killed and how it was skinned):
http://stevewestdiscgolf.com/Documents/Disc%20Golf%20Throw%20Model.pdf

[Camgolfer]

Mind explosion.

[Steve West]

Actually, I am not missing that. Here is a simulation of this actual hole (http://maps.google.com/maps?hl=en&q=42.211093,+-87.835355&ie=UTF8&hq=&hnear=%2B42%C2%B0+12'+39.93%22,+-87%C2%B0+50'+7.28%22&gl=us&ei=us3FS5TKNIn8NdT89PIN&ved=0CAgQ8gEwAA&ll=42.211103,-87.835492&spn=0.001184,0.002411&t=h&z=19)(marker is near the basket,
tee is north, OB Street down right of fairway and OB path to left). Picture (http://www.dgcoursereview.com/course_pics/1179/c5bf71aa.jpg) from Tee.

http://www.dgcoursereview.com/forums/attachment.php?attachmentid=4654&stc=1&d=1271255072

I have thrown this hole probably 100 throws and gone OB once or twice (1-2%).
I am a Blue level player, and the simulator is showing me going OB 36-40% of the
time....and OB again on my recovery shot 2%! It shows Open level players going
OB 25-27% of the time.

On top of that, the tee pad is made of wood chips! :doh: :D

The model players were throwing from grass.

Anyway, I really appreciate you looking closely at the model. Nothing good comes without peer review.

I checked a number of things. I set the model up as you had it. Then, I froze the set of random numbers to get a baseline run which came out to be 38.7% out of bounds.

Then, I set the ratings of all the players in the group to 950: you had a range of 800 to 925. (That's an input-able figure, by the way.) That only made the percentage out of bounds go down to 34.3%.

I then adjusted the aim of the players to optimize the number that would land inbounds. If they aimed a little over 2 feet to the right, a few more could get inbounds, but that only brought the figure down to 33.8%.

So none of those explain the gap between your 1% or 2% experience and the 40% or so the model gives. So, I went back to look at the raw data that went in to the model.

(Before we get to that, note that on the hole you modeled, to stay in bounds, a player needs be accurate to less than 4.4 degrees to the right, or 10.8 degrees to the left.)

We measured 18 throws from Blue level (average rating 947) non-open, non-women players that were from a distance of less than 350 feet. I threw one out as an outlier. Of the 17, two were more than 10.8 degrees to the left, and four were more than 4.4% to the right. So, 35% would be out of bounds on this 298 foot hole.

My conclusion is that the model reflects the data we collected.

If that doesn't match your experience, we probably need more data. Grab a laser rangefinder and come to Minnesota.

Either that, or all Blue players throw more accurately than the average Blue player.

[Dave242]

Thanks for your reply. I think that the nature of your data collected on that 757.9' wide open hole leads to more variation in accuracy than in the majority of real-life situations. And I think your model reflects that.

At that length, I am assuming pretty much all Blue level players on up (and probably over half the intermediates....all though there were none there that day I do not believe) are trying for a 3. That takes two 375' throws, but I think most people's strategy would be to bomb their first one as far as they can to shorten the 2nd throw to increase the odds of parking the upshot. Since there is no punishment for being off line, very little effort or attention would be paid to it. Even if you totally screw up your first throw, recovering for par (4) is pretty easy (assuming calm conditions).

......just re-read the thread and saw CK's post saying the same thing. Woops - shoulda just copied that.

One other thing....I stopped by that hole this evening and my first two throws at the basket went OB right. :D I throw LHBH and was throwing drivers for aces....I normally throw mid-ranges if I am trying to park the hole.

[Cgkdisc]

I'm thinking we might need to get data on a wide open hole around 375ish (CP Adams 4 from short tee or BRP 24) for white and blue level and maybe 415ish for gold level (Lions 10?) to get better spray pattern info.

[Steve West]

Thanks for your reply. I think that the nature of your data collected on that 757.9' wide open hole leads to more variation in accuracy than in the majority of real-life situations. And I think your model reflects that.

......just re-read the thread and saw CK's post saying the same thing. Woops - shoulda just copied that.



Less than half of throws we measured were from the tee. We also measured second and third shots. The 17 throws I mentioned in my last post were all from within 350 feet. I don't see how that would not be a real-life situation at those distances. After all, they were actually keeping score for prizes.

I don't doubt that some players would throw differently if there is a hazard within range. My guess is that this would tighten the accuracy and possibly reduce the distances.

However, some players will still throw the same as they would for a wide-open hole. Therefore, you cannot rely on the presence of a hazard to reduce the size of the landing area you need have for safety. And that's the primary purpose of the model.

[Dave242]

I saw where you said you analyzed the margin of error for 17 Blue level throws, but I guess I did not comprehend how that data was factored into the simulator. I get your point that 35% of the 17 throws would have been OB right on my example 168' hole with OB only ~25' right of the basket.

I also understand the purpose of the tool, but I still think it would be cool if there was a version where the data was based solely on people trying to park their drive (at the basket or intended landing zone). I know it is a ton of work, and I do not think I fully appreciate the level of work (and sacrifice) for 3 guys to stand around all day recording data (laser range finder readouts and identifying characteristics of the competitors) and then going back and entering the data, triangulating all the positions, culling through and discarding errors and outliers, writing formulas and compiling the results into a usable format. Your simulator is 9MB in size for goodness sake!! I am majorly impressed by our work and by no means am "demanding" anything.....just expressing a "wouldn't it be cool if......."

[Steve West]

I saw where you said you analyzed the margin of error for 17 Blue level throws, but I guess I did not comprehend how that data was factored into the simulator. I get your point that 35% of the 17 throws would have been OB right on my example 168' hole with OB only ~25' right of the basket.

The data on the 17 throws was just a "smell test". These 17 were not treated special when they were incorporated into the data.

I also understand the purpose of the tool, but I still think it would be cool if there was a version where the data was based solely on people trying to park their drive (at the basket or intended landing zone). I know it is a ton of work, and I do not think I fully appreciate the level of work (and sacrifice) for 3 guys to stand around all day recording data (laser range finder readouts and identifying characteristics of the competitors) and then going back and entering the data, triangulating all the positions, culling through and discarding errors and outliers, writing formulas and compiling the results into a usable format. Your simulator is 9MB in size for goodness sake!! I am majorly impressed by our work and by no means am "demanding" anything.....just expressing a "wouldn't it be cool if......."

I feel like that old soup (or spaghetti sauce?) commercial where they keep saying "It's in there!"

We have data on several throws where the players were within the range where they are not throwing for distance. I separated these out in the table below. (Inaccuracy is the standard deviation of the number of degrees each throw was off-target.)

/ / Drive Inaccuracy / / Approach Inaccuracy
Rating /Throws/ Model/ Measured/ Throws/ Model/ Measured
Green / 34 / 13.2 / 12.0 / 5 / 11.8 / 18.2
Red / 59 / 11.1 / 11.0 / 19 / 11.2 / 7.7
White / 96 / 10.3 / 11.0 / 22 / 10.3 / 8.6
Blue / 115 / 9.7 / 10.3 / 55 / 9.5 / 8.2
Gold / 54 / 8.6 / 7.6 / 38 / 8.4 / 6.9

(Sorry. Is there a table feature? Or at least a way to keep extra spaces?)

Having separated it out like this, I see that there is potential to add a parameter which would reduce inaccuracy if a player is within Range. If I did that, it would reduce the number of OB's on the example hole from about 39% to 32%. If MN weather gets back to normal, I might do that.

It's only "work" if someone is paying you.

I don't agree that measuring the throws of players who are "trying to park it" would be helpful. That might not be the way they would actually play the hole. The throws in the right three columns reflect the actions of players who are relatively close to the hole, and are trying to get a low score. For most players, that might have meant "trying to park it" – but I wouldn't want to instruct the test subjects to something artificial.

I do think more data about players playing from close range would be helpful. But, that's the most difficult kind to collect. They play pretty fast when they get close.

(The 9MB version is the stripped-down EZ version for mass consumption. The one that makes the archery-target graph is 40MB and takes about 10 minutes to run one simulation.)

In the meantime, I made a picture of all the throws that we measured.

http://stevewestdiscgolf.com/Documents/AllThrows.jpg

On another note: It is interesting that no one is commenting on the variance in distances (short or long throws). Yet, these contribute just as much to the shape of the "blob" of throws. If your perception of the accuracy of these players were correct, there would be a relatively thin "red carpet" of landing points along the line of throw. Yet, whenever I see a pattern of throws around the target, they always seem to be more circular than linear.

[Dave242]

I feel like that old soup (or spaghetti sauce?) commercial where they keep saying "It's in there!"

Yes, it is in there, but you said it was less than half the data. So while it is in there it is diluted greatly by the other data based on players just chucking as far they can with little regard to accuracy.

(Sorry. Is there a table feature? Or at least a way to keep extra spaces?)

Use the tag. Let me try:

[code]
DriveInaccuracy ApproachInaccuracy
Rating Throws Model Measured Throws Model Measured
Green 34 13.2 12 5 11.8 18.2
Red 59 11.1 11 19 11.2 7.7
White 96 10.3 11 22 10.3 8.6
Blue 115 9.7 10.3 55 9.5 8.2
Gold 54 8.6 7.6 38 8.4 6.9



I don't agree that measuring the throws of players who are "trying to park it" would be helpful. That might not be the way they would actually play the hole. The throws in the right three columns reflect the actions of players who are relatively close to the hole, and are trying to get a low score. For most players, that might have meant "trying to park it" – but I wouldn't want to instruct the test subjects to something artificial.

I should have been more specific/accurate in my jargon to convey what I meant. Instead of "park it" I should have said, "get within putting range". If a hole is reachable, with small risk involved, what player is going to try to do anything but that in tournament play?


In the meantime, I made a picture of all the throws that we measured.
http://stevewestdiscgolf.com/Documents/AllThrows.jpg


Sweet!


On another note: It is interesting that no one is commenting on the variance in distances (short or long throws). Yet, these contribute just as much to the shape of the "blob" of throws. If your perception of the accuracy of these players were correct, there would be a relatively thin "red carpet" of landing points along the line of throw. Yet, whenever I see a pattern of throws around the target, they always seem to be more circular than linear.

Interesting point about the circular patterns! I gotta think about that (I have a couple knee jerk thoughts, but need to chew on it a little). For me, what I find intriguing is that you have done really good work towards quantifying accuracy. I hear lots of talk about raw distance and there is plenty of talk regarding design (and par) models built around understanding the distance of various skill levels of players. Seeing pictures and data of accuracy is what is new and interesting.

[LetsPlayGolf]

First of all I want to say thanks to Steve West for putting this thing together. I am trying to get final approval for a course installation and I believe this tool is going to help.

I do have a question for all of the experts on here:

The course I am proposing is on a university campus. What 'minimum' to 'maximum' rating should I use on the 'Throw Simulator' that would simulate the average college student playing for the 1st time through maybe the 20th time. In other words, most of the people that I believe will use this course will be playing for the first time... then once they play for the first time, hopefully they keep coming back to play.

Thanks for the help in advance.

[Steve West]

First of all I want to say thanks to Steve West for putting this thing together. I am trying to get final approval for a course installation and I believe this tool is going to help.

I do have a question for all of the experts on here:

The course I am proposing is on a university campus. What 'minimum' to 'maximum' rating should I use on the 'Throw Simulator' that would simulate the average college student playing for the 1st time through maybe the 20th time. In other words, most of the people that I believe will use this course will be playing for the first time... then once they play for the first time, hopefully they keep coming back to play.

Thanks for the help in advance.

You're welcome.

The primary use of the model is to see where discs might land - to locate fairways away from trouble. I suggest using a minimum of 600 and maximum of 1050, with the categories weighted evenly, because you never know who is going to use the course.

This wide spread will show you where discs MIGHT land. Every spot on the graph represents a 1 in 1500 chance of landing there. So, if you have 150 players per week, and the course is open 10 weeks, you can plan on a disc landing there (or around there) once a year. If that spot is in a busy road, it's not acceptable. Re-run the model of few times and pay attention to the outliers.

If you want to try to use it to figure out a good landing zone for the first throw, then you can narrow down your definition of the players. I would suggest 800 to 850 Advanced players as one group, and 700 to 750 female as another (the model thinks that females and intermediates play the same).

Others may have different ideas about the ratings to design for, and I hope they share their thoughts here.

Not many new players will actually reach an 800 rating after eight times, but you don't want to design a course to be easy for someone who has only played eight times. Give them something to work toward.

Run the two groups separately and find a place that will work for both; don't run them together and find a place that works for the average of the two.

If you want to go farther and generate appropriate scoring spreads, then you should join the Disc Golf Course Designers group so you can have access to the Hole Forecaster.

Actually, you should join the DGCDg (http://www.discgolfcoursedesigners.org/discgolfwiki/index.php5?title=Main_Page) anyway.