I've never gotten so mad during a casual round that I've wanted to quit forever, but my first couple of tournaments made me want to never ever play in tournaments. My first was the St. Louis open '08. I'd been playing a few months, and the open was held at Endicott. I was carded with a couple local pros (and I knew it) for my first round, and I proceeded to hit every tree. Every. Tree. Every tree. Every hole. I was carding 8's like they were minors in prom dresses. Just mortifying. Second round was no better. The next tournament? Not much better. I realized that I had no mental game whatsoever, and the prospect of playing with people I didn't know intimidated me. But I got better, and now, while I am not "tournament player" per se, I have learned to not care about a crappy hole, which allows me to move on to the next one without the baggage of anger and shame. And PRESTO, I have fewer crappy holes.
But casual, no, I've never wanted to stop. Angry, yes sometimes, but finished? Nope.