That is totally, absolutely wrong. Just because teachers etc say it doesn't mean you should believe it. I'm sure you've read how Steve Jobs forced himself to go to college because it was what was expected, and then dropped out. Personally, I missed the course I wanted to do, started something else, dropped that and finally ended up doing a computer science degree (which was something I'd ignored for 3 years). (Edit: and it all worked out better than if I'd done the degree I'd initially wanted to do)
There are plenty of examples of the opposite - people who don't go to college straight out of school.
Think it through - there are very, very few decisions that "decide your future" in a way that can't be changed, and the few that are usually are crimes.
Sure, work hard and go to college or whatever, but DON'T THINK "MISTAKES" CAN'T BE FIXED".
When someone tells you "this decision decides your future" nod politely, and ignore the implied pressure. They are incorrect.
I had a very similar experience that took this geeky computer nerd into the crazy and scary world of politics for a couple years. I ended up running a state senate campaign and worked for the WI Senate Majority Leader for about 2 years. It was crazy and completely different from anything I would have experienced had I stuck to Computer Engineering and graduated in the standard 4 years like everyone expects. It happened by accident (because I was purposefully wandering) and was far more educational than anything I could have done in college. It was is by far the best thing I ever did.
Don't be afraid of a curvy path through life. There's a lot of value in the curves.