As someone who's been socially awkward in the past (yet good with math, a typical nerd type I guess) but who eventually taught himself to be social to the extent that I can arguably surpass most of my former friends at this game (yet who is nowhere near Paris Hilton yet), here is my hard-won perspective:
1. Being truly social requires two things: (a) social experience, (2) a sophisticated theory of mind, a brain "faculty" that is incredibly resource-intensive, possibly more so than the faculty which performs abstract reasoning. In evolutionary biology, there is an influential theory which claims that it was theory of mind (which is being able to figure out what others think) that developed due to the pressure of living in groups and not tool-making that was the original cause of humans becoming sentient. This is why I don't believe for a second those who say that women are less smart than men (most women were simply taught to be dependent on others from the early age).
2. Our education system especially tends to value social skills less than hard problem-solving skills and therefore convinces us at early age that being social requires less brain power than doing math (because "brainy" is always "good"). This is because of two things (1) people with high social skills are somewhat more likely to steal money from you or cheat or use you in some other way without you ever knowing it, (2) most people except the autistic minority are capable of powerful theory-of-mind type thinking given enough social exposure, while only a select minority (whether because of education or biology) is capable of high-level abstract reasoning.
3. At one point in my life I felt I could be very social if I wanted (and I indeed could), yet I still avoided social interactions because being social felt incredibly draining to me (all those neurons firing consume glucose...) and I did not learn yet to derive pleasure from being social.
4. People who are very good at being social (past a certain threshold) derive a lot of pleasure from it, to the extent that they forgo most non-social intellectual pursuits which they perceive as less rewarding. This is actually a dangerous trap to fall into (similar to a drug addiction) if your main area of work requires quiet contemplation.
5. People who are truly brilliant (I'm not there yet) learn to balance the amount of interaction with others (and pick those they interact with carefully) because they understand that social withdrawal can give them a serious advantage, since most of the society (perhaps except tiger-educated Asian kids) falls squarely into the social-driven category.