At some schools the athletics program gets considerable tuition revenue as well (in the form of a tacked-on "athletics fee"). UF seems to have one of the lower fees ($57/yr), though when multiplied by 50,000 students, that still works out to a bit under $3m/yr. It appears (see sibling comment) that athletics contributes back some surplus to academics, though, so may be a net positive at UF. Not true at all schools, some of which have much higher than $57/yr charges (e.g. UVA's $650).
Last I heard, LSU and Nebraska were the only two schools with athletic programs that were both financially and nominally self-sufficient. This was in 2009, I think. This link gives some financial stats on athletics programs starting with the 2004-2005 academic and ending with the 2009-2010 academic year.
In other words, the difference between the athletics budget and the academic budget represents the priorities of Americans in general, not any specific group such as a universities administrators.
Is that the case, or it sounds like from the other commentors here that it also represents the effectiveness and focus on fundraising? If instead people were given a single fundraising plea form with 2 checkboxes (one for academics, one for athletics) -- which way would most people go?
Athletics gets money from donors, ticket revenues and private funding. This allows them to spend as they see fit.