You would be surprised. University of Michigan Publishing, one of the largest and wealthiest of the publics only has a staff of ~30, some of whom are shared with other departments. The university's annual contribution is well under $2 million. The entire University Library, the second largest research library in the U.S. after Harvard, has an annual budget of $64m (0.58% of the entire budget).
I think historically the larger presses were expected to be close to revenue neutral. They do well enough to limp along, but don't get the resources really needed to build new infra. As a cost center without significant tuition or grant funding, the library has little pull when provosts form their priorities.
Talking about "the budget" is meaningless in this context, you could similarly argue that Bill Gates can't afford to buy his kids bubblegum on any given day. After all his self-imposed budget for them is $10/day.
Look at the size and growth of the university's endowment.
Unfortunately talking about the "size and growth of the endowment" is similarly meaningless in this context for a number of reasons. Even if the funds were not restricted in a way that excluded funding the University Press, it's probably a poor measure of whether a University could "afford" to support a press.
I think historically the larger presses were expected to be close to revenue neutral. They do well enough to limp along, but don't get the resources really needed to build new infra. As a cost center without significant tuition or grant funding, the library has little pull when provosts form their priorities.