Most obviously, if you invite the military in and ask them to secure a portion of the building, you're exposed to the risk that the commander-in-chief will order them out (or order them to stand aside); it's unlikely that forces could be redeployed fast enough to respond to such a defection. (And no, you can't avoid this danger by having all the forces working together everywhere; far too many command-and-control issues arise.) If you don't invite the military to assist in the first place... well, then you're not relying on them to guard your back.
There's also a fundamental democratic issue at stake: It's not by coincidence that the United States Capitol Police answers to the legislature and not to the executive -- indeed, this is seen around the world (e.g. Canada's Parliamentary Protective Service answers to the Speakers of the House and Senate) and arguably the principle that military forces should not be brought to the seat of legislative power dates back to the Roman Republic... which swiftly became the Roman Empire after Caesar crossed the Rubicon with an army at his back.
Those are all valid points. Though I will argue that the assumption that the Capitol Police answers to the legislature is shaky, at best (in practice). If I had to guess who is more likely to refuse an unlawful order, I'd say a member of the military, rather than the police, based on my limited knowledge of their respective cultures. Combined with the idea of police officers who believe they are part of a semi-secret, ad-hoc, patriot's army, things can go wrong indeed.
Let's do a thought experiment: let's say there are a few elements in the police who are active QAnon believers, sprinkled in at various levels. Let's also assume some more force members are not believers, per se, but sympathize with the cause, and are willing to look aside since they may dislike some legislators who they see as enabling BLM, Antifa and other un-American actors (in their eyes) and believe that something "weird" happened with the elections and/or the whole establishment is dirty. Would these individuals not listen to the orders of the commander in chief, even when not delivered via the official chain of command?
> Though I will argue that the assumption that the Capitol Police answers to the legislature is shaky, at best (in practice).
It's a matter of law that they answer to the legislature, this isn't an "assumption." Individually they have answered to a not very sharp police chief, and the Sergeants-At-Arms of the House and Senate, who are all in the process of resigning because of how badly they recently screwed up, if that's what you mean.
> It's a matter of law that they answer to the legislature, this isn't an "assumption."
I could have expressed myself better there - I was distinguishing between them being answerable to the legislature (de jure) in the logical, org chat way, and them "answering" to a mercurial president they ideologically agree with (de facto), in the here and now.
Hmm, why turn down an offer of assistance from a military whose commander-in-chief wants to overturn the election? I can think of a few reasons...