Startups do well when they improve things for people who vote with their wallets (buying this or that service).
Bureaucracies though are a completely different beast. Better value? Happier customers? They care not a bit. You need different levers to (sloooowly) move them.
Exactly. You can't enter the market and provide better service for "government administration." You could offer a service to help a person, but the person still has to go through it all.
I think one common way is to get a high ranking bureaucrat to mandate the change. It is not a guarantee, but a good enabler. I suspect pride, shame (you still use paper receipts for this?), media or even bribes are used.
For examples, I would look at automating standard government functions (e.g., car registrations, toll collections, etc.). Putting those online leaves some clerks unemployed and is thus often resisted by default, so someone found a working path for each of those
I think competition among countries actually helps. Case in point: this visa program. They are trying to make it more streamlined and more attractive with a longer expiration. It is improvement in the right direction.
Absolutely not. The reason things are like described ( although it's not as bad today), is because many sectors are heavily regulated, and belong to a few actors abusing their monopolistic position. There's a reason the european union gave a 500millions fine to the french bank a decade ago ( but things didn't move a lot).
As an example, all the online banking start ups belong ( or were created as subsidiaries of) big banks. All of them.