This is called full reserve banking, which is a step further than narrow banking (where you only invest in essentially zero risk assets like short term government bonds or let money sit in a federal reserve account).
That says narrow banking means you park it at the fed. In theory, couldn’t you keep the value of all deposits in cash in a vault and not break any rules?
To be clear, I’m not suggesting this is a good idea. I’m just trying to clarify if this is allowed. Because at face value the idea that keeping all deposits liquid is a systemic risk seems counter intuitive. It only begins to make sense with the specific context of parking that money at the Fed.
Right, so you can think of it as a spectrum where full reserve banking (all deposits held directly, you're most likely going to have to pay for the privilege of secure storage and transactions) is on one end, fractional reserve banking (a small portion of deposits are held liquid, the rest is loaned out to generate interest and pay for operations) is on the other end, and narrow banking (park money in short term treasury notes and at the fed) is in the middle.
In our current system, every bank or bank-like institution does fractional reserve banking. The systemic risk posed by a narrow bank or a full reserve bank is that during any potential banking crisis, there is a strong prisoner's dilemma style incentive for depositors to "defect" by withdrawing all of their money from normal banks and moving it to narrow or full reserve banks all of a sudden. This is broadly why regulators won't let you run one.
Now, to answer your core question AIUI: "what stops me from running a full reserve bank [equiavalent]?"
In theory, you could offer a secure storage service like safety deposit boxes etc and not be subject to finance regulatory controls. In fact, most cities do have businesses that offer secure storage facilities that you can rent space in to store your property and retrieve it as needed. This is perfectly legal and honestly such a business doesn't usually know or even need to know the contents of such storage lockers etc.
The problem comes when you want to integrate with the larger financial system. Finance regs are far reaching in a "viral" sort of way; they don't just restrict how you do business, they also typically restrict whom you can do what business with based on how and what kind of business they do. The result of this is that you end up under the purview and subject to the approval of regulators one way or another, at which point they disallow your business.
As a simple example, if you want to make it so your customers can easily send or receive money, you'll probably want to be able to process ACH and wire transactions, at which point you fall under extensive regulation administered primarily by the Treasury and the Federal Reserve. I imagine the same will be true for things like debit cards on the standard networks since they're already subject to regulation of a viral nature and so on and so on.
AIUI regulators do not allow this due to systemic risk: https://www.bloomberg.com/opinion/articles/2019-03-08/the-fe...