Banks don't do anything useful that cannot be provided by other means. They are completely useless. The only thing that banks do is to provide credit against things you already currently own, like mortgages and commercial loans. But these things could very easily be provided by an official agency. For example, if you don't pay your monthly mortgage you just lose the house and an official agency will deal with the sale of the property (maybe through third-party services). Same thing for commercial loans. There is no need to create a private, for-profit bureaucracy to handle this.
It doesn't need to be the public sector directly. What I mean is that banks are not using their own money to do what they do, so it does't make sense to let them charge interest. There could just be a set of regulated institutions that charge fixed fees (independent of the amount of the operation) to perform these tasks, which basically consist of converting one form of wealth (real state or business assets) into monthly obligations.