Something like this may grow out of the GGML project, which is gaining traction....

DougBTX on July 6, 2023 | parent | context | favorite | on: InternLM – new open source 7B LLM

Something like this may grow out of the GGML project, which is gaining traction. They already have a weights format which can be loaded with mmap, though AFAIK the model architecture still needs to be defined in C++.