I suspect you'd need adoption before authors would want to maintain a Pluginfile.json, and before there's a decent selection in your repo there's not a selling point to gain adoption.
A tool that can help me find things like git-series would be cool, but I think you'd need to work downstream rather than having every git plugin know about your package manager. If you're gonna make repos to wrap upstream projects into your package format (which is what Linux distros do), then that could work.
It's called a plugin, but it's actually a manager for git aliases.
Integrity verification of distributed packages (ideally cryptographically) is a core and required feature of a package manager.
At an absolute bare minimum you need a Lockfile style mechanism to allow users to ensure two installs are the same.