I don't have anything particularly concrete but I'd say find an open source implementation of something similar that has had a track record of running without too much incident and carefully copy its implementation, design and configuration.
Disable ability to make submissions until you have more solid plan
Decide what is your goal. Do you want to make a judge system, do you want create tasks or do you want a platform with specific kind of tasks.
Get in touch with people involved in ICPC an IOI contests in your country. Even if you are not interested in those kind of algorithm tasks, there will be some people who are familiar with similar existing systems and could point you in the right direction.
Assuming your primary goal isn't to make a judge system itself, some other options are:
* Evaluate the existing online judge systems. There are some open source ones like DOMjudge(https://www.domjudge.org/), CMS(https://cms-dev.github.io/index.html) and others . Consider if you reuse or extend them to suite your desired format. In the worst case maybe just the execution part can be reused. At least learn from their experience and mistakes creating and maintaining such systems.
* Many programming languages now have online REPL environments. Some of them open source. This is one more source of projects that provide sandboxed execution.
* If you have some budget, there are platforms that provide sandboxed exection as service oriented at your exact use case. Some examples are Sphere Engine https://sphere-engine.com/enterprise used by Sphere online judge and Kattis .
There a lot more different platforms with different style of programming tasks than what you listed in your FAQ. Some of looking for problem setters. Maybe one of them fits your type of tasks more. Or it can be a one off contest with slightly unusual problem set. Or maybe it can be a separate category on their system
and you can advertise this category on your website.
Thanks for the suggestion. Thread is too deep to reply to your actual message, but I will look around to see how other "online judge" software run arbitrary code securely.
There's probably some low hanging fruit in configuring Docker properly.
This website is a free open-source side project and not business critical.