Ah do we run all user-submitted code in Docker containers. The "engine" that runs the code is written in Python and we do different things for different languages.
For running Javascript and Julia, it goes something like Python objects -> JSON -> read JSON in Javascript/Julia -> run code -> output JSON -> read user output from JSON in Python.
For C, we can call C functions directly from Python with some code for dealing with different types.
Not sure if this is the best approach (it's not super fast) but we've been learning as we go. We might be due for a refactor in case the next language we want to support doesn't fit into this pattern. I'm personally excited to learn Rust and maybe add support for it.
From a quick glance it looked quite worrying, many red flags. I didn't look too carefully so some of this might be wrong or I missed where it's done.
* Results from the untrusted part inside container are returned using pickle. Which can be used to achieve arbitrary code execution outside container.
* no time limiting
* no memory limiting
* Untrusted code is run as root in the container which by default is same user as root outside container. From what I understand it isn't as bad as it was in earlier docker versions but still not great.
* untrusted code is run in the same process as semitrusted run_lang code, which means that the untrusted code with little bit of effort can manipulate reported execution time and memory usage
* for some languages correct_output is copied into the untrusted execution environment which means that solution could potentially just read the correct answers instead of calculating themselves
* none of the default capabilities are dropped which is probably more than what solution needs
That's a good attitude to have but people are probably going to take over your host[s], vandalize your site and run up your bills long before you get a chance to do all the necessary learning. I don't think your current approach is actually all that easy to secure.
I don't have anything particularly concrete but I'd say find an open source implementation of something similar that has had a track record of running without too much incident and carefully copy its implementation, design and configuration.
Disable ability to make submissions until you have more solid plan
Decide what is your goal. Do you want to make a judge system, do you want create tasks or do you want a platform with specific kind of tasks.
Get in touch with people involved in ICPC an IOI contests in your country. Even if you are not interested in those kind of algorithm tasks, there will be some people who are familiar with similar existing systems and could point you in the right direction.
Assuming your primary goal isn't to make a judge system itself, some other options are:
* Evaluate the existing online judge systems. There are some open source ones like DOMjudge(https://www.domjudge.org/), CMS(https://cms-dev.github.io/index.html) and others . Consider if you reuse or extend them to suite your desired format. In the worst case maybe just the execution part can be reused. At least learn from their experience and mistakes creating and maintaining such systems.
* Many programming languages now have online REPL environments. Some of them open source. This is one more source of projects that provide sandboxed execution.
* If you have some budget, there are platforms that provide sandboxed exection as service oriented at your exact use case. Some examples are Sphere Engine https://sphere-engine.com/enterprise used by Sphere online judge and Kattis .
There a lot more different platforms with different style of programming tasks than what you listed in your FAQ. Some of looking for problem setters. Maybe one of them fits your type of tasks more. Or it can be a one off contest with slightly unusual problem set. Or maybe it can be a separate category on their system
and you can advertise this category on your website.
Thanks for the suggestion. Thread is too deep to reply to your actual message, but I will look around to see how other "online judge" software run arbitrary code securely.
There's probably some low hanging fruit in configuring Docker properly.
There should be a limit on how long the Docker container can run code for, but it might be unnecessarily long right now.
I'm not a web developer by trade or anything so I'll have to learn how to secure the Docker container from malicious code. Hopefully Docker provides some amount of protection for now...
For today it's going to be old school htop + F9 haha.
For running Javascript and Julia, it goes something like Python objects -> JSON -> read JSON in Javascript/Julia -> run code -> output JSON -> read user output from JSON in Python.
For C, we can call C functions directly from Python with some code for dealing with different types.
Not sure if this is the best approach (it's not super fast) but we've been learning as we go. We might be due for a refactor in case the next language we want to support doesn't fit into this pattern. I'm personally excited to learn Rust and maybe add support for it.
PS: Our stack is on GitHub in case anyone is interested in having a look: https://github.com/project-lovelace