Ask HN: How can I run untrusted Python code?

[billpg]

Hello HN. There's a side-project I've been wanting to work on for years, but there's one thing that keeps holding me back because it won't work without it.

It all started when I read about people running poker-bots. Systems that play poker on poker websites. I wanted to try my hand at writing a robot but I didn't want to play against humans. Those websites have rules against robotic players and I didn't want to spend time fighting that arms race. I wanted a system with an API designed for robots to interact with. I also didn't want to gamble with actual money. This would be an intellectual effort for me so I'd want all the "chips" the robots gamble with to all evaporate once one robot has been declared the winner.

Not being able to find such a venue, I wanted to try building one. Wanting to avoid human players meant that a tournament would need to be held in a closed system where all the players are completely autonomous, allowing no interaction with the outside world.

I sketched out my plan. Each developer would upload a ZIP file to my server that contained the program code. The controller code on my server would unzip and launch each player process, connecting to the stdin and stdout of each process. The controller would send event notifications into the stdin ("You have been dealt this hand.") and read player instructions from the stdout ("Discard card #3.")

That was when I realised my problem. If I allow strangers to upload any program they like, people are going to send me malicious programs. Bitcoin miners. Spam senders. All sorts of potential for evil.

I considered JavaScript as that's designed to run inside a restricted environment but it was impractical. Do I launch a web browser from my server? How does my service code communicate with the JS code? Can I stop JS code from connecting to the internet?

Even if I could work through to a practical solution, I didn't really want JS, I wanted Python.

Googling for answers led me to a set of projects that didn't quite fit the bill. Most hadn't been touched for years and others had a different security model to what I needed. If this project was going to get anywhere I'd need a partner who knows exactly how to run a process on a Linux server in a way that can't do anything other than be a poker bot.

So here I am. I'm hoping that projects like the one I described already exist and that'd be great. Maybe someone knows of a way to launch the Python interpreter but won't allow code to start reading files or connect to other services on the internet.

Or maybe I should stick with my plan B where player code runs on the developers own machines and I accept the possibility of human players as an unavoidable reality.

[al2o3cr]

You could use a short response timeout (<1s) to make it impractical for human players to participate - for instance, that's what Battlesnake does (https://play.battlesnake.com/). Players provide an endpoint that follows the required API and their code never leaves their machines.

[billpg → al2o3cr]

Thank you. I'll have a look at that API and see what I can learn from it.

[astroid]

Just my $0.02, but as someone who made a living for a while poker botting before the Black Friday banking changes, I can't help but feel like you are approaching this in a bizarrely complex manner.

I guess I don't get why it is important to filter 'real' players for starters, and I definitely don't get why you'd land on just running uploaded Python code... the entire approach seems so needlessly convoluted.

I also don't understand why you are so worried about the cat and mouse game with the 'big sites' if you are just using play money. I haven't followed developments here in years, but back when I was earning a living it was enough to just run a screen scraper on your machine, that then passed of table state data to a seperate device (which is running the actual bot logic / storing player hand history databases, etc.) -- and even that wasn't strictly necessary unless you really raised some red flags.

Maybe take a look at some of the open source botting platforms like OpenHoldem, and use that as inspiration to re-evaluate your approach?

https://github.com/OpenHoldem/openholdembot/releases

This just seems like such an unsafe and half-cocked approach -- even if you got this python bot site up, I can tell you as a former pro poker botter that I would never ever have considered uploading my bot to your site, aside from your code execution concerns. You'd then see all my logic, and that's the biggest leak a poker botter could have.

Even if you pull this off, no one doing serious work on their bot logic is going to trust you with it -- for you it may be an intellectual exercise, but for that community it's a lucrative job where the only thing that makes you stand above the rest is a dynamic strategy that you are going to protect just as tightly as the key to a bitcoin wallet. The only submissions you will get are toy bots that have no chance of being competitive anyway... which kind of defeats the purpose of 'bots only' I think.

I don't mean to be discouraging, but as someone who made a living doing this for a few years I can say without a doubt no one who has put even a moderate amount of work tuning their bot is going to upload it for you to be able to rip the logic off.

If nothing else, you should be re-evaluating this in a manner where the only data exchanged between you and the client is the table state as it changes, as well as an API for submitting their action on their turn. Otherwise this is dead on arrival.

[billpg → astroid]

Thank you very much for the thorough answer.

I hadn't considered that angle, that someone might not want to reveal their player logic to me. What you suggested was my "plan B" where instead of communicating over stdin/stdout, the players connect to m server over TCP/TLS and communicate over that channel instead.

I didn't want to do it this way because I liked the idea of running in a closed autonomous system. That ZIP file has to contain all the state of the player. Your robot is out there in the world alone and separated from their owner. It isn't just communication with a human I wanted to restrict, but any external system. Databases, helper systems, anything like that.

The idea of doing this way stemmed from playing CRobots in my teens in the late 80s. The robot has to be completely autonomous without the ability for a human to step in. I found programming a fighting robot rather frustrating in the end but I recalled the experience when I stumbled upon poker bots.

Your post may just cause me to pick up that plan B and leave trying to find that sandbox mode for Python. I'll be sure to thank you if I ever build something bigger than some rough plans in a text file.

[astroid → billpg]

I'd love to hear where this project goes, and I'm glad my feedback was actually helpful.

Something else I want to point out though since you mentioned "It isn't just communication with a human I wanted to restrict, but any external system. Databases, helper systems, anything like that."

This is going to be another deal breaker for true poker bot players -- they are all tapping into hand databases, recording every play they have ever participated in. I'm not sure if this is still possible since I have been out of the game, but I even ran a separate machine just for observing games I was not in.

Hand History is necessary for a proper bot logic - it needs to be able to 'bucket' player behavior, and that doesn't exist in a vacuum. A huge part of gaining an edge is programming in that behavior.

For example -- how can you possibly assign a player to be considered 'aggressive', 'passive', 'heavy bluffer', or 'only goes in on nuts' etc. without a hand history database? On top of that, there are other applications such as Sit N Go Analyzer that can help give you 'by the books correct' plays for end games if you are playing tournaments for example.

Any serious botter is going to be taking advantage of -some- historical data, absolutely no one is playing a blind strategy -- that's not poker.

I hope this isn't discouraging, and I'm glad my initial comment was helpful -- I'm genuinely curious to see where this goes, as I really miss those days. It felt so wonderful to be living off the logic I wrote, possibly the freest I have ever felt. And the rush of being at the point where it was time to raise the stakes/buy-in, which always results in tweaking to the playerbase at those levels. It was so fun. Had I been maybe 3-6 months ahead of where I was when Black Friday hit, I probably would have left the US to keep playing. So close..