I'm pretty active on both. What threads/subreddits are you reading?
Pretty funny considering the content of the post he wrote that on.
I do and don't agree with you. Whats really going on here is that development time scales linearly with the number of decisions you need to make. Decisions can take the form of a product questions - "what are we making?" and development questions - "how should we implement that?".
There are three reasons why people feel confident saying "I could copy that in a weekend":
- When looking at an existing thing (like r/place), most of the product decisions have already been made. You don't need to re-make them.
- If you have a lot of development experience in a given domain, you don't need to make as many decisions to come up with a good structure - your intuition (tuned by experience) will do the hard work for you.
- For most products, most of the actual work is in little features and polish thats 'below the water', and not at all obvious to an outsider. Check out this 'should we send a notification in slack' flowchart: https://twitter.com/mathowie/status/837807513982525440?lang=... . I'm sure Uber and Facebook have hundreds of little features like that. When people say "I could copy that in a weekend", they're usually only thinking about the core functionality. Not all the polish and testing you'd actually need to launch a real product.
With all that said, I bet I could implement r/place in less than a weekend with the requirements stated at the top of that blog post, so long as I don't need to think about mobile support and notifications. Thats possible not because I'm special, and not because I'm full of it but because most of the hard decisions have already been made. The product decisions are specified in the post (image size, performance requirements) and for the technical decisions I can rely on experience. (I'd do it on top of kafka. Use CQRS, then have caching nodes I could scale out and strong versions using the event stream numbers. Tie the versions into a rendered image URL and use nginx to cache ... Etc.)
But I couldn't implement r/place that quickly if reddit didn't already do all the work deciding on the scope of the problem, and what the end result should look like.
In a sense, I think programming is slow and unpredictable because of the learning we need to constantly do. Programming without needing to learn is just typing; and typing is fast. I rule out doing the mobile notifications not because I think it would be hard, but because I haven't done it before. So I'd need to learn the right way to implement it. - And hence, thats where the scope might blowout for me. Thats where I would have to make decisions, and deciding correctly takes time.
But in all seriousness, I agree with most of what you said - I think I'm just more bearish on people's ability to infer those many decision points without being given the blueprint like we were in this article.
If you are a senior engineer at FB and you decide to make Twitter in your spare time, I buy that lots of experience and knowledge gained at your job can probably get you going fairly quickly. But I have never seen an example of engineers discussing sophisticated systems like these where crucial aspects of its success in terms of implementation didn't rely on some very specific knowledge/study of the particular problem being solved that could only be gleaned after trying things out or studying it very carefully. The representation of the pixels is a great example in this case -- they go into wonderful detail about why they decided to represent it the way they did, which in turn informs and impacts how the rest of the stack looks like.
I think at one point Firebase had it as one of their example apps something which very closely mirrored what they did with r/Place, so I agree that one could probably build some "roughly" like it somewhat quickly. I agree that in general knowledgeable individuals could probably grok things which would in some form resemble popular services we know today. The devil is in the "roughly," though. I think that often what makes them be THE giant services we know are things that let them have the scale which very few of us have ever needed or know how to deal with, or because they have combined tons of "below the water" features and polish like you mentioned. When really basically most web apps that we use are CRUD apps which we all know how to build, I think maybe we need to give more weight to these "below the water" features in terms of how much they actually contribute to the success of the applications we use.
That's funny because I thought (assuming you're referring to the bit-packing of the pixels) that seemed to me one of the more obvious choices (to do something else would have been more remarkable to me).
Beyond that though, I have zero experience with real-time networked systems. Especially with a gazillion users and that everybody gets to see the same image, that seems hard.
The cleverest solution that I read in the article, that I really liked but probably would never have thought of myself (kinda wish I would, though) was the part where the app starts receiving and buffering tile-changes right away, and only then requests the full image state, which will always be somewhat stale because of its size, but it has a timestamp so they can use the buffered tile-changes to quickly bring its state up to time=now. Maybe this is a common technique in real-time networked systems, I don't know, but it was cool to learn about it.
It's getting the nitty-gritty details hammered out and then implemented that takes time and people. I'm currently trying to do something similar but directed in a very different way. Facebook feels like a CRUD app. But there's a lot of different moving parts that you have to deal with on top of the CRUD pieces.
And every single one of those things--while not necessarily difficult to implement individually--add up to tons of decision time.
When the expected outcome is already decided, the problem becomes much much easier to solve for a developer.
Assuming that you're dealing with some kind of a reasonable web stack, implementing any individual feature is often not that big a deal. Deciding to do it, and also perhaps making the choice to have a reasonable web stack to begin with, are potentially problematic.
And of course, building something that can massively scale is hard as well.
But from my experience with side projects and also my day job, the hard part is making decisions.
The challenge is: Meet the main requirements of r/place: 1000x1000 image, a web based editor, 333+ edits / second, with an architecture that can scale to 100k simultaneous users (although that part will be hard to actually test). I won't implement mobile support or notifications, and I won't implement any sort of user access control (thats out of scope). The challenge is to do it & get it hosted online before I go to bed on Saturday (so, I'm allowing myself some slop there). But its already 1:30pm on Friday, so I think that easily qualifies as "less than a weekend".
As a stretch I'm going to write all the actual code in nodejs while aiming for well into the thousands / tens of thousands of writes per second territory.
I'm willing to accept some fun stakes if anyone wants to propose them (ice bucket challenge level stuff). I'll live-tweet progress here - https://twitter.com/josephgentle
(Edit: fixed days)
Polish and perf tuning tomorrow.
Code: https://github.com/josephg/sephsplace (warning: contains evil)
I'll post a writeup about it all once I've eaten some real food and had a rest. But for now, enjoy!
It would be kind of neat if the place experiment could become something of a micro-benchmark for online customer facing distributed platforms.
Keep up the good work man, it's looking good.
Don't forget to hydrate and consume pizza.
Mobile support was a huge factor for /r/place. To reimplement the project while completely ignoring mobile is basically taking out maybe 30% of the effort required. It's a bit of a copout to claim "can implement in a weekend", while discounting one of the main reasons why it is nearly impossible to hammer out solo in a single weekend.
The other "big ticket item" is integrating such a system into an existing live stack. It's very convenient that you get to pick the tools you think are most efficient for the job from scratch; it's quite another to restrict yourself to an existing set of tech.
>> an architecture that can scale to 100k simultaneous users (although that part will be hard to actually test)
Also this. Reddit had a single opportunity to deploy live. You can't throw out many days of required dev/staging benchmarking for a system that must work on first deploy. You're not "done" after a weekend unless your first deploy both works 100% and scales as required.
tldr; Were I to do what you are doing, it would be out of interest and to put it live for others to use. What seems disingenuous is to do so with the primary goal of proving that "see, child's play - it's not difficult at all!". The goal should be to build something that works, not to rush the job as some way to show off and take something away from Reddit's efforts.
From my post above:
> But I couldn't implement r/place that quickly if reddit didn't already do all the work deciding on the scope of the problem, and what the end result should look like.
If anything, this only increases my motivation to replicate the project myself, whether it's during a weekend or two full weeks. It's interesting enough and at the right level of complexity - kind of simple, but not too simple - to make it a fun side project.
I thought I knew timezones... But that's 40+ hours ahead of me.
But there's no reason I couldn't use multiple partitions if I had to. To make it fully consistent I'd just need a consistent way to resolve conflicting concurrent writes on different partitions. Either adding a server timestamp to the messages, or using the partition number to pick a winner would work fine for this. It would just make the server logic a little more complicated.
Yes - today's Facebook could not be built by a dev in a weekend, but Facebook 1.0? Not so hard.
The secondary goal of pushing pixel updates to lots of users would probably want separate hardware. I would probably fake it so its not really realtime. If you send the web client batches of updates to replay on the board it looks realtime and doesn't require persistent connections.