The whole point of a side project is to over-engineer it! The thought of holding myself to deadlines and driving myself to "ship" is totally antithetical to the entire purpose of a hobby project. It's a welcome relief from the 9-to-5 constant necessity to cut corners and finish things without ever properly understanding them because there's no time.
For my side projects, I actually want to go down rabbit holes. I will try the same thing 7 different ways and benchmark the hell out of them. I'll read all the documentation and every blog post I can find. Learn a new language just to see if its features make things ever so slightly easier. Learn a new framework, write a new framework, throw it away and write a new one.
The point isn't to ship code, it's to learn. Honestly I don't care if it ever ships. And even when I do release one, I don't care if nobody uses it except me.
HN is made up of two groups in ever constant warfare, the yin and yang, as it were- the group who thinks that programming should be fun, experimental and enjoyable, and the group that thinks that programming is a means to the end of getting a product to market and nothing more.
Whenever a post from one group (such as the article) comes out, a post from the other (such as OP) will quickly appear. And everyone loves it :)
For what it's worth, I think that both perspectives are valid. It depends only on your goals.
This is mostly off-topic, but being able to identify or root out the value mismatches between people in a discussion (as you've done) seems like an essential ingredient to productive discussion.
I've noticed that people who can do this are disproportionately better at achieving their goals than those who can't, especially in engineering organizations. Exceptions of course for zealots and revolutionaries.
"Observe, this is the beginning of philosophy, a perception of the disagreement of men with one another, and an inquiry into the cause of disagreement, and a condemnation and distrust of that which only `seems', and a certain investigation of that which `seems' whether it `seems' rightly..."
- Epictetus, The Discourses.
Absolutely. This even works at a somewhat lower level... I remember hearing an argument between two cow-orkers, and realizing that each was thinking different (but similar!) things, but assuming the other was thinking the same thing. Angriness ensued...
This skill is one of the key differences that I've noticed among philosophy students (majors) and everyone else, with their skill being far more developed than others'.
I'm not sure if persons with the skill are more disposed towards studying philosophy or the study of philosophy prompts the development and demonstration of that skill more, but hang out with the philosophy kids and you might come to refine this skill by social osmosis.
The yin and yang becomes more amusing when you think that the site's title suggests an appeal to the first group: the techies for whom this is a passion.
Yet it would not exist without the second group because it was started by an investment fund/startup incubator.
I agree with you, I love both perspectives and the passions they ignite. I am squarely in the first group, and I love reading about the real life realities of product design/management, pitching, etc.
I don't think these are mutually exclusive. At work, my goal is to create products that people use (and pay for). At home, my goal is to learn and have fun. Different approaches for different contexts.
This is not specific to programming, but personalities in general. There are people who love the journey and others who love the destination.
People rarely switch from one to the other. It's part of our core ideals.
However, there is a large, vocal, percentage from both these groups that will always voice their opinion against an opposition opinion. It's not a competition and there's no correct answer. It's purely subjective.
I think its embedded into the "maker" mindset in general and does not necessarily have to be one way or the other. It's both.
A writer can have lots of poetry, personal expository essays, journals, unfinished drafts lying around OR s/he can "ship" a best-selling novel, publish a high-traffic blog or something.
Similarly, a painter would experiment with lots of techniques, colors, textures "on the side," again having lots of personal work lying around, yet would probably treat something s/he would put in an exhibition differently.
In the first case, you are experimenting, discovering, breathing; while in the second case you have to think hard about your audience, your readers, or your customers.
A writer can have lots of poetry, personal expository essays, journals, unfinished drafts lying around OR s/he can "ship" a best-selling novel, publish a high-traffic blog or something.
There are also writers who can do all of these: my personal favourite is Robert Graves (possibly because once many years ago I got him to autograph his Selected Poems and his memoir Goodbye to all That).
And then there is everyone else in between, including me who both want programming to be fun and enjoyable and on the other hand simultaneously care about getting a product to market.
There are things in-between, too - Enjoy the fun software development process to make an end product successful! And this is exactly my own feeling and goal, though I'm not doing it very well as of now ;)
Yes! My last hobby project (I explicitly point out "hobby" project instead of "side revenue", but the article mentions "hobby" in the first sentence as well) was exactly that.
Because it was a hobby project, I went completely mad on it. I built my own SPDIF interface, from scratch, using an FPGA. I let the hardware broadcast the sound frames over Ethernet. I also re-implemented Sony's proprietary consumer electronics control protocol, in hardware as well (a microcontroller could have easily done that instead). I greatly overengineered a circuit board, despite having a functional "test setup" already that did the job more than well enough. I wrote largely unnecessary client software, using language extensions I've never used before.
The task at hand could have been solved with buying a cheap USB sound adapter from eBay and some manual work instead. Had I gone down that route, I probably would have completed the task in a fraction of the time that building this project took.
But I had so much fun along the way and learned so much about electronics, FPGAs, SoCs, static typing for python etc., all because it was a project for me and nobody else.
One big lesson I've learned, though, is that having an actual task you want to solve greatly helps you in finishing your project. I ran out of steam a number of times and the project fell to the wayside, but because the underlying task was not fulfilled yet, I always went back to it and quickly got completely focused on it again. Projects that I only did for playing around and learning most often got abandoned.
Agreed. Generally, at any point of time I have two types of side projects going on:
#1. One that I am very clear about how to implement. One that I want build, ship and grab users as fast as I can. The real goal of the project is to get it out their in the market and make a tiny dent in the universe ;)
#2. One that I am not very clear about what it's gonna become. This one I want to take it slow. Experiment. Play with tech that I haven't had a chance to flirt with. The goal is to enhance my toolbelt and learn new concepts/tech.
Clearly keeping these projects separate helps me keep things clear. I quickly take decisions and move fast with one. I play and experiment a lot, no matter what the outcome is, with the other one.
There is also the third way -not to earn, not to necessarily learn, but simply to feel pleasure from making a software art and experimenting with things. When playing with my side project I feel like a painter adding new lines to the same canvas for years, like Gaudi spending his 15 years of life constantly redesigning Sagrada Familia.
Well, I also learn stuff this way but often I get bored with them once challenge is completed ;)
Totally agree. I had a problem which required finding the minimum distance between an ellipse and a point. If that had been my day job I would have settled for Newton's method and hacked around the regions of bad convergence. Instead I spent a week developing to be what I still believe to be a novel numerical method that is much more stable over all regions.
Turns out I was wrong in some earlier maths, and the curve I wanted to find the distance to wasn't an ellipse. I was able to generalise the method for arbitrary curves, and now I've solved my problem!
>I had a problem which required finding the minimum distance between an ellipse and a point. If that had been my day job ...
OT: I had the problem as part of my 1st day job in 1990 on Parasolid.
IIRC, there was more than one way of converting the problem into solving a quartic polynomial, which is solvable by radicals and so doesn't need Newton's method.
If the point is inside the evolute of the ellipse then the quartic has four real solutions, which gives two local minima and two local maxima of distance.
If the point is outside or on the evolute of the ellipse then the quartic has two real solutions, which gives one local minima and one local maxima of distance.
Having said that, all methods using quartic polynomials could be unstable if the ellipse was pathological, for example very long and thin. Maybe that would be a time to revert to Newton's method.
There are certainly multiple ways of doing it. I'm curious, do you remember if the specific quartic had a nice "trick" to the formulation? I played with it a little and couldn't see anything that would cancel in a nice way.
I try to avoid using quadratic solutions in code because of numeric instability issues. Granted I never implemented the direct solution, but the number of opportunities for catastrophic cancellation gave me shivers.
One approach is to use the rational quadratic parametrization of the ellipse and substitute that into the squared distance function to get a quartic.
>I try to avoid using quadratic solutions in code because of numeric instability issues.
Sensible!
Back then the quartic solver had been written already and extensively tested by someone else - probably for line X torus intersection.
Even so there were numerical problems introduced because the quartic was in power basis and so the coefficients didn't have geometric meaning. I guess phenomena like Wilkinson's polynomial could occur. [0]
If I was doing it today, I would probably proceed as follows:
1. special case for the point exactly on an axis.
2. otherwise, choose the quadrant containing the point and represent that as a rational quadratic Bezier curve. This parametrizes the quadrant from 0 to 1 and is guaranteed to contain the global minimum.
3. substitute the quadratic Bezier curve into the square distance function to get a Bernstein basis quartic polynomial.
4. differentiate to get a Bernstein basic cubic.
5. solve the roots of the cubic numerically in the range 0 to 1: either Newton's or clipping or some other hybrid numerical method. A good reference is [1].
>Is there a reason not to use the cubic formula in this situation?
Numerical stability in edge cases, which will occur for code exercised as extensively as Parasolid's.
I guess you could a Bernstein version of the cubic formula, though I've not seen one derived in the wild. It would be nice to have as well as Bernstein quartic formula. The Bernstein quadratic is easy.
> The point isn't to ship code, it's to learn. Honestly I don't care if it ever ships. And even when I do release one, I don't care if nobody uses it except me.
The objectives for a side project are subjective, and so they should be. However I doubt most people truly have no intention to ever ship a side project or "don't care if nobody uses it", as admirable as that would be.
I look back at my naive, recently graduated self and remember how I truly wanted to ship my side projects but loved to play around with new toys so much that I rarely shipped anything. I wish I had read a few more articles like this that would have helped me strike a balance between trying new things but still having enough experience in the stack I'm using to actually ship.
In short, if you have a great idea you want to ship then deprioritise learning to stand the best chance of getting it out there!
I think there's an important conflation of what a side project is to many people. If I mention to any of my non technical friends that I'm building an app their minds go to the movie "The Social Network" or to Steve Jobs or Bill Gates. They become excited and interested - up until the point where they realize it has nothing to do with Facebook or Tinder or Instagram.
I have no doubt that this effects the side projects developers choose, or how they present them. Places like HN will always to a certain extent push people to "launch" and monetize their projects.
Having done this a number of times with side projects, I can say I both agree and disagree. I've learned so much from delving down black holes: new languages, frameworks, design patterns, tools, etc.--stuff that stimulates my brain and helps me feel like I'm becoming a better engineer. ...but not realizing an idea in my head for a tool that I just want to use because I can't find it anywhere else is really annoying. I kinda hesitate to even start projects anymore because I fear I'll spend hours and hours, burn out, then end up with nothing I can use.
Agree fully. Personal projects let you experiment, they let you try things out to see just how hard or easy building that set of features you were contemplating is.
Now, if you personal project is intended as a gateway to your bootstrapping of commercial venture you _must_ make completely different choices in what/how you build (lean, and minimum useful/viable project) tec.
I agree; I've been working on my mess of a side project for years now. I let the scope grow tremendously, and started over more times than I can remember. It's fun to experiment with potentially better or more interesting ways of doing something in a side project without worrying how much time it "wastes" in the long run.
The premise of the story is that you want to deliver an MVP. Your premise is that side-projects are a nice way to geek out on tech. Both are valid, but you have to make sure what you're trying to accomplish before spending quality time with rabbit holes.
I offer a middle path , a bhudist interpertation of the problem. A project can be both fun/innovative and productive. You iterate on the fun , then you iterate on the productive. Youll never acheive completeness in both. Youll always suffer from needing to improve on aspect. Life is suffering , learn to accept and not let that reality stop you from creating.
But some projects are more innovative or productive than others. Maybe you should abstract those projects away from eachother? You could put them in seperate classes. You could have the productive project use the innovative project.
Lots of times I use side projects as a way to explore new stacks, deployment approaches, etc. The goal of the project is not to deliver anything, it's to learn so that when I go to do my next "real" project at work, I know what works well and what doesn't.
Exactly. When I see a coworker start irrationally focusing on a new technology at work the first thing I think is "This person probably has no side projects to play with"
> "This person probably has no side projects to play with"
Exploring new technologies that could be beneficial (or harmful) to the company shouldn't be a side project. Perhaps we can shift the thinking from "this person doesn't have time for side projects" to "how can we make exploring new technologies a part of the job?"
How about making wednesday mornings, a 4hr 'personal project' time? Make it a requirement that everyone has to work with tools at the edge of their skillset, that is also relevant to work, but let them choose what, why, how. They could work on a personal project, or experiment with a new library or tool they would like to incorporate.
Start the day with a "not really a standup", where people can quickly explain what they want to accomplish in the next four hours.
Once a month, you spend a couple of hours of the time demoing the work you have done for each other, explaining the technology and what you have learned, and then discussing and evaluating our individual technical weaknesses and deciding what to do next. This rest of this monthly meeting could also be spent improving social coding skills - code and commit hygiene, team processes etc.
Not only would this keep people technically sharp but it keeps them from playing with toys in production and breaks up the work week, while still fulfilling company goals.
If you have to start giving reports, it kills creativity. The point is to do something everyone else may think is irrational, but you feel would lead somewhere interesting.
We call this development time at my workplace. Every two weeks, I spend the first half of my shift doing whatever I please, with encouragement for it to be work related, but no real guidance.
Last week I wrote a utility in Python to parse a configuration file that we had so far been editing by hand, or with bad, slow and cumbersome tools. Now our Linux guys can make the changes is seconds across multiple machines, and the parser is much safer than the bash hacks we had been tempted to use before.
Point being, development time can be great! More companies should embrace the idea of letting their employees improve their own workflow. Some of our best ideas were just that: a little spark that an employee had the opportunity to actually explore.
It doesn't even have to be something "productive" ... what's being proposed here is that companies invest in literally having developers research new technologies, without expectations that said research will result in anything immediately profitable.
Some companies are already doing this. When I worked at Intuit we would have Whitespace days and Innovation days, roughly every month or so where you could spend 1-3 days doing whatever you want.
While I have a lot of freedom at work to explore side projects and random stuff and I spend a lot of time doing so, there is a part of me that feels like there's a reason software engineers, at least in the US, are paid so damn much. To me, it's kind of implied that I'm paid <ridiculous amount compared to virtually any other job with similar requiring similar amount of training> is because I'm going to be doing more than the 35-40whatever hours I spend at the office.
Part of that is support (though about half of the jobs I've had involved little to no on call duty), part of it is training one self, in addition to everything I have to do during the day.
Let's call it discretionary overtime baked in pay since we're usually salaried.
Having a solution to the problem and having people advocate for implementing that solution in their companies are too different things.
We know about lots of best practices but if no one is advocating for them at their companies those practices we'll just be shining cities on a hill that we look to from a deep, dark valley.
I recently built a C++ meta-programming monster for a side project. I want to make an article about that in the near future, and I am somewhat afraid that some people might think of reusing it in a real project.
I had a lot of fun doing it, and indeed learnt to not do it in prod, the compiler errors are a nightmare to read and good luck maintaining that in the long run.
That seems like a great side project. The one that let's you understand things NOT to do. At the same time, it helps engender basic traits like curiosity, try-until-you-fail-and-try-again, engineering skills etc.
There are two types of side-projects though: one such as yours, where you try and understand a concept or engineer something just for the fun of it.
And there is second one: a game that will mature into something; a hobby project that will actually develop into a company. I think it's good to disambiguate these two as the OP was more about the latter where you may never get to a point to see it run or build let alone flourish into something meaningful - as overengineering kills the fail fast fail often mentality and the project itself.
Maybe the real lesson here is that the last open task on your project is something that can present human-readable digests from parsing the compiler errors...
My constant on-again/off-again side project is something similar but in C#. Spending a week building a preprocessor and then discarding it because I didn't need it was both infuriating and strangely pleasing at once. :)
To be fair, many of us have plenty of side projects that are just not computer related. Reality is those should also provide teaching opportunities. (Indeed, for many, teaching children is essentially a full time job in terms of time commitment.)
Still, I completely agree with the point that "experiments" should be kept off the critical path as much as possible. It all depends where your risk/reward is, though. The people that took the risk on a stack and got rewarded with success are highly visible. Just as many took the risk and failed. (Probably more?)
Now that's a good point - I've noticed a tendency to use side projects as a buffer for frustrations being experienced at work. Maybe there is a fundamental amount of true engineering I need to do on a weekly basis; if I'm not getting my fix at work it'll spill over to hobbies.
Lately I'm pulling back from technical deep diving at work. A lot. My role as an enterprise architect (bleh, don't judge too much plz) means I necessarily have to shift to big picture guff and stop spending time on the details pieces. In short, I don't really get to engineer as much as before.
Of course, I'm an engineer at heart so the idea of ignoring the details is anathema and it frustrates me. When I'm deep in whiteboards, board meetings and shitty powerpoint at work, the side projects and home lab become my release valve. It's a coping mechanism, I think.
Likewise, when I'm sucked into low-level grind at work which happens from time to time, side projects stagnate. I don't need them for a while.
Or they're bored out of their minds and trying to make their work not be so much of a waste of time.
Most of the stuff I've seen done at typical development companies are intellectually interesting maybe to people who really started learning to code on the job, and generally don't have much experience. They're discovering new things every day, so it's fun! However, if someone has already few languages and various types of projects behind them, then the job becomes much less technically challenging and much more repetitive.
IMO, there's room for both approaches, perhaps with the exclusion of the most the irrational situations as you describe. It's unreasonable to expect engineers to bring battle-tested solutions to the table 100% of the time.
I have a side project where I'm using the new Crystal language and even wrote a post on my company blog about it but we don't actually use it, nor do I try to get us to use it.
Me too, I think there are two types of side projects, this article focuses on those which I believe people will try to generate a business out of, or at least some form of passive income.
But there are also some side projects where you just want to get your hands dirty with some different tech, if you do deliver something then you can showcase it on your portfolio. I know when the app store gold rush first started a lot of people did this as experience to get iOs dev jobs.
Exactly - I enjoy side projects precisely because they're not bound by pragmatic concerns. I can delve as deep as I like into getting the CI pipeline working just right, making sure the servers and cacheing are set up ok and all those other things that I usually have to leave to colleagues.
More often than not the skills I learn doing this kind of thing end up being very useful to me in the day job.
I'd like it if the side project was successful enough to justify all of that yak shaving, but I don't really expect it to be and I'm not that fussed when it isn't.
But if your primary goal is that your side project should become your day job then the article is pretty good advice. Leave your yak unshaven :)
99% of the time, if you want to go pragmatic, you should buy a ready-made solution (and then possibly adapt it to your use case). It's efficient (especially if you need the problem solved to make money) - but it's no fun.
I agree, but there are good reasons why it is often good to combine both types.
Considering this is only a side project, you have a high likelihood of abandoning it out of boredom, lack of traction or simply burn out. However, if your side project is "interesting" to you because you are experimenting with various tech, you come back to it because it is fun and it will keep you motivated.
It is totally cool to build something to learn and throwaway, but why not build something to learn that might also generate some income in the future?
Nailed it with this comment. My side projects are for learning. There was a moment or two where I was aiming to monetise but I'm not at that stage and I now realise that, I just want to play with technology.
Actually starting to livestream them now as I figure if others can see what I'm trying to do I might be able to get instant feedback on how to do things better - crowd sourced learning :)
I have never regretted taking the time to make sure my projects have a proper logging infrastructure with every possible thing fed to the logger. I've built logging support libraries to set up AOP hooks to auto-log the call stack at all times, and it's been a lifesaver.
I can't imagine not over-architecting this kind of stuff. I know I'd be in a world of pain several times if I hadn't.
Not 'amyjess, but the rightest answer I've found (and I do a lot of this stuff) is to use a Bunyan-style one-JSON-object-per-line log file and ship to something that can consume it (ELK, CloudWatch Logs, whatever). AWS CWL is dirt cheap and the awslogs daemon, while pretty far from perfect, does work pretty well. You can also couple it with CloudWatch Metrics.
That's completely dependent on the format of your infrastructure. There is no answer that will fit everybody, and asking around for advice of strangers may even bias you in a damaging way.
This is exactly how I approach side projects as well. I use them as iterative learning experiences, with each new project building on the last to more complex ideas. Learn a TON that way, and potentially build something that people want to use. You can always go back, and add things you've learned down the line.
One thing I do not do, is just rebuild something someone else has already done in a new "stack", just to see if I can. Any project I undertake is a new idea I've not seen done, or with features I want.
Agreed; in fact, the title falsely (or maybe ironically) conveys a positive attitude about how it's an "art," thereby attracting people who, like me, wanted to hear about some gloriously and baroquely over-engineered side project that will never see the light of day. A large investment of time, effort and craft into something, for its own sake, with no heed to whether it will ever "ship": That's what art is, man!
Should've been titled something like "Don't Over-Engineer Your Side Projects" or "How to Make your Side Project Resemble Your Day Job (If You Work for Sensible People)"
I often explore new stacks by building a product that already exists complete with identical css! For example: Wanna try out aiohttp, vue & css grids? Let's build a Slack clone!
Yep. That's why I've rebuilt my little blog about a dozen times. I've built CMS's from scratch before (actually built a white label CMS generator) and it's a non-complicated process that I know deeply enough to push the pain points when I try things out.
I do that too, but make that my secondary goal. If it fails, at least I got to improve my skills on whatever technology I chose to build it in. It helps me push along when it gets to the dull parts.
When you are doing a side project you typically have a number of risk factors such as: (A) using new tools, (B) working in a new area, (C) building a new type of application, (D) uncertain marketing, etc.
Any kind of project has a "risk budget". If you have few other risks, then you can spend your risk budget on new tools. If you are facing high risks in other directions, it makes sense to stick with what you know.
Yea a lot of my side projects are started out with the understanding they're not going to become anything more than toys for me to play with ideas.
This article feels like it's written from the perspective of using a side-project to bootstrap a business rather than as a learning experience, hobby or toy.
Most of my side projects are like that -- just an excuse to learn something and more often than not abandoned when I have learned what I wanted. Not the best way to develop software but a great one to learn new technology.
By "deployment approaches", I mainly meant CI/CD and infrastructure. So Jenkins, Travis CI, etc. to do the build work and testing, then automatically push to Kubernetes, Docker Swarm, AWS Lambda, etc. on the infrastructure side.
For a side project, pick one on that you're interested in on each side. For example, maybe try out CircleCI and deploy to Google App Engine. Then you'll learn two new things, maybe one is great but the other is a pain in the ass. You can take the one that works well to a "real" project and talk knowledgeably to the architecture.
“I know: I’ll create a triple zoned redundant architecture with pub/sub database replication, a 32 node Kubernetes cluster and private networking across all regions – that way, I can handle anything!”
The good news is, once you've done that, you can get a pretty decent job.
The majority of software engineers seem to be under the illusion that potential customers care about the stack they are running.
This is not entirely true. The technology do surface in terms of user experience. It does in the details: app start, ui responsiveness, etc. I'm writing a game using go right now, and the end user can tell it's different because the user's cpu seems more powerful.
All in all, I think those are good mistakes, not bad ones. I wish more engineers could spot over-engineering, and they don't until they did the mistake a couple times. Side projects are, I believe, how one really learn to write programs. I think the right advice is: do whatever you want, do all the mistakes. How can you write a successful side project anyway if you never learn how to write a program? Also, the worse type of programmers are the ones telling you about over-engineering because they read about it in a blog. Code more, read less, I guess should be a good motto.
Potential customers do care about the stack you're running. They care if they have to download Java or Flash. They care if they have to be connected to the internet or can use your product offline. They care if your program starts quickly. They care if your program runs quickly once started. They care if your program has bugs (let's not pretend different languages aren't more or less error-prone). They care if your program is cross-platform. They care if your UI is consistent with their platform. They care if your product releases new features quickly.
I think the point was that user doesn't care about your stack, but they care about the consequences of the stack you've chosen. It's a subtle but important distinction.
Of course. My point was that it's fairly common to write slow/unsuitable applications in spite of a stack's strengths, while stack limitations (or strengths) are generally overstated, depending on trends.
Proper engineering and design are required no matter what you choose.
Have you figured out a way to make a mature implementation of the JVM start up fast? Nobody else in the industry has.
Is there some way I know of to package Flash games so that Firefox users can play them without downloading Flash?
I get that done of the examples I mentioned are arguable, but some of these are pretty unambiguously stack intrinsic. Some usability concerns cannot be addressed in some stacks.
Notice how just about none of them are the user caring about your stack.
In other words, when confronted about your high bug rate, you're not going to get anywhere going "heh, you see, Python was just more convenient than a statically-typed language and this lead to..."
"A customer will not know or care if you are using Ruby, Go, PHP or any other language as long as what you have written is performant and is fit for purpose (which all modern languages are)."
Where games are concerned I'd tend to agree (though Minecraft was built in Java and was TERRIBLE for the first couple of years, now look at it).
> Minecraft was built in Java and was TERRIBLE for the first couple of years, now look at it
Well, it still takes absurd amounts of memory for what it's doing, which I guess is on par for Java. (On my notebook with 4GB RAM, modpacks fall into one of three categories: "works well", "have to shut off as many OS services as possible before starting" and "forget it".)
4GB? I wouldn't expect most open world games to perform well with 4GB. I think 2010 was the last time I had a laptop with less than 8GB. My current laptop has 16GB. Game developera and certainly mod developers don't optimize for bargain basement hardware.
comparing minetest (a c++ minecraft clone) and minecraft gives the following memory usage upon starting on a blank new world. Also, minecraft takes five times more time to launch.
well only because java has no value types/struct like data structure.
the reason why this makes memory so heavy is because even a simple int list, needs boxing.
So you end up with a big chunk of garbage which you don't need. On the server, this is not a problem because most of the time the list is a young gen object and die really fast, so G1GC solves most problems.
The problem of course is really big, because arrays are a pain to work with and not many people do that.
However as soon as java 10 hits, we might get value types and maybe this changes the game (but we will see, since it would've taken way to long for that thing).
Also there are more and more types inside the standard library who would be great value types, i.e. LocalDate/LocalDateTime, but at the moment they take way more memory than needed, especially since they only contain 2 immutable shorts and 1 immutable int.. in the perfect case it would be something like 8 byte, however it takes way way way way more. (basically a simple class at least takes 16 bytes, which is already double the field size, so you end up with 24 bytes [probably more due to various other stuff])
btw. for a game like minecraft you probably have a lot of types that follow the same stuff like LocalDate, small class with immutable int's/short's for position, etc. these have the same problem and will fill the memory more quickly than needed
You'll likely have to anyway :) I still make a difference between writing a side project and starting a startup. The way I see it, there is no failing on a side project.
The weird thing is the same situation works in reverse as well.
Publicly declaring your intention to quit a habit or lose weight has also been shown to increase success in some instances.
I think it depends on whether people reward you with positive regard for stating an intention to do something, and whether you look foolish if you don't do the thing you stated. When you can get the reward just for the statement and there is no consequences for failure, pre-stating your intention seems harmful. On the other hand, if you are not rewarded for pre-statement, and failure is embarrassing, pre-stating your intention seems helpful.
I have observed this vastly "bipolar" results amongst my friends as well. Some take public declaration to heart and they ram through their goals. However I quickly lose interest of the project after I told everybody. I would go extra length to surprise and dish out cool stuff more often I care to finish side project only because I said I will do it.
I somehow concluded that I don't care about approvals on my 'integrity' yet I highly value my 'creativity' viewed by others.
Fascinating. I've tried to use talking about a project as a commitment device.
Looking back on my many projects, the feeling of delivering on an external commitment seems much weaker than the urgency of "wait 'til my colleagues see this!"
I don't agree with this reasoning. Expressing in public your projects is a compromise and it can help you keep developing it when people ask if it is ready. If you are going to abandon your projects, not telling people will not help you succeed, it will only spare you the shame and effort explaining why you droped another project.
I don't think people start projects with the conscious intention of building identity, but with a genuine intention of delivering. I do think, though, that there's a small (or big!) part of your unconscious brain that binds these efforts to an overarching persona-building (both inwards and outwards) project. Although unconscious, it contributes to this net energy.
I guess that this communicating your intentions satisfies this portion of your brain, and reduces the amount of energy involved in the project.
Sure, this makes sense. I just don't think (using only myself as a reference, which is totally biased) that this effect has a bigger impact than the social pressure the public compromise creates.
On another note, I believe integrity could be unnecessarily valued sometimes.
My (limited) experience in business field told me that there is no friend or enemy, but profit. Profit trumps integrity almost every time in business decisions.
I don't think VC deals with integrity of founder in absolute term either. If potential benefit outweigh risk after calculation, then greenlight it is.
Very much perceived as disloyal Steve Jobs, I don't think he got around by integrity.
I find that exclaiming publicly what it is I want to do has a level of gratification (hey, look at this cool idea!) that would otherwise turn into excitement/anticipation during development. That initial gratification might be enough to stall/kill the project.
I've had plenty of ideas that I'd love to see through and think are great, I just don't have the time to do them. Since my time is so limited, I have to choose to focus on one project, or I end up with a ton of unfinished ones (which is the case). So unless I have strong motivation to complete a project (which can be compromised by some sort of gratification), it stalls.
But the majority of mine stall for other reasons; I don't usually announce/discuss them.
Yep, as soon as you start talking about it, people start congratulating you, your reward system kicks in in your primitave lizard brain, and you don't bother actually doing it because you've already been rewarded.
Good points in there. I have a little trouble with "mistake #5" though.
I found that worrying about deployment automation early on is immensely valuable, especially for side projects. I don't always find time or interest to keep working on side projects after a regular day of work. Quite often there are weeks or even months where I abandon a side project only to come back at a later point when motivation is back or things have settled down and leave more time for my side projects. If I didn't take the time to automate testing and deployments (which makes a lot of what continuous delivery is about) I find myself struggling with getting things up and running and become frustrated before I even get started.
Then there's only one way to make sure that my test and deployment automation keeps working: running it regularly, e.g. in a pipeline.
So, yes, don't over optimise your deployment pipeline early on. But having one in place early can pay off really soon -- and maybe even keep your side project alive.
Agreed. Having a deployment pipeline in place dramatically reduces the friction of returning to an old project.
I have a back-burner project that touch very infrequently, but I can jump in, make a quick change, and have it deployed to "production" in just a few seconds.
I definitely automate locally by way of Makefiles, running tests then using the GCloud/aws CLI to deploy from local. Things like TravisCI, webhooks, etc come much later if at all.
At risk of a tar and feathering by the 'lean' crowd, I am going to declare that I don't do MVP's at all, and never really have, with any of my projects.
Why? Well, because almost all my projects are designed for businesses and enterprises, and with that audience - things better work well and they better work first time during the demo or you don't stand a chance of getting a customer.
Perhaps if I was writing a social media app for sharing cat pictures, then yeah, I would slap together an MVP and show it to cat lovers. After all - what is the worst that can happen if your cute cat picture doesn't upload? Just try it again or delete the app and walk away, and there are no consequences.
But what if my HR app doesn't send that crucial reminder email to a department manager that one of your electrical tradesmen's license expires tomorrow, and you end up sending him out to a job next week and he makes a mistake that ends up getting your company sued and eventually shut down, putting hundreds of others out of work?
I am sure as sh*t going to ensure that my reminder email system is absolutely over engineered to ensure reliable email delivery and tracking. You don't get ANY chances to blow that kind of stuff with a business of enterprise audience.
Related to this, I've been following a practice I call "Release Notes Driven Development", which is perfect for side projects. Don't worry too much about technical debt (although, keep it under control), but focus on the little time you have and how to make that as efficient as possible.
Sounds good. Early on, a quick list of bragables should be able to tell you how awesome the idea is. Have new bragables for the next version. If its not awesome at some stage (and the bugs are fixed) you are probably done?
It reminds me of many release notes describing a ton of work without any woah.
The best thing about side project is that I may be wonderfully inefficient, so I can do it the way I want and enjoy the way. Even mentioning the word 'deadline' or 'planing' or 'efficient' or 'customer' in the same sentence as 'side project' is contradictory to me.
E.g., I love to program in assembler. Preferably 8 bit SoCs like AVR. The point is not to finish the project fast or even at all by using a more efficient language that may have libraries for everything, but to enjoy programming in 8 bit assembler. Reading the absurdly detailed manuals. Comparing chips at register level. Obviously, I need to write my own libraries/frameworks, because, I want to do that myself, in assembler.
Even if not using assembly, I may still want to write the library myself in order to, well, do that.
Wood workers often seem to have similar ideas about side projects, like when they build a project without power tools, without glue, from old scrap wood, etc., just because. But for some reason, I seldom meet hackers who understand the fun in doing stuff from scratch.
Yeah, I read the title and I thought it was going to be an ode to the joy of trying new things with your side projects, and spending a ton of time working something just because it is fun.
I guess the author and I have a different purpose for side projects. It sounds like the author is trying to actually start a business while still employed, while I am just trying to mess around and have fun.
> But for some reason, I seldom meet hackers who understand the fun in doing stuff from scratch.
Go visit your local hackerspace :).
That said, it's probably because in tech, there's much more vertical space for one to pick their definition of "from scratch" from. You go low, to assembly level, but you don't fab your own chips :).
I absolutely use my side projects as a vehicle to learn new things... though I always try to pursue ideas that at least appear to have a possible path toward monetization at some point (though I haven't had much success with turning much of a profit from any projects thus far) :)
For my current side project, which I've been working on for a year (and is probably a month or two away from being "beta-able"), I've learned Clojure, Elasticsearch, ES6, mithril.js, Ansible, and have expanded/refined my skills with RabbitMQ, Postgres (or SQL in general), and distributed architecture (not to mention getting to figure out how systemd service and timer configuration files work).
I've really been working hard on it because my current day job doesn't involve any programming, and by the time I get home in the evening I'm really jonesing for some code.
Have my side projects made me a better developer? Without a doubt. Am I any more employable? No, not really - I still haven't figured out how to improve my 'Cultural Fit' via side projects.
I always overengineer my side projects, because it's way more interesting that way. I do side projects for fun and to learn something, both of which happen more easily when the project is overengineered.
Agreed. Part of the fun of my side project is that it's well built. At work I have to deal with poor quality code practices from artificially tight timelines, contractors, etc. From one perspective it's very wasteful to have higher quality code for an app that generates $150/mo. vs multimillion dollar projects I manage at my day job, but I enjoy that $150, the code and customers more than what I get from my day job. I wouldn't if it was an MVP.
I tend to underengineer my side projects because the stuff that I touch at work is frequently much too overengineered to begin with (cough OpenStack cough).
When I first read the title, I had thought that this would be a humorous example of a delightfully over-engineered side project. It turns out that this is instead a list of 5 common mistakes of over-engineering with some commentary.
A major roadblock I find is that when you want to release your side project and start charging for it, I feel it becomes much harder to make changes after. This means you get caught in a loop of trying to polish it more and more before biting the bullet and releasing it.
If you change the interface a lot or how features work, you'll annoy users that don't like the change and you'll have to be extra careful not to introduce any bugs. If users can save data, the new version will have to work with the old data format. If you want to move features from a free version to the paid version you'll annoy users. So to me, releasing a MVP you charge for that might change a lot later might lead to more headaches than developing the idea further.
Are there ways around this?
Side projects you release for free are one thing but when you want to charge for them the amount of polish and attention to detail you have to pay to marketing jumps by an order of magnitude. People will tolerate bugs and rough edges for free projects but not for paid ones. I agree with the advice about not wasting time with CI, switching frameworks etc. though.
I'm currently working on a side-project, and I think I've figured out how to beat over-engineering and just launch a MVP which is actually a "minimum viable product". I'm building the product for my own use, and I'm just building the bare minimum that I actually need to complete the task. After that, I'm planning to bring on my first 10 customers and be available 24/7 to answer any questions and build the features that they need. But only if they're going to pay me money to use those features.
I've often fallen in the trap of "oh that would be cool", but this time I'm going to try to only build the necessary features that actually bring in customers, instead of wasting time on a bunch of useless stuff.
> Over-architecting infrastructure
Yep, I think it's a good idea to just put everything on Heroku until you're spending at least $100 per month. It's just so easy to get up and running.
I'm pretty sure if I wasn't over-engineering my side projects I wouldn't be technically competitive as I am now. So over-engineering is a very good strategy for learning and mastering any field. You can't really lose anything, either your project will succeed or you will acquire/strengthen your skills.
IDK, I over-engineer my side projects every single time. Like tweaking for days something which is so clearly premature optimization.
But sometimes I need that knowledge again for the next side project. And this over-engineering is often a way to simplify a reference architecture and be much faster at the end.
So for me over-engineering equals learning but is also a big part of the motivation.
I have similar experiences. I was working with an embedded platform that uses C exclusively and I decided I wanted to use C++. I ran into mountains of hurdles, had to stare at disassembled code, tweak stuff, build a class library, etc. I never actually finished the project I was working on but doing all that stuff was the fun part.
#3 hits home. I've been "working" on my side project for 3 years now with almost no progress because I'm constantly intrigued by new frameworks and BaaS. I spend all my time tinkering with them until I get bored with it all together. If I had just built the thing in Rails from the start I'd probably have a solid app under my belt by now.
I'm in the same position. I've built my own game engine from scratch and while it was a really good learning exercise I recently started looking at Unity and wow... So many things I've always wanted in my engine that I will get for free. I don't regret a thing though.
I've stopped the urge to create repos, buy domain names etc, and started just diving into Sketch to do mockups to flesh out ideas instead.
It gives me pause to think about features and to show people to get feedback. You can also use it as a sales tool to sell an app you are thinking of building without investing weeks and months.
Nowadays its so simple to do nice mockups its so easy to find stencils for SemanticUI, icons etc. Makes it a breeze to get an interface going and I have never been this productive in the 'idea' stage before.
If your goal is to make money on something, then yes I agree with the author's points. However the whole goal of side projects is to have fun with new languages, frameworks etc.
> A customer will not know or care if you are using Ruby, Go, PHP or any other language
Users will absolutely notice if your product crashes or breaks, which is more likely with some languages than others. The single most likely reason for me to stop using a nominally useful product is that it doesn't work reliably. I hate the fact that so much technology today is broken, partly because people have this cavalier attitude towards choosing technologies that help to keep things working.
I spent many years before I learned not to over-engineer side projects. I'm a c guy, mostly, so I don't have the stack issues (make and the standard c library and I'm g2g). Now I just start writing code to do the first basic feature, then refactor if needed to handle the second one, rinse and repeat. Once the project takes shape, it gets easier to predict what changes might be needed in the future, and what concessions I can make in the code for them now without slowing me down.
Working this way is much nicer for me. It's more organic. I end up with code that evolves, and takes shape of it's own accord. It makes me feel more creative. The best thing is, I have code that does things. It's not out in the wild, but when I come home from work I have a couple of projects that are coming along nicely to work on and play with.
Probably the best thing I have come to understand in building side projects is that you need to get it out in front of people as soon as possible. After that, just get feedback, use analytics, and iterate quickly to fix issues.
Even more important, go look for existing problems, whether it be in online forums or talking to people in person. If you making a unicorn detector and you spend several years with your nose down, its a rough lesson to learn that no one wants a unicorn detector.
Nice! I actually wanted to do something like for my own food obsessed self. Were you able to get good traction with your project? Or did you just do it for fun?
My biggest hurdle was friction between using instagram and my app. But may that's not what you're targeting.
If you are doing hobby side projects that are solely for learning, then this article's advice is (mostly) not intended for you. If your goal is to explore new tech, try new approaches, go down rabbit holes, then by all means, go for it!
The author's use of "side project" is referring to "I'm building something outside of my day job that I hope will turn into a business." IF your goal is to make money, then the author's advice is spot on. Get to MVP. Get users. Listen to them. Iterate. All the advice we hear that applies to startups, applies here.
I actually find that unless I do some amount of project management (i.e. use Trello or Pivotal Tracker), I easily get myself sidetracked. I'll use other tools for longer running timelines, but splitting things out into small stories actually helps me stay focused without getting bogged down in considering the implications to every design decision I make.
For a lot of software, making the best decision today without worrying about tomorrow, while still following tried and true engineering practices (i.e. test driving, etc), leads to a project that continues to progress.
I write a SVG path string optimizer for fun on the side. So far I wrote a custom decimal floating point type because of the 0.1+0.2 problem which I threw out afterwards, a zero allocation filter for printing rounded f32 without leading zeroes (.1 instead of 0.1) and dropping of unneeded spaces (.1.1 is valid and equal to 0.1 0.1), which I now want to replace with a custom float formatting routine.
All I did I did for fun and maybe learning something. So I think over-engineering side projects is fine.
I can tell this article was written by a very wise man who wasted time on lots of projects or saw a lot of other people waste time on projects. He's described behaviors I have dabbled in many, many times.
In fact, I have a side project currently where I just needed to fabricate a tiny fixture for a physical device. I caught myself over-engineering the prototype, went to sleep, woke up, saw this article and came to my senses. Consider this article bookmarked!
We just did a talk on this exact thing at Laracon last month. And raised many of the same points. We however also talked a lot about the business side of things.
> "Develop your project. Then, put it on the bare minimum architecture that it can run on (be that a 512MB instance from DigitalOcean or a medium instance on AWS). As you get more users, monitor, and modify the infrastructure appropriately to account for load and redundancy."
That's a pretty ignorant and potentially short sighted statement. Look at what happened to Instagram. They almost lost it all because they hadn't planned for the possibility of handling scale quickly
Sometimes you simply don't have the luxury of monitoring and modifying to account for load and redundancy. Sometimes you're monitoring says "hey traffic" shortly followed by "OMG YOU'RE SCREWED" because you can't possibly think up and implement an appropriate scalable architecture, and modify the codebase to handle it and and and while your servers are melting and everyone is yelling at you.
Yes, this is very unlikely to happen, but when it does it can cost you your business if you're not prepared. Obviously apps with no networking and no viral component are far less likely to have this problem. Got a macApp that creates icon sets for XCode? Yeah, you can ignore this scaling issue. Writing the next Instagram? You're a fool if you don't have plans in place, in advance of launch, for . handling the onslaught of people that will make your business successful.
> Look at what happened to Instagram. They almost lost it all because they hadn't planned for the possibility of handling scale quickly
That's a problem that essentially nobody has. You're talking such a very small percentage of start-ups or projects, as to be meaningless as a reference point.
Simply put, that isn't a problem you're going to have.
If by some small miracle you have that kind of scaling problem, you can very likely fix it thanks to the fact that you've struck gold.
The number of homeruns that failed because they couldn't scale, is few and far between. There are so few of them, that you're naming one that became a huge success instead of listing some huge prominent failure. First, start with the fact that there are so few of these Instagram rapid hyper-scaling scenarios to begin with, then narrow it down to the percentage that failed due to poor scaling, then narrow it down to the number of those who failed due to poor scaling in the modern cloud era (ie not Friendster from 2004-05). You're very likely down to close to zero examples.
It's interesting that you choose Instagram - a highly successful company/product - as an example to warn people against.
It may be harder to come up with examples of people who failed, for the obvious reason - they failed and are therefore not well known, but I really don't find the statement that Instagram "nearly lost it all" very compelling. Did they? Really? If they had had even more problems scaling, would they really have lost users? Look at Twitter, they took years to resolve issues.
I actually think the importance of stability and uptime are overblown in the early stages. If I think about the times downtime has significantly impacted my view of a company or product, it's when large established services I'd come to rely on professionally went down. You need to establish a reputation for reliability before most people expect it of you. That's not to say it's not important, but you need to look at context. Noone relies on Instagram minute by minute for their professional work, so the priorities are different. UX, marketing, network effects, etc. are going to trump stability.
The article has good points. But the term "side project" implies working on something that isn't part of your job. I think looking at your project from that perspective can also be a step in the wrong direction---if you enjoy what you do, work is just one such application of your knowledge and ability. Projects you do outside of work are another, and may even be more important to you than work.
A number of commenters point out that many side-projects to them are to explore new concepts/libraries, or as a means of learning. If it's for the sake of learning for your job, then yes, it'd be a side project.
But if you're looking to take a project to completion (the author talks about products; my projects aren't), I find that using it as a learning environment extremely detrimental. Yes, you do learn things over the course of development---through research, struggle, and growth. But if too much of your time is spent on things you don't have a good foundation on, you may burn out too early, wind up frustrated, and wind up with a mess that leads to refactoring. Use a foundation that you have experience with and know well, and learn the parts that are necessary.
(I do distinguish, though, a research project with a project for playful learning. The former is much more formal and disciplined. But you can still burn out early.)
I thoroughly enjoy over-engineering side projects, and writing about them (linked below). The trick for me is in managing expectations. It's completely possible for a learning project to evolve in to something widely used, just don't go in with that expectation.
He so aggressively avoids overengineering that he goes the other way - if you make ANY effort to properly engineer something he gets really annoyed and says time is being wasted.
It's almost like he feels software MUST be engineered to NOT scale in order to be an "MVP".
It's strange because for me, thinking about how something is built means that it can be built right the first time, without extra effort, just by thinking it through.
This is my attitude to the React PATENTS file - yeah okay I probably won't be big enough to worry about it, but just in case I am, why take the risk with React?
I have a personal Jenkins that's been running for years. Adding new jobs isn't yak shaving; it's a two minute task that gives me CD.
Same with my AWS "framework". And my CSS "framework" and my React component superclass and lots of other stuff I carry from one side project to another.
Some would (and have) accused me of over engineering; but I can discover my project is pointless very quickly.
Man, I think this article is dead on, except it doesn't identify the original reason, for me at least. In a normal environment, there are constraints on time, money, goals, resources, etc. Suddenly, when there are no constraints, a side project, you try to do everything perfectly, which leads to the issues in this article.
>You’re so hyped for this project and confident that it will succeed that you start thinking about the future: “how will I scale my application for the millions of users it will have?”
That's one of my side project vices!
Every time I write a new login database... I wonder if I should make the ID a MEDIUMINT or INT.
Why do you need a deployment system if you have nothing to deploy? Build the project locally, then use continuous delivery when you have something to deliver.
"Build the project locally" implies some kind of pipeline system, even if it's not "deployment" or "delivery".
Triggering that pipeline from a commit trigger instead of manually just means copying the build command into a config file. It pays for itself in the first hour.
Agreed. The best part of side projects is that I can take as long as I want to build it to my satisfaction, rather than to the minimal acceptable state for business purposes.
I really like over-engineering my side projects. I have three frameworks, one in PHP, one in JS, and one in PHP/JS (a bit abandoned) that no one really uses. But you know what? Because no one uses them, I'm not burdened by any opinion except mine. When I change something, I don't read death threats because someone's solution stopped working. And since I don't have any deadlines, I can spend as much time as I want to get the exact solution as I planned. I don't need to compromise. That's quite opposite thing to what I do at worktime and get paid for.
I feel like there should be a solution that covers 99% of online projects. out of the box it scales, deploys, upgrades, does continuous integration, testing, logging, supports all major logins, has an admin console, metric console, etc... maybe it starts with the traditional Todo app and you just start modding.
I find it incredibly frustrating that much of this isn't a solved problem that I can then just insert my side project into.
hacking is fun until I have to maintain it and then I start wishing for all of those features instead of having to spend days or weeks implementing each of them from scratch
I can only speak for myself here but I feel that belonging too strongly in either camp is a bad thing.
Ideally, you'd want to understand both worlds.
As such, I feel that the projects I work on in my own time and the crazy things I do in them help me to better understand what kind of things really do need to happen when working under tight deadlines in the workplace.
I.e. It's easier to produce an effective product quickly if you know from personal experience exactly which approaches or things really are essential.
Also, note I haven't read the actual article. I'm replying mostly in response to the comments :)
> I know: I’ll create a triple zoned redundant architecture with pub/sub database replication, a 32 node Kubernetes cluster and private networking across all regions – that way, I can handle anything!
Well, maybe not a 32 node k8s cluster. However, spinning up K8s clusters is so easy now, that it costs almost no time to deploy stuff there. And so many things are taken care for you: configuration, secret storage, networking, versioning, keeping services up, etc.
If on AWS, there's Kops. If on GCP, that's even easier, a couple of clicks and you have a cluster.
One of the ways I learn is via code review. In a side project I'm often doing new things that I am not already an expert in. Any ideas on how to get feedback that things are ok vs not?
Side projects are meant to be challenging because they are a learning opportunity so it's possible that they end up over-engineered sometimes.
You can always refactor the parts that are over-engineered. There's a very fine line between highly-engineered and over-engineered.
One of my favourite projects is Kubernetes; some might consider it over-engineered but it's extremely powerful.
"Perfection is achieved not when there is nothing more to add, but when there is nothing left to take away"
I've always taken that quote to apply only when you have been taking things away; not when you've ended up with a colossal complex monstrosity where removal of any single strand would cause total collapse.
As in everything: figure out what you are trying to get out of your side project.
If you want something to generate a nice supplementary income stream, or practice for starting a company when you get "that great idea" then this article is very good advice.
On the other hand, if you are trying to learn a new technology stack, develop experience in building scalable architectures or even beefing up your project management skills then this is terrible advice.
So, the real art is figuring out what you want out of your side project.
If I don't overengineer my side projects -- if I stick to what's simple and practical -- how am I going to gain marketable skills with the modern frameworks?
Set your goals high for the project instead, achieving your goals can never be considered over-engineering. Don't let the scope creep into "oh I should totally spend 30 hours on this form design"
As an amateur coder/noob I'm happy I'm not unique in this idea. I constantly start projects in Flask/Python and just try to make them as complex as possible (to the limit of my current capabilities). When I'm at work (im an Incident Manager for a NOC/SOC) I can somewhat hold my own in conversations about infrastructure issues. I'm also hoping to go into the security field soon (OSCP cert).
There's probably another reason hidden in the first sentence, which is that there's a transition from building the product (engineering) to shipping the product (marketing, customer development, sales) that requires different skill sets and experiences. Was just discussing this with a software engineer friend of mine who realizes that he "has no clue how to market my product the right ways."
When I do side projects it's normally because I want to explore something which is too risky for someone to do (or it would most likely have been done). In my experience that normally requires some sort of overengineering because your problem and solution isn't common.
A few of them turned into great side businesses but most of the times it help me explore technologies and projects while still solving real problems.
Just like in real life, the art of engineering is iteration. So I always get quick and dirty results with my side projects, and iterate if it's worth iterating. But my side projects are just for myself, not for others, so it's things like writing a multi-thread stock strategy backtester, a dashboard for my IoT devices, etc. It's never that I think will make money.
This makes a lot of sense -- and makes me feel like I'm guilty of over engineering a non-side project I started working on at work ...
I'm always looking to make the code as easy to change as possible -- the idea being that change is a precondition for any form of improvement ... Reading this article makes me wonder if I've turned that pursuit into an anti-goal
I'm going to try another approach with my next project - not sure if it's a good idea, but I'm going with the Minimum Unviable Product. As soon as the project is bootstrapped and compilable, I'm going to release a version.
I figure this way at least I can say I released something, even if it's not that feature rich.
I recently was thinking about 1).
At my current company, I got to know agile methodologies for the first time, I liked it and started planning my side project, but it instantly started to felt like work! I just realized if I write the parts whatever interests me the most, I will be more inclined to finish it!
I think https://qbix.com/platform is the epitome of this. It came out of me wanting to abstract away once and for all everything that made developing social networks hard. And I just kept going lol.
If the side project is for learning purposes then taking time to explore different approaches is OK.
If your side project goal is to build a product, then you should approach it as a more traditional project, budgeting your time, and focusing on time-sensitive tradeoffs, etc.
Every enterprise client I have worked with has asked about our tech stack. I will agree that your groupon clone users do not care about your stack, but respond to an enterprise SaaS RFP and stack, security, DR will all be discussed.
The definition of a side project is not clear. Is it just for fun/learning about new tech, and can be thrown away? Or is it a serious endeavor, and supposed to be the next big thing? These require completely different mindsets.
I have been successfully applying Jonathan Blow principles into my side projects. It's been working great. If you are not familiar I highly suggest you to listen to a few of his presentations about software development.
I think deferring the CD part until the end can be really painful when you are "ready". I think it's a worth upfront investment even if you have few users, will make your life much easier and less stressful.
All those rules applies to our new non-side projects as well.
I can't even count how many times I seen projects with awesome infrastructure and basically zero functionality after depleting whole budget (and usually some more).
on point! cannot agree more - I have been down this road before with my last project - Jabid.com. We tried to boil the ocean even though that was not the goal of the project. This time around, for https://veganfutura.com, I am taking a complete opposite approach, that closely aligns with the list you have provided. Keep it simple, focus on the problem you are trying to solve and the user. Marketing and User acquisition > Tech initially.
For the record (it's not mentioned but a lot of time it happens): security (e.g. password storage) should not be something left for later. Do it or use a framework that does it for you.
In case I want to actually deliver a side-project, I usually write some basic tests. As such I can focus more on adding new (fun) features instead of hunting down regressions.
If you want to release your project and maybe turn it into a start up, then these are all great points. However, many times the purpose of a side project is to just learn new things.
I'm in the opposite camp - personal side-projects are where you can safely implement the MVP and/or ruthlessly chop things/ideas that never* get used without people harassing you about bells and whistles.
I think the majority of engineers only do side projects to learn things they couldn't at work or explore architectures they wouldn't be able to in a work environment.
heh, most of this stuff is extra wastes of time past the very first hurdle that you should be working on: make something that works and is useful to people.
sure it would be great to have scaling to 10,000 users already sorted by the time you get there but odds are your project will never get there, when you get to 5k users you need to start worrying about that
if you're just doing it to learn or for fun then obviously this article or my comment don't apply
guilty of all of the above. my thinking is always "i'll kill 2 birds with one stone, side project and learning the latest framework!" and yah.. after struggling with the framework for 2 weeks the project loses steam and then just thinking about a stupid challenge i have to solve BEFORE getting back to my project is crippling.
it does have a nice side benefit however of actually learning a new framework :)
Setting up a CI on AWS using git, CodePipeline, Code Build, and elastic beanstalk takes no more than a few hours. It's worth the few hours. Otherwise you're manually pushing builds out which wastes more time in the long. In the beginning your CI should be nothing more than ==> I check in code, it deploys to environment. Later on you can worry about triggering tests and preventing builds for deploying if they fail. Also if you use elastic beanstalk, you can scale as necessarily with a few configuration changes.
For my side projects, I actually want to go down rabbit holes. I will try the same thing 7 different ways and benchmark the hell out of them. I'll read all the documentation and every blog post I can find. Learn a new language just to see if its features make things ever so slightly easier. Learn a new framework, write a new framework, throw it away and write a new one.
The point isn't to ship code, it's to learn. Honestly I don't care if it ever ships. And even when I do release one, I don't care if nobody uses it except me.