
The art of over-engineering your side projects - elsyms
https://elsyms.com/the-art-of-over-engineering-your-side-projects/
======
czep
The whole point of a side project is to over-engineer it! The thought of
holding myself to deadlines and driving myself to "ship" is totally
antithetical to the entire purpose of a hobby project. It's a welcome relief
from the 9-to-5 constant necessity to cut corners and finish things without
ever properly understanding them because there's no time.

For my side projects, I actually want to go down rabbit holes. I will try the
same thing 7 different ways and benchmark the hell out of them. I'll read all
the documentation and every blog post I can find. Learn a new language just to
see if its features make things ever so slightly easier. Learn a new
framework, write a new framework, throw it away and write a new one.

The point isn't to ship code, it's to learn. Honestly I don't care if it ever
ships. And even when I do release one, I don't care if nobody uses it except
me.

~~~
johnfn
HN is made up of two groups in ever constant warfare, the yin and yang, as it
were- the group who thinks that programming should be fun, experimental and
enjoyable, and the group that thinks that programming is a means to the end of
getting a product to market and nothing more.

Whenever a post from one group (such as the article) comes out, a post from
the other (such as OP) will quickly appear. And everyone loves it :)

For what it's worth, I think that both perspectives are valid. It depends only
on your goals.

~~~
iamnafets
This is mostly off-topic, but being able to identify or root out the value
mismatches between people in a discussion (as you've done) seems like an
essential ingredient to productive discussion.

I've noticed that people who can do this are disproportionately better at
achieving their goals than those who can't, especially in engineering
organizations. Exceptions of course for zealots and revolutionaries.

~~~
chris_st
Absolutely. This even works at a somewhat lower level... I remember hearing an
argument between two cow-orkers, and realizing that each was thinking
_different_ (but similar!) things, but assuming the other was thinking the
_same_ thing. Angriness ensued...

~~~
Chris2048
> Angriness ensued...

This usually only happens when one begins to make personal insults, i.e. "only
a poor programmer would think that!"..

------
caleblloyd
Lots of times I use side projects as a way to explore new stacks, deployment
approaches, etc. The goal of the project is not to deliver anything, it's to
learn so that when I go to do my next "real" project at work, I know what
works well and what doesn't.

~~~
alexandercrohde
Exactly. When I see a coworker start irrationally focusing on a new technology
at work the first thing I think is "This person probably has no side projects
to play with"

~~~
markcerqueira
> "This person probably has no side projects to play with"

Exploring new technologies that could be beneficial (or harmful) to the
company shouldn't be a side project. Perhaps we can shift the thinking from
"this person doesn't have time for side projects" to "how can we make
exploring new technologies a part of the job?"

~~~
WalterSear
How about making wednesday mornings, a 4hr 'personal project' time? Make it a
requirement that everyone has to work with tools at the edge of their
skillset, that is also relevant to work, but let them choose what, why, how.
They could work on a personal project, or experiment with a new library or
tool they would like to incorporate.

Start the day with a "not really a standup", where people can quickly explain
what they want to accomplish in the next four hours.

Once a month, you spend a couple of hours of the time demoing the work you
have done for each other, explaining the technology and what you have learned,
and then discussing and evaluating our individual technical weaknesses and
deciding what to do next. This rest of this monthly meeting could also be
spent improving social coding skills - code and commit hygiene, team processes
etc.

Not only would this keep people technically sharp but it keeps them from
playing with toys in production and breaks up the work week, while still
fulfilling company goals.

~~~
jeffdavis
If you have to start giving reports, it kills creativity. The point is to do
something everyone else may think is irrational, but you feel would lead
somewhere interesting.

------
quadcore
_“I know: I’ll create a triple zoned redundant architecture with pub /sub
database replication, a 32 node Kubernetes cluster and private networking
across all regions – that way, I can handle anything!”_

The good news is, once you've done that, you can get a pretty decent job.

 _The majority of software engineers seem to be under the illusion that
potential customers care about the stack they are running._

This is not entirely true. The technology do surface in terms of user
experience. It does in the details: app start, ui responsiveness, etc. I'm
writing a game using go right now, and the end user can tell it's different
because the user's cpu seems more powerful.

All in all, I think those are good mistakes, not bad ones. I wish more
engineers could spot over-engineering, and they don't until they did the
mistake a couple times. Side projects are, I believe, how one really learn to
write programs. I think the right advice is: do whatever you want, do all the
mistakes. How can you write a successful side project anyway if you never
learn how to write a program? Also, the worse type of programmers are the ones
telling you about over-engineering because they read about it in a blog. Code
more, read less, I guess should be a good motto.

~~~
kerkeslager
Potential customers do care about the stack you're running. They care if they
have to download Java or Flash. They care if they have to be connected to the
internet or can use your product offline. They care if your program starts
quickly. They care if your program runs quickly once started. They care if
your program has bugs (let's not pretend different languages aren't more or
less error-prone). They care if your program is cross-platform. They care if
your UI is consistent with their platform. They care if your product releases
new features quickly.

All of these are stack-related.

~~~
scott_karana
Those are all client usability concerns, and none of them are stack intrinsic.

~~~
mark-r
I think the point was that user doesn't care about your stack, but they care
about the _consequences_ of the stack you've chosen. It's a subtle but
important distinction.

~~~
scott_karana
Of course. My point was that it's fairly common to write slow/unsuitable
applications _in spite_ of a stack's strengths, while stack limitations (or
strengths) are generally overstated, depending on trends.

Proper engineering and design are required no matter what you choose.

No silver bullets ;-)

~~~
kerkeslager
There are no silver bullets, but there are wooden bullets that are totally
ineffective.

------
xwvvvvwx
For me it's very important that I don't discuss projects with friends / family
/ colleagues etc. until I really have something to show.

I have always found it very easy to talk all the energy out of a good idea
before I've even got any real work done.

~~~
hectorlorenzo
That's a thing. Expressing in public your desire to achieve something makes
you less likely to deliver.

[http://www.psych.nyu.edu/gollwitzer/09_Gollwitzer_Sheeran_Se...](http://www.psych.nyu.edu/gollwitzer/09_Gollwitzer_Sheeran_Seifert_Michalski_When_Intentions_.pdf)

~~~
CuriouslyC
The weird thing is the same situation works in reverse as well.

Publicly declaring your intention to quit a habit or lose weight has also been
shown to increase success in some instances.

I think it depends on whether people reward you with positive regard for
stating an intention to do something, and whether you look foolish if you
don't do the thing you stated. When you can get the reward just for the
statement and there is no consequences for failure, pre-stating your intention
seems harmful. On the other hand, if you are not rewarded for pre-statement,
and failure is embarrassing, pre-stating your intention seems helpful.

~~~
juiyout
I have observed this vastly "bipolar" results amongst my friends as well. Some
take public declaration to heart and they ram through their goals. However I
quickly lose interest of the project after I told everybody. I would go extra
length to surprise and dish out cool stuff more often I care to finish side
project only because I said I will do it.

I somehow concluded that I don't care about approvals on my 'integrity' yet I
highly value my 'creativity' viewed by others.

------
hamvocke
Good points in there. I have a little trouble with "mistake #5" though.

I found that worrying about deployment automation early on is immensely
valuable, especially for side projects. I don't always find time or interest
to keep working on side projects after a regular day of work. Quite often
there are weeks or even months where I abandon a side project only to come
back at a later point when motivation is back or things have settled down and
leave more time for my side projects. If I didn't take the time to automate
testing and deployments (which makes a lot of what continuous delivery is
about) I find myself struggling with getting things up and running and become
frustrated before I even get started.

Then there's only one way to make sure that my test and deployment automation
keeps working: running it regularly, e.g. in a pipeline.

So, yes, don't over optimise your deployment pipeline early on. But having one
in place early can pay off really soon -- and maybe even keep your side
project alive.

~~~
bitwisebob
Agreed. Having a deployment pipeline in place dramatically reduces the
friction of returning to an old project.

I have a back-burner project that touch very infrequently, but I can jump in,
make a quick change, and have it deployed to "production" in just a few
seconds.

------
cyberferret
At risk of a tar and feathering by the 'lean' crowd, I am going to declare
that I don't do MVP's at all, and never really have, with _any_ of my
projects.

Why? Well, because almost all my projects are designed for businesses and
enterprises, and with that audience - things better work well and they better
work first time during the demo or you don't stand a chance of getting a
customer.

Perhaps if I was writing a social media app for sharing cat pictures, then
yeah, I would slap together an MVP and show it to cat lovers. After all - what
is the worst that can happen if your cute cat picture doesn't upload? Just try
it again or delete the app and walk away, and there are no consequences.

But what if my HR app doesn't send that crucial reminder email to a department
manager that one of your electrical tradesmen's license expires tomorrow, and
you end up sending him out to a job next week and he makes a mistake that ends
up getting your company sued and eventually shut down, putting hundreds of
others out of work?

I am sure as sh*t going to ensure that my reminder email system is absolutely
over engineered to ensure reliable email delivery and tracking. You don't get
ANY chances to blow that kind of stuff with a business of enterprise audience.

~~~
jamesmishra
I fully agree with this. Big enterprise MVPs are often about cutting scope --
not cutting corners.

Of course, by the very nature of enterprise customers, sometimes there isn't a
whole lot of scope that you can cut.

------
Mojah
Related to this, I've been following a practice I call "Release Notes Driven
Development", which is _perfect_ for side projects. Don't worry too much about
technical debt (although, keep it under control), but focus on the little time
you have and how to make that as efficient as possible.

I wrote about it here; [https://ma.ttias.be/release-notes-driven-development-
rndd/](https://ma.ttias.be/release-notes-driven-development-rndd/)

~~~
dispo001
Sounds good. Early on, a quick list of bragables should be able to tell you
how awesome the idea is. Have new bragables for the next version. If its not
awesome at some stage (and the bugs are fixed) you are probably done?

It reminds me of many release notes describing a ton of work without any
_woah_.

------
beeforpork
The best thing about side project is that I may be wonderfully inefficient, so
I can do it the way I want and enjoy the way. Even mentioning the word
'deadline' or 'planing' or 'efficient' or 'customer' in the same sentence as
'side project' is contradictory to me.

E.g., I love to program in assembler. Preferably 8 bit SoCs like AVR. The
point is not to finish the project fast or even at all by using a more
efficient language that may have libraries for everything, but to enjoy
programming in 8 bit assembler. Reading the absurdly detailed manuals.
Comparing chips at register level. Obviously, I need to write my own
libraries/frameworks, because, I want to do that myself, in assembler.

Even if not using assembly, I may still want to write the library myself in
order to, well, do that.

Wood workers often seem to have similar ideas about side projects, like when
they build a project without power tools, without glue, from old scrap wood,
etc., just because. But for some reason, I seldom meet hackers who understand
the fun in doing stuff from scratch.

~~~
cortesoft
Yeah, I read the title and I thought it was going to be an ode to the joy of
trying new things with your side projects, and spending a ton of time working
something just because it is fun.

I guess the author and I have a different purpose for side projects. It sounds
like the author is trying to actually start a business while still employed,
while I am just trying to mess around and have fun.

------
jcadam
I absolutely use my side projects as a vehicle to learn new things... though I
always try to pursue ideas that at least appear to have a possible path toward
monetization at some point (though I haven't had much success with turning
much of a profit from any projects thus far) :)

For my current side project, which I've been working on for a year (and is
probably a month or two away from being "beta-able"), I've learned Clojure,
Elasticsearch, ES6, mithril.js, Ansible, and have expanded/refined my skills
with RabbitMQ, Postgres (or SQL in general), and distributed architecture (not
to mention getting to figure out how systemd service and timer configuration
files work).

I've really been working hard on it because my current day job doesn't involve
any programming, and by the time I get home in the evening I'm really jonesing
for some code.

Have my side projects made me a better developer? Without a doubt. Am I any
more employable? No, not really - I still haven't figured out how to improve
my 'Cultural Fit' via side projects.

~~~
rochak
Could you tell me how you go about knowing what all you might be needing.

------
GhostVII
I always overengineer my side projects, because it's way more interesting that
way. I do side projects for fun and to learn something, both of which happen
more easily when the project is overengineered.

~~~
kentt
Agreed. Part of the fun of my side project is that it's well built. At work I
have to deal with poor quality code practices from artificially tight
timelines, contractors, etc. From one perspective it's very wasteful to have
higher quality code for an app that generates $150/mo. vs multimillion dollar
projects I manage at my day job, but I enjoy that $150, the code and customers
more than what I get from my day job. I wouldn't if it was an MVP.

------
mixedmath
When I first read the title, I had thought that this would be a humorous
example of a delightfully over-engineered side project. It turns out that this
is instead a list of 5 common mistakes of over-engineering with some
commentary.

------
seanwilson
A major roadblock I find is that when you want to release your side project
and start charging for it, I feel it becomes much harder to make changes
after. This means you get caught in a loop of trying to polish it more and
more before biting the bullet and releasing it.

If you change the interface a lot or how features work, you'll annoy users
that don't like the change and you'll have to be extra careful not to
introduce any bugs. If users can save data, the new version will have to work
with the old data format. If you want to move features from a free version to
the paid version you'll annoy users. So to me, releasing a MVP you charge for
that might change a lot later might lead to more headaches than developing the
idea further.

Are there ways around this?

Side projects you release for free are one thing but when you want to charge
for them the amount of polish and attention to detail you have to pay to
marketing jumps by an order of magnitude. People will tolerate bugs and rough
edges for free projects but not for paid ones. I agree with the advice about
not wasting time with CI, switching frameworks etc. though.

------
nathan_f77
I'm currently working on a side-project, and I think I've figured out how to
beat over-engineering and just launch a MVP which is actually a "minimum
viable product". I'm building the product for my own use, and I'm just
building the bare minimum that I actually need to complete the task. After
that, I'm planning to bring on my first 10 customers and be available 24/7 to
answer any questions and build the features that they need. But only if
they're going to pay me money to use those features.

I've often fallen in the trap of "oh that would be cool", but this time I'm
going to try to only build the necessary features that actually bring in
customers, instead of wasting time on a bunch of useless stuff.

> Over-architecting infrastructure

Yep, I think it's a good idea to just put everything on Heroku until you're
spending at least $100 per month. It's just so easy to get up and running.

------
bojanvidanovic
I'm pretty sure if I wasn't over-engineering my side projects I wouldn't be
technically competitive as I am now. So over-engineering is a very good
strategy for learning and mastering any field. You can't really lose anything,
either your project will succeed or you will acquire/strengthen your skills.

------
thinbeige
IDK, I over-engineer my side projects _every single time._ Like tweaking for
days something which is so clearly premature optimization.

But sometimes I need that knowledge again for the next side project. And this
over-engineering is often a way to simplify a reference architecture and be
much faster at the end.

So for me over-engineering equals learning but is also a big part of the
motivation.

~~~
wvenable
I have similar experiences. I was working with an embedded platform that uses
C exclusively and I decided I wanted to use C++. I ran into mountains of
hurdles, had to stare at disassembled code, tweak stuff, build a class
library, etc. I never actually finished the project I was working on but doing
all that stuff was the fun part.

------
pgm8705
#3 hits home. I've been "working" on my side project for 3 years now with
almost no progress because I'm constantly intrigued by new frameworks and
BaaS. I spend all my time tinkering with them until I get bored with it all
together. If I had just built the thing in Rails from the start I'd probably
have a solid app under my belt by now.

~~~
0xfeba
Yeah I've been working on a game for 5+ years that went thru C++/DirectX, then
C# and Managed DirectX, and now Java and LibGDX.

Thinking about using Unity now...

~~~
Kiro
I'm in the same position. I've built my own game engine from scratch and while
it was a really good learning exercise I recently started looking at Unity and
wow... So many things I've always wanted in my engine that I will get for
free. I don't regret a thing though.

------
dev360
I've stopped the urge to create repos, buy domain names etc, and started just
diving into Sketch to do mockups to flesh out ideas instead.

It gives me pause to think about features and to show people to get feedback.
You can also use it as a sales tool to sell an app you are thinking of
building without investing weeks and months.

Nowadays its so simple to do nice mockups its so easy to find stencils for
SemanticUI, icons etc. Makes it a breeze to get an interface going and I have
never been this productive in the 'idea' stage before.

------
ropman76
If your goal is to make money on something, then yes I agree with the author's
points. However the whole goal of side projects is to have fun with new
languages, frameworks etc.

------
wyager
> A customer will not know or care if you are using Ruby, Go, PHP or any other
> language

Users will absolutely notice if your product crashes or breaks, which is more
likely with some languages than others. The single most likely reason for me
to stop using a nominally useful product is that it doesn't work reliably. I
_hate_ the fact that so much technology today is broken, partly because people
have this cavalier attitude towards choosing technologies that help to keep
things working.

------
godshatter
I spent many years before I learned not to over-engineer side projects. I'm a
c guy, mostly, so I don't have the stack issues (make and the standard c
library and I'm g2g). Now I just start writing code to do the first basic
feature, then refactor if needed to handle the second one, rinse and repeat.
Once the project takes shape, it gets easier to predict what changes might be
needed in the future, and what concessions I can make in the code for them now
without slowing me down.

Working this way is much nicer for me. It's more organic. I end up with code
that evolves, and takes shape of it's own accord. It makes me feel more
creative. The best thing is, I have code that does things. It's not out in the
wild, but when I come home from work I have a couple of projects that are
coming along nicely to work on and play with.

------
yigitozkavci
I think side projects and hobby projects need to be thought of separately.

In the context of hobby projects, over-engineering can even serve purpose of
learning new things you otherwise could not.

For side projects, or startup candidates, however, you just ship it

------
dtzur
"Continuously Delivering Nothing" is a mission statement for several companies
I know.

~~~
bpicolo
Tar it up, cat it to devnull

~~~
cryptos
It is important to compress it with bzip2 or your even better custom algorithm
before finally archiving it in /dev/null

------
CJefferson
I recently had this with a html5 project. Thought it would be nice to use es6
and transpile, then disappeared down a hole of gulp and webpack.

Eventually backed up and used Make to run Babel and then cat the files
together.

------
tmaly
I have committed almost all of these mistakes with my food side project
[https://bestfoodnearme.com](https://bestfoodnearme.com) .

Probably the best thing I have come to understand in building side projects is
that you need to get it out in front of people as soon as possible. After
that, just get feedback, use analytics, and iterate quickly to fix issues.

Even more important, go look for existing problems, whether it be in online
forums or talking to people in person. If you making a unicorn detector and
you spend several years with your nose down, its a rough lesson to learn that
no one wants a unicorn detector.

~~~
kamikaz1k
Nice! I actually wanted to do something like for my own food obsessed self.
Were you able to get good traction with your project? Or did you just do it
for fun?

My biggest hurdle was friction between using instagram and my app. But may
that's not what you're targeting.

Either way, I was just curious...

~~~
tmaly
I am still iterating to improve it, but I was doing it for fun and to scratch
my own itch.

There is not much traction, the 1% internet rule is real in terms of the
number of people that contribute verse those that consume content.

------
jgable
If you are doing hobby side projects that are solely for learning, then this
article's advice is (mostly) not intended for you. If your goal is to explore
new tech, try new approaches, go down rabbit holes, then by all means, go for
it!

The author's use of "side project" is referring to "I'm building something
outside of my day job that I hope will turn into a business." IF your goal is
to make money, then the author's advice is spot on. Get to MVP. Get users.
Listen to them. Iterate. All the advice we hear that applies to startups,
applies here.

------
Spartan-S63
I actually find that unless I do some amount of project management (i.e. use
Trello or Pivotal Tracker), I easily get myself sidetracked. I'll use other
tools for longer running timelines, but splitting things out into small
stories actually helps me stay focused without getting bogged down in
considering the implications to every design decision I make.

For a lot of software, making the best decision today without worrying about
tomorrow, while still following tried and true engineering practices (i.e.
test driving, etc), leads to a project that continues to progress.

------
legulere
I write a SVG path string optimizer for fun on the side. So far I wrote a
custom decimal floating point type because of the 0.1+0.2 problem which I
threw out afterwards, a zero allocation filter for printing rounded f32
without leading zeroes (.1 instead of 0.1) and dropping of unneeded spaces
(.1.1 is valid and equal to 0.1 0.1), which I now want to replace with a
custom float formatting routine.

All I did I did for fun and maybe learning something. So I think over-
engineering side projects is fine.

------
shortoncash
I can tell this article was written by a very wise man who wasted time on lots
of projects or saw a lot of other people waste time on projects. He's
described behaviors I have dabbled in many, many times.

In fact, I have a side project currently where I just needed to fabricate a
tiny fixture for a physical device. I caught myself over-engineering the
prototype, went to sleep, woke up, saw this article and came to my senses.
Consider this article bookmarked!

------
thecodemonkey
We just did a talk on this exact thing at Laracon last month. And raised many
of the same points. We however also talked a lot about the business side of
things.

For those who are interested: Video: [https://streamacon.com/video/laracon-
us-2017/day-2-mathias-a...](https://streamacon.com/video/laracon-
us-2017/day-2-mathias-and-michele-hansen) Slides:
[https://speakerdeck.com/minicodemonkey/launching-and-
scaling...](https://speakerdeck.com/minicodemonkey/launching-and-scaling-a-
side-project)

------
masukomi
> "Develop your project. Then, put it on the bare minimum architecture that it
> can run on (be that a 512MB instance from DigitalOcean or a medium instance
> on AWS). As you get more users, monitor, and modify the infrastructure
> appropriately to account for load and redundancy."

That's a pretty ignorant and potentially short sighted statement. Look at what
happened to Instagram. They almost lost it all because they hadn't planned for
the possibility of handling scale quickly

Sometimes you simply don't have the luxury of monitoring and modifying to
account for load and redundancy. Sometimes you're monitoring says "hey
traffic" shortly followed by "OMG YOU'RE SCREWED" because you can't possibly
think up and implement an appropriate scalable architecture, and modify the
codebase to handle it and and and while your servers are melting and everyone
is yelling at you.

Yes, this is very unlikely to happen, but when it does it can cost you your
business if you're not prepared. Obviously apps with no networking and no
viral component are far less likely to have this problem. Got a macApp that
creates icon sets for XCode? Yeah, you can ignore this scaling issue. Writing
the next Instagram? You're a fool if you don't have plans in place, in advance
of launch, for . handling the onslaught of people that will make your business
successful.

~~~
adventured
> Look at what happened to Instagram. They almost lost it all because they
> hadn't planned for the possibility of handling scale quickly

That's a problem that essentially nobody has. You're talking such a very small
percentage of start-ups or projects, as to be meaningless as a reference
point.

Simply put, that isn't a problem you're going to have.

If by some small miracle you have that kind of scaling problem, you can very
likely fix it thanks to the fact that you've struck gold.

The number of homeruns that failed because they couldn't scale, is few and far
between. There are so few of them, that you're naming one that became a huge
success instead of listing some huge prominent failure. First, start with the
fact that there are so few of these Instagram rapid hyper-scaling scenarios to
begin with, then narrow it down to the percentage that failed due to poor
scaling, then narrow it down to the number of those who failed due to poor
scaling in the modern cloud era (ie not Friendster from 2004-05). You're very
likely down to close to zero examples.

------
mikegerwitz
The article has good points. But the term "side project" implies working on
something that isn't part of your job. I think looking at your project from
that perspective can also be a step in the wrong direction---if you enjoy what
you do, work is just one such application of your knowledge and ability.
Projects you do outside of work are another, and may even be more important to
you than work.

A number of commenters point out that many side-projects to them are to
explore new concepts/libraries, or as a means of learning. If it's for the
sake of learning for your job, then yes, it'd be a side project.

But if you're looking to take a project to completion (the author talks about
products; my projects aren't), I find that using it as a learning environment
extremely detrimental. Yes, you do learn things over the course of development
---through research, struggle, and growth. But if too much of your time is
spent on things you don't have a good foundation on, you may burn out too
early, wind up frustrated, and wind up with a mess that leads to refactoring.
Use a foundation that you have experience with and know well, and learn the
parts that are necessary.

(I do distinguish, though, a research project with a project for playful
learning. The former is much more formal and disciplined. But you can still
burn out early.)

------
CarrotCodes
I thoroughly enjoy over-engineering side projects, and writing about them
(linked below). The trick for me is in managing expectations. It's completely
possible for a learning project to evolve in to something widely used, just
don't go in with that expectation.

[https://blog.skywelch.io/2017/06/over-engineering-bunnies-
io...](https://blog.skywelch.io/2017/06/over-engineering-bunnies-io/)

------
hoodoof
I have built projects with a friend in the past.

He so aggressively avoids overengineering that he goes the other way - if you
make ANY effort to properly engineer something he gets really annoyed and says
time is being wasted.

It's almost like he feels software MUST be engineered to NOT scale in order to
be an "MVP".

It's strange because for me, thinking about how something is built means that
it can be built right the first time, without extra effort, just by thinking
it through.

~~~
robertlagrant
This is my attitude to the React PATENTS file - yeah okay I probably won't be
big enough to worry about it, but just in case I am, why take the risk with
React?

------
hashkb
I have a personal Jenkins that's been running for years. Adding new jobs isn't
yak shaving; it's a two minute task that gives me CD.

Same with my AWS "framework". And my CSS "framework" and my React component
superclass and lots of other stuff I carry from one side project to another.

Some would (and have) accused me of over engineering; but I can discover my
project is pointless very quickly.

------
redm
Man, I think this article is dead on, except it doesn't identify the original
reason, for me at least. In a normal environment, there are constraints on
time, money, goals, resources, etc. Suddenly, when there are no constraints, a
side project, you try to do everything perfectly, which leads to the issues in
this article.

------
GigabyteCoin
>You’re so hyped for this project and confident that it will succeed that you
start thinking about the future: “how will I scale my application for the
millions of users it will have?”

That's one of my side project vices!

Every time I write a new login database... I wonder if I should make the ID a
MEDIUMINT or INT.

~~~
imhoguy
I go full and make them BIGINT straight away. One less thing to worry once I
will become billionaire.

------
myhf
> 4\. Use frameworks and customise them – only refactor/build your own when
> absolutely neccessary.

> 5\. Build your project first – then worry about continuous delivery.

These contradict each other. Building your own deployment system is just as
much of a distraction as building your own CSS framework.

~~~
MichaelBurge
Your "deployment system" could be a 20-line systemd service and a git
checkout. You deploy with a "git pull" and "sudo service restart".

~~~
staticelf
You could even skip the systemd and just rely that the server will never be
restarted in the very beginning.

------
klokoman
But that's all the fun

~~~
wyager
Agreed. The best part of side projects is that I can take as long as I want to
build it to my satisfaction, rather than to the minimal acceptable state for
business purposes.

------
n1vz3r
I really like over-engineering my side projects. I have three frameworks, one
in PHP, one in JS, and one in PHP/JS (a bit abandoned) that no one really
uses. But you know what? Because no one uses them, I'm not burdened by any
opinion except mine. When I change something, I don't read death threats
because someone's solution stopped working. And since I don't have any
deadlines, I can spend as much time as I want to get the exact solution as I
planned. I don't need to compromise. That's quite opposite thing to what I do
at worktime and get paid for.

------
greggman
I feel like there should be a solution that covers 99% of online projects. out
of the box it scales, deploys, upgrades, does continuous integration, testing,
logging, supports all major logins, has an admin console, metric console,
etc... maybe it starts with the traditional Todo app and you just start
modding.

I find it incredibly frustrating that much of this isn't a solved problem that
I can then just insert my side project into.

hacking is fun until I have to maintain it and then I start wishing for all of
those features instead of having to spend days or weeks implementing each of
them from scratch

------
elderK
I can only speak for myself here but I feel that belonging too strongly in
either camp is a bad thing.

Ideally, you'd want to understand both worlds.

As such, I feel that the projects I work on in my own time and the crazy
things I do in them help me to better understand what kind of things _really
do need_ to happen when working under tight deadlines in the workplace.

I.e. It's easier to produce an effective product quickly if you know from
personal experience exactly which approaches or things really are essential.

Also, note I haven't read the actual article. I'm replying mostly in response
to the comments :)

------
outworlder
> I know: I’ll create a triple zoned redundant architecture with pub/sub
> database replication, a 32 node Kubernetes cluster and private networking
> across all regions – that way, I can handle anything!

Well, maybe not a 32 node k8s cluster. However, spinning up K8s clusters is so
easy now, that it costs almost no time to deploy stuff there. And so many
things are taken care for you: configuration, secret storage, networking,
versioning, keeping services up, etc.

If on AWS, there's Kops. If on GCP, that's even easier, a couple of clicks and
you have a cluster.

------
SadWebDeveloper
> Then, put it on the bare minimum architecture that it can run on (be that a
> 512MB instance from DigitalOcean or a medium instance on AWS).

Or just test it in a VM and stop wasting money on side-projects.

------
p4lindromica
One of the ways I learn is via code review. In a side project I'm often doing
new things that I am not already an expert in. Any ideas on how to get
feedback that things are ok vs not?

------
jondubois
Side projects are meant to be challenging because they are a learning
opportunity so it's possible that they end up over-engineered sometimes. You
can always refactor the parts that are over-engineered. There's a very fine
line between highly-engineered and over-engineered. One of my favourite
projects is Kubernetes; some might consider it over-engineered but it's
extremely powerful.

"Perfection is achieved not when there is nothing more to add, but when there
is nothing left to take away"

~~~
zimpenfish
> "[...] when there is nothing left to take away"

I've always taken that quote to apply only when you have been taking things
away; not when you've ended up with a colossal complex monstrosity where
removal of any single strand would cause total collapse.

------
spott
As in everything: figure out what you are trying to get out of your side
project.

If you want something to generate a nice supplementary income stream, or
practice for starting a company when you get "that great idea" then this
article is very good advice.

On the other hand, if you are trying to learn a new technology stack, develop
experience in building scalable architectures or even beefing up your project
management skills then this is terrible advice.

So, the real art is figuring out what you want out of your side project.

------
bitwize
If I don't overengineer my side projects -- if I stick to what's simple and
practical -- how am I going to gain marketable skills with the modern
frameworks?

~~~
parmesan
Set your goals high for the project instead, achieving your goals can never be
considered over-engineering. Don't let the scope creep into "oh I should
totally spend 30 hours on this form design"

------
old_chap
As an amateur coder/noob I'm happy I'm not unique in this idea. I constantly
start projects in Flask/Python and just try to make them as complex as
possible (to the limit of my current capabilities). When I'm at work (im an
Incident Manager for a NOC/SOC) I can somewhat hold my own in conversations
about infrastructure issues. I'm also hoping to go into the security field
soon (OSCP cert).

------
forthelove
There's probably another reason hidden in the first sentence, which is that
there's a transition from building the product (engineering) to shipping the
product (marketing, customer development, sales) that requires different skill
sets and experiences. Was just discussing this with a software engineer friend
of mine who realizes that he "has no clue how to market my product the right
ways."

------
ThomPete
When I do side projects it's normally because I want to explore something
which is too risky for someone to do (or it would most likely have been done).
In my experience that normally requires some sort of overengineering because
your problem and solution isn't common.

A few of them turned into great side businesses but most of the times it help
me explore technologies and projects while still solving real problems.

------
pfarnsworth
Just like in real life, the art of engineering is iteration. So I always get
quick and dirty results with my side projects, and iterate if it's worth
iterating. But my side projects are just for myself, not for others, so it's
things like writing a multi-thread stock strategy backtester, a dashboard for
my IoT devices, etc. It's never that I think will make money.

------
breatheoften
This makes a lot of sense -- and makes me feel like I'm guilty of over
engineering a non-side project I started working on at work ...

I'm always looking to make the code as easy to change as possible -- the idea
being that change is a precondition for any form of improvement ... Reading
this article makes me wonder if I've turned that pursuit into an anti-goal

------
yummybear
I'm going to try another approach with my next project - not sure if it's a
good idea, but I'm going with the Minimum Unviable Product. As soon as the
project is bootstrapped and compilable, I'm going to release a version. I
figure this way at least I can say I released something, even if it's not that
feature rich.

------
jmkni
Duck it, it's your side project.

You own it, nobody it paying you to work on it, over-engineer the shit out of
it if you feel like it.

------
Walkman
I recently was thinking about 1). At my current company, I got to know agile
methodologies for the first time, I liked it and started planning my side
project, but it instantly started to felt like work! I just realized if I
write the parts whatever interests me the most, I will be more inclined to
finish it!

------
ratsimihah
"Continuously delivering nothing" – exactly what I'm doing, building a product
no one will use.

------
EGreg
I think [https://qbix.com/platform](https://qbix.com/platform) is the epitome
of this. It came out of me wanting to abstract away once and for all
everything that made developing social networks hard. And I just kept going
lol.

------
partycoder
Depends what the goal of the project is.

If the side project is for learning purposes then taking time to explore
different approaches is OK.

If your side project goal is to build a product, then you should approach it
as a more traditional project, budgeting your time, and focusing on time-
sensitive tradeoffs, etc.

------
FatAmericanDev
Every enterprise client I have worked with has asked about our tech stack. I
will agree that your groupon clone users do not care about your stack, but
respond to an enterprise SaaS RFP and stack, security, DR will all be
discussed.

------
real-hacker
The definition of a side project is not clear. Is it just for fun/learning
about new tech, and can be thrown away? Or is it a serious endeavor, and
supposed to be the next big thing? These require completely different
mindsets.

------
datpuz
This article ignores the fact that many of us work on side projects for fun.
PHP? Nah.

------
binthere
I have been successfully applying Jonathan Blow principles into my side
projects. It's been working great. If you are not familiar I highly suggest
you to listen to a few of his presentations about software development.

------
wreath
I think deferring the CD part until the end can be really painful when you are
"ready". I think it's a worth upfront investment even if you have few users,
will make your life much easier and less stressful.

------
dejv
All those rules applies to our new non-side projects as well.

I can't even count how many times I seen projects with awesome infrastructure
and basically zero functionality after depleting whole budget (and usually
some more).

------
mmphosis
Minimum viable product (MVP)
[https://en.wikipedia.org/wiki/Minimum_viable_product](https://en.wikipedia.org/wiki/Minimum_viable_product)

------
funnyenough
on point! cannot agree more - I have been down this road before with my last
project - Jabid.com. We tried to boil the ocean even though that was not the
goal of the project. This time around, for
[https://veganfutura.com](https://veganfutura.com), I am taking a complete
opposite approach, that closely aligns with the list you have provided. Keep
it simple, focus on the problem you are trying to solve and the user.
Marketing and User acquisition > Tech initially.

------
esseti
For the record (it's not mentioned but a lot of time it happens): security
(e.g. password storage) should not be something left for later. Do it or use a
framework that does it for you.

------
rubenbe
#Mistake 6: not writing tests.

In case I want to actually deliver a side-project, I usually write some basic
tests. As such I can focus more on adding new (fun) features instead of
hunting down regressions.

------
treyhuffine
If you want to release your project and maybe turn it into a start up, then
these are all great points. However, many times the purpose of a side project
is to just learn new things.

------
aetherspawn
Doing #5 is chronic because I'm too lazy to handle my own deployments and it's
easier to copy paste a generic firebase deploy.

Then you can send the link to anyone and it always just works.

------
scadge
I believed over-engineering IS the point of personal side-projects.

~~~
zimpenfish
I'm in the opposite camp - personal side-projects are where you can safely
implement the MVP and/or ruthlessly chop things/ideas that never* get used
without people harassing you about bells and whistles.

------
auserperson
I think the majority of engineers only do side projects to learn things they
couldn't at work or explore architectures they wouldn't be able to in a work
environment.

------
twelvedogs
heh, most of this stuff is extra wastes of time past the very first hurdle
that you should be working on: make something that works and is useful to
people.

sure it would be great to have scaling to 10,000 users already sorted by the
time you get there but odds are your project will never get there, when you
get to 5k users you need to start worrying about that

if you're just doing it to learn or for fun then obviously this article or my
comment don't apply

------
SonicSoul
guilty of all of the above. my thinking is always "i'll kill 2 birds with one
stone, side project and learning the latest framework!" and yah.. after
struggling with the framework for 2 weeks the project loses steam and then
just thinking about a stupid challenge i have to solve BEFORE getting back to
my project is crippling.

it does have a nice side benefit however of actually learning a new framework
:)

------
viach
If only it was applied to side projects exclusively...

------
darod
Setting up a CI on AWS using git, CodePipeline, Code Build, and elastic
beanstalk takes no more than a few hours. It's worth the few hours. Otherwise
you're manually pushing builds out which wastes more time in the long. In the
beginning your CI should be nothing more than ==> I check in code, it deploys
to environment. Later on you can worry about triggering tests and preventing
builds for deploying if they fail. Also if you use elastic beanstalk, you can
scale as necessarily with a few configuration changes.

------
mehh
Love the title of this post.

My side projects have all been over engineered, an unsuccessful. They have
also helped my career enormously :)

------
AngeloAnolin
Bottomline is, you need to deliver something usable - regardless of its size
or the technology behind it.

------
arrty88
Depends on the goal. Do you want to learn a lot or launch a MVP in the least
amount of time possible.

------
dudul
Oddly enough, I would apply some of these practices to real projects as well.

------
awjr
A lot of time this can be applied to any new project not just side projects.

------
kyberias
These apply to every project.

------
zhuzhu
the best real joke of month!

------
fundabulousrIII
Back in the 90's and early oughts no one thought of writing code this way. It
was counter productive. I'd suggest that it is still counter productive to
consider SDLC concepts when writing pet code: great idea or not.

This whole agile sdlc idea is a distraction from writing good code.

------
jlebrech
this should apply to the start of any project.

------
polote
This post should be renamed : The art of over-engineering

 _" those software engineers often have the tendency to over-engineer their
side projects"_

When your first sentence is false, so do is the rest of the article !

~~~
tw1010
"When your first sentence is false, so do is the rest of the article !"

Not necessarily.

------
rthomas6
Triggered. Leave me alone and let me make my Hugo blog migration with my
custom theme using bulma.io and webpack. Never mind that the WordPress setup I
have now could handle 100x the traffic I get.

