
Ask HN: What are some rite-of-passage style projects for programmers? - linuss
Hi,
I&#x27;m trying to compile a list of projects, algorithms and maybe data structures that I feel every well-rounded programmer should have implemented at least once in their career. I&#x27;m hoping this list can then function as a guide for programmers to challenge themselves and maybe fill gaps in their knowledge they didn&#x27;t know they had.  So far, I&#x27;ve come up with the following list:<p>* Data structures
- Linked list
- Hash map
- Several types of trees<p>* Algorithms
- The common sorting algorithms
- Dijkstra
- Graph-traversal algorithms<p>* Projects
- Ray tracer
- Parser&#x2F;Interpreter
- Compiler
- Virtual Machine
- Small kernel
- Neural Network
- Web server<p>The algorithm and data structures section I believe should be covered by any decent computer science education (but may be useful to self-taught programmers). The projects are slightly more advanced and may take up to several weeks to implement completely. 
I realize this list is far from complete, so that&#x27;s why I&#x27;m turning to you: do you have any projects that you&#x27;ve worked on that turned out to be very educational and made you a better programmer once you completed it?
======
jasonkester
Odd list. You can have a very successful developer career without ever needing
to write any of those ever. I'm a little over 20 years in now, and have never
needed to do any of that.

Maybe in the 90s, I might have actually built a linked list in C for whatever
reason, and I guess you could define graph traversal to include walking a tree
pulled out of a database. But the rest are things that _The Universe Provides
For You_ (tm), which one would only ever reproduce in School, a Job Interview,
or a Poorly Chosen Hobby.

As to actual rites of passage? Ship Something that real people actually use.
Build an entire _thing_ , be it a piece of desktop software, video game, web
application, mobile app, etc. from bottom to top and send it out in to the
world fully formed. That, in my mind, is what we're here for.

~~~
jnbiche
> You can have a very successful developer career without ever needing to
> write any of those ever.

You write this, but then you go on to mention job interviews, which are
becoming more and more algorithmic each passing year. Software engineers are
_frequently_ asked to write these during interviews. If you've missed the
experience of being asked to implement a hash table or linked list in an
interview, or at the very least asked to write a function somehow manipulating
some data structure, you've either been lucky or else stayed in the same job
for a long time (not a bad thing, but not typical).

I highly recommend _all_ software developers practice implementing data
structures (linked lists, hash tables, trees, graphs) and algorithms (sorting,
combinatorics, etc) on a regular basis. If you fail to master these, and to
understand their underlying rates of growth (ie, "Big O"), you'll be forever
pigeon-holed as a lightweight "webdev" forever, even if you have the
programming chops required to build complex, reliable, and useful software.

Plus, they get really interesting the more you study them, and the math behind
them.

~~~
jmcgough
> You write this, but then you go on to mention job interviews, which are
> becoming more and more algorithmic each passing year.

It's been the opposite experience for me in SF, actually. More startups are
realizing that they want people who can build good products quickly, and with
more people coming from non-CS backgrounds, they'd rather test for that than
see if they can whiteboard a breadth-first search.

I took enough CS coursework to learn the stuff you mentioned, but I've worked
with a number of developers who don't know any CS theory. Several of them are
stronger engineers than me and have had very successful careers, and I'd
rather work with someone who can communicate well and solve hard problems than
someone who's read through CLRS.

------
mynameisjody
Some real world rites of passage... * have your project cancelled * get
outsourced * build the same thing twice (billing system, e-commerce, etc) *
underestimate a project * get promoted to management....realize you hate it *
hire a net-negative * implement a cms

We could totally make career bingo outnof this...

~~~
asimuvPR
\- Stick long enough at a job that the product becomes a social network.

\- Attend the same convention twice in a row for different jobs.

\- Work with an ivy league MBA.

\- Eventually settle on a Java or C# job.

\- Build a wonderful product that people won't use. For whatever reason...

\- Burnout and dream of working in construction...

~~~
ljk
> _\- Eventually settle on a Java or C# job._

why is Java considered "settling"? relatively inexperienced here

~~~
khedoros
They've got the reputation as being kind of boring, corporate languages, among
some people. Neither one is usually in the list of things that people would
get very excited about using.

~~~
ljk
what's an "exciting" language in your opinion?

~~~
asimuvPR
I know the answer was not meant for me, but I'm very excited about Verilog. It
allows me to build my own chips with FPGAs (not that I'm doing any at the
moment). Plus that means I can build my own language around a custom chip
architecture.

~~~
khedoros
I've considered that. At the very least, it would be cool to have an OS-less
piece of hardware with known performance characteristics that I could program
as a custom coprocessor. I feel like I've got so many projects on my hands
that I don't have time to learn hardware design on top of the software stuff
I'm already working on, though.

------
sheraz
My rights of passage:

1\. your own MVC framework You should do this to appreciate why developers of
other frameworks make the decisions they do. I gained so much wisdom from
this.

2\. Parsing HTML with regex (see here - [http://blog.codinghorror.com/regular-
expressions-now-you-hav...](http://blog.codinghorror.com/regular-expressions-
now-you-have-two-problems/))

Seriously, just DONT DO IT(tm) -- but if you do, you will eventually learn why
you don't want to do it this way, and you might get pretty good at regex
expressions

3\. Your first mobile app published to the app store Publishing apps to the
Apple app store has given me a deeper appreciation for paying attention to the
little details. Also, making native apps is a completely different paradigm
than web apps because shipping code with logic errors has such a high cost and
delay to fixing them.

4\. Port an existing library to a new language I long time ago I ported a
recipe parsers from Ruby to Python for a paid gig. It was such a good learning
experience because I had a perfectly functioning reference implementation,
which allowed me to go deep on getting the details right.

I had to replicate test cases, documentation, scaffolding, and the code itself
while being aware of the gotchas of Python.

~~~
madmax96
>Seriously, just DONT DO IT(tm)

Yes, you'll learn that this task is literally impossible, as well as why:) I
think we've all used regex in a spot where it's impossible to do so correctly
at some point.

~~~
falcolas
Well, considering the PCRE regexes are nearly Turing complete, you can parse
(well formed) HTML.

Someone better versed than I: [https://nikic.github.io/2012/06/15/The-true-
power-of-regular...](https://nikic.github.io/2012/06/15/The-true-power-of-
regular-expressions.html)

Of course, the money quote is: Just because you can, doesn’t mean that you
should.

~~~
madmax96
I had never seen PCRE regexes. I was thinking about the plain ones. This was
pretty cool, thanks for sharing:)

------
fitzwatermellow
Snake. Can pretty much pack all of CS 101 into that one little game ;)

My "hello world" for learning new languages and platforms. Integer based or
floating point. 2d or 3d. Sound effects, sprite animation, physics, procedural
particle systems, global leaderboards, digital skins and so on ad infinitum.
Allows you to experience nuances in packaging and deploying WebGL vs Android
vs Steam. Continue polishing it, and you may end up with something fun that
others will love!

Can also be refactored into a full Tron Light Cycle style simulation. Which is
a great way to learn AI. Good luck!

------
madmax96
For me, it was writing my own Lisp. I'd __strongly __recommend giving it a try
-- you 'll learn about tokenization, parsing, garbage collection, and a bunch
of little software development lessons.

~~~
macintux
Useful for getting started:
[https://github.com/kanaka/mal](https://github.com/kanaka/mal)

------
paulmooreparks
For me, it was implementing a large bit of the C standard library. I had just
blown an interview at Microsoft in 2001, where I stumbled over a whiteboard
implementation of strtol, and I thought that I needed a serious upgrade of my
understanding of library implementations and the algorithms that are used in
them.

Every developer should also implement all the basic data structures and
algorithms -- and then never write their own again! The process, however,
definitely improves your chops, helps you understand the trade-offs inherent
in choosing among data structures and algorithms, and gives you an
appreciation for what's going on "under the hood."

~~~
paulmooreparks
Also, writing a compiler covers a lot of the list above. Even a simple
language like Brainfuck (my choice for this rite of passage) can teach you a
lot. It also gets you into the world of assembly language, which it seems that
fewer and fewer new devs are exposed to these days.

(Yep... I'm officially old now. I just played the "kids these days" card.)

------
borplk
Maybe compiler-related stuff. Create a small dummy language that compiles down
to JVM bytecode and things like that (learn about tokenizing, parser, abstract
syntax tree, type checking, operator precedence, code generation, etc...).

------
broodbucket
The classic one I always heard of was a shell. You learn about input parsing,
fork/exec, and usually some syscalls. Lots of features can be added on, like
running processes in the background, autocompletions etc.

Writing a shell in C (you can use Readline if you don't care to learn much
about parsing) teaches you lots of stuff about the operating system you're
working on.

------
FLGMwt
This may be colored by me being self-taught, but aside from interviews, why
algorithms?

Seems to me that the best way to test your well-rounded skills as a programmer
is to build and launch a product. Even if you don't aim to be an entrepreneur,
the holes you find while taking an idea from inception to launch are much
bigger holes than you'd find building this tree vs that tree.

~~~
framebit
Building fundamental data structures and implementing classic algorithms is a
phenomenal way to learn about unit testing. How else are you going to make
sure your doubly-linked list is doing the right thing?

It's much more difficult to learn about unit testing when you have the
complications of a project stack: databases, front ends, etc. The fundamental
data structures are self contained and their behavior down to the last detail
(the spec) is fully described. That's an ideal way to learn about writing good
unit tests.

~~~
collyw
Unit testing is not the be all and end all of writing an application.

------
aappleby
Common ones from when I started writing video games in C in the 90's -

* Write malloc() and free().

* Write a gzip decompressor.

* Write a triangle rasterizer and use it to draw a spinning cube on the screen.

Each of those is a decent but manageable amount of work for a new dev and will
teach you a variety of useful low-level skills.

------
contingencies
The phrase 'rite-of-passage', to me, carries negative connotations - something
you _have_ to do, something _tedious_ , something _necessary_. In truth, there
are no such projects.

However, in terms of projects popularly considered to be commonly implemented
by newer programmers that do hold benefit, in network programming, I would say
a traceroute implementation. Server-side, I'd say any multi-node cluster
system, preferably diskless. Any embedded system. An RDBMS system. A NoSQL
system. An open source intelligence system. Any computational linguistic
system. Any i18n/l10n heavy project.

~~~
linuss
Ah, the context/connotations of rite-of-passage might have been lost on me as
English is not my first language. Thanks for your suggestions though!

~~~
justin66
I don't think it's a language thing. To many the notion of an ordeal endured
or an adventure undertaken in order to pass into manhood or womanhood (or
whatever) is still appealing. "Rite of passage" does not have a universally
negative meaning in English, it's just that many of us today are... well,
yeah.

~~~
fao_
I think you might be confusing a "Rite of Passage"[0] with a "Hazing
Ritual"[1]

[0]:
[https://en.wikipedia.org/wiki/Rite_of_passage](https://en.wikipedia.org/wiki/Rite_of_passage)
[1]:
[https://en.wikipedia.org/wiki/Hazing_ritual](https://en.wikipedia.org/wiki/Hazing_ritual)

~~~
justin66
I cannot imagine why you think so. Or why you believe the links you shared
support such a statement.

~~~
fao_
In retrospect, I think I misread your posts

------
d--b
1\. Backtracking algorithm 2\. Any kind of NP hard problem really 3\. A
recursive decent parser 4\. An expression evaluation library 5\. A plugin
mechanism 6\. Fixing memory leaks 7\. Speed optimization

------
PeterisP
Quine -
[https://en.wikipedia.org/wiki/Quine_(computing)](https://en.wikipedia.org/wiki/Quine_\(computing\))

------
spacemanmatt
Learn enough sql to write an application entirely in the db, except
presentation layer, of course. Use OO concepts to hide data, provide
accessors, etc. Write a wrapper library for accessors in your front end
language of choice (CLI, Web, GUI, whatever).

You won't often see a big project structured this way, but it is very
effective for a first hand experience of data driven design.

~~~
Clubber
I assuming you mean writing all the business logic in stored procs and
functions. Every time I've seen this, it ended up horribly. I use the database
for storage and data validation only. I write a middle tier for the business
logic, and of course the front end layer.

I'm not saying it can't been done well, but it poses enough problems that are
difficult to avoid, like calling an external web service inside a stored
procedure, and the general mess of maintaining a bunch of chained stored
procedures, functions.

It also guarantees vendor lock-in.

~~~
spacemanmatt
I should also mention, I found it common and well-received to use this pattern
for sustaining development on an enterprise app. Some logic _has_ to be in the
database anyway.

~~~
collyw
In my experience, keep the logic in / close to the data model and the
application code becomes very clean and easy. If you have a crap data model,
then lots of implicit rules start appearing at the application level, bugs
start to appear and no one knows exactly what the application should be doing,
as all the rules are tied up in some crappy crufty code.

Like Linus said, bed programmers worry about the code, good programmers worry
about the data and its relationships (or something to that effect).

~~~
spacemanmatt
That experience is why I propose pushing as much business logic down into the
data layer as possible as a matter of implementing a rite-of-passage code. For
organizations, this puts a premium on db programmers, but it also protects
their data from less-skilled programmers.

------
brightball
On the web:

1\. Build your own CMS

2\. Build your own framework

3\. Performance tune SOMETHING intensely so that you can observe the
bottlenecks and their causes across the stack

Just off the cuff there.

------
Aeolun
1\. Anything involving two different forms of encoding, one of which should
preferably be something obscure like ShiftJIS.

------
wayn3
* Projects - Ray tracer - Parser/Interpreter - Compiler - Virtual Machine - Small kernel - Neural Network - Web server

Among these, do the ray tracer and neural network seem way easier than the
competition to me due to my math background or are they just weirdly chosen?
Those two are essentially algorithms, not "projects".

------
gorbachev
Go to an interview and get asked an algorithm question after another realizing
you've never had to implement a meaningful algorithm from scratch since you
graduated College, because there's a library for (almost) everything.

------
reacweb
To learn web security and encryption, Strip CTF2
([https://stripe.com/blog/capture-the-flag-wrap-
up](https://stripe.com/blog/capture-the-flag-wrap-up)) is fabulous.

------
Zekio
Encryption, so you learn something about how it works.

Some kind of game.

Edit: Something that uses an API.

~~~
PeterisP
The problem with learning encryption by writing it yourself is that you will
_not_ know if you got it right or totally wrong.

For most types of problems, the feedback is clear - if it works and gives you
the expected result, then it's (at least mostly) correct. For implementing
cryptographic systems, a passing test suite doesn't mean anything, you'd need
extensive expert review to tell you where you went totally wrong, and without
it you'd just likely learn untrue things.

~~~
fao_
> and without it you'd just likely learn untrue things.

I prefer this to the more common 'Just don't do it', as you actually state
_why_. The reason is something I knew intuitively but couldn't really (And had
actually never devoted any time to) put[ting] it into words. Thank you.

------
erroneousboat
not really an answer to your question, but it could give some guidelines:
[http://matt.might.net/articles/what-cs-majors-should-
know/](http://matt.might.net/articles/what-cs-majors-should-know/)

~~~
linuss
Wow, this is awesome. Thanks for the link.

------
adav
Code something in an afternoon that makes your friends go "wow" the next day!

------
fishnchips
Project management application.

------
jrochkind1
anything involving concurrency.

~~~
linuss
I agree that concurrency is a nearly essential skill nowadays, but would you
have any concrete projects or implementations that use concurrency? There's
quite a difference between writing a parallel merge-sort and say, a whatsapp
back-end clone.

~~~
lightcatcher
Write an IRC bot with sockets. The program needs to listen to multiple
networks inputs (the connections to different IRC servers) as well as respond
to user input (or at least signaling). This is a great playground to try out
different models such as thread per connection, your own busy waiting or
sleeping event loop, and OS event loops such as poll.

If you want to do more parallelism rather than just concurrency, you could
have some inputs make the bot start doing some work (like computing the
billionth digit of pi) and queue the work up into a threadpool. If you really
want to get into parallel programming, you could write a multiproducer-
multiconsumer queue to allow the IO threads to communicate with the pi
computing worker pool.

Hope this is a good concrete project! I did the first part (multiple
connection IRC bot) a few years and it definitely helped me understand
concurrency and network programming.

------
CptMauli
A timesheet application

------
hga
Dive into someone else's imperfect at best code and make it work. After
finishing the final project for my 2nd programming class, I helped someone
else make their's work. In the context of what could run on a PDP-11/70
running Version 6 UNIX™ with 24x80 CRT terminals, it was rather neat, a semi-
real time air traffic control simulator game, with planes coming from the
upper corners and crossing paths as they went down to the catercornered
runway.

Great idea, better than mine, but the code was awful, had three global state
variables X, XX, and XXX ... plus U, UU, and UUU for the UFOs he'd added to
make the game more fun ^_^.

I helped him reduce the complexity by removing the UFOs (it was still quite
challenging enough), and getting it to work in general. This prepared me for
the many future jobs I took working on the code bases of others (one of which,
for example, taught me red-black trees for real), and which soon enough led to
the extremes of software archaeology when you can't even ask anyone about the
code.

Not that you necessarily want to seek out such work, it's hard and often
thankless, but at its best it's also what paying down technical debt is about.
And code you've written long ago can also be rather foreign when you come back
to it....

------
askyourmother
Find a personal itch and scratch it. In my case, I knew little about reverse
engineering closed source bins and drivers but had some hardware I was not yet
ready to toss out. I learnt a lot about writing new device drivers based on
the specs I managed to gtok from poking and peeking and analysing the original
ones.

------
bbcbasic
Looking for a job as a fresh graduate. With no work history it is all bullshit
multi-day interviews, giving presentations, showing you have 'people skills',
rediculously high applciation to interview and interview to offer ratio.

My favourite was a small company that told me they 'like to get girls in for
interviews' on the phone and then I stupidly still went to a face to face (I
was desperate) and I got rejected - the reason was I used a loop to implement
something but didn't suggest I could copy and paste the loop innards 10 times,
and therefore was 'holding things back'.

