
The Architecture of Open Source Applications - letientai299
https://aosabook.org/en/index.html
======
eatonphil
I've had the idea for a while to start a club to read through open source
projects, compile them, extend them, note architectures and design decisions,
look at documentation and CI/testing organization, etc.

Do something like one project per month with one topic among the above per
week. Haven't found the right group of folks to experiment on with this yet.

Edit: Send me an email if you'd like and I'll think about hosting this
virtually.

~~~
jcranmer
On a similar note, I've thought about putting together a presentation on
debugging and understanding large codebases by fixing a bug on a large project
whose source code I've literally never seen before. I figure that by picking a
codebase I have 0 experience with, there's less risk of skipping over steps.

The biggest hurdle for me has been actually finding a project I'm interested
in where I haven't yet gone poking around the source code for some reason or
another.

~~~
eatonphil
Yeah, I've also been interested in writing more about orienting yourself in
novel codebases. I don't think it's explored very much.

It's a pain to write about because you have to find some contrived error (real
or not) in a contrived app. I've got a draft of a post but without the
examples filled in.

Would like to read (and share) what you've got.

~~~
monksy
It's not explored very much because most new developers try to push for
greenfield development and they're trying to split everything up even when it
doesn't need it.

Now we're having multiples of codebases to comb through.

~~~
gnufx
So true. Also with respect to documentation, which you have to put in a blog
rather than contributing it to the project where it can be maintained. It's
bizarre to be literally the last port of call for work you've actually done,
when people would rather refer to obsolete blogs on things you addressed years
ago.

------
chubot
FWIW the Oil project was partly (negatively) inspired by the very helpful
description of bash's architecture:

[https://aosabook.org/en/bash.html](https://aosabook.org/en/bash.html)

The maintainer expresses some regret about the parser architecture. I
referenced this in several 2016-2017 blog posts:

 _OSH Parses Shell Scripts Up Front in a Single Pass. Other Shells Don 't_ \-
[http://www.oilshell.org/blog/2016/10/13.html](http://www.oilshell.org/blog/2016/10/13.html)

 _OSH Can Be Parsed With Two Tokens of Lookahead_ \-
[http://www.oilshell.org/blog/2016/11/17.html](http://www.oilshell.org/blog/2016/11/17.html)

 _The Thinner Waist of the Interpreter_ \-
[http://www.oilshell.org/blog/2017/01/26.html](http://www.oilshell.org/blog/2017/01/26.html)

 _Word Evaluation_ \-
[http://www.oilshell.org/blog/2017/03/09.html](http://www.oilshell.org/blog/2017/03/09.html)

\----

This principled architecture ended up paying off in a couple unexpected ways,
which I wrote about earlier this year:

 _Oil Uses Its Parser For History And Completion_ \-
[http://www.oilshell.org/blog/2020/01/history-and-
completion....](http://www.oilshell.org/blog/2020/01/history-and-
completion.html)

 _Oil 's Parser Doesn't Worry About Aliases and Prompts_ \-
[http://www.oilshell.org/blog/2020/01/alias-and-
prompt.html](http://www.oilshell.org/blog/2020/01/alias-and-prompt.html)

Summary: in addition to the regrets expressed by the maintainer in the
article, bash actually has multiple ad hoc, incorrect parsers for its own
language! They are incorrect to the point that a separate project (bash-
completion) has code to paper over the errors.

Oil uses the same parser for all use cases. Moreover, its parser isn't
littered with orthogonal concerns like the interactive prompt and expanding
aliases.

\----

However I still need help finishing the project -- see what I've cut here:
[http://www.oilshell.org/blog/2020/08/risks.html](http://www.oilshell.org/blog/2020/08/risks.html)

And feel free to contact me (address on home page and in profile)

------
dang
If curious see also

2019
[https://news.ycombinator.com/item?id=20731206](https://news.ycombinator.com/item?id=20731206)

2015
[https://news.ycombinator.com/item?id=9418788](https://news.ycombinator.com/item?id=9418788)

2013
[https://news.ycombinator.com/item?id=6404929](https://news.ycombinator.com/item?id=6404929)

2013
[https://news.ycombinator.com/item?id=6613748](https://news.ycombinator.com/item?id=6613748)

2011
[https://news.ycombinator.com/item?id=2904425](https://news.ycombinator.com/item?id=2904425)

2011
[https://news.ycombinator.com/item?id=2598643](https://news.ycombinator.com/item?id=2598643)

2011 (a bit)
[https://news.ycombinator.com/item?id=2578071](https://news.ycombinator.com/item?id=2578071)

There have been many other submissions
([https://news.ycombinator.com/from?site=aosabook.org](https://news.ycombinator.com/from?site=aosabook.org)),
but those seem to be the nontrivial threads about the book itself.

------
astatine
I was inspired by the AOSA many years ago and I had studied (and held out) as
an example the Apache architecture (about 18-20 years ago). It is indeed a
pity that so many younger software engineers really have no idea of how
heavily used real-world systems get built. Nor do they understand all the
various tradeoffs and corner cases that need to get handled to make products
work well in real life.

Building Apache from source (including a few key addon modules) and
configuring it to work for a small set of tests was step 1 at my organization
then. Those successful at this would then get into the source itself. Fun
times!

~~~
krisgenre
Unfortunately deep technical depth is unwarranted and unwelcomed in many
organizations. In fact you repeatedly get told that "the customer doesn't care
about your code". At the end of the day adding yet another 'if' condition to
patch up things is what is usually appreciated.

~~~
non-entity
> I once worked at a place where we had an endpoint on an application taking a
> long time because of a weird DB query, like 12+ seconds to query a few
> hundred rows. Was pretty much told if we didn't figure it out today it
> doesn't matter, we'll ship it like that.

------
dllthomas
I've been working through the first volume in a reading group with a few
friends. Some chapters are definitely better than others, and it often
requires awareness that it captures a snapshot of thinking from ~10 years ago,
but it's been a good experience.

~~~
p1nkpineapple
Could you please elaborate on the chapters that you feel are the best to
study?

Cheers!

~~~
dllthomas
So far we've read the chapters on Asterisk, Audacity, Bash, BerkeleyDB, CMake,
Continuous Integration, Eclipse, and Graphite.

Of these, I liked the chapters on Bash, BerkeleyDB, and Graphite better than
those on Asterisk, Audacity, CMake, and CI. I had trouble getting through the
chapter on Eclipse, although there were some interesting bits.

The chapter on Bash suffered a little, in my personal experience, because
about 70% of it was a rehash (with a slightly, but not sufficiently, different
perspective) of information present in the Bash man page, which I had
previously read in a reading group with some of these same people.

But really, the chapter that's best to study is probably the one you're
currently most interested in, even if it's "not as good" as another considered
in the abstract. Just remember not to assume that the entire book is of the
same quality, whatever your experience.

------
rektide
Telepathy is one of the coolest most useful most modular pieces of software
i've ever used, a modular pluggable framework for chat, & I am always so glad
to have a chapter of this book talking about that all.

------
gnufx
It's a pity it's "open source" and not free software including Emacs, in
particular, which has been rather successful itself and as a model long-term.

I wonder how some of these architectures have really helped with maintenance.
For instance, I'm familiar with Open MPI and its rather sorry maintenance
history with continual breakage and at least partially unnecessary lack of
compatibility. Then it still seems the best bet for general purpose MPI...

------
site-packages1
Very good book, but I would love a version working with modern tooling and
software products. I remember going through these maybe 5-7 years ago and
thinking at the time that they were using some really old paradigms and
software. Would love to see some covering software with modern languages as
well, like Rust.

------
legulere
I think the most important you can take away from it, is that a good
architecture takes the problem into account that the project tries to solve.
Too often I hear calls for some kind of architectural holy grail, that solves
all problems.

------
aabbcc1241
Want to double star this resource. It covers many interesting (and advanced?)
topics like distributed system, building your own database, meta programming,
and template engine.

------
pkphilip
This is a brilliant idea! It would be great if they start with something
everyone uses such as editors - example: Vi, Emacs, Code, browsers such as
Mozilla etc.

------
pattato
Is all the content on that website representative of what is in the book when
you buy it?

------
spicyramen
This should be part of any serious computer science program subject.

------
amelius
I'd like to see Kicad, Freecad and Inkscape covered.

~~~
tepitoperrito
Can't speak about the other projects listed but FreeCAD has a great landing
page on their wiki.

[https://wiki.freecadweb.org/Developer_hub](https://wiki.freecadweb.org/Developer_hub)
(I especially liked the forum post from the contributor who detailed the
geometric constraint solver - very neat)

------
unixhero
Great book! I think I will read the whole thing.

