

How I organise a large project - vegancap
http://ewanvalentine.io/how-i-organise-a-large-project/

======
Ixiaus
I've learned my lesson trying to structure "large projects" inside of a single
"project" or logical repository.

A developer I hired at one point had read an article written by someone in the
OSS community promulgating the virtues of treating your "projects" like OSS
projects and if you have a "large project" breaking it down into smaller
constituent projects with their own source repository, dependency defs,
README's, tests, Vagrant / Docker environments etc...

This fits well with the ideas behind small composable units of work and it
also increases clarity.

[EDIT] Essentially, if you have a large project, it's probably time to re-
think how you're coupling or de-coupling functional units. The goal being lots
of small, documented, and tested units composed to create a "large software
product".

~~~
vegancap
Yeah I agree! I used to commit everything into one repo, and I found I was
wasting a lot of time downloading thousands of files and vm's etc when I only
needed to work on say one component.

I think this structure lets you take either approach, needs some tweaking
though for sure.

------
dbpatterson
A large project? I'm not sure what's particularly large about this. Perhaps it
should be "how I organize a project I'm going to have running on a server at
some point"?

~~~
vegancap
By large I just meant a project which houses multiple components. Of course
the term 'large' is subjective.

------
BillinghamJ
Imo, it's better to keep all components separated in their own repos for any
project which can really be described as 'large'.

What if, for example, you completely rewrite your backend system without
changing its API spec at all? Having the history of the backend and mobile app
linked becomes a bit odd at this point.

What if you're using a system which deploys your repo? It may expect certain
things to be in the root of the repo, and you certainly don't want your mobile
app code to end up on your application servers.

Honestly, I'm not really aware of any instance where it is actually better to
lump together multiple parts of a system into one repository. Maybe you made
some bad architectural decisions which makes it easier to do so, but I don't
consider that 'better'.

~~~
taeric
This does go counter to the way that it is described that both Google and
Facebook have been reported to do things.

The specific problem usually comes down to the extra work necessary to
correlate changes between repositories.

~~~
BillinghamJ
I kinda get the impression that some of the very large, very fast growing
companies have ended up like that by necessity because they've dug themselves
into a hole they can't get out of. e.g. Facebook building HHVM instead of not
using PHP.

~~~
taeric
I do not at all disagree with this impression. The question, though, is why
are the major successes counter to the "best practice" that so many folk like
to cite?

Consider, even the linux kernel is counter to the best practices as they were
thought at the time.

I also realize that they could just be an unrelated data point. I just have
grown weary of every success story I know about being "in spite of terrible
practices." At some point, the best practices theory bows to the results of
actual practices.

------
chriswarbo
This looks somewhat similar to what we're doing at my current job. We've
writing APIs for our internal systems to decouple things a bit, and have ended
up with a frontend router, a "middle-end" message queue and backend scripts to
do the processing.

However, our Vagrant setup just took a massive turn for the worse: due to some
compatability issues, these VMs have now all been hooked together using
Puppet! During provisioning, if the git repos for the other machines aren't
checked out to a particular location, everything dies; which seems like a
really cargo-culty choice to me.

I'd be interested in doing things like this with Nix: Disnix to define the
relationships between the servers and NixOps to define the VM provisioning.
That may be because I'm a functional programming nut who's just switched to
NixOS though ;)

~~~
vegancap
I've only been using this structure for say, an app and an API. I haven't
tried it with say caching vm's or a message server thrown in the mix.

Your idea about using Nix might be worth me looking into next! It kinda feels
like spinning plates trying to organise vm's and codebases haha.

I thought this structure might be a good start for new comers though :)

------
liotier
In the name of all corporate grunts, thanks for reminding of how subjective
'large' is...

~~~
vegancap
Haha not a problem! I've worked with a lot bigger (being a corporate grunt
also), but this was just a starting point. In theory you could extend this
structure without things getting _too_ messy :)

------
paukiatwee
The structure petty interesting, but which language/framework do you use? I
think different language/framework might have their standard.

~~~
vegancap
This was about a PHP/Symfony API and a angularjs/phonegap app. I've split it
between a PHP app, Node app and a phonegap app previously as well and it felt
like a nice workflow and I felt like I was getting more done. You're right it
might not be ideal for everything. But for new comers I think it's a solid
place to start. Especially for PHP, which can get a bit messy!

~~~
paukiatwee
Never really work on PHP before, but I think you should include those info so
people like me will understand this setup is optimize for which language.

------
_random_
Pretty standard for ASP.NET Web API + Angular projects.

------
tootie
Are the API and app not separate deploys?

