Melvin Conway: "Organizations which design systems ... are constrained to produce designs which are copies of the communication structures of these organizations."
The way that we organise our filing system is nothing less than the (asynchronous) communications structure of the organisation, and needs to be architected with the same care and attention to detail.
By the same token, no single scheme will fit all organisations, and each team should seek to find a way of structuring their work that plays to their strengths and gives them a unique competitive advantage.
Here are my thoughts from over the years:
Nowhere is this clearer than when the structure is hierarchical. Divisions close to the root of the tree implicitly carry a greater weight of importance than those close to the leaves.
This property means that it is difficult to find compromise within a single hierarchy; a driving force behind the prevalence of flat 'tagging' mechanisms.
In spite these drawbacks, the very rigidity of hierarchies offers us an unparalleled communications opportunity: The distinctions that we make at the root of the tree are highly visible and send a clear message about the divisions and items that we consider important.
Is it important that tests be kept separate from product logic? Is it important that projects be kept distinct from one another? Is it important that each component or service be kept isolated? How about maturity? Is it important that research code be kept distinct from production code?
We have the opportunity to form a strong opinion, to make a lasting choice, and to shape the culture and the attitudes of those who follow.
In our houses we tend to use the same things in the same ways each day. There's just the one razor, keep it next to the shaving cream.
In a workshop, we need to cooperate with other people, and even for ourselves, it's important to be able to find a tool when we need it. All the X-acto blades go in the same drawer, next to the drill bits.
Which one is more like your codebase? If it's a blog, might be the house. I think more code is workshop-like, but YMMV.
That said, when a project gets big enough I tend to end up relying on memory and search (e.g. IntelliJ's find class) to find files anyway - at that point file name conventions are more important than anything else.
Regardless if the method changed is in the same file or another directory my IDE will find it for me right away.
This is an age old discussion but I think it has been made somewhat moot by the speed and power of our IDE's. Essentially there have been two approaches...
1 ) Put things that are the same together. Tools go in the tool-shed, food in the kitchen.
2) Put things that you need to use at the same time together. Some tools can go in the kitchen because you need a few tools there. Maybe a small fridge in the tool-shed is not a bad idea.
A new third approach opens up as our IDE's got more powerful. Put the files anywhere. This is akin to having a robot in your house that regardless of where you are, it will make a tool or food appear for you on request. This is similar to how I no longer spend time organizing old emails because I just rely on the email system's powerful searching features to find what I need.
For example, my IDE (Netbeans) has CNTRL-O, which lets me go to any class file directly without going to any directories. It also has usage search and implementation search. If you digging around directories, then you doing it wrong. I am still not sold on putting your files in random places, but I just feel where you put them matters less for most purposes.
What you're describing are the facilities that modern IDEs and editors provide to support searching. Those are great, and like you, I use those search facilities to jump directly to a specific point of interest a lot of the time when I'm developing.
However, searching is useful only if you already know what you want to search for. If you're trying to explore the overall structure of a large program, having the files (and the contents within those files) organised systematically still makes browsing easier.
However..in my experiences thus far, this type of cohesion becomes increasingly complex and open to individual interpretation when you are dealing with more complex business domains. Say for example, several different types of customers engaging in several different types of transactions that have just enough differences in implementation to be several distinct models. Often there are several reasonable ways to interpret "functionality", and you end up with a situation where you are actively building what can be a fairly tricky form of technical debt over the course of years as developers come and go. From what I have experienced, the best way to avoid this, is of course more formality on the development team about structuring what goes where, but then you just end up with a different but still just as rigidly enforced structure as the first example from this article. It also has the downside of being non-standardized.
If I were going into some creature's "house", when they may very well be an alien creature whose domain and existence I understand little of, I personally feel most safe in the house that had the more "absurd" structure. I may have to walk down a few more hallways, but I won't blast myself with a ray gun looking for a faucet.
Making a code layout like this is effectively making a prediction about what future changes are likely to entail. Predictions can be wrong, so we want to be able to change anyway.
Of course, as presented, having lots of files named “posts.py” is also fairly terrible in text editors.
One solution is for the filename to include more, e.g. posts/posts-controllers.py. This helps some tools at the cost of others and greater verbosity.
* unique filename: filename.ext
* common filename, different directories: filename.ext<dirname>
* common filename and immediate parent directory name: filename.ext<grandparent/parent>
> But since the file is not located near the change, you need to search through the entire code base to see where the changed function is called in order to update the calls.
You can't really rely on all change being locally restricted, so you have to check the wider code base, anyway. Search through the entire code base should be easy, simple, and fast. If it isn't, that's a bigger problem than the directory structure of the project not being ideal.
It might be tempting to try to maintain multiple directory structures, making use of hard or soft links into the code base, but this sounds like it might be fragile, not to mention hard work to maintain.
I am sure others have also tried to abandon hierarchical file systems altogether, placing the design into a database instead. This also seems like a lot of work, although I can see the attraction.
My solution is to embrace the conflict and take an opinionated approach to the hierarchy -- one that enhances and supports the development process -- but to supplement that with generated HTML documentation with multiple different hyperlink-based navigation schemes.
There are a lot of projects organised around automating this maintenance—though they tend to pop in and out of existence, so I'd agree we're not at the state of having a reliable solution that'll be around next year, or next month. TagFS (https://github.com/marook/tagfs) is the system with which I've been familiar most recently, but it appears to have been dormant for four years.
We've already layered the app between state and UI. UI components are grouped by features. We've not done this with state but it makes sense to. But we can't see a benefit to merging state and UI layers. There are multiple UI features that make use of a set of actions and selectors.
Thank you @kroltan for sharing this.
I know the mobile experience and accessibility of my site have some issues. I have a change coming t fix it, but feel free to report any issues you encounter.
This layout is similar to how I organize a fairly large real world project in my Flask course.
For example, here's some folders that house various functionality:
Each one has templates, views (route definitions), background tasks and models that are associated to that specific thing.
Sharing code is super simple between the 2. If you find yourself creating a general function that could be used in both, then you drop it in a lib/ folder that lives outside of those folders.
Overall this makes it very easy to skim a code base and see what it does. I've scaled this same pattern to having about 15 of those folders for distinct app functionality and hundreds of models, etc..
so if you lay out your code like you'd lay out your house your code will not be very DRY.
But I found the first motivating example (refactoring is hard) to be a poor argument in favor of cohesion. When making a breaking public API change, types and the ensuing type errors are going to be a way more powerful guiding force for identifying affected usage sites than cohesion. Of course, his examples were in Python, so maybe the author doesn’t have such luxury.
We could setup our code to self-organize under similar principles perhaps.
I had made an api controller, but then we needed to expand what our api covered to cover everything the app did (should have gone api first but again, my mistake, also partially inherited code base)
So of course I should have put the parts of the api that handled users into the user controller and the parts that handled images into the image controller and so on and so forth, but instead I thought (and here I can point to overwork dulling ones thinking processes) 'I have an api controller, this stuff now needs to be moved over from these other controllers into api controller' and in the end it became a horrible mishmash with some duplicated functionality I needed to fix later.