I catch myself just throwing everything in inappropriate data structures myself these days, because the hardware lets me get away with it, and many recent languages makes hashes so ridiculously simple to work with and don't provide much more than that + arrays.
I guess that's kind of the problem though right? The hardware lets you get away with it, only until it doesn't. Then if you don't know the fundamentals, you're in trouble or you end up spending an inordinate amount of time and money trying to spread the task across dozens of systems in a couple racks of in some datawarehouse somewhere.
And then the blame often gets shifted to things like the language or framework and rewrites start being bandied about instead of analysing algorithm usage because the problem often isn't apparent when looking at profiles etc. unless you're used to looking for exactly this class of problems.
And people start talking about "Big Data" and Hadoop and similar when their data are really middle of the tree and can easily fit on a single server..
I'm at a company now where our default deployment is something like 6 fairly heavy VMs. You can jam it all into 4 lighter VMs and service a single user (which is what we do for testing on our laptops) and it runs..."ok".
Looking at what they're actually doing, basically a few postgres or solr requests, file serving and generating a some web pages...the data for one of our larger deployments barely breaks 1 GB when dumped out of the databases (It doesn't handle more than 100 users at a time)...it should be running cleanly off of a $699 mid-range all-in-one desktop from Costco and have enough headroom for 5,000 more users.
Yet somehow between all the framework, JVM and various other nonsense, there's murmers about moving that system to a more web-scale infrastructure because of all the performance issues we're seeing. Everytime somebody uploads some content to the server 4 of the VMs peg out their cores for several seconds. As I look around in engineering meetings at all the fresh young engineers working on this stuff, nobody seems to think this is a problem.
On the client side the problem is even worse, constrained to the 1990s era performance we're stuck with inside of browsers.
Machines capable of crunching billions of calculations per second, and putting tens of billions of bytes into main memory are brought to their knees dragging and dropping a half-dozen items in a GUI (something my Commodore 64 could actually handle) or running two programs at once, something my Amiga 500 could do.
It's bad and it makes me feel bad. I think growing up when computers were basically terrible has broken me to where things that young engineers think is cool and amazing I'm finding more and more disappointing...it's not that old computers were great, it's that modern computers aren't but they should be.
I'm every now and again hacking on AROS - a reimplementation (and in some ways extension) of the AmigaOS API's.
The tragicomical part of it is that "booting" the Linux hosted version of AROS (there's also a native version) on my laptop, with a custom Startup-Sequence that starts FrexxEd (a scriptable "emacs like" editor, but without - by default at least - Emacs keybindings) lets me "boot" AROS straight into FrexxEd in a fraction of the time it takes Emacs to start on my laptop...
Of course Emacs is kinda famous for being a resource hog, but anyway - FrexxEd can run comfortably in 1MB of RAM, and that's including a bytecode compiled C-like extension language and full AREXX integration.
I'm wondering what I can do to either get a smoother integration between AROS and Linux, or run it "full time". There are a few missing pieces still holding me back.
But it still irks me that mainstream OS's lack so much that AmigaOS has (e.g. FrexxEd exposes all open buffers as files, so you can e.g. use diff and patch between your last saved version of a file and the in-memory buffer, or run gcc on the in-memory buffer to get an error report into another buffer. Of course you can do the same with tmp files, but why should you need to?)
I don't like it, though.