Are you asking why software engineers don't memorize the lookup complexity of various data structures and how to implement (e.g.) sorting algorithms? It seems like asking "Why do many civil engineers not know truss types and stiffness matrices?". That is, there are high-quality repositories of this information that can provide it on-demand and free up cognitive overhead for creative problem solving rather than rote memorization.
A lot of programming can be done by piping packages together and routing them to a frontend. I remember a post on AskHN where the OP couldn't even solve basic assignments his GF got in uni yet worked for years as a developer. So the reason is simple: you don't need to know them to get a job. Personally, I think this is a trend that's detrimental to software quality, but apparently demand drives costs up as well as it drives down quality. There are already replies that suggest theory is unnecessary. These people are wrong. I'm sure they wouldn't go to a surgeon that merely learned to operate a surging robot. Btw I don't think you need to get a degree to learn the theory.
> Btw I don't think you need to get a degree to learn the theory.
What are your recommendations for doing this? I've started getting into some kernel programming and its slowly helping me enjoy programming again, but I very often find myself getting lost in theory and other thing; however, after running some estimations going back to school is too expensive.
Data structures and algorithms were manipulated into a way to give IQ tests that can be studied for. Companies use these as hiring criteria and inflate the worth of memorized data structures and algorithms problems/knowledge. In short, they don't know it because mostly they don't need to, and there are obvious exceptions to that.
If you are talking about fundamental data structures and algorithms, then these are within the purview of computer science and anyone else who creates libraries, database management systems and the like.
High quality libraries and tools are hard to write and are optimised over a long period of time to provide reliable and fast answers within available resources.
Software engineers use these libraries and tools to solve business and other higher level problems. When they do work on data structures and algorithms, it related to the job at hand rather than needlessly duplicating existing good code.
There is just no need for them to worry about low level detail.
It is due to the abundance of open-source libraries. To some extent it has removed the incentive for developers to re-implement closed-source libraries and they just want to use the library and get on with their product. The need to look at the intricate data structures or algorithms is always an excercise for the developer reading the technical implementation which is worth looking at but not a hard-requirement in getting the product out there (unless specified). I would say it was essential in the 1990s era (OSes, drivers, etc) but for creating web apps in 2019? I don't think so.
It is highly unlikely that a company asking me to implement a data-structure is actually going to use it or is explicitly using it. Unless they want to improve it in a research paper or fund a proof-of-concept that improves the state-of-the-art. My point here is that we as software engineers should know these DS and algorithms, but to companies who ask such questions in a interview, you should be prepared to use them explicitly.
I'm my experience they do. They know the cost of using a list instead of a hashmap, know the difference between a bubble sort and a quick sort, know the difference between a binary tree and a linked list. And for 99.9% of the job this is more what you need. I wísh they know more about composition instead of inheritance...
As a self taught developer, where should I start with this? I feel like I have gaps in my knowledge and would like to learn more without being overwhelmed. I like the web because it's easy to make things, not because of maths
Do you need it for the type of work you want? If not, you probably don't need to learn it.
That being said, if you want to get into CS, I recommend getting a textbook or two about computational complexity (big-O notation) and the theory of computation (nondeterministic finite automata and regular expressions, stack machines, and Turing machines). I liked "Theory of Computation" by Michael Sipser for the second topic (didn't like my textbook for the first), though it's very dense so you may want to look into additional material if you want a gentler introduction.
Honestly, you can probably find everything online. The main things to look for are:
- big-O notation (how many steps it takes to solve a problem given some input size)
- data structures (linked lists, b-trees, and hash maps are the big ones)
- the list of terms above
Work out some proofs until you feel comfortable with the material.
But it really doesn't matter of you just want to do web stuff. Most of the hard problems have been solved by the standard library or database you use, so unless you want to work on those, you can choose to be ignorant of the inner workings. I'm the type that doesn't like to not understand something, but I have plenty of friends who care more about other things than learning CS/EE stuff that isn't practical to their line of work.
Because at least 50% of software work doesn’t require knowing them. And of the remaining 50% at least 50% can be done with suboptimal ds/algos. But I agree everyone should know the basics. (Beyond basics is nearly worthless.)
I've been traveling around the world earning money as a freelance software developer. I have had no need to use any data structures or algorithms to complete my work.
So far, I've been paid to ship machine learning tutorial content for an education startup. I got the client through upwork.
Think of all the people that know less about site administration/programming than you. These people would rather spend their time working on their business and pay you to work in it.
It also helps that cost of living in South America/SEA is much lower than the US. I was paying $150/month rent in Medellin, Colombia, for example.
Can you expand on what data structures and algorithms specifically you're talking about? And what is your use-case for needing to know these by memory?
Knowing it exists at all is the long tent in the poll. If I need some way to get my next item based on some form of priority, knowing about heaps and priority queues helps me not build something kinda similar that is objectively worse.
Sure, knowing that there's something out there to look for might be helpful. (Might not, too. Googling "how do I remove items from a list by priority" might get you info on priority queues.)
But if I know that "priority queue" is what I'm looking for, I can then totally forget how to implement a priority queue. I can look it up as many times as I'm going to need it in the rest of my career faster than I can memorize it. (And if I need it very many times, I'll memorize it just by using it.)
