I think it's a bit of both, and varies significantly with the domain.
Quick, how do you write an SMTP server, and what are the challenges of making it scale?
Most developers won't know how, to start with. That's fine - that's besides the issue. So they need to look it up.
Here starts the performance gap, even if you deal with people with the same lack of knowledge of the relevant RFC's.
In my experience, there's a vast difference in developers ability to read even relatively simple specs and ensure they develop something that follows it. I mentioned SMTP because it genuinely is a simple standard compared to many of the alternatives. But it has enough edge cases that you'll have a big gap out of the gate instantly between the people who have a problem mentally picturing what the spec is describing and those that can easily and systematically map it out.
Secondly in this case you'd start to see experience gaps. Even assuming most people won't have written an SMTP server, you will start seeing a gap between a group of developers that at least have in-depth knowledge of part of the domain or type of service. That will account for a very substantial difference.
In this case, understanding on how to write efficient, scalable network services makes the difference between the guy that will do horribly inefficient stuff like read()'ing a byte at a time to get a line from the client (I mention this because the MySQL C client libraries did that for years instead of the vastly more efficient solution of non-blocking larger reads into a temporary buffer to avoid the context-switches, so it's not like this is something that only rent-a-coder's with no experience will do).
Just the gap between those that understand the tradeoffs of context switches and threads vs. processes vs. multiplexing connections will account for a fairly substantial factor in many types of problems like this.
Then comes the thorny issue of queues. Most otherwise relatively competent developers will struggle to get this right in a way that is not either slow or full of race conditions. Most competent developers never have to deal with really optimizing disk-IO. Witness the wildly different approaches and performance in established mail servers to see that doing queueing well is hard, and those are the good ones.
That does not mean they won't be able to figure out how to do it well enough for typical use cases.
(I used this example, because I've written several SMTP servers, and managed teams working on mail software, so it's an area where I know the tradeoffs and problem points particularly well)
Then again, when writing your typical cookie-cutter web app, the difference probably won't be 10x because so much more of the time will be spent mediating stakeholder requests vs. solving hard problems.
Quick, how do you write an SMTP server, and what are the challenges of making it scale?
Most developers won't know how, to start with. That's fine - that's besides the issue. So they need to look it up.
Here starts the performance gap, even if you deal with people with the same lack of knowledge of the relevant RFC's.
In my experience, there's a vast difference in developers ability to read even relatively simple specs and ensure they develop something that follows it. I mentioned SMTP because it genuinely is a simple standard compared to many of the alternatives. But it has enough edge cases that you'll have a big gap out of the gate instantly between the people who have a problem mentally picturing what the spec is describing and those that can easily and systematically map it out.
Secondly in this case you'd start to see experience gaps. Even assuming most people won't have written an SMTP server, you will start seeing a gap between a group of developers that at least have in-depth knowledge of part of the domain or type of service. That will account for a very substantial difference.
In this case, understanding on how to write efficient, scalable network services makes the difference between the guy that will do horribly inefficient stuff like read()'ing a byte at a time to get a line from the client (I mention this because the MySQL C client libraries did that for years instead of the vastly more efficient solution of non-blocking larger reads into a temporary buffer to avoid the context-switches, so it's not like this is something that only rent-a-coder's with no experience will do).
Just the gap between those that understand the tradeoffs of context switches and threads vs. processes vs. multiplexing connections will account for a fairly substantial factor in many types of problems like this.
Then comes the thorny issue of queues. Most otherwise relatively competent developers will struggle to get this right in a way that is not either slow or full of race conditions. Most competent developers never have to deal with really optimizing disk-IO. Witness the wildly different approaches and performance in established mail servers to see that doing queueing well is hard, and those are the good ones.
That does not mean they won't be able to figure out how to do it well enough for typical use cases.
(I used this example, because I've written several SMTP servers, and managed teams working on mail software, so it's an area where I know the tradeoffs and problem points particularly well)
Then again, when writing your typical cookie-cutter web app, the difference probably won't be 10x because so much more of the time will be spent mediating stakeholder requests vs. solving hard problems.