I disagree, I'd say the main reason has been pride. Otherwise we could have built reusable rockets in a similar fashion as early as the 1970s, maybe earlier.
There are a few reasons why this is "hard" but few of them are actually underlying engineering issues, most of them are at higher levels. A big problem is that, as I alluded to, the natural and traditional optimizations for expendable rockets lead you away from reusability. Those optimizations tend to push you toward lowering the number of engines on each stage and also concentrating more cost on the upper stages. Incremental rocket development tends to be easiest by improving the upper stage, because that takes lower upfront costs and is sometimes less complex (lower R&D costs, smaller capital investment, able to leverage existing launch and rocket processing infrastructure). But the result of those optimizations tends to be rockets (like the Delta IV, Atlas V, Ariane 5, etc.) that are difficult and costly to reuse (due to an inability to throttle the first stage thrust enough, among other reasons) and where reuse of the first stage, the easiest stage to reuse, is not advantageous because that's not where the costs are concentrated. Indeed, if you look at the Shuttle system, that involves reuse of the upper stage, at extraordinary cost and with extraordinary complexity. It takes intentional choice to design a rocket, like the Falcon 9, which is both highly amenable to first stage reuse and also where that reuse has the most benefit (due to that stage being most of the cost of the launcher). It's even more difficult to achieve that without being vertically integrated the way SpaceX is.
The Falcon 9 is fundamentally based on a 1950s era rocket design, and the main design goals aren't high performance or technical achievement but low costs. Using well-proven basic designs (LOX/Kerosene open cycle engines, two stage rockets) meant that SpaceX could concentrate a significant amount of their development efforts on working on reusability alone. That's actually surprisingly difficult for traditional rocket development orgs to do. For one, using a "simpler" basic design would seem like "going backwards". For another, doing the basic engineering to develop reusability would seem rudimentary. More importantly, any perceived failure would be very, very difficult for those orgs to accept. Experimental rockets are hard to get right, and often anytime things don't work exactly as expected on the first flight can easily be perceived by the public as a "failure". This was true even for SpaceX when they took several attempts to achieve first stage landings even though those stages already achieved their main purpose and normally they'd just get thrown away. NASA and traditional rocket makers are enormously prideful and sensitive about perceptions of failure.
This pushes those orgs to avoid "messy" development projects and to also tend to focus on "beyond the state of the art" developments. This protects those orgs from injuries to their pride because it helps avoid messy failures but also gives them an excuse if things don't work out, because they were working on something innately challenging. This is how we got the Shuttle, for example, which was a typical "beyond the state of the art" development effort. It's also how we got the X-33/VentureStar which relied on several key beyond the state of the art technologies.
It's also why we turned away from the DC-X design, which demonstrated VTVL sub-orbital flight in the 1990s. But something as simple and clunky as a 2-stage launcher using old rocket engine technology and lower stage reusability wasn't sexy enough for NASA et al back then, they had to have something like an RLV SSTO that was sexy enough to justify any risk of program failure.
If you could go back in time to the 1960s/'70s and tell them what to build you could absolutely produce a system similar to SpaceX's. It wouldn't have the same performance margins, it wouldn't have the same capabilities, it probably wouldn't be able to do barge landings, but you could absolutely build a reusable rocket using contemporary technology. And we'll see in the near future how many others copy the design once it's proven. The hard part isn't really the technology, it's the risk taking and the dogged pragmatism.
There are a few reasons why this is "hard" but few of them are actually underlying engineering issues, most of them are at higher levels. A big problem is that, as I alluded to, the natural and traditional optimizations for expendable rockets lead you away from reusability. Those optimizations tend to push you toward lowering the number of engines on each stage and also concentrating more cost on the upper stages. Incremental rocket development tends to be easiest by improving the upper stage, because that takes lower upfront costs and is sometimes less complex (lower R&D costs, smaller capital investment, able to leverage existing launch and rocket processing infrastructure). But the result of those optimizations tends to be rockets (like the Delta IV, Atlas V, Ariane 5, etc.) that are difficult and costly to reuse (due to an inability to throttle the first stage thrust enough, among other reasons) and where reuse of the first stage, the easiest stage to reuse, is not advantageous because that's not where the costs are concentrated. Indeed, if you look at the Shuttle system, that involves reuse of the upper stage, at extraordinary cost and with extraordinary complexity. It takes intentional choice to design a rocket, like the Falcon 9, which is both highly amenable to first stage reuse and also where that reuse has the most benefit (due to that stage being most of the cost of the launcher). It's even more difficult to achieve that without being vertically integrated the way SpaceX is.
The Falcon 9 is fundamentally based on a 1950s era rocket design, and the main design goals aren't high performance or technical achievement but low costs. Using well-proven basic designs (LOX/Kerosene open cycle engines, two stage rockets) meant that SpaceX could concentrate a significant amount of their development efforts on working on reusability alone. That's actually surprisingly difficult for traditional rocket development orgs to do. For one, using a "simpler" basic design would seem like "going backwards". For another, doing the basic engineering to develop reusability would seem rudimentary. More importantly, any perceived failure would be very, very difficult for those orgs to accept. Experimental rockets are hard to get right, and often anytime things don't work exactly as expected on the first flight can easily be perceived by the public as a "failure". This was true even for SpaceX when they took several attempts to achieve first stage landings even though those stages already achieved their main purpose and normally they'd just get thrown away. NASA and traditional rocket makers are enormously prideful and sensitive about perceptions of failure.
This pushes those orgs to avoid "messy" development projects and to also tend to focus on "beyond the state of the art" developments. This protects those orgs from injuries to their pride because it helps avoid messy failures but also gives them an excuse if things don't work out, because they were working on something innately challenging. This is how we got the Shuttle, for example, which was a typical "beyond the state of the art" development effort. It's also how we got the X-33/VentureStar which relied on several key beyond the state of the art technologies.
It's also why we turned away from the DC-X design, which demonstrated VTVL sub-orbital flight in the 1990s. But something as simple and clunky as a 2-stage launcher using old rocket engine technology and lower stage reusability wasn't sexy enough for NASA et al back then, they had to have something like an RLV SSTO that was sexy enough to justify any risk of program failure.
If you could go back in time to the 1960s/'70s and tell them what to build you could absolutely produce a system similar to SpaceX's. It wouldn't have the same performance margins, it wouldn't have the same capabilities, it probably wouldn't be able to do barge landings, but you could absolutely build a reusable rocket using contemporary technology. And we'll see in the near future how many others copy the design once it's proven. The hard part isn't really the technology, it's the risk taking and the dogged pragmatism.