The best legal writing has the same goal as an other form of expository writing - that is, in general, to communicate clearly and concisely with an intended audience.
Legal writing can (and often does) also carry with it the goal of persuading a target reader in some fashion and in this sense is best deployed using not only clarity and concision but also the tools of rhetoric, which can often be quite refined and which can but most often do not include obfuscatory tactics. The shyster model of lawyering says play for advantage to the disregard of truth. Carried to the extreme, it can include amazing tricks aimed at dissembling and hucksterism but it does not necessarily involve using language that is badly written - indeed, the finest forms of flim-flammery come from top legal echelons, where very high-priced lawyers are often paid to craft misleading arguments that will benefit their clients (including the government).
The so-called Darwinian theory alluded to in the OP, however, is not a good explanation for shoddy legal writing or, indeed, for legal writing that is not shoddy but rather unduly complex.
Many years ago, when much of law consisted of small-town practice and when very little information was inter-linked or easily accessible, hack lawyers would draft contracts using case digests and or similar third-rate resources (often not even written by lawyers) for guidance of what was or was not acceptable to the courts in matters that might eventually be litigated. That sort of practice got the defenestration treatment several decades ago and survives today primarily in narrow specialty areas (such as certain forms of insurance contracts) where "tested" language is sometimes used simply because it has been upheld by the courts.
In the vast portion of the business-law field, however, contracts are generally drafted with the aim of making them clear and unambiguous and a very good way to get fired at a large law firm (or any other firm that maintains high standards) is to put together a shoddy contract that shows you have no ability to state things clearly and unambiguously.
Of course, in a negotiation over a contract, one might try to slip in some "snake oil" language which has as its goal the aim of putting something over on someone. In such cases, a deliberate choice is often made to use ambiguous language in order to further that goal. A good lawyer on the other side will usually pick up on this and come back with cleaner language, but many times people miss the subtleties and someone gets had. This scenario, however, doe not define any significant portion of the contract drafting that is done but rather constitutes the exceptional case in customized deals that are negotiated in private and usually kept private. In the event such a clause is litigated, believe me, in modern law, it will not serve as a model for future drafting.
By far the greatest reason why legal drafting can become hard to read has to do with the complexity of the subject matter and the nature of the deal and not with any goal of obfuscation. For example, one can indemnify (hold harmless) another party with simple contract language that says, "I will indemnify and hold you harmless from all damages resulting from x." That same simple clause can be expanded to fill two or more pages of a single-spaced legal contract if enough is at stake and if the parties are motivated to protect their interests with a detailed contract that covers all the possibilities. In the example just cited, for instance, you can add elaborate clauses describing the exact types of claims against which indemnity will be provided and you can specify the types of claims against which it will not be provided. You can include notice requirements and specify consequences if they are not met. You can include caps and limits of various sorts. You can state that indemnity must be made when a claim is first asserted or you can state that it will only apply if there is a final judgment. These sorts of details can be replicated in great number, depending on how important it is to the parties to have them in the deal. This can easily lead to a fairly complex clause with a lot of legal terms of art in it, but the overriding goal is to keep it clear and unambiguous, not to obfuscate. This sort of contract language gets drafted every day and does not normally get litigated. It is an example of lawyering at or near its best, provided a deal warrants it. If not, you can always say "I will indemnify you from all claims" and leave it at that.
Thus, the point made in the OP is either wrong or at least vastly overstated as applies to modern business lawyering.
As an anecdotal example, when I purchased my house, the purchase agreement, loan documents, and associated paperwork were nearly a hundred pages for a standard purchase. I read all of it, and I had to ask numerous questions of my real estate agent before I thought I understood all that I needed. I think I can say as a simple matter of fact that I am more educated than an average person in the United States (I hold a bachelor's with some graduate level work completed and am currently applying to law school.) If I found it this challenging, then even if the intention was to be clear and unambigious it is likely to be highly confusing to an average purchaser.
On a slight tangent, since you mentioned the importance of rhetoric, would you have any advice on learning the art? You have previously recommended practice and study of the classics and of P.G. Wodehouse in particular as good ways to improve writing skills, but do you have any specific recommendations for rhetoric?
Concerning rhetoric, classical resources are vast, ranging from works by Aristotle to Cicero to many others. An excellent modern resource is Edward P.J. Corbett, Classical Rhetoric for the Modern Student (Oxford University Press 1965) (this work was updated later by editors who sought to make it "relevant" for the 1970s, and much diminished as a result - get the earlier edition through Abebooks if possible).
how's that working out for you?
Natural language, in a normal context, is (usually) co-operative, to the extent that the speaker and the listener both have the common goal that the listener understands what the speaker is saying. This is what gives natural language a higher tolerance for ambiguity, as the listener will be using generally amenable heuristics to discover meaning.
A programming language involves a human communicating with a generally imbecilic partner. This is like a game against a deterministic opponent, or even a single-player game, trying to optimize expression under the constraints of the language. This is what gives programming its literalness. Language designers try to solve the dual problem by designing the constraints. Arguably, the goal of a well-designed language is for the expression-under-contraints to approach the cooperative communication of natural language.
Legal writing is best modeled as a game against an adversarial opponent, which is actually not far from the truth. In a sense, it's also dual to language design, in that the goal is to design constraints that limit the expressive ability of an adversary.
While this is not explicitly mentioned in the OP, it's worth pointing out that bureaucratic writing is slightly different than legal writing, although it has a similar eliminate) the need for the judgement or ad-hoc decision-making of the individual contributors to the system, by spelling out the results of all possible decisions beforehand. This system may best be modeled as imagining an imbecilic adversarial opponent, rather than a powerful one, making it share the worst qualities of both legal contracts and FORTRAN.
- litigation, where the aim is to persuade a judge
- court opinions, where the aim is to persuade the parties and other judges
- I'd add law journals as well, which are written for peers
I'd say that the reason legal writing gets singled out is that people are expected to read and understand it, while people are not expected to read and understand equally difficult writings from different fields, such as something from a medical journal.
I would say that any form of persuasion benefits from imagining an adversarial opponent (in face, they're usually called a Devil's Advocate!). In this case, your judge you are persuading can be modeled as an adversarial opponent, characterized as being Maximally Skeptical of your position, within the constraints of still being Perfectly Rational.
You may also be confusing "adversarial" with "confrontational." An adversary can still be very dry. In fact, in game theory, they probably are very dry because they're assumed to be perfectly rational. Adversarialism defines the other participant's goals, not their approach.
Court opinions are not technically adversarial, but the same issues apply. It is virtually guaranteed that other parties down the line will try to misinterpret court opinions to get a judgement they are not entitled to.
It is true that journals are probably not subject to active intentional misinterpretation, but people that write legal journals are already lawyers and thus are already trained in that particular adversarial style of writing.
When you write a patent you have to explain a complex system in such a way that a non-technical judge or juror will understand it while someone else is actively trying to confuse said judge or juror by intentionally misinterpreting the explanation. It is not an easy task.
But I have to point out that one reason why legal language tends to be complex, is because natural language is very ambiguous. Thus, it takes a lot of careful writing to actually remove all the ambiguities from something. What often happens when one writes a contract is one writes a first draft and then starts thinking "how can this be misinterpreted or wiggled out of." And then one continues to add or change wording to prevent misinterpretations, and then finds new ambiguities with the new wording etc., etc. And in the end we usually end up with something long and rather difficult to read.
To put this in a software perspective, this is the same reason why there will never be a natural english programming language.