
Top lawyers beaten by legal AI - eaguyhn
https://hackernoon.com/20-top-lawyers-were-beaten-by-legal-ai-here-are-their-surprising-responses-5dafdf25554d
======
nickelcitymario
Having worked in a law office, I can say confidently that the future of legal
AI is human+AI. AI can rapidly spot all the issues, saving expensive lawyers
from spending their time wading through the documents. But the human lawyer
brings strategy, judgement and face-to-face consultation to the process.

In other words, I don't think lawyers are in danger. But law clerks? Junior
staff? The people who are usually forced to do all the grunt work to allow the
top lawyers to focus on their strengths? Those jobs are at definitely on the
chopping block. There's still an insane amount of rote repetitive work in your
standard law firm.

~~~
atonse
What I've heard (but many years ago) is that the legal profession in general
is extremely tech-hostile (some is just cultural, but some is that it gets in
the way of billable hours).

I don't know how much of that has changed now, especially with younger lawyers
growing up with tech, getting in and wanting to do things a certain way.

~~~
avinium
I'm (somewhat) qualified to comment, having presented to an audience of
lawyers in SF literally hours ago, and having been discussing/testing the
market for most of this year.

I think there's roughly three groups - the old guard (senior partners at big
firms), the younger ones who have gone out on their own, and the in-house
counsel.

In my opinion, the old guard aren't tech-hostile, just disinterested. They
haven't ever _really_ needed to compete, and most have retirement on the
horizon, so there's little incentive to invest money for a pay-off in five
years' time.

Corporate counsel are still pretty unsophisticated when it comes to new
tooling/platforms, except when it comes to cost management (where there's an
enormous amount of interest).

The most tech-savvy are the smaller (3-30) firms who have broken off and are
looking for ways to compete with their former employers.

That being said, it's still a very hands-on, enterprise-y market to sell to.
I've been told that LawGeex, for reference, start at the (multiple) six-figure
range to begin with.

We ([https://lexico.io](https://lexico.io)) didn't really want to go down that
path, so we pivoted from automating legal drafting and review inside Word, to
a broader paperwork automation assistant.

------
kemitchell
Time wasted reviewing pointless variations on the same old mutual NDA is a
scourge. It's hard to take the profession's performative technophilia
seriously while the most obvious, widespread inefficiency continues every day.

On the other hand, teaching computers to parse pointless variations of the
same substantive terms in legalese accommodates bad practice, instead of
correcting it. It's symptomatic treatment at best, enablement at worst.

Lawyers shouldn't be sending and receiving a thousand different NDAs for the
same fundamental deal. They should be defining and evolving a shared
vocabulary of common forms, and invoking them whenever cost-benefit favors a
standard.

That's the idea behind a very recent project of mine, The Canting Tribe NDA:

[https://nda.cantingtribe.com/](https://nda.cantingtribe.com/)

It's a "viral" NDA. The website publishes a versioned form. The form includes
a signed certificate guaranteeing that the proposal is identical to the
version published on the site. On the site, newcomers see a list of firms
using and accepting the form. The experience _shows_ , rather than _tells_ ,
how standardization can work.

I soft launched the project just a couple weeks ago. A couple firms signed on
almost immediately. Several more are reviewing. My top priority is more firms
on the website. Once the site provides enough baseline social proof, the form
can spread word and create new users on its own.

~~~
SturgeonsLaw
When it comes to lawyers notoriously slow uptake of technology, the cynic in
me says that one man's inefficiency is another man's billable hours

~~~
cylinder
The biggest issue is clients aren't demanding it, they don't want to leave
Outlook to do anything.

~~~
kemitchell
Funnily enough, some _lawyers_ demand it.

I, for one, really hate interrupting my workflow to read yet another MNDA form
on short notice, only to find nothing warranting comment. Meanwhile, my
client's deal is held up, waiting on me, which makes me seem a drag on Getting
Business Done.

Businesspeople have to wait for review, but lawyers have to do the review.
Both come away knowing that they've wasted precious time.

------
caffeine5150
I am a transactional lawyer and I definitely would find value in an
application that could issue spot an agreement in seconds. That said, just
yesterday I spoke on a panel on the topic of how things can go wrong in a
contract. We spent the majority of time talking about the dynamics and
challenges that exist outside the agreement in the process of trying to
memorialize the parties’ intent in a clear, concise, precise and reasonably
complete manner. There are often significant challenges in terms of clearly
obtaining the intent and relevant issues from the various stakeholders. And
there are dynamics like relative negotiating leverage and psychology or other
issues that can drive what the deal will look like regardless of pure legal
issues. Also, since one never starts with a blank page, there is the contract
template one starts with that must be evaluated against all this – what stays,
what goes, what must change and how. Navigating these requires intangible
skills, instincts, sensitivity to human dynamics, etc. It’s very much a human
endeavor. So a key question is to what extent AI could help with all of these
external issues. I have to think that’s much farther down the road. But having
help assessing purely legal issues within the document would be a great
supplement.

For a good thread on training to be a lawyer through apprenticeship, see this
[https://news.ycombinator.com/item?id=16255023](https://news.ycombinator.com/item?id=16255023)

------
dctoedt
With an AI, you don't know what it is that the AI _doesn 't_ know. A trivial
example: Some "nondisclosure" agreements (NDAs) also include invention-
assignment agreements [0] and/or non-competition covenants. _I 'll_ recognize
such provisions if I see one in a putative NDA, but I don't know whether the
AI will recognize it. Sure, the AI could provide a list of everything that it
_does_ know, but when I review the list, I won't necessarily notice that a
missing item is missing.

[0] Stanford University got trapped by an invention-assignment agreement
contained in an "NDA" that one of its researchers signed when he visited
someone at Roche — see
[https://en.wikipedia.org/wiki/Stanford_University_v._Roche_M...](https://en.wikipedia.org/wiki/Stanford_University_v._Roche_Molecular_Systems,_Inc).

~~~
caffeine5150
This is an important point, although I can imagine AI could be trained to spot
things that don't belong or are unusual provisions in an NDA. This may be what
you're getting at, but it's easy in a contract review to focus on reacting to
what's there and it's harder to know what's missing. An NDA is a relatively
simple, cookie-cutter types of agreement with widely agreed elements. Other
agreements not so much. How would AI figure out what's missing?

~~~
dctoedt
> _How would AI figure out what 's missing?_

That's an important point, but I assume an AI would use some sort of
checklist, perhaps seeded by human lawyers, to spot expected-but-missing
provisions.

------
tshadley
Statement from Professor Yonatan Aumann, advisor to Tel Aviv-based LawGeex, on
LawGeex AI:

“The technology has been developed through a combination of supervised and
unsupervised learning techniques. Unsupervised learning was used for teaching
the AI engine the core legalese language. Thereafter, supervised learning,
using deep learning multi-layer LSTM and convolution technology, was used to
train the system for the fine-tuned issue-spotting. Supervision was performed
based on human-annotated documents, using legal experts. A unique augmentation
algorithm was applied to boost learning from these examples. The overall
result is the most advanced technology for the automatic analysis of legal
documents. The p-value for the statement that accuracy of AI is above that of
these lawyers is 0.0068 (using MannWhitney’s U test).

[https://images.law.com/contrib/content/uploads/documents/397...](https://images.law.com/contrib/content/uploads/documents/397/5408/lawgeex.pdf)

------
joshuaheard
Clicking on the link, I expected to discover a watershed moment akin to AI
beating a human at Chess or Go. Alas, since contract review is not an
adversarial process, no human lawyers were "beaten". A less click-baity
headline would be: AI slightly more efficient at issue-spotting contracts than
lawyers.

~~~
Digory
And "issue spotting" here means "identify if there are unusual clauses in this
NDA."

When lawyers talk about "issue spotting," they usually mean "given these
facts, what legal principles apply?" This is not that.

~~~
kemitchell
> When lawyers talk about "issue spotting," they usually mean "given these
> facts, what legal principles apply?" This is not that.

Accurate.

------
overthemoon
The labor cost savings aren't going to make it to clients. I've worked in a
law office, I know how billing goes. Those extra billable hours will be found
elsewhere. I hope I'm wrong, and maybe I'm just cynical, but my opinion of
lawyers and the legal profession in general took a hit after 3 years of
working in it.

~~~
scotty79
You are right. Price of every item has nothing to do with the cost of making
it and everything with how much people agree to pay for it.

Driving down the cost of making a thing doesn't lower the price for everyone,
but it widens the market downwards serivicing new el-cheapo customer base.

------
sandworm101
>> It took the lawyers an average of 92 minutes to complete the NDA issue
spotting, compared to 26 seconds for the LawGeex AI.

Issue-spotting an NDA is not a typical task for an actual lawyer. It is the
sort of thing handled by a legal intern or paralegal... then signed off on by
a lawyer. It isn't typical legal work. The results are therefor of limited
application in the real world.

Better test: Client asks "Do I need an NDA?" or, conversely, "can I break this
NDA I signed last year?". Ask C3PO to handle that question. That is the sort
of problem where the real legal expertise happens.

~~~
kemitchell
> Ask C3PO to handle that question.

I had a good laugh here. Thanks!

------
DanHulton
I just joined a company last year that works in this space almost exactly -
due diligence contract review - and we are hearing this pretty constantly from
people who evaluate our software. Not only is it much faster, but frequently
it finds clauses that they had missed themselves.

It's a pretty exciting time to be working in this space.

------
YeGoblynQueenne
Can I be a skeptic? The study was carried out by the people who sell the
software. How reliable is that?

------
casper345
"Top mathematicians were beaten by calculator AI that computes equations
faster"

------
RichardBurtLaw
NDAs tend to be a straightforward type of document with limited scope and a
fairly well-known universe of issues. So having a computer program parse the
document and itemize issues seems like a good idea.

But what does it mean that the AI “beat” the humans? That the program simply
spotted the issues and listed them faster and more completely than the
lawyers? Not that impressive a victory. It’s how the issues get resolved that
are important.

Did the AI interact with a lawyer on the other side and get issues resolved?
Not likely, but the story doesn’t tell us exactly what the test consisted of
and how “victory” was determined. (It is possible to download the 40-page
report and analyze it, but I am not willing to spend the time to do that, and
I bet most commenters, like me, are just responding to the headline take-
away.)

While AI can be a great tool, one of the problems with it is that often no one
knows how the program makes decisions. Not knowing what you don’t know can
present serious blind spots. For example, it may be possible for a counter-
party to a transaction who has access to the same AI as you to construct a
document with text containing an issue that the AI won’t spot but that the
counter-party planted in the document for his or her advantage. If the
document is reviewed only by AI (and given a cursory review by a bored low-
paid worker who expects AI to catch all the issues), then there is an open
back door to your legal castle.

Computers haven’t put accountants out of business, and they won’t put lawyers
out of business either. They will make lawyers more efficient, but whether
that translates into lower fees remains to be seen.

~~~
kemitchell
Greetings from up north, in Oakland.

Your points on "beating" and decision making are very well taken. Same on the
resilience of the professions. My own take here is basically that it's
accommodating inefficient behavior---a thousand implementations of the same
basic deal terms---rather than correcting it. The cure for the common NDA is
standardization, which is something the profession can do for itself.

Along those lines, I recently launched an initiative to stop pointless waste
on review of routine MNDAs:

[https://nda.cantingtribe.com/](https://nda.cantingtribe.com/)

The idea was not to write the world's greatest NDA, or to offer whiz-bang
features with software, but to create a boring, unsurprising, neutral NDA with
a distinctive name that spreads word about itself, creating opportunities to
sign routine NDAs without waiting for redundant legal review.

Would love to read your thoughts, here or by e-mail.

~~~
RichardBurtLaw
I have not carefully read the entire NDA, but I offer the following comment.

If a recipient of information protected by an NDA is a legal entity, such as a
corporation, Section 4 of your form requires the recipient to enter into a
confidentiality written agreement with each employee (and others working for
the recipient) who receive information protected by the NDA. This is both too
much and too little!

It is too much because it is burdensome on the recipient to require employees
to sign a written agreement every time the corporation receives receive
information protected by an NDA. There might be 20 employees who need to see
the information, and keeping track of who has signed and who hasn’t and making
sure that an employee who has signed doesn’t share information with an
employee has not yet signed is an administrative burden.

I don’t ask for this provision when representing the disclosing party, and I
would resist this when representing the recipient.

In any event, how does the disclosing party police this requirement?

It is too little because the recipient fulfills its obligation by entering
into a written agreement with the employee as required by your form of NDA. If
there is a written agreement with the employee and the employee improperly
discloses the information protected by the NDA, the recipient points to the
written agreement and says, “I did what you asked.” So why would the recipient
be liable for the wrongful disclosure by its employee?

To whom is the employee liable if the employee improperly discloses the
information protected by the NDA? Presumably the disclosing party would want
to sue based on the fact that the employee signed a written agreement agreeing
to keep the information confidential, but unless the disclosing party is a
third-party beneficiary of the recipient’s written agreement with the
employee, the disclosing party has no rights under that agreement.

In the absence of an express statement in the NDA that the recipient’s
agreement with the employee is intended for the benefit of the disclosing
party (i.e., the disclosing party is a third-party beneficiary of the
agreement), it is a litigable issue whether the disclosing party a third-party
beneficiary. There is nothing in the NDA form that requires the recipient’s
agreement with the employee to contain such a clause.

In some situations, both the disclosing party and the recipient might prefer
not to identify the source of the information protected by the NDA.

Even assuming that the written agreement with the employee allows the
disclosing party to sue the employee for wrongful disclosure based on that
agreement, does the employee have adequate resources to pay a judgment?

Instead of requiring such agreements between a recipient and an employee, it
might be more effective to state in the NDA that the recipient is required to
instruct its employees on the confidential nature of the information, that the
recipient is responsible for any disclosure of information by its employees in
violation of the NDA, and that the recipient will be liable for any damages
resulting therefrom.

~~~
kemitchell
Thanks very much for this comment. I'm glad I looked back on my threads page
and saw it!

First, I should make clear that I did _not_ set out to write the very best NDA
possible. Rather, I set out to write the NDA that I thought companies and
their counsel would find the quickest and easiest to approve. In other words,
to reflect current practice, which we might both to agree falls well short of
optimum.

Alas, my experience from prior projects is that better terms don't provide
enough incentive to standardize. Everybody likes their NDA, and the difference
between OK and better isn't compelling. But my hope is that if we standardize
otherwise, through CT's self-propagating mechanism, that standard can then
become a platform for better terms.

Now to your points on employees and confidentiality.

The covenant to sign NDAs with employees is essentially a commitment to best
practice. The parties are confirming that NDAs, or more likely CIIAAs, are
part of the hiring packets for both sides. Note the exception for
professionals under non-contractual confidentiality obligations, like lawyers.
That tracks reality, in my experience.

You are correct that absent express language, and without privity, disclosers
could end up without direct claims against breaching employees. That's an
absolutely fair point, though I'd hasten to add that the company, rather than
its employee, is usually the deep pocket, and an injunction to the company is
usually what's needed to stop any bleeding.

The terms addressing employer responsibility are in section 4(m) of version
1.1.0:

> (m) Compliance and Oversight. > > (i) Receiving Party shall ensure that its
> Advisers abide by the confidentiality obligations of Receiving Party under
> this agreement. If Receiving Party is a legal entity, Receiving Party shall
> also ensure that its Personnel abide by the confidentiality obligations of
> Receiving Party under this agreement. Breach of Receiving Party obligations
> by Receiving Party Personnel or Receiving Party Advisers will be deemed
> breach of this agreement by Receiving Party itself. > > (ii) If Receiving
> Party is a legal entity, Receiving Party shall provide Disclosing Party
> copies of confidentiality agreements with Personnel who receive Confidential
> Information on Disclosing Party request.

Again, thank you very much. Writing what you did took time, I know.

If you'd like to discuss further, please do e-mail me. Otherwise, I can't be
sure I'll see it. I'd be more than happy to make time for a call, or arrange a
meal, coffee, or libations next I'm down to San Jose from Oakland.

------
mv4
I've always felt that a good contract should look like code - that it should
"compile" and "run" without errors.

~~~
Skrillex
I can imagine system with drag and drop sentence fragments and modifiable
variables designed for the construction of contracts. When a document is
finished, it would produce a summary of the consequences with as few
interpretations as possible, possibly with a toggle for Wikipedia-style
keyword linking in the result. The document could then be tweaked until the
summary exactly matches what both parties want, at which point the _document
and summary together_ become the contract. Although, if such a system were to
become widely used, I feel like lawyers will still be used to either discover
or introduce any other remotely plausible interpretations that exist in the
generated contract.

------
pmlnr
There's a plethora of science fiction that using only AI in court _will_ go
wrong.

~~~
atrus
There's also a plethora of writings showing that using only using humans in
court _will and has_ go wrong.

There's probably a plethora of writings showing that using _both_ humans and
AI _will_ go wrong.

The question isn't _will_ they go wrong (they will) it's a question of will
they do it less.

------
jacobkg
Is this technology somewhat akin to a code linter? It automatically finds that
problems that would be tedious for humans to find and greatly speeds up the
work of the human expert?

------
buboard
Lawyers have traditionally been able to secure their own payout by law, so
they ll just thakn you for the extra free time , but will this lead to actual
reductions of legal costs?

~~~
kemitchell
I'm a solo transactional attorney, and I launched a "viral" NDA designed to
get routine deals signed without legal review a couple weeks ago:

[https://nda.cantingtribe.com/](https://nda.cantingtribe.com/)

I don't want another dime for reviewing standard mutual NDAs. There is way,
way better work for me to do.

------
browsercoin
this is interesting point in time for me....im thinking about becoming a
lawyer....im caught between my love for machine learning and deep learning and
the desire to practice law.

------
pcunite
Computers can compute numbers faster than 20 professors too ... can we please
stop the _AI is everything_?

~~~
njharman
Write AI that better curates news items.

------
ax0ar
The green header is very repulsive and distracting. I found it difficult to
read the story.

~~~
theonemind
Firefox's Reader mode works pretty well for things like that.

uBlock Origin also has an eye-dropper to pretty easily block selected
elements. You can typically remove floating headers like that.

This bookmarklet also usually kills sticky headers:
[https://alisdair.mcdiarmid.org/kill-sticky-
headers/](https://alisdair.mcdiarmid.org/kill-sticky-headers/)

Anyway, common problem. You can probably find a stock solution.

------
known
Soft skills?

------
paulgrant999
"Predictive Coding".

You don't need AI.

You just need collusion between law firms i.e. to write a program, and then
agree, that whatever it interprets, is the way it will be semantically
enforced in court.

To be clear: the study of law is bullshit; so why not take advantage of it,
and simply agree, that computer generated bullshit, is what determines it...

... of course while still retaining high rates and billable hours for
"attorney review" (for billable hours/padding).

~~~
paulgrant999
and once you get the Courts ok'ing it, then why do you even need lawyers or
courts?

If no-one is going to look at the actual evidence/agreements/law....

------
caffeine5150
51-156 minutes is crazy to review an NDA. If an NDA is well drafted, I can do
it in about 10 minutes. If it's a bit of a mess, maybe 30 min. If it's worse
than that, I can assess that in about 5 minutes and propose using a better
form.

~~~
cf141q5325
Its the number for all 5 NDAs combined, so fitting exactly with your numbers.

~~~
caffeine5150
Ah, thank you for the correction.

