
The Paradox of Source Code Secrecy (2019) - BerislavLopac
https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3409578
======
gazzini
The gist:

IP law lets software have the best-of-both-worlds. Source code is patentable &
protected by copyright, but is also a trade-secret (because it's not publicly
disclosed).

This is problematic when these black-box algorithms are used to assist in the
processing of legal cases.

\-----

My 2 cents:

\- The overall concern that our lives are being governed by the invisible hand
of black-box abstractions really resonates with me. Richard Stallman has been
warning us about this for a while [0], and I've personally started considering
my role in this as a software engineer [1], although I don't have any
actionable advice yet.

\- I don't feel as strongly that all of this code should be open-source. I
think it might be sufficient to just strictly define a series of tests --
measure the inputs & outputs, describe the requirements... that kind of thing.

It's inherently difficult whenever there's a constantly-evolving ML model
behind the curtain. It reminds me of the issues with using AI in the aviation
industry, for instance [2]. As I recall from my undergraduate classes,
aviation control systems have been unable to use "fuzzy" logic to pilot planes
due to strict deterministic testing requirements.

[0] [https://www.gnu.org/philosophy/free-software-even-more-
impor...](https://www.gnu.org/philosophy/free-software-even-more-
important.html)

[1]
[https://gazzini.com/essays/posts/tools/](https://gazzini.com/essays/posts/tools/)

[2] [https://www.nlr.org/areas-of-change/introduction-non-
determi...](https://www.nlr.org/areas-of-change/introduction-non-
deterministic-approaches-nda-artificial-intelligence-self-learning-aviation-
systems/)

~~~
Cerium
For actionable advice, I believe that the government should make it a first
choice priority to use and improve open source software. Open software is
digital infrastructure and critical in the success of our society.

~~~
ChuckMcM
I largely agree with this. I have suggested in the past that the mission of
the US Digital Corps be amended to include the development and maintenance of
infrastructure critical software in a transparent way.

Imagine for example if the software used in voting machines was managed by
such a system. Always published, always analyzable, maintained and
authenticated by people whose job is only to make sure it does what it is
supposed to do.

------
buzzkillington
>Today, the government relies on machine learning and AI in predictive
policing analysis, family court delinquency proceedings, parole decisions, and
DNA and forensic science techniques, among other areas, producing a
fundamental conflict between civil rights and automated decisionmaking.

Most of the methods used for this are basically large matrices with no human
understandable way to represent them. Getting the values would only let you
recalculate the results, not discover why the results were produced.

~~~
krona
Climate models and recent epidemiological modelling (outliers in terms of
their effect on public policy making) are far simpler than say the ML this
article seems to suggest. But you don't need a human-understandable
representation to understand their limitations.

~~~
p_l
Climate models don't use ML, similarly, most sensible epidemiological
modelling doesn't.

ML is, afaik, considered too much of "curve fitter" to actually simulate or
analyze anything novel instead of rehashing of past events, making it useless
in cases like this.

------
D13Fd
In my view, the fundamental problem is that software doesn't fit neatly into
either the patent or the copyright regimes.

Patents are not supposed to cover abstract ideas. In reality, all software is
abstract.

Copyrights are supposed to cover an expression of an idea. It is not meant to
cover inventions or functional devices.

Patent terms are relatively short, 20 years from filing, because technology
needs to advance. It's great that you invented a drug to cure cancer, but you
shouldn't be able to own it forever.

Copyright terms are functionally infinite (currently the life of the author
plus 70 years, or 120 years for corporate works, but likely to be extended
going forward).

It doesn't make sense that software is most similar to inventions protected by
patent, but gets the essentially infinite term of protection of copyright.

I would propose that Congress pass a law limiting the term of copyright
protection for software (object code, source code, or otherwise) to 20 years
from date of creation. That puts it in a middle ground between
patent/copyright, where it should be.

Trade secret is fine as it is.

------
x3blah
For those not using Javascript (SSRN uses third party scripts), link to PDF:
[https://poseidon01.ssrn.com/delivery.php?ID=5550640311191050...](https://poseidon01.ssrn.com/delivery.php?ID=555064031119105004095090069123012096018036019060022069010066122091125090029069004088027107042044056060004120006096108020007111107084089093088002103070121101024110091077082121019107094019001015013087122096098070066086118083116112093099022075122123026&EXT=pdf)

------
gerland
Putting aside the topic of how easy is it to understand a trained ML model, I
really hope we won't end up in a SEO-like situation.

~~~
umvi
> Putting aside the topic of how easy is it to understand a trained ML model,

I wouldn't say it's _easy_ to understand a trained ML model. If you don't have
access to it, it's a black box that can be hard to penetrate. But if you _do_
have access to it, it's fairly easy to figure out how to _trick_ it into doing
what you want.

Example scenario: in a dystopian 2030 America, judges are replaced by ML
models where you just input a bunch of parameters and the model spits out the
judgment. Soon:

Divorced Husband: "Why do you keep calling the non-emergency police line to
concern troll?"

Divorced Wife: "So the ML judge will give me custody of the kids"

(number of calls to police as an input to ML model is being gamed to alter
outcome in one party's favor)

~~~
TheOtherHobbes
You've just invented a new service industry: AILegal, a combination of legal
expertise to play the human side, and ML expertise to play the machine side.

Probably available by subscription for a monthly fee that corresponds to the
probability of a positive result.

~~~
lliamander
The court system will likely retain the use of human judges. More likely this
would be used as a cheaper means of arbitration.

