
Why do we need modules at all? (2011) - tosh
http://erlang.org/pipermail/erlang-questions/2011-May/058768.html
======
msluyter
Whatever you think of the idea, I think the reasoning here is interesting and
instructive.

1\. Start by questioning a long held assumption ("Do we need modules? Why? Is
it because we stored functions in files?" Etc...)

2\. Consider alternatives / what ifs. ("What if we used a database instead of
files?")

3\. Think through implications of the alternatives ("How would we identify
functions?")

IMHO, this is a powerful technique for generating new ideas. Step 4, not
explicitly performed here, might be to take ideas generated from steps 2 & 3
and apply them to other domains. ("Ok, perhaps a database of functions
wouldn't be great for code, but perhaps it (or something similar) would be
great for some other domain...")

~~~
ajavascriptdude
and voila, serverless is born

~~~
Anonymous4C54D6
Serverless is about running functions this is about distributing their code. I
agree that there is some overlap but different problems are being solved.

------
pjc50
This is one of those ideas that sounds very tempting at the small to medium
scale but will definitely fall over at the large scale. Zooko's Triangle is
one obvious problem as the namespace gets full.

It also looks like this is only suitable for functional languages, and
possibly just non-OO languages?

What about the type signatures of these functions? What do they look like? How
well can different types be substituted? Do we have to version the types as
well?

How do we handle testing in this scenario? Every version of every function?
How about combinations of functions?

How about deprecation? This is hard enough to do with standard library
functions; how long has gets() been deprecated, for example?

~~~
jstimpfle
Consider that if you control a large codebase, the need for "decentralisation"
(as in Zooko's triangle) is not there. You can just rename things.

When integrating across maintenance boundaries, renaming isn't possible but a
simple prefix is almost as good as a namespace.

~~~
pjc50
At the point where the article talks about ""the open source Key-Value
database of all functions"", you can't rename things.

~~~
pharke
The name shouldn't matter. What I mean is, we should only care about the
output produced by a given input. The keys could simply be unique hashes but
we also associate a full description of the interface, tests, and text
describing how it should be used. We would just need tooling that makes it
easy to search, run the tests with our data, and locally alias the function to
something we understand.

------
xixixao
On the topic of "where do I put this function", for "library/utility"
functions, I really like the Hack Standard Library approach: Organize
namespaces by the types of return values (or arguments). So we have
Vec/Keyset/Dict namespaces for all library functions which return
vec/keyset/dict, and the C (Collection) namespace for functions which take any
collection and return something else (either element or a number). I'm sure
this can be extended, and I'm always toying with the idea of replacing lodash
with a library that follows this convention. Here the objective is really
discoverability - you usually know the types of return value/argument, while
you often don't remember the function name.

~~~
wruza
Also maybe mark arguments which are essential or auxiliary (and defaults),
like functions receiving Lists of Foos are not really List-related, but Foo-
related.

I always wanted a code completion tool which I could ask: hey, I have these
two values, say, one field and one pango layout, but forgot what to do next.
And it would figure out that I want

    
    
      pango_layout_set_text(
        layout,
        gtk_entry_get_text(entry),
        -1);
    

among the others. Soo cool.

------
nickjj
I really like the idea of a no module system.

If you look at the /usr/bin directory of a Linux system there aren't any
namespaces. There's 1,000+ binaries thrown into 1 folder, each with a distinct
name and things work nicely.

Imagine if you tried to group those binaries into 1 or more folder driven
namespaces. Where would things go? The amount of bike shedding would be
endless and I don't think it would be a better solution in the end even if you
managed to pull it off.

All of Joe's points in that post are spot on.

~~~
dmitriid
There are namespaces, you just don't see them. That's why there's python and
python3, cmake and cmake2 (IIRC) etc.

With functions it's worse. In a flat namespace you will inevitably run into a
problem of two people coming up with the same name.

~~~
nickjj
> There are namespaces, you just don't see them. That's why there's python and
> python3, cmake and cmake2 (IIRC) etc.

IMO python and python3 is a very welcome feature not a negative side effect of
no namespaces. At call time it's super explicit so you know exactly what
version you're using.

But in most module driven programming languages you might import the foo
function at the top of your file and then use foo in 5 different places within
that file. However, you spend 99% of your time working with the code in the
file not glancing at imports, so now you're left wondering not only where foo
is coming from, but who provided foo. Is it from the standard library, your
own code base or a third party author? Suddenly you need to keep all of this
in your head and it sucks.

Phoenix (a web framework for Elixir) has been taking steps to remove a lot of
loose function imports so you know what module they are coming from at call
time, but that's because Elixir as a language has modules. But even still,
explicitness where you're using it is so much better in the long run for
maintenance, even if it involves typing a few more characters.

~~~
Anonymous4C54D6
As way-too-often, I think the solution should be a good IDE. There is nothing
stopping your IDE from completing "foo" as "module1.foo" and offering
"module2.foo" as well if it exists. Alternatively the IDE could just display
"module1.foo" instead of foo because that is what the import dictates.

As always with dynamic things and editing code for them, this will be more
difficult to get right when imports are dynamic.

------
christophilus
When people criticize NPM for having millions of downloads for single-function
packages, this is exactly the letter I always think of.

When Joe Armstrong proposed this, I thought, "Hm, that's an interesting idea."
But I think NPM has shown that it isn't such a great idea in practice.

~~~
Crinus
Assuming "package" is not just another name for "module" and that it goes away
once "installed" (i do not know about NPM), what would be the difference
between 1000 single function packages and a single 1000 function package?

~~~
TeMPOraL
1000x as much metadata, and with small packages, you get more metadata than
real code.

------
kristiandupont
I think this idea makes a lot of sense.

I recently published a tiny library on NPM for changing the case of strings
(snake_case, etc.). It felt so small and silly that I was tempted to add some
other string manipulation functions, to make it a more "serious" library. But
it struck me that this would just be silly. How often do you need a whole
collection of string manipulation functions? Never! It would just add to the
size of dist folders everywhere, with no gain whatsoever. I suspect this
applies in many situations.

~~~
Ididntdothis
" How often do you need a whole collection of string manipulation functions?
Never! "

Always! The Javascript world lacks a good standard library or something like
STL that covers most of what you need during normal development. Instead you
have thousands of little packages of varying quality. I would much prefer a
few big libraries with more power and consistency.

------
dang
Thread from 2015:
[https://news.ycombinator.com/item?id=10409507](https://news.ycombinator.com/item?id=10409507)

2014:
[https://news.ycombinator.com/item?id=8572600](https://news.ycombinator.com/item?id=8572600)

2011:
[https://news.ycombinator.com/item?id=2580383](https://news.ycombinator.com/item?id=2580383)

------
rexpress
The proposed key-value function database mentioned in the article sounds like
the recently launched Function Repository in the Wolfram Language...

[https://resources.wolframcloud.com/FunctionRepository/](https://resources.wolframcloud.com/FunctionRepository/)

------
choeger
What about module-level programming? Has the author ever heard of functors?
How about information hiding and coupling data structures with their
functions? I need just a glance to the ML world to find lots of valid use
casea for modules, way more than just "provide a compilation unit structure".

~~~
dmix
I'm sure Joe Armstrong is familiar with functors.

~~~
choeger
Then why is he completely ignoring this topic in his argument?

------
leblancfg
The whole thread has interesting discussions. Keep in mind it's 8 years old:
[http://erlang.org/pipermail/erlang-
questions/2011-May/thread...](http://erlang.org/pipermail/erlang-
questions/2011-May/thread.html#58772)

------
z3t4
I love lexically scoped modules. Its like global functions, but without the
complexity. Node.js got this right, where modules are like an extension of the
standard lib (node.js framework) too bad its too slow. But in compiled
languages like for example D there is no performance penalty! Some programmers
make fun of small pure functions/modules ... Then they cut their own code up
into tiny small pieces that all depend on each other, without any reusability.

------
wazHFsRy
I think the Unison language goes exactly in that direction

[http://unisonweb.org/](http://unisonweb.org/)

------
jstimpfle
> all functions have unique distinct names

> To avoid all mental anguish when I need a small function that should be
> somewhere else and isn't I stick it in a module elib1_misc.erl.

I can totally relate. It doesn't reduce collisions significantly if a function
is named Foo::Bar() instead of Foo_Bar(). But the latter has the advantage
that we are free to move the function (avoiding initial decision paralysis,
and later it's easier to clean up) and being required to reference this
function with fully qualified name leads to much more readable code (IMO).

What I've been doing lately is I just make descriptive function names. I don't
even use a prefix, but may still include type names in the function name.
Example, slightly long name: _copy_from_StreamsChain_to_Textnode()_. That's
totally fine in terms of avoiding (the FUD concerning) collisions and it flows
very naturally.

Just like with Objects/methods, not doing namespaces leads to a significant
reduction in decision paralysis (time consuming taxonomic concerns - "where
should I put...?") for me.

~~~
pjc50
The converse question to "where should I put" is "where could I find".
Namespaces and objects make it easier to find related functionality so you
don't have to reinvent it. Without good techniques for locating relevant
useful functions, you're just writing write-only code. And the technique of
"if I was going to write this, where would I put it" doesn't even work when
I'm trying to find my keys, let alone among all the developers of the world.

~~~
Oreb
“Where could I find” is indeed an important question, but it is not obvious to
me that hierarchical namespaces are the best solution. Couldn’t something like
a list of tags attached to every function/type/whatever be at least as
effective?

~~~
alexisread
I think the end result of something like this is call-by-meaning. Alan Kay and
co. Have published several papers on this subject, but in a nutshell the code
looks for a matching function to a signature and return type, then calls it.

Obv. There are issues with this approach if fully automatic (how many sum
functions take two ints and return int?) but there's no reason why we can't
use a sig/return to filter on ie. The same function can be in 2 modules if
filtering by sig or by return type.

With typing similar to Haskell we can be even more strict - the sig/return is
a contract, and we can have multiple types like userInputInt, randomInt which
can differentiate similar inputs.

With good typing and behavioural provability we may be able to move towards a
call-by-meaning system

------
Mikhail_Edoshin
An interesting thing about XML namespaces is that they are basically a way to
autonomously create globally unique names that are still human-readable and
writable. (As opposed to GUIDs, for example.) Like `foo:bar`, where `foo` is a
local alias for a long and unique string, often an URN. (Of course, this
wasn't an original idea of XML.)

------
tabtab
Table-Oriented-Programming (TOP) philosophy is that while it's good to apply
categories to snippets of code, one shouldn't be forced to have one and only
one category per snippet. The hierarchical and file-based view of grouping or
categorizing code is seen as obsolete under TOP's way of looking at things.

We use RDBMS to manage large quantities of physical objects, so why not also
use them to manage large quantities of code snippets, such as event handlers.
Early databases were hierarchical also, but over time hierarchies ran out of
steam. TOP believes eventually the same lesson will be learned about code
management.

Take MVC for example. Some code maintenance tasks are by entity and not just M
or V or C. For instance, adding a new column to a given table. So if one could
run a little query to list links to all modules/files/snippets for a given
entity, then entity-based tasks are easier.

------
sgt101
Hello Julia - and it's unreasonably effective multiple dispatch system.

[https://www.youtube.com/watch?v=kc9HwsxE1OY](https://www.youtube.com/watch?v=kc9HwsxE1OY)

f(x1,x2){} < x1.f(x2){x1} < f(x1,x2){x1,x2}

But modules are useful for organising code and choosing what to bother the
compiler with.

------
twhitmore
Nope. Not convinced this is an idea, or even a good thought provoker.

Organization of code at a larger granularity than individual functions, is
something that's really useful and important in most non-trivial applications.

Topicality and meaningfulness of organization is something that should be
valued. Removing or skipping it because Joe can't be bothered separating his
String, Text File and Process funcs would be foolishness. Resulting in only a
new form of spaghetti.

------
jakear
“Global key-value store of functions with their metadata” sounds a lot like
Hoogle to me. As for getting rid of modules and replacing them with
namespaces... I’m not sure I see what is gained. You still have function names
plus an additional identifier.

------
soheilpro
We already have this in JavaScript: isArray, is-even, left-pad, etc.

And the (searchable) database is called npm.

------
pezo1919
So the "database" is the new module? So an entity can share its whole database
or nothing? Everything smaller than the whole database seems a module to me.

------
zachrose
How about a global K/V store for data structures?

------
dgulino
Anyone have a copy of his elib1_misc.erl?

------
interactivecode
Isn't a major major point that package registries and or modules don't need
review board? thus eliminating gatekeeping?

~~~
pjc50
The gatekeeping can be a feature; centralising security review is definitely
worthwhile.

------
cammil
I feel the same way about scope

~~~
tudelo
explain?

------
makz
Why do we need anything at all?

------
jbverschoor
Please stop the insanity called software “engineering”

------
kasperni
That's a great idea! Let's try and deploy this 'left-pad' function that
everyone can use...

But seriously, I know his description is a bit different, but I think NPM has
shown that such a model is really difficult to maintain.

~~~
tosh
Functions in a global persistent database would solve the "dependency suddenly
disappearing" problem.

~~~
jakear
“Global persistence” is a lie. The second copyright law or malware become
involved, practice separates from theory. Look at the hackage rollback, for
instance. ([https://hackage.haskell.org/package/Facebook-Password-
Hacker...](https://hackage.haskell.org/package/Facebook-Password-Hacker-
Online-Latest-Version-1.0.1) <\- downloads what used to be a Hackage package
to your computer. Now, its a text file with the word "Gone")

~~~
tosh
bittorrent, blockchains

