
Scala: Understanding implicits - maddening
https://kubuszok.com/2018/implicits-type-classes-and-extension-methods-part-4/
======
paulddraper
I've seen uses of implicits fall into three categories:

(1) Dependency injection, e.g. ExecutionContext

(2) DSLs/syntax sugar, e.g. custom string interpolation, specs2

(3) Typeclasses and type-level programming, e.g. Play JSON Reads/Writes, cats,
shapeless

The more I have seen Scala, the more I realize that #1 is cancer.

Implicits should _not_ be used to reduce the number of characters typed. If
used at all, they should be used for a better reason, e.g. it is a necessary
part of a typesafe pattern, like typeclasses.

~~~
bad_user
Implicits are great.

People unfortunately go to great length to avoid typing function parameters so
will do all kinds of shit to reduce arguments in function calls. This includes
all kinds of insane dependency injection frameworks that depend on runtime
introspection or bytecode manipulation, or stateful objects being passed
around instead of plain values.

There's even this best practice around that says " _have functions with 3
params or less_ " which is complete bullshit. I mean, you actually have
company wide policies with enforced lint rules on maximum number of params and
people end up passing around giant records (maps, objects) that then get
unwrapped.

I cannot blame implicits, if anything, if people are going to go to such great
lengths to avoid passing arguments in functions, I'd rather see implicits
instead of Spring or Guice, because at the very least implicits are compile
time.

The actual cancer are the best practices that drive people to doing this.

~~~
forkerenok
I wouldn't say that best practices drive people to doing stupid stuff, at
least not directly. And they usually do have merit.

The issue is that people apply them without understanding the motivations
behind.

F.x. the "too many method or constructor args" warning signals that a class or
a method is probably doing too much, so please decompose. A person without
understanding the motivation will think it's a sort of a game and wrap some
params into a map or move some them into globals :-\

~~~
mlthoughts2018
I find it insane that someone would use an actual lint rule to enforce a best
practice about few function parameters (loosely defended by this thin
complexity argument).

Functions of appropriate complexity routinely have more than 3 arguments, even
in standard library functions.

Why not just use common sense and code review? If a certain function actually
needs 10 arguments for customization (for example, this is common in plotting
libraries), so be it. If during code review the writer can give a reasonable
account of why the complexity is useful, then just move on.

I cannot see any way to defend putting this is a linting rule.

~~~
forkerenok
I'm not a fan of those either (and 3 arguments is insanely low indeed), but I
see a way it can work: you always have a choice to suppress a linter finding
if your case is "legitimate".

F.x. in our case the limit is 7. I think most people would agree that 7+ args
constructor or method is worrying. But there are cases where it can be ok
(with or without some stretch), f.x. if it's some sort of a glue code that
does a very mechanical thing and breaking it down would be nothing short of
gold-plating.

Edit: In general I find the signaling that linter gives useful. At the same
time I agree that mature teams can do without it just fine.

~~~
mlthoughts2018
Yes, but then you just encourage people to litter the code with noisy linter
suppressing directives or comments.

The point of directives that suppress specific linter rules is that it should
be used in rare, exceptional cases when an otherwise common lint rule needs to
be violated.

But having a large number of function parameters is not rare or exceptional:
you have that _for legitimate reasons_ all the time.

If a lint rule causes lint overrides to become a daily experience, I’d say
that clearly tells you that the lint rule itself is wrong, not the software
practices.

To me, someone who would support a lint rule reducing function parameters
would almost certainly be someone who had a deeply unpragmatic and unrealistic
obsession with certain kinds of design patterns and refactoring patterns.

You code professionally long enough and you start to realize that most design
patterns, especially OO cookie cutter patterns, are pretty crap and only have
a few narrow use cases.

Introducing some type of abstraction purely to allow refactoring into function
signatures with fewer parameters would be a nasty bad code smell to me.

~~~
dragonwriter
> But having a large number of function parameters is not rare or exceptional:
> you have that for legitimate reasons all the time.

That doesn't match my experience. IME, virtually most cases where I've run
into asignaturas with more than about 3 parameters, there have been one or
more groups that should have been passed to another function whose result
should have been passed to the present function.

But that may be context dependent.

> If a lint rule causes lint overrides to become a daily experience, I’d say
> that clearly tells you that the lint rule itself is wrong

Sure, so in a context where you need to do that daily, the lint rule is bad
for that context.

------
harpocrates
I heartily recommend [splain][0] to anyone debugging non-trivial implicits. It
is a scalac compiler plugin that, among other things, will swap out the
horribly unhelpful "implicit not found" or "diverging implicit expansion"
messages for an indented and annotated representation of the search tree that
the compiler went through before giving up.

`-Xlog-implicits` is good to use every now and then, but it quickly becomes
unreadable for any decent sized project.

    
    
      [0]: https://github.com/tek/splain

~~~
bad_user
Well grown libraries are using the `@implicitNotFound` annotation to generate
better library-controlled compiler errors, with advice given for what to do.
It has its limitations of course, but proved very useful in many situations.

A search tree is cool, but it's not going to explain how to get a value when
it has restrictions (dependencies).

~~~
harpocrates
`@implicitNotFound` still only gives you a top-level failure message. Also, my
comment was aimed more at developers of libraries with complex implicit
derivations not the consumers of such libraries.

------
weego
The best summary is why bother when coding explicitly and without "magic" is
far more suitable to long term maintainable code

~~~
bad_user
Implicit parameters in Scala are solved at compile time and lexically scoped
so do not qualify for "magic".

> _why bother_

Because software development is hard, we're solving hard problems and it's
good to have better and better tools to do that.

In particular Scala's implicit parameters, along with its support for higher
kinded types and functional programming in general allow for modeling your
problem domain via tools imported from Haskell and other FP languages, because
implicit parameters are in fact a generalization of _type classes_. And _type
classes_ are many times a superior alternative to OOP for building polymorphic
code.

Plus the landscape is very competitive and the market is global. I remember in
2000 when the bubble burst the jobs of thousands of developers disappeared
over night. To stay competitive, a " _why bother_ " attitude is not going to
cut it when the current bubble bursts, just saying.

~~~
couchand
> solved at compile time and lexically scoped so do not qualify for "magic".

That seems like too limiting a definition of "magic" to me.

> implicit parameters are in fact a generalization of type classes.

Well, not exactly, as far as I can tell. Type classes (in Haskell at least)
are significantly more ergonomic. Since they semantically describe the user's
types, they allow you to perform their operations directly with the user's
values. Implicit parameters have to be pulled out of the implicit environment
and used explicitly themselves, as far as I can tell.

It's true that the underlying mechanism is (roughly) the same, but the
surface-level behavior is what really matters, and there they are very
different.

~~~
bad_user
> " _That seems like too limiting a definition of "magic" to me_"

What is your definition of "magic" then? Things that are unfamiliar?

Please don't mention "explicit vs implicit", because Scala's implicits are in
fact explicit, plus we can always talk of assembly language and how it lacks
any magic.

> " _Well, not exactly, as far as I can tell. Type classes (in Haskell at
> least) are significantly more ergonomic._ "

Ergonomics has nothing to do with whether a concept is a generalization of
another.

> " _Since they semantically describe the user 's types, they allow you to
> perform their operations directly with the user's values. Implicit
> parameters have to be pulled out of the implicit environment and used
> explicitly themselves, as far as I can tell._"

Nope, both type clases and implicit parameters allow for describing functions
that turn _type names_ into _values_. That's all there is to it. Or in other
words, both type classes and Scala's implicits are about _return type
polymorphism_.

The only actual differences are that:

1\. type classes in Haskell are supposedly _coherent_ , i.e. the compiler is
supposed to force a single type class instance in the whole project, however
GHC does in fact not do that so you can end up with multiple instances of the
same type in the same project

2\. Scala's implicits are more flexible in what they allow, for example it has
priority rules in case of conflicts, which allow say those implicits to work
in the presence of _OOP subtyping_ , or to work with multiple type params for
describing things that are difficult to describe via type classes (but not
necessarily desirable, e.g. the CanBuildFrom pattern)

3\. working with Haskell's type classes is nicer, because for hierarchies in
Scala you end up dealing with OOP issues (i.e. in modelling Haskell's type
class hierarchies, we ended up debating on inheritance vs composition), plus
we do a lot of manual plumbing in libraries like Typelevel Cats

But trust me when I say this, fundamentally they are the same ;-)

You can also read Martin Odersky's paper about it:
[http://ropas.snu.ac.kr/%7Ebruno/papers/TypeClasses.pdf](http://ropas.snu.ac.kr/%7Ebruno/papers/TypeClasses.pdf)

~~~
tome
> 1\. type classes in Haskell are supposedly coherent, i.e. the compiler is
> supposed to force a single type class instance in the whole project, however
> GHC does in fact not do that so you can end up with multiple instances of
> the same type in the same project

Can you give an example?

------
AheadOfTime295
It will be interesting to try out implicits in the upcoming Twitter's
Reasonable Scala Compiler [0]

According to [1] a working version will be ready before originally planned:

> Recently, Stu Hood came up with a groundbreaking idea. He suggested > that
> Rsc can become useful much earlier that we initially thought.

[0] [https://github.com/twitter/rsc](https://github.com/twitter/rsc)

[1]
[https://github.com/twitter/rsc/commit/ebc3019c06c9e175311541...](https://github.com/twitter/rsc/commit/ebc3019c06c9e175311541637ec31047fb914c4b)

~~~
doikor
There is also some interesting work (for dotty only atm) on doing the type
checking in two phases which would allow the second phase to be parallelized.
This could be paired together with the Rsc type checker for the first pass for
even bigger improvements.

[https://github.com/lampepfl/dotty/pull/4767](https://github.com/lampepfl/dotty/pull/4767)

------
justinjlynn
For when you've completed that here's an application (with an amusing name):
[http://eugenezhulenev.com/blog/2017/04/26/type-level-
instant...](http://eugenezhulenev.com/blog/2017/04/26/type-level-instant-
insanity-in-scala/)

~~~
wging
Reminds me of [https://aphyr.com/posts/342-typing-the-technical-
interview](https://aphyr.com/posts/342-typing-the-technical-interview)

~~~
justinjlynn
As to technical interviews I've always believed the rule was: ask a silly
question, get a silly (but still enlightening) answer. That series of blog
posts is the height of this precept.

------
AheadOfTime295
Re implicits and their interplay with macros: Achieving 3.2x Faster Scala
Compile Time

Diagnostics:

\- A method takes an implicit parameter to be filled in by the compiler

\- the Scala compiler synthesizes an argument for that parameter using macros

\- instead of using an existing value in the implicit scope

Conclusion: that's why a small source file took so long to typecheck.

Details at [https://jobs.zalando.com/tech/blog/achieving-3.2x-faster-
sca...](https://jobs.zalando.com/tech/blog/achieving-3.2x-faster-scala-
compile-time/)

------
joshlemer
Also see Martin Odersky's recent talk about abstracting context using
implicits [https://youtu.be/uiorT754IwA](https://youtu.be/uiorT754IwA)

------
cutler
If you keep an eye on Indeed.com and Angel.co you'll notice a decline in Scala
adoption over the last few years. I can't help thinking implicits are partly
responsible for this. They're certainly the main reason I ditched Scala. More
than any other language there seem to be more exmaples, with Scala, of a team
trying it then switching to something simpler.

~~~
dionian
people write terrible java code, java is fine. no surprise scala suffers from
that worse since it has a lot more power

------
hderms
I think implicits are fine as long as they're used primarily for the typeclass
pattern. The extension methods and conversions aspect of implicits should be
confined to library DSLs

------
hnbroseph
i wonder if implicits could present a security risk, insofar as the compiler
may grab some [perhaps mistakenly in-scope] identifier which has sensitive
data on it.

------
emodendroket
In general I feel like implicits are not a great feature.

------
namuol
(Would be helpful to include "Scala" somewhere in the title)

~~~
alexanderdmitri
That it's about Scala is implicit in the title :P

~~~
Randgalt
Touche :D

------
rajman187
Off topic but annoying (and pedantic on my part) nonetheless, "begging the
question" does not mean what the author seems to think.

------
peeyek
Scala is like Rust and Rust is like Scala. They are proud of their complexity.

~~~
masklinn
I commonly see this assertion but it never made much sense to me: Rust is not
that complex, and it's certainly nowhere near the complexity of Scala or C++
let alone worse (to me the sole topic of C++ constructors & assignment feels
more complex than the entirety of Rust).

What it is is highly front-loaded, especially through the early learning cliff
of integrating the borrow checker.

~~~
joshlemer
Meh I would say Rust is just as complex as Scala. I actually don't think Scala
is all that complex, it has few features, they're just write powerful and
general ones which take some getting used to I think.

