Hacker News new | comments | show | ask | jobs | submit login
Dependency Injection for Python (github.com)
37 points by mace 1573 days ago | hide | past | web | 52 comments | favorite



I don't get dependency injection. I come from languages that I am told don't have it. But when I read the Wikipedia page for DI, I get:

"a dependent consumer

a declaration of a component's dependencies, defined as interface contracts

an injector (sometimes referred to as a provider or container) that creates instances of classes that implement a given dependency interface on request."

Wait, I still have to declare dependencies, just like the import statement in python, require.js in JS, etc. which are apparently not 'dependency injection' systems? And this just adds, er more code?

That same wikipedia page mentions dynamically loaded dependencies while also mentioning 'a declaration of a components dependencies' which sure doesn't sound dynamic to me.

Does anyone care to provide a better explanation of what dependency injection gives me over the existing common dependency mechanisms?


Python itself has DI built in throughout the stack.

Building Java like DI mechanisms is cargo cultism. The whole point of DI in Java is to work around its static typedness, so one can perform runtime binding of components in a dynamic manner.

Three quarters of cool Java and .NET "design patterns" are just band aids around semantic limitations of these languages.


> The whole point of DI in Java is to work around its static typedness.

So, coding the configuration of your application is an ideal way to work. Interesting.


Yes!

http://thedailywtf.com/Articles/Soft_Coding.aspx

If Java guys wouldn't have stuck their heads up their asses so hard, they could use something like scripting language, with the whole debugging and library stack to configure their applications instead of building byzantine XML based DSL's.

Frankly my python (code!) based config files work a lot better than similar XML based config files in Android. Except mine are debuggable and readable.

The whole point of software is that it gets changed on demand. That's why it is called SOFTware.

Edit: Or perhaps you are going to enlighten me why XML config files are not code? Then you might be able to explain where configurations cross the border of "code". Is configuring Sharepoint coding or configuring?


Readable to who? Machine readable? Which is easier to change or debug: dependencies which are in predictable locations OR buried within your source code? Could you say the same thing about arbitrary python code? Now I have to learn the way you decided to implement what's already been done dozens of times before I even get to reading the actual logic of the application.

Also, for the record, if you're debugging the infrastructure of how objects are created, you're doing it wrong. You probably don't understand the framework you're working in.


You are missing the point. I am not saying that one should make mess of his code.

I am merely saying that configuration files are code too. They need to be tested, debugged and refactored. Writing code in XML seems a retarded way to go about it. If my language would prevent me from writing configurations in itself in a sane manner, I would rather use another language with a solid toolchain for this part of the job. Than go and build my own DSL in XML of all things.

If you are of an open mind, go check how Django is configured and extended and compare it to your run of the mill enterprise framework.

And let me tell you that I did my fair share of enterprisey "configurable and extendable" framework shit. And its all a lie. You cannot make "simplified" languages that will enable you to be more "flexible" or to let amateurs to program system behavior. Always these kinds of interfaces gobble up way more developer hours than they save in the long run. Often programmers resort to writing programs that offer a sane API for writing these kinds of configs.

One should simply build a quality API and document it well.


> So, coding the configuration of your application is an ideal way to work. Interesting.

You're coding it one way or another; whether you do the whole thing in Python, or most of it in Java and some of it in XML (or YAML, JSON, etc.) because Java is inconvenient.

The main motivation for using XML (YAML, JSON, etc. are equivalent for this purpose) "configuration" over Java "coding" for certain pieces is that: 1. XML, unlike Java, doesn't need to be AOT compiled 2. XML is more naturally suited than Java to a succinct declarative style 3. Its easier for external tools to work with and modify XML than Java, which may be useful for configuration tools.

In many other languages, one or more of these advantages of XML over the main coding languages is less significant or entirely absent (interpreted/JIT-compiled languages avoid #1; functional and many modern multi-paradigm languages avoid #2; Lisps and other easily-parseable homoiconic languages avoid #3.) Depending on the language you are using and which of these factors is important to your use case, there may be no significant reason to bear the cost of a second language for configuration.


> You're coding it one way or another;

Yes. I agree, most of these frameworks are in some sense a DSL. I cannot agree that writing in a different language changes the underlying purpose and function of separating configuration from the code.


> I cannot agree that writing in a different language changes the underlying purpose and function of separating configuration from the code.

Language features effect the degree to which the underlying purpose and function of separating service identification from other code requires framework support, and the style of framework which is most sensible to support it to the degree that framework support is necessary.


I can agree with that. Untyped languages seem to allow for better gradients of separation as well as tend to work with a wider variety of tools. It's not always good, but it usually quicker to start with.


Thank you for treating .NET as a language and ignoring languages like F#


I was aware of this dichotomy when writing the post. And I find .NET stack much better than JVM stack.

In fact while I couldn't stomach working in Java I would be much more willing working with C#, not to mention other .NET citizens. But in general as a spectrum, the toolkits and libraries of .NET and Java share architectural and design patterns.

However the .NET stack does have built in languages (Javacript and F#) that can be used to glue together the rest of the system in a sane manner. The capability is there, however standard operating procedure is sadly still using shitload of XML's.


In a stackoverflow thread I found a reference to this classic:

'"Dependency Injection" is a 25-dollar term for a 5-cent concept.'

http://www.jamesshore.com/Blog/Dependency-Injection-Demystif...

I do think it is a useful concept - but I sometimes think it gets a bit blown out of proportion.


At its core DI is basically constructing instances of objects with the help of other objects. So you have constructor injection , setter injection , property injection ,etc ...

Now that you got these huge object graphs, you need a way to wire them without pain.

That's where service locators and IOC containers are handy.

A service locator is basically a registery that holds the dependencies and you instanciate objects that way

     container.createDefinition("MyClass").injectProperty("prop1","MyOtherClass").injectConstructor("MyThirdClass","MyLastClass")
then

    myInstance = container.create("MyClass")

or whatever

The question is while dependency injection is good practice in general , do Pythonists need IOC containers or Service Locators? etc ...

AMD in javascript has nothing to do with dependency injection. AMD is not about instanciating objects but resolving file dependencies.


"At its core DI is basically constructing instances of objects with the help of other objects."

Why would you wan t that in python? Doesn't python have higher order functions?


Exactly, in a dynamic language with first-class functions you don't need to create object factories just to pass around static methods between scopes.


sigh

DI is not the same as an IOC container. Get it right ffs, if you want people to take you seriously.

DI:

    class Blah():
      def __init__(self, dependency):
        self.dep = depedency
IOC:

    my_blah = container.resolve(Blah)

Anyway, onto my main complaint:

    "Pinject is a pythonic dependency injection library. Python ports of other libraries, like 
     Spring or Guice, retain the feel (and verbosity) of being designed for a statically typed
     language. Pinject was designed from the ground up for python."
Wow, I'm impressed. Designed from the ground up for python. That's why you needed ~20 pages of documentation on how to use it?

Intuitive.

Also, what's wrong with annotations? I'm rather fond of this sort of code:

    ### implements, resolve, Scope
    class IType(object):
      prop1 = None
      def call(self):
        pass

    @implements(IType)
    class ImplType(object):
      prop1 = "Value"
      def call(self):
        pass

    scope = Scope()
    scope.register(ImplType)

    @resolve(scope)
    class UsesType(object):
      def __init__(self, t=IType):
        self.t = t

    instance = UsesType()
Or in pinject:

    obj_graph = pinject.new_object_graph(binding_specs=[SomeBindingSpec()])
    needs_provider = obj_graph.provide(NeedsProvider)
But wait, I want a test. Ok~

    instance = UsesType(my_mock)
Or in pinject:

    ?????? 
In fact, pinject doesn't even mention tests in its documentation. Whew~ That's only the whole reason for using DI in the first place.

Self important library is self important. Not amazed.


"IOC: my_blah = container.resolve(Blah)"

That's not IOC. That's service location. Which is only used at the "composition root" (look it up) in IOC.

"In fact, pinject doesn't even mention tests in its documentation. Whew~ That's only the whole reason for using DI in the first place."

No, testing accounts for just one of the more minor reasons to be using DI.


I would argue the single greatest benefit (possibly, only benefit) that DI provides over other service location options (eg. factories) is that you get a nice means to test independent units of code by providing mock implementations.

In dynamic languages with no real 'private' members, the benefit is actually pretty minimal because you can assign mock service instances after the object is created.

eg.

    test_instance = Blah()
    test_instance.service = MockService()
vs.

    test_instance = Blah(MockService())
...which means, there isn't a lot of point in using it, unless you're using an IOC container which gives you some other synergy bonuses like singletons, per thread and per request instances, configuration file driven implementation binding, etc.


No the single greatest benefit that a IOC container provides is object lifetime and scoping management. The next greatest benefit is the ability to decompose your code base into a single responsibility adhering components with good decoupling and namespace cohesion characteristics.

Testing is very low down on the list of advantages of a IOC container. And even then, most people that think they're doing testing properly with a IOC container are doing it wrong. Hint: "Re-binding" your container by forcing it to drop an existing service implementation and replacing it with your mock/test implementation is "doing it wrong".

The reason there is always so much discourse around DI and IOC is because very few programmers actually bothered to learn, and grok it, even if they think they already have.


> Testing is very low down on the list of advantages of a IOC container. And even then, most people that think they're doing testing properly with a IOC container are doing it wrong. Hint: "Re-binding" your container by forcing it to drop an existing service implementation and replacing it with your mock/test implementation is "doing it wrong".

What part of that is wrong? I may be missing your focus, but that sounds right for unit testing.


It's wrong because your unit tests, just like your application, should not be aware of the presence of an IoC container except at the absolute composition root.

Your unit test libraries should have their own composition roots of their own and it is at this level, in configuring the IoC "modules" (as they are often called) that special mock services would be provided.

If your test cases are littered with crap concerning your IoC container, re-binding stuff, then something is badly wrong. Sadly, pretty much all projects I've seen that use IoC end up writing their tests in this way. Someone wasn't thinking.

Your tests should be treated with the same adherence to SOLID principles as your application code. They are first class citizens of your code base. So why dirty your tests by having them coupled to your IoC container?


I agree. Testing with heavily mocked implementations is basically useless. Especially under time pressure, I tend to focus more on system and integration tests than on cute unit tests that tend to find issues that never occur in practice and don't find issues that occur when the first user has his first minutes with the system.

"Hooray, my setName sets the name correctly!" - well, my congratulations.


The direct opposite of Dependency Injection is the Service Locator pattern.

    my_blah = container.resolve(Blah)
Notice how there is no injection of dependencies at all.

I think you aught to learn what DI actually is... when you say that "the single greatest benefit (possibly, only benefit) that DI provides over other service location options" you are saying that DI is a form of service location (sic), which isn't the case.


> The direct opposite of Dependency Injection is the Service Locator pattern.

You're right in spirit, but the direct opposite of dependency injection is obviously direct dependency construction, not service location.

Service location is one step up from direct dependency construction, and can be enough for a lot of cases, but as you say, it's not dependency injection.


You're right in spirit, but the direct opposite of dependency injection is obviously direct dependency construction, not service location.

Fair call!


another thing - there are multiple ways to inject dependencies - not just constructor injection.


:)

hint: the DI is in the constructor to Blah when the container returns an instance. (all IOC frameworks have a top level resolve call somewhere).


Yeah, to startup the container and manage the object graph, sure. But not to directly instantiate the objects like you've given in your example!

The whole point behind DI is to decouple objects, any IoC container worth it's salt gets started and works it out for you. You shouldn't need to resolve the individual object.

Another poster noted this is known as composition root, a good explanation can be found here:

http://blog.ploeh.dk/2011/07/28/CompositionRoot/

Note the following:

"A DI Container should only be referenced from the Composition Root. All other modules should have no reference to the container."

Words to live by :-)


True indeed, however, the point remains:

IOC is resolving an instance with it's dependencies.

How you do that is irrelevant.

The point is that the action of resolving dependencies is NOT dependency injection.

If you're writing an IOC container, call it an IOC container.


Excuse me, but the IoC Container is doing the injecting. That's the whole point.


aught = something. ought = should.


I never realized! Thanks, I'll remember that.


Less documentation is better? That's not something I hear every day.


The author could refactor the documentation a little, starting with removal of the word "Pythonic" and perhaps ending only after 90% of the text has been replaced with a single section entitled "WHY GOD WHYYYYY".

    def python_di_copyright_2069_hosay123(dep, *args, **kwargs):
        modname, _, classname = dep.rpartition('.')
        mod = importlib.import_module(modname)
        return getattr(mod, classname)(*args, **kwargs)
> Though Google owns this project's copyright, this project is not an official Google product.

We just published it under Google's Github to give a terrible project implied legitimacy that it likely doesn't even have internally.


I understand dependency injection, but I never understood dependency injection frameworks. For me the whole point of DI is explicit composability. There should only be a few well defined places in my application where all my classes are assembled into a working program. If my classes cover my domain nicely, I should be able to compose all kinds of programs (including tests) and reuse code.

If my objects get injected to each other automatically by their name then my dependencies are just as predetermined, which results in coupled code, except the coupling is now difficult to read or modify.

Once I decided I am only using explicit injection, I don't need a framework, I just instantiate my classes in the right order and pass them into each other. What can possibly be more pythonic than that?

I have the same issue with Angular.js btw. They are trying to make is easy to start by adding dependency injection magic, but in reality all I see is "this can't possibly work (and indeed it doesn't)", so I just need to spend more time digging and understanding the magic before I can start writing ANY code.

Every documentation should start by explaining the explicit way of doing things, then adding the sugar afterwards.

import this


I'd prefer by type to by name, but there's still a small decoupling advantage: suppose you have some componentA in your application and you want to add some more functionality that depends on some serviceB. What DI should make possible is you modify only componentA, to state that it now depends on serviceB, and as long as you have a serviceB available then you don't need to change your "wire up my application" code at all (but of course if there is no serviceB then this should fail as fast as possible and tell you to fix it).

/has just written a compile-time DI framework in scala using type-level programming. It's pretty sweet except for the compile times.


> What DI should make possible is you modify only componentA, to state that it now depends on serviceB, and as long as you have a serviceB available then you don't need to change your "wire up my application" code at all

So... instead of modifying just the "wire up my application" code, you modify just the componentA code. In return you get to fill your code with magic so nobody can see where serviceB is created.


You always have to modify componentA - you're adding a new feature to it, and declaring that it takes a serviceB. If you're not using DI you have to change your "wire up my application" code as well.

The choice is between writing the wiring code yourself, which means a lot of boilerplate that's basically irrelevant to the functionality of your application, or have it happen implicitly, which yes involves a certain amount of magic. But even polymorphic functions are "magic" on some level.


In practice, the automatic "wiring-up" often breaks, and in any sizable system it quickly becomes very difficult and time consuming to track down and fix such breakages.

Things can become even worse when the libraries or frameworks you're using end up having bugs in them that interfere with the dependency injection they're performing.

Too many times I've seen very experienced and capable developers lose hours, and even days in some cases, dealing with dependency injection issues. Just one of these incidents, which will happen in any serious project, will far exceed the time and effort it would've taken to write the equivalent code by hand.


Dependency Injection (DI) and Dependency Injection Containers (DIC) are the "in-thing" in PHP at present, and to speak against them [0] is - like disagreeing with anything fanboys latch onto - ridiculed, without a sensible discussion or explanation.

From a testing perspective I sort-of get it; from a "this is less tightly coupled" I don't: all you've done is move the coupling to a different level. Code always has to be coupled at some level, so where's the advantage?

With the amount of noise about DI I'm sure it's something I should pay attention to rather than be turned off (as happened with Rails), but I really don't et it. Can anyone provide the missing piece in my thinking?

[0] http://till.klampaeckel.de/blog/archives/154-Dependency-Inje...


It's interesting to follow the flow of ideas among languages/platforms.

We've repeatedly seen concepts that were popularized by the enterprise Java crowd flow down to the .NET crowd, and eventually to the PHP crowd.

IoC and DI is a good example of this. It was extremely hyped within the Java community during the early- and mid-2000s. Then this flowed into the C# community, with it getting a lot of attention during the late-2000s and early-2010s. Now it has finally made its way to the PHP community.

An astute PHP user who recognizes this flow of ideas will look at how the trend has progressed within the Java and C# communities. What they'll see is that there has be movement away from IoC and DI. The purported advantages never materialized, but there was a lot of pain and bad code left around.

Many in the Java community have moved on toward simpler frameworks and techniques, or even toward a far more static JVM-targeting language like Scala. Many C# users have similarly started moving away to other approaches.

At some point, perhaps around 2016 or so, the PHP community will come to the same conclusions about the suitability (or lack thereof) of these techniques. After experiencing the problems first hand, they'll too move back to simpler, more explicit approaches.


Interesting, I had wondered if this was the case but don't know where to look in the Java/.NET community for current best-practices (or What's Currently Hot). Where should I look?

> Many in the Java community have moved on toward simpler frameworks and techniques

What have they moved onto?

> At some point, perhaps around 2016 or so, the PHP community will

Oh so true :D


Actually, that is the point. If you decouple the objects via the container at the composition root, then you have actually achieved loose-coupling of objects.


advantage is when you manage to couple things at composition root (which is the very beginning of your application). that way there is one convenient place where coupling takes place.

read "Dependency injection in .Net" by Mark Seemann. it's the best resource to understand wtf DI is all about and what are benefits.


Yes even if you don't program .NET, the first half of the book is just about manual DI principles that could apply to almost any language. His blog also has good stuff about DI.


Pinject is a pythonic dependency injection library.

+

Implicit class bindings

...ummm...


Implicit class bindings sound like a really bad idea and hard to debug.

  Pinject creates implicit bindings for classes.
  The implicit bindings assume your code follows PEP8
  conventions: your classes are named in CamelCase, and
  your args are named in lower_with_underscores. Pinject
  transforms class names to injectable arg names by
  lowercasing words and connecting them with underscores.
  It will also ignore any leading underscore on the class
  name.

  If two classes map to the same arg name, whether those
  classes are in the same module or different modules,
  Pinject will not create an implicit binding for that arg
  name (though it will not raise an error).


that looks pretty good.

not sure if i overlooked it in the notes or not, but i have a couple questions.

you mentioned copyright at the end. what sort of license are you releasing it under?

Does it support nested creation of objects? ( ie if you ask for an instance of class 'FOO' and FOO has a DI or 'BAR' will BAR be first created and then injected into FOO)


Seems like it's released under the Apache License 2.0 (cf. LICENSE file).


DI is about breaking constructor coupling. I wish people would understand that.


Stop writing Java code in Python. No. Bad. Stop it.




Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | DMCA | Apply to YC | Contact

Search: