This is the second in a series of posts in which I’ll be examining common patterns found in OO, the problems they are trying to solve, and then how these same problems might be approached in Clojure. The first part can be found here.

Today I’m going to approach the topic of Dependency Injection (DI).

In most sufficiently large systems that I’ve encountered in Java, the decision of whether or not use a DI framework isn’t something we think about. Instead we head straight to the question of “which one shall we use?”. It’s pretty much a given that we’ll be using DI.

Given that it’s used so often, it’s only natural for us Java developers to feel the need to map this concept to some analogous concept on FP.

In this post I explore a few different ways of achieving DI in Clojure, in order of increasing complexity:

  1. Passing a ‘system value’ to functions
  2. Reader monads
  3. Component library

What does DI give us?

DI gives us quite a bit:

  1. Enables smaller units of code: there is almost no overhead of piecing it together. This means smaller and more focussed classes.
  2. Less “infrastructure” code: the DI framework will take care of stitching our system together at runtime.
  3. Flexibility: ability to switch out implementations and behaviours via config.

There are more things, but I feel these are the most important. We’ll be referring back to the above 3 items as we explore our solutions.

Sample Java code

Let us set the scene in Java. Today we’ll ‘pretend write’ some code that takes a string and searches the major social networks using that string as a query.

Let’s define the interface:

public interface SocialNetworkSearch {
  List<Item> search(String query);
}

In our application - we will have multiple implementations of the above wired up at runtime. Furthermore, let’s pretend we wish to log all search queries into a database. Take a look below:

public class AggregateSearch {

  private final List<SocialNetworkSearch> searchers;
  private final Database database;

  @Inject
  AggregateSearch(List<SocialNetworkSearch> searchers, Database database) {
    this.searchers = ImmutableList.copyOf(searchers);
    this.database = database;
  }

  /**
   * Returns an aggregate list of search results from each social network
   */
  public List<Item> getResults(String query) {
    database.logQuery(query);
    List<Item> allItems = Lists.newArrayList();
    for (SocialNetworkSearch search : searchers) {
      allItems.addAll(searchers.search(query));
    }
    return allItems;
  }
}

(We are avoiding all threading/concurrency discussions for now. A topic for another day.)

With some configuration, our framework will analyse the types being requested for injection by our AggregateSearcher class, and inject them in at runtime.

When we’re writing tests, we can provide mock implementation(s) of the SocialNetworkSearch and database and instantiate the above class the old-fashioned way with new [1]. In a staging environment, we might only be hooked up to Twitter but no others. In production, we might be hooked up to Twitter, Facebook and Google+.

A DI container helps us achieve this flexibility by simply changing the DI configuration, rather than changing our code. This would result in the DI container injecting different searcher instances and/or database implementations into our searcher at runtime, with no code changes.

DI in Clojure I - system value

So let’s explore one simple way to achieve this in Clojure.

In this first approach, we’re going to dispatch functions based on a configuration value. To dispatch on a system value, we’ll need a value that represents our ‘system’. For the above, we could imagine a configuration that looks like this:

(def config {:enabled-searchers [:twitter :facebook :google+]
             :db-uri "jdbc:......"
             :broker-uri "some://message-broker"})

The nice thing about having a system config value such as above is that we can store it as a simple .edn file and just read it at startup.

We’d use the above config to create a “system value” at startup. To achieve this, we define a function that takes the config as an input, and returns the system value:

(defn create-system 
  "Connects to db and messaging layer, and returns configured system."
  [config]
  (-> config
      (assoc :db-conn (connect-to-db (:db-uri config))
             :request-queue (messaging/request-queue (:broker-uri config))
             :response-queue (messaging/response-queue (:broker-uri config)))))

The system value would then be passed as an argument to any functions that needed it. Let’s take a look at how that might look.

But first - let’s define our search function as a multimethod (using defmulti). This multimethod will dispatch on the type of searcher being asked for (:google+, :twitter, etc).

(defmulti social-search identity)

(defmethod social-search :twitter [network query]
  (println "Searching twitter for " query)
  ...)

(defmethod social-search :facebook [network query]
  (println "Searching Facebook for " query)
  ...)

(defmethod social-search :test [network query]
  (println "[TEST] Searching for " query)
  [{:content "Hey there!"
   :user "Sandipan"}])

Finally, we define our aggregating search function, which takes the system value as a parameter, and pulls out the parts of interest in to do its duty:

(defn aggregate-social-search [system query]
  (log-to-database! (:db-conn system) query) 
  (for [portal (:enabled-searchers system)]
    (social-search portal query)))

We could imagine utilising the above function in an area of the code that serves the search requests:

(defn serve-requests
  "Listens on the request-queue for queries and serves them."
  [system]
  (for [query (:request-queue system)]
    (let [result (aggregate-social-search system query)]
      (messaging/send! (:response-queue system) result))))

Revisiting our 3 points from “What does DI give us?” in light of the above code:

  1. Enables smaller units of code: We have a single function for each of our search portals. Our aggregated searcher is also defined independently - each can be updated largely independently.
  2. Less “Infrastructure” code: Clojure’s multimethod dispatch is our “infrastructure” code for choosing different implementations of the searcher. Dependencies are just function arguments, our create-system function is essentially our DI container.
  3. Flexibility: We’re able to change how the system behaves just by changing the config.

DI in Clojure II - reader monads

Part I gave us a solution which might work well for smaller systems. But it did pose a slight problem. Every function that had a dependency needed to accept the whole system value as an argument, rather than just the parts of the system that concerned it. This leads to a lot of areas of the code needing to be ‘system value aware’, including our tests.

Reader monads give us a potential solution for this. They enable us to define a function that, given an “environment” value, will read something from that environment, and do something with that value.

I’m not going to write deeply about monads in this post. Rather I’ll focus on how you can use reader monads specifically to solve this problem. The monad implementation we’ll be using is algo.monads.

Continuing with our social searching example - we shall now strip the aggregate-social-search function of its knowledge of the system value. We define a function builder that, given the enabled searchers and database, returns the function that performs the search:

(defn aggregate-social-search
  [enabled-searchers db]
  (fn [query]
    (log-to-database! db query) 
    (for [portal enabled-searchers]
      (social-search portal query))))

We then use the reader monad to define a function that accepts an environment, reads stuff from it, and uses those values to construct our system:

(def create-system
  (domonad reader-m
           [db-uri (asks :db-uri)
            enabled-searchers (asks :enabled-searchers)
            db (m-result (create-db db-uri))]]
           {:db db
            :searcher (create-searcher enabled-searchers db)})

It’s important to understand that the create-system var above will end up as a function that accepts an environment as an input. When called, it will thread this environment into each of the readers defined with asks, and finally return what’s in the body of the computation. So we can use it like this:

(let [system (create-system {:db-uri "jdbc:foo:bar"
                             :enabled-searchers [:twitter :facebook]})]
  ...)

Obviously this system is largely contrived, and in a real system we’d have multiple interdependent components in our system. We could use reader monads to instantiate those components with the right dependencies as we have done above.

So let’s revisit our 3 points:

  1. Enables smaller units of code: Not a massive change from part I.
  2. Less “infrastructure” code: Minimal change from part I - function dependencies are explicit rather than within a monolithic ‘system’ parameter.
  3. Flexibility: Minimal change from part I, defining our environment/config differently will change the behaviour.

DI in Clojure III - Component

Whilst the DI in part I and II was mainly contained within one function, we still need to maintain the order in which components are brought up manually. For instance, in the above code we must create the database before creating the searcher (due to its dependency). We cannot just declare that a searcher needs a database and be done with it.

This is where Stuart Sierra’s component library can help us out. It gives us the ability to define components, declare their dependencies and define their lifecycles. The Component library then stitches things together for us at runtime - bringing up components in the correct order according to their dependencies.

I won’t go deeply into implementation since one can simply read the documentation, but rather I’ll talk about the usage in the context of what we’re doing here.

First we define our aggregate searcher component:

(defprotocol AggregateSearch
  (search [this query]))

(defrecord AggregateSearchComponent [db-conn enabled-searchers]
  AggregateSearch
  (search [this query]
    (log-to-database! db-conn query) 
    (for [portal enabled-searchers]
      (social-search portal query))))

(defn aggregate-search-component [enabled-searchers]
  (AggregateSearchComponent. nil enabled-searchers))

Note we haven’t implemented any ‘lifecycle’ protocol methods in our record above. This is because in the case of the aggregate search, there’s nothing really to bring up or down. In other components however, like a database, there would be connections to make/close upon startup and shutdown [2].

Once we’ve defined all our components, we define our ‘system map’:

(defn create-system [config]
  (component/system-map
    {:searcher (component/using
                  (aggregate-search-component (:enabled-searchers config))
                  [:db-conn]) ;; <-- these are the dependencies
     :db-conn (db-component (:db-uri config))}))

The key take-away from this is that declaring our aggregate searcher’s dependency on ‘db-conn’ above is declarative, and Component takes care of injecting (really just assoc’ing) the db-conn into our aggregate searcher component at runtime so that it’s available to the search function.

Quickly revisiting our 3 points:

  1. Enables smaller units of code: We still have this, though there is a little more ceremony involved in working with component when comparing to our other 2 solutions.
  2. Less “infrastructure” code: Dependencies are now stated declaratively, and Component takes care of the lifecycle for us (provided we implement the lifecycle protocol appropriately in our components).
  3. Flexibility: Minimal change from part I or II. Defining our environment/config differently will change behaviour with no required code changes.

Conclusions

In this post, we explored 3 ways to achieve DI in Clojure.

Firstly, we stayed within the bounds of the Clojure language features. We supplied a system value to functions as an argument - so that functions could make use of any parts of the system it was interested in. We defined a create-system function to stitch together the system at runtime.

Secondly, we optimised this further by having functions that depend only on the parts of a system they are interested in, and used reader monads to help us here. This allowed us to refine create-system to accept the config/environment and then call functions appropriately to bring up our system [3].

Finally, we used the Component library to implement a (slightly) more heavy-weight solution that gave us a neat way of being declarative with our dependencies. It also gives us fine-grained control over the lifecycle for each component that makes up our system, and takes care of bringing things up in the right order.

End notes

  1. We could also use Mockito’s @InjectMocks annotation.
  2. There’s definitely a level of understanding required for implementing the component/Lifecycle protocol - beyond what’s in this post. I’d strongly recommend referring to the github page.
  3. There’s nothing to say we couldn’t have achieved this without reader monads quite easily, but I thought I’d scratch the surface of what monadic workflows can do for us.