Configuration with types and style

Passing by a Config object is common practice in Scala. But here is why I do not like this pattern...

Configuration with types and style
Photo by Dmitriy Demidov / Unsplash

If you are a Scala developer, you must certainly know of Typesafe Config which is I guess, the most widely library used to deal with configuration parsing and loading in Scala applications. If you do not know it yet, the library offers a language called HOCON which is a JSON superset with convenient features for configuration (fallback, comments, ...). The library in itself is 100% Java by the way, but several libraries exist to make the most of it in various languages, including Scala. In this article, I won't talk much about those libraries, but instead focus on some usage patterns.

If you happen to use Alpakka and its Kafka library, you will notice that creating a Kafka producer requires a ProducerSettings and this is created using directly a Config. Same for the Cassandra library : creating the settings object is done via Config and there is no other way to create those than passing around a Config. The Alpakka libraries are not the only ones to use this style, which is used widely in the Scala ecosystem. This is somehow frustrating to not have the option to create settings by hand, and to have to create a Config to then be able to transform it into the appropriate settings object. In my humble opinion this usage of typesafe config is not optimal for several reasons.

Reasons of why I don't like this pattern

Mixing concerns

First of all, this approach mixes several things: extracting parts from the configuration, validating it, and even using it. If we take a closer look at the source code of ProducerSettings, there are calls like config.getXXX("key"). If we read the documentation of such method, like getDuration for instance, we can see that these methods throw exceptions, which in turn, show that they are fail fast : if getting a key fails, then the exception is escalated and the retrieval of configuration items is stopped. An implication of this is that it is not possible in this approach to get all the errors in the configuration.

Strong dependency on typesafe config

Looking at the example of ProducerSettings closely, one question may arise : why does it need a Config in the first place ? In this case, I guess it is because Alpakka is used in conjunction with akka which in turn summons typesafe config, so the assumption is that typesafe config is in the classpath anyway. But, this pattern does not hold outside of the akka ecosystem and passing around a Config is almost the same as using a Map[String, Any]. Config provides some nice wrappers around cast exceptions and does a great job at hiding them from the users, turning them in more comprehensive exceptions like WrongType. As a consequence, this highlights how typesafe config is a library to build around to load and parse HOCON files, but it is clearly not a replacement for methods parameters.

Rigid configuration

One other issue I have with this pattern is that it sometimes imposes how to find the configuration by hard-coding a path. Let's say I have a component which does this:

object MyComponent {
  def apply(cfg: Config): MyComponent = {
    val a = cfg.getString("mycomp.a")
    val b = cfg.getString("mycomp.b")
    ???
  }
}

What if I want several instances of MyComponent? As a caller, I have to know the underlying path were the config is read, then transform a Config to point wherever I need to get two instances. This is much more complicated than (again and always, imho) :

// A config object
final case class MyComponentConfig(a: String, b: String)

object MyComponent {
  def apply(cfg: MyComponentConfig): MyComponent = {
  	// Use directly the config object
    //...
  }
  
  // Also works with basic parameter passing
  def apply(a: String, b: String): MyComponent = {
    // ...
  }
}

// Somewhere else
val myCompA = MyComponent("a", "b")
val myCompB = MyComponent("c", "d")

Solving those issues

Well, the last snippet showed a bit of the solution : do not rely on typesafe config in your components by using case classes to hold this configuration. It then appears that we need a layer between the raw Config and our configuration case classes. This hole is filled by libraries like Pureconfig, but as an exercise, you could write your own layer and play with the applicative functor techniques I already talked about before. For completeness it is worth noting that ProducerSettings in Alpakka Kafka offers some builder methods which are available once your have a ProducerSettings instance. It's just sad to have to use a Config object to get access to theses withXXX methods.