Implementation of dependencies using the Cats-effect Resource monad

Implementation of dependencies using the Cats-effect Resource monad

Cats-effect monad Resource provides a great monadic abstraction over the pattern try-with-resource. For example, it allows you to manage the lifecycle of dependencies, including closing/finalizing a resource when it is no longer needed (closing the connection to the database, freeing the cache when it is finished). Combined with monad composability, this has become a very popular approach to dependency management—so much so that Scala libraries such as http4sprovide their dependencies wrapped in the Resource monad.

Offtop: One of the main advantages of the Resource monad is that it allows you to release resources in a way that is not instantiated, something even monsters like Google Guice don’t do.

Nowadays, every developer of applications on Scala, when creating their application, in one way or another encounters dependency management. There are libraries that help manage dependencies, e.g macwire. But usually it’s quite simple to use a monad to implement dependencies Resourceas, for example, in the following example of a simple http service:

import cats.effect.*
 
 // Репозиторий, два сервиса и api:
 class Repository(conn: ConnectionFactory) {}
 
 class ServiceA(repo: Repository) {}
 class ServiceB(repo: Repository) {}
 
 class HttpServerTask(serviceA: ServiceA, serviceB: ServiceB) {
   def run: IO[Unit] = ???
 }
 
 // Часть с внедрением зависимостей:
 object Dependencies {
   private val conn: Resource[IO, ConnectionFactory] = ???
   
   private val repo: Resource[IO, Repository] = for {
     conn <- this.conn
  } yield new Repository(conn)
    
   private val serviceA: Resource[IO, ServiceA] = for {
     repo <- this.repo
  } yield new ServiceA(repo)
   
   private val serviceB: Resource[IO, ServiceB] = for {
     repo <- this.repo
  } yield new ServiceB(repo)
   
   val server: Resource[IO, HttpServerTask] = for {
     serviceA <- this.serviceA
     serviceB <- this.serviceB
  } yield new HttpServerTask(serviceA, serviceB)
 }
 
 // Точка входа в приложение
 object Main extends IOApp.Simple {
   def run = Dependencies.server.use(_.run())
 }

Let’s understand what we see here. We can say that our object Depencies is a dependency implementation container (DI-container), a server is an exit dependency, that is, a dependency that will be used outside the container. The rest of the other dependencies are internal dependencies.

This code has one major problem: through lazy nature cats-effectthough it’s nice to think that conn – this val, it will actually be executed as many times as it is referenced, in this example twice, which we don’t want because then we’ll have two connection pools to the same database in the same application. The default dependency implementation scheme is to have only one instance of each dependency, unless otherwise specified for specific cases.

So usually this problem is solved in terms of functional thinking, and the first approach that comes to mind is to turn all this wiring into a function with a big for-comprehension that returns the original dependency, like here:

...
 
 // Здесь мы указываем наши выходные зависимости, оборачивая это все в case-класс для добавления новых выходных зависимостей в будущем
 case class Dependencies(server: HttpServerTask)
 
 object Dependencies {
   val conn: Resource[IO, ConnectionFactory] = ???
 
   def apply: Resource[IO, Dependencies] = for {
     conn <- this.conn
     repo <- new Repository(conn)
     serviceA <- new ServiceA(repo)
     serviceB <- new ServiceB(repo)
     server <- new HttpServerTask(serviceA, serviceB)
  } yield Dependencies(server)
   
 }

Because of its simplicity, this approach is widely used for small applications (example). But it has some disadvantages:

  • The presence of one large for-comprehension – does not expand the solution, which begins to look very bad when it grows due to the addition of new dependencies;

  • Adding additional source dependencies requires instantiating all of those dependencies as far as the dependency graph is concerned. Let’s say we have 2 APIs and 2 entry points (which will be deployed as 2 separate services), with this approach we need to either write 2 separate for-comprehensions, or instantiate them all together and use only one, which to say the least , not optimal;

  • If we want every dependency to be an output dependency, for example, to be able to use every dependency outside of a DI container, which makes java libraries like Google Guice easy to do, this approach is not for us.

Another approach is to take a step back to the individual dependencies and just try to initialize them, but track shutdown functions of initialized dependencies to be able to end them gracefully when the program stops.

For this we need a helper class Allocatorwhich will be responsible for initializing dependencies, tracking initialized dependencies and finalizing them:

import cats.effect.*
import cats.effect.unsafe.IORuntime

class Allocator(implicit runtime: IORuntime) {
  // Ссылка, которая будет отслеживать финализаторы
  private val shutdown: Ref[IO, IO[Unit]] = Ref.unsafe(IO.unit)

  // Метод аллокации зависимостей
  def allocate[A](resource: Resource[IO, A]): A =
    resource.allocated.flatMap { case (a, release) =>
      // Закрываем этот ресурс, а после его закрытия - все предшествующие
      shutdown.update(release *> _).map(_ => a)
  }.unsafeRunSync()

  // Завершение зависимостей
  def shutdownAll: IO[Unit] = {
    shutdown.getAndSet(IO.unit).flatten
  }

}

This class allows us to do non-idiomatic deployment of resources and monitor finalizers to complete them in the correct order. The non-idiomaticity will be fixed later, but for now, here’s how the code looks like using this simple class:

class Dependencies(val allocator: Allocator) {
  lazy val conn: ConnectionFactory = allocator.allocate {
     ???
  }
   
  lazy val repo = new Repository(conn)
    
  lazy val serviceA = new ServiceA(repo)
   
  lazy val serviceB = new ServiceB(repo)
   
  lazy val server = HttpServerTask(serviceA, serviceB)
}

As you can see, only one dependency was needed Allocatorbecause she had something to finalize (ConnectionFactory). The code has become much simpler and allows us to achieve what we want:

  • Dependency instantiation is still lazy, but now laziness is not achieved by laziness Resourcebut due to the mechanism lazy val Scala.

  • All dependencies can now be exposed as source dependencies. Each dependency can be freely used externally.

  • And again, the code is simple and expandable. We can group similar dependencies in separate classes Dependenciesusing only one instance method for them.

Now let’s deal with the problem, which is that we have violated the idiomaticity of the pattern. Resource inside the DI container. If you look at the class Allocatorthen it has a method in itself shutdownAllwhich allows us to now wrap our entire DI container into a single monad Resource:

object Dependencies {
  // Безопасный метод для создания зависимостей:
  def apply(runtime: IORuntime): Resource[IO, Dependencies] =
    Resource.make {
      IO(unsafeCreate(runtime))
   } {
      _.allocator.shutdownAll
   }
 ​
  // Небезопасный метод, используйте его с осторожностью, так как здесь не выполняется завершение:
  def unsafeCreate(runtime: IORuntime): Dependencies =
    new Dependencies(new Allocator()(runtime))
}
​
class Dependencies(val allocator: Allocator) {
  ...
}
​
// Запускающий класс
object Main extends IOApp.Simple {
  // Теперь мы можем использовать любую зависимость, а не только server, так как все они могут быть выходными:
  private val dependencies = Dependencies(IORuntime.global)
  
  override def run = dependencies.use(_.server.run)
}

That’s all! This method is my favorite way to implement dependency injection using the Cats-Effect in Scala to date. Although breaking the idiomatic border Resource inside the container, it allows us to achieve dependency reuse in a simple and extensible way, and we still go back to the pattern Resource for external users that will call the DI container dependency.

This pattern also makes it easy to split dependencies into multiple containers, for example:

class AwsDependencies(val allocator: Allocator, config: Config) {
  val s3: AwsS3Client[IO] = allocator.allocate {
    Resource.fromAutoCloseable(IO.blocking {
      S3AsyncClient.builder()
        .region(config.region)
        .build()
    }).map(new AwsS3Client(_))
  }
}
 
class MainDependencies(val allocator: Allocator) {
  
  lazy val config = ???
  
  lazy val aws: AwsDependencies = new AwsDependencies(allocator, config.as[Config]("aws"))
  
  lazy val httpRoutes: Routes = new Routes(aws.s3)
  
}

Here is a complete example of the final code:

import cats.effect.*
import cats.effect.unsafe.IORuntime
import org.postgresql.core.ConnectionFactory

// repository, two services, and api:
class Repository(conn: ConnectionFactory) {}

class ServiceA(repo: Repository) {}

class ServiceB(repo: Repository) {}

class HttpServerTask(serviceA: ServiceA, serviceB: ServiceB) {
  def run: IO[Unit] = ???
}

class Allocator(implicit runtime: IORuntime) {
  // Ref that will keep track of finalizers
  private val shutdown: Ref[IO, IO[Unit]] = Ref.unsafe(IO.unit)

  // Method to allocate dependencies
  def allocate[A](resource: Resource[IO, A]): A =
    resource.allocated.flatMap { case (a, release) =>
      // Shutdown this resource, and after shutdown all previous
      shutdown.update(release *> _).map(_ => a)
    }.unsafeRunSync()

  // Shutdown dependencies
  def shutdownAll: IO[Unit] = {
    shutdown.getAndSet(IO.unit).flatten
  }

}

// Dependency Injection part:
object Dependencies {
  // Safe method to create dependencies:
  def apply(runtime: IORuntime): Resource[IO, Dependencies] =
    Resource.make {
      IO(unsafeCreate(runtime))
    } {
      _.allocator.shutdownAll
    }

  // Unsafe method, use it carefully as no shutdown is executed:
  def unsafeCreate(runtime: IORuntime): Dependencies =
    new Dependencies(new Allocator()(runtime))
}

class Dependencies(val allocator: Allocator) {

  lazy val conn: ConnectionFactory = allocator.allocate {
    ???
  }

  lazy val repo = new Repository(conn)

  lazy val serviceA = new ServiceA(repo)

  lazy val serviceB = new ServiceB(repo)

  lazy val server = HttpServerTask(serviceA, serviceB)
}

// Runner class
object Main extends IOApp.Simple {
  // Now we can use any dependency and not just `server` as all of them are exposed:
  private val dependencies = Dependencies(IORuntime.global)

  override def run = dependencies.use(_.server.run)
}

Finally, we invite all Scala developers to open classes at OTUS:

Related posts