Salute, Habre!

Go Kit provides a standardized way of creating services, with its help you can realize the compatibility of services. With its help, you can easily integrate various transport protocols such as HTTP, RPC, gRPC and many others, as well as implement common patterns: logging, metrics, tracing. In general, Go Kit is good for developing microservices on go.

The developers described the motivation for creating this lib as follows:

Go has become a server language, but it’s still underrepresented in so-called “modern enterprise” companies like Facebook, Twitter, Netflix, and SoundCloud. Many of these organizations have turned to JVM-based stacks to build their business logic, thanks in large part to the libraries and ecosystems that directly support their microservices architectures.

To reach the next level of success, Go needs more than simple primitives and idioms. It needs a comprehensive set of tools for sequential distributed programming in general. The Go Kit is a collection of packages and best practices that provide a comprehensive, reliable, and robust way to build microservices for organizations of all sizes.

It is also worth mentioning that the developers do not aim to implement the following functions:

• Support for messaging patterns other than RPC (for now) such as MPI, pub/sub, CQRS, etc.

• Re-implementation of functionality that can be provided by adapting existing software.

Installing the Go Kit:

go get -u github.com/go-kit/kit

Go Kit requires Go version 1.13 or higher.

Go Kit components

Services – This is the basis of microservice architecture. Each service is a separate component that performs a specific function or set of functions. In Go Kit, services are developed as sets of interfaces and implementations that separate business logic from the rest of the system.

Transport layer is a bridge between your services and the outside world. It is responsible for receiving requests from customers, processing them and transmitting data back to customers. Go Kit offers a system of transport layers supporting multiple protocols including HTTP, gRPC, Thrift, etc.

Endpoints are endpoints that clients call for implementation certain operations. Endpoints are responsible for processing incoming requests, executing corresponding service calls, and returning results back to clients.

Basic functions

Service layer

The service layer begins with the definition of the interface. The service interface defines the operations or actions that can be performed within the given service. It is an abstraction that hides the implementation details, allowing you to focus on what what does the service, not as he does it.

Suppose you need a microservice for user management. At the interface level, it might look like this:

package userservice

// userService определяет интерфейс для нашего сервиса управления пользователями.
type UserService interface {
    CreateUser(name string, email string) (User, error)
    GetUser(id string) (User, error)
}

UserService provides two operations: CreateUser to create a new user and GetUser to obtain information about the user regarding his ID. Return values ​​and errors indicate the result of each operation.

After defining an interface, the next step is to implement that interface. An implementation is the specific code that executes the logic described by the interface:

package userservice

import "errors"

// userService представляет реализацию нашего UserService.
type userService struct {
    // здесь различные зависимости, ссылки на бд и т.п
}

// NewUserService создает и возвращает новый экземпляр userService.
func NewUserService() UserService {
    return &userService{}
}

// CreateUser реализует логику создания пользователя.
func (s *userService) CreateUser(name string, email string) (User, error) {
    // логика создания нового пользователя.
    // проверка валидности данных и запись пользователя в базу данных
    return User{Name: name, Email: email}, nil
}

// GetUser реализует логику получения пользователя по ID.
func (s *userService) GetUser(id string) (User, error) {
    // логика поиска пользователя по его ID в базе данных.
    // если пользователь не найден, возвращается ошибка.
    return User{}, errors.New("user not found")
}

// ser представляет модель пользователя в нашей системе.
type User struct {
    ID    string
    Name  string
    Email string
}

userService is a private structure that implements the interface UserService. FunctionNewUserServicewe need to hide the details of creating a service instance and return the interface, and not the specific type.

Endpoint layer

Suppose we have a service UserService with the method GetUser, which we want to expose over HTTP. First, let’s define the endpoint:

import (
    "context"
    "github.com/go-kit/kit/endpoint"
)

// GetUserRequest определяет структуру запроса к endpoint.
type GetUserRequest struct {
    UserID string
}

// GetUserResponse определяет структуру ответа от endpoint.
type GetUserResponse struct {
    User  User   `json:"user,omitempty"`
    Err   string `json:"err,omitempty"` // ошибки не сериализуются по JSON напрямую.
}

// MakeGetUserEndpoint создает endpoint для метода GetUser сервиса UserService.
func MakeGetUserEndpoint(svc UserService) endpoint.Endpoint {
    return func(ctx context.Context, request interface{}) (interface{}, error) {
        req := request.(GetUserRequest)
        user, err := svc.GetUser(req.UserID)
        if err != nil {
            return GetUserResponse{User: user, Err: err.Error()}, nil
        }
        return GetUserResponse{User: user, Err: ""}, nil
    }
}

An endpoint that accepts the request has been created GetUserRequestpulls out UserID and uses it to call the method GetUser our service UserService. The response from the service is then turned into GetUserResponse.

Middleware allows you to add interception logic to request processing, for example, for logging, monitoring, authentication verification, etc., without changing the logic of the endpoints themselves.

Simply put, middleware is a function that takes an endpoint and returns another endpoint:

import (
    "context"
    "github.com/go-kit/kit/endpoint"
    "github.com/go-kit/kit/log"
)

// LoggingMiddleware возвращает Middleware, которое логирует запросы к сервису.
func LoggingMiddleware(logger log.Logger) endpoint.Middleware {
    return func(next endpoint.Endpoint) endpoint.Endpoint {
        return func(ctx context.Context, request interface{}) (response interface{}, err error) {
            logger.Log("msg", "calling endpoint")
            response, err = next(ctx, request)
            logger.Log("msg", "called endpoint")
            return
        }
    }
}

Here, the middleware logs messages before and after calling the original endpoint. You can apply this middleware to any endpoint service by passing it through MakeGetUserEndpointfor example, or to any other end.

Endpoints can be grouped. The grouping of endpoints is achieved by creating a set of endpoints, which is an aggregation of all endpoints associated with a certain service.

Let’s define several basic endpoints for our service example, which we will group. For example, there is ProfileServicewhich provides functionality for managing user profiles:

type ProfileService interface {
    CreateProfile(ctx context.Context, profile Profile) (string, error)
    GetProfile(ctx context.Context, id string) (Profile, error)
}

For each method of the service interface, we define the corresponding endpoint.

func makeCreateProfileEndpoint(svc ProfileService) endpoint.Endpoint {
    return func(ctx context.Context, request interface{}) (interface{}, error) {
        req := request.(createProfileRequest)
        id, err := svc.CreateProfile(ctx, req.Profile)
        return createProfileResponse{ID: id, Err: err}, nil
    }
}

func makeGetProfileEndpoint(svc ProfileService) endpoint.Endpoint {
    return func(ctx context.Context, request interface{}) (interface{}, error) {
        req := request.(getProfileRequest)
        profile, err := svc.GetProfile(ctx, req.ID)
        return getProfileResponse{Profile: profile, Err: err}, nil
    }
}

There are now multiple endings that can be grouped together. This is usually done by creating a structure that contains all of these endings as fields:

type Endpoints struct {
    CreateProfile endpoint.Endpoint
    GetProfile    endpoint.Endpoint
}

func MakeEndpoints(svc ProfileService) Endpoints {
    return Endpoints{
        CreateProfile: makeCreateProfileEndpoint(svc),
        GetProfile:    makeGetProfileEndpoint(svc),
    }
}

Structure Endpoints now aggregates all endpoints associated with ProfileServiceapparently – convenient.

After endpoints are grouped, they can be used in the transport layer (more on transport layers below). For example, when creating an HTTP server, you can reference these endpoints directly from the framework Endpoints:

func NewHTTPHandler(endpoints Endpoints) http.Handler {
    r := mux.NewRouter()

    r.Methods("POST").Path("/profiles").Handler(httptransport.NewServer(
        endpoints.CreateProfile,
        decodeHTTPCreateProfileRequest,
        encodeHTTPGenericResponse,
    ))

    r.Methods("GET").Path("/profiles/{id}").Handler(httptransport.NewServer(
        endpoints.GetProfile,
        decodeHTTPGetProfileRequest,
        encodeHTTPGenericResponse,
    ))

    return r
}

Transport layer

To create an HTTP server, transport functions are defined that transform HTTP requests into calls to your service and service responses back to HTTP responses:

package main

import (
    "context"
    "encoding/json"
    "net/http"
    "github.com/go-kit/kit/endpoint"
    "github.com/go-kit/kit/transport/http"
)

// сервис
type MyService interface {
    Add(a, b int) int
}

type myService struct{}

func (myService) Add(a, b int) int { return a + b }

// endpoint
func makeAddEndpoint(svc MyService) endpoint.Endpoint {
    return func(ctx context.Context, request interface{}) (interface{}, error) {
        req := request.(addRequest)
        v := svc.Add(req.A, req.B)
        return addResponse{V: v}, nil
    }
}

type addRequest struct {
    A int `json:"a"`
    B int `json:"b"`
}

type addResponse struct {
    V int `json:"v"`
}

// decode и encode функции
func decodeAddRequest(_ context.Context, r *http.Request) (interface{}, error) {
    var request addRequest
    if err := json.NewDecoder(r.Body).Decode(&request); err != nil {
        return nil, err
    }
    return request, nil
}

func encodeResponse(_ context.Context, w http.ResponseWriter, response interface{}) error {
    return json.NewEncoder(w).Encode(response)
}

func main() {
    svc := myService{}

    addEndpoint := makeAddEndpoint(svc)
    addHandler := http.NewServer(addEndpoint, decodeAddRequest, encodeResponse)

    http.Handle("/add", addHandler)
    http.ListenAndServe(":8080", nil)
}

We create a simple service MyService with the method Add, which is two numbers. An endpoint is then created that handles the request-response transformation logic. We use HTTP to process requests and responses http.NewServer.

A similar abstraction is used to create an HTTP client:

package main

import (
    "context"
    "encoding/json"
    "net/http"
    "github.com/go-kit/kit/endpoint"
    "github.com/go-kit/kit/transport/http"
)

func makeHTTPClient(baseURL string) MyService {
    var addEndpoint endpoint.Endpoint
    addEndpoint = http.NewClient(
        "POST",
        mustParseURL(baseURL+"/add"),
        encodeHTTPRequest,
        decodeHTTPResponse,
    ).Endpoint()

    return Endpoints{AddEndpoint: addEndpoint}
}

func encodeHTTPRequest(_ context.Context, r *http.Request, request interface{}) error {
    // код для кодирования запроса
}

func decodeHTTPResponse(_ context.Context, resp *http.Response) (interface{}, error) {
    var response addResponse
    if err := json.NewDecoder(resp.Body).Decode(&response); err != nil {
        return nil, err
    }
    return response, nil
}

Go Kit offers built-in support for gRPC:

Let’s define the service interface and the data structures it uses, .proto file fe:

syntax = "proto3";

package example;

service StringService {
  rpc Uppercase (UppercaseRequest) returns (UppercaseResponse) {}
  rpc Count (CountRequest) returns (CountResponse) {}
}

message UppercaseRequest {
  string str = 1;
}

message UppercaseResponse {
  string str = 1;
  string err = 2;
}

message CountRequest {
  string str = 1;
}

message CountResponse {
  int32 count = 1;
}

using protoc compiler with a plugin for Go, you can generate Go code that will be used to create a gRPC server:

protoc --go_out=. --go-grpc_out=. path/to/your_service.proto

Next, we implement the Go service using interfaces generated from .proto file:

package main

import (
    "context"
    "strings"
    "google.golang.org/grpc/codes"
    "google.golang.org/grpc/status"
    pb "path/to/your_service_package"
)

type stringService struct {
    pb.UnimplementedStringServiceServer
}

func (s *stringService) Uppercase(ctx context.Context, req *pb.UppercaseRequest) (*pb.UppercaseResponse, error) {
    if req.Str == "" {
        return nil, status.Errorf(codes.InvalidArgument, "Empty string")
    }
    return &pb.UppercaseResponse{Str: strings.ToUpper(req.Str)}, nil
}

func (s *stringService) Count(ctx context.Context, req *pb.CountRequest) (*pb.CountResponse, error) {
    return &pb.CountResponse{Count: int32(len(req.Str))}, nil
}

run:

package main

import (
    "log"
    "net"
    "google.golang.org/grpc"
    pb "path/to/your_service_package"
)

func main() {
    lis, err := net.Listen("tcp", ":50051")
    if err != nil {
        log.Fatalf("failed to listen: %v", err)
    }
    var opts []grpc.ServerOption
    grpcServer := grpc.NewServer(opts...)
    pb.RegisterStringServiceServer(grpcServer, newStringService())
    grpcServer.Serve(lis)
}

It is here as well as in the endpoints Middlewarewhich in the transport layer allows you to embed additional processing logic for incoming and outgoing requests/responses:

package main

import (
    "context"
    "fmt"
    "github.com/go-kit/kit/endpoint"
    "github.com/go-kit/log"
)

// LoggingMiddleware возвращает Middleware, которое логирует детали запроса
func LoggingMiddleware(logger log.Logger) endpoint.Middleware {
    return func(next endpoint.Endpoint) endpoint.Endpoint {
        return func(ctx context.Context, request interface{}) (response interface{}, err error) {
            logger.Log("msg", "calling endpoint")
            defer func() {
                logger.Log("msg", "called endpoint", "err", err)
            }()
            return next(ctx, request)
        }
    }
}

Other possibilities

He has a cat logging abstractionwhich allows you to easily integrate any logging system with your services. For example, my belovedlog.Loggerwhich is distinguished by its minimalism:

import (
    "github.com/go-kit/log"
    "github.com/sirupsen/logrus"
)

type logrusLogger struct {
    *logrus.Logger
}

func (l logrusLogger) Log(keyvals ...interface{}) error {
    // здесь может быть реализация адаптация аргументов keyvals
    // для logrus или другой логики адаптации.
    l.Logger.WithFields(logrus.Fields{"keyvals": keyvals}).Info()
    return nil
}

// экземпляр Logger Go Kit, используя logrus
logger := logrusLogger{logrus.New()}

It is possible, perhaps integrate with metrics systemsFor example with Prometheus:

import (
    "github.com/go-kit/kit/metrics/prometheus"
    stdprometheus "github.com/prometheus/client_golang/prometheus"
)

var requestCount = prometheus.NewCounterFrom(stdprometheus.CounterOpts{
    Namespace: "my_namespace",
    Subsystem: "my_subsystem",
    Name:      "request_count",
    Help:      "Number of requests received.",
}, []string{"method"})

It is possible, perhaps integrate with tracking systemsfor example Jaeger:

import (
    "github.com/go-kit/kit/tracing/opentracing"
    "github.com/opentracing/opentracing-go"
    "github.com/uber/jaeger-client-go/config"
)

// сеттинги Jaeger
cfg, _ := config.FromEnv()
tracer, _, _ := cfg.NewTracer()

// трассировка в endpoint
tracedEndpoint := opentracing.TraceServer(tracer, "my_endpoint")(myEndpoint)

In the examples, error processing was already implemented, but I think this function should be included in this section, for example, user processing.

import (
    "errors"
    "net/http"
    "github.com/go-kit/kit/transport/http"
)

var ErrInvalidArgument = errors.New("invalid argument")

// прнбразование ошибки в HTTP статус
errorEncoder := func(ctx context.Context, err error, w http.ResponseWriter) {
    code :=

 http.StatusInternalServerError
    if err == ErrInvalidArgument {
        code = http.StatusBadRequest
    }
    w.WriteHeader(code)
    json.NewEncoder(w).Encode(map[string]interface{}{
        "error": err.Error(),
    })
}

Thus, typical problems in distributed systems and application architecture can be solved with go kit. Go Kit on GitHub, Go Kit site

---

The article was prepared on the eve of the start of the course “Microservice Architecture” by OTUS.