High-performance caching using Redis and Go

High-performance caching using Redis and Go

Go is an excellent language for building high-performance web applications, and high-performance web applications often require centralized caching.

Using Redis in Golang improves application performance by accelerating data access. In addition, Redis provides many features such as caching the results of database queries, saving intermediate calculations and storing temporary data, which can significantly reduce the load on the database server.

The de facto standard for centralized caching is RedisHowever, popular libraries do not support memory-efficient data streaming.

In my telegram you will find many projects with tutorials, guides and code for Golang developers

I also collected a useful folder for Golang developers, there is everything a Go programmer needs.

Instead, they offer []byte APIs that you interact with as follows:

// В этом коде используется https://github.com/redis/go-redis, но те же
// ограничения действуют для Rueidis и Redigo.
func redisHandler(w http.ResponseWriter, r *http.Request) {
ctx := context.Background()
// Извлечение ключа из RequestURI
key := strings.TrimLeft(r.RequestURI, "/")

// Получить значение из Redis в виде байтового фрагмента
val, err := rdb.Get(ctx, key).Bytes()
if err == redis.Nil {
	http.Error(w, "Key not found in Redis", http.StatusNotFound)
	return
} else if err != nil {
	http.Error(w, err.Error(), http.StatusInternalServerError)
	return
}

_, err = w.Write(val)
if err != nil {
	http.Error(w, err.Error(), http.StatusInternalServerError)
}

}

This pattern works well if you are caching small objects, but if you are caching objects larger than 1kb, []byte-oriented APIs do not perform as well.

There is nothing in the Redis protocol that prevented you from creating a streaming API. That’s why we present redjet, a performance-oriented Redis library.

Redjet is a high performance Go library for Redis. Its distinguishing feature is a streaming API with low resource allocation. More detailed information can be found in the benchmarks section.

Installing the library:

go get github.com/coder/redjet@latest

By using redjet you can write the above code like this:

func redisHandler(w http.ResponseWriter, r *http.Request) {
ctx := context.Background()
// Извлечение ключа из RequestURI
key := strings.TrimLeft(r.RequestURI, "/")

// Передаем значение непосредственно из Redis в ответ.
_, err := rdb.Command("GET", key).WriteTo(w)
if err != nil {
	http.Error(w, err.Error(), http.StatusInternalServerError)
}

}

The code has become simpler and more efficient.

Streaming recording

Popular Redis libraries suffer from the same problem when it comes to writing values ​​in Redis. They require you to store the entire value in memory as a form []byte before sending it.

With redjet, you can pass Redis values ​​as follows:

fi := strings.NewReader("Содержимое файла")
err := rdb.Command("SET", "key", fi).Ok()
// обработка ошибки

There is an important caveat here. In the protocol Redis values ​​are prefixed with length, so we can’t stream the record via io.Reader. I guess this is one of the main reasons why popular libraries don’t support streaming.

To get around this, redjet necessary redjet.LenReaderWhich is defined as:

type LenReader interface {
Len() int
io.Reader
}

and can be created using redjet.NewLenReader:

Conveniently, some types in the standard library, such as bytes.Reader, strings.Reader, and bytes.Buffer, implicitly implement redjet.LenReader.

Benchmarks

Redjet compared with benchmarks:

Time for surgery. If we consider a read of 1 kb, then the results

All benchmark results are available here.

Related posts