Event logging for analytics and monitoring of Telegram bot

Event logging for analytics and monitoring of Telegram bot

Hello, Habre! My name is Artem.

As the variety of work with the Telegram bot of our corporate Telegram bot grew, optimizing and maintaining it became a more complex task. At some point, I realized that I needed a reliable way to keep track of events happening in the application. This is important not only for solving problems and bugs, but also for optimizing productivity and improving overall work efficiency.

In this article, we will look at how to implement event logging for a Telegram bot.

Environment settings

For those who are into the topic of the telebot library and generally know about creating telegram bots, this block can be skipped.

Let’s consider in more detail what technology stack we will need:

  1. Python: The obvious choice Python is an easy-to-learn and powerful programming language that allows you to create bots with minimal effort.

  2. Telebot Library: To interact with the Telegram API, we will use the Telebot library (or similar libraries such as python-telegram-bot). It is a convenient tool that makes interaction with the Telegram API simpler and more intuitive.

Before we start working with the bot, let’s make sure we have the necessary libraries installed. To do this, follow these steps:

  1. Installing Telebot: To install Telebot, use pip, Python’s standard package manager:

    pip install pyTelegramBotAPI
  2. Telegram Bot settings: You will need a Telegram account to create a bot and receive a token needed to interact with the Telegram API. Create a new bot through the official Telegram bot BotFather by following the instructions.

After successfully setting up your environment and creating your bot through BotFather, you will be provided with a unique token for your bot. This token will be the key to interact with the Telegram API. Let’s create a simple example of connecting to the Telegram API using Telebot:

import telebot

# Замените 'YOUR_BOT_TOKEN' на фактический токен вашего бота
bot = telebot.TeleBot('YOUR_BOT_TOKEN')

# Пример обработчика команды /start
@bot.message_handler(commands=['start'])
def handle_start(message):
    bot.send_message(message.chat.id, "Привет! Я твой телеграм-бот. Как я могу помочь?")

# Запуск бота
bot.polling(none_stop=True)

This code creates a bot that responds to a command /start and sends a welcome message. replace 'YOUR_BOT_TOKEN' to your bot’s actual token. At this point, you can already send messages to your work via Telegram.

Implementation of event logging

The event logging class will be the central part of our logging system and will help us organize log messages into a structured form. Let’s create an example of a logging class in Python:

import logging

class BotLogger:
    def __init__(self, log_file):
        self.logger = logging.getLogger("bot_logger")
        self.logger.setLevel(logging.DEBUG)
        formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
        file_handler = logging.FileHandler(log_file)
        file_handler.setFormatter(formatter)
        self.logger.addHandler(file_handler)
    
    def log(self, level, message):
        if level == 'info':
            self.logger.info(message)
        elif level == 'error':
            self.logger.error(message)
        elif level == 'debug':
            self.logger.debug(message)

This class BotLogger creates a logger that writes log messages to a file. You can configure the logging level (eg logging.DEBUGto record all events) and message format as you see fit.

Defining events and logging levels helps us categorize messages and highlight the most important aspects of the robot’s performance. Some examples of events and their levels:

  • Information event (INFO): Here we can log important events such as user requests, sending messages and other bot actions.

  • Error (ERROR): In case of errors or exceptions in the operation of the bot, logging with the level ERROR will help us quickly identify and resolve the issue.

  • Debug event (DEBUG): Using DEBUG level, we can log detailed information about the bot’s internal processes, which is especially useful during development and debugging.

Basic logging methods

  1. Information logging (INFO):

def log_info(self, message):
    self.log('info', message)

Example of use:

logger.log_info("Получен запрос от пользователя: /help")
  1. Error logging (ERROR):

def log_error(self, message):
    self.log('error', message)

Example of use:

try:
    # Ваш код, который может вызвать ошибку
except Exception as e:
    logger.log_error(f"Ошибка: {str(e)}")
  1. Debugging data logging (DEBUG):

def log_debug(self, message):
    self.log('debug', message)

Example of use:

# Подробная информация о внутренних процессах бота
logger.log_debug("Получено обновление от Telegram API")

Organization of the structure of log messages

Organizing the structure of log messages helps us to easily read and analyze the logs. In our class BotLogger we used log message formatting like this:

%(asctime)s - %(name)s - %(levelname)s - %(message)s
  • %(asctime)s: Date and time of the event

  • %(name)s: Logger name (in this case, “bot_logger”)

  • %(levelname)s: Logging level (INFO, ERROR, DEBUG).

  • %(message)s: Own message

This structure allows us to easily identify and analyze events in the logs. You can customize the message format to suit your needs.

Integration with bot

How to connect the previously created logging class to your robot:

# Импортируем наш класс логирования
from bot_logger import BotLogger

# Создаем экземпляр логгера
logger = BotLogger('bot.log')

# Далее в коде бота
@bot.message_handler(func=lambda message: True)
def handle_message(message):
    # Обработка сообщений от пользователей
    user_message = message.text
    logger.log_info(f"Получено сообщение от пользователя: {user_message}")
    
    # Отправка ответа
    bot.reply_to(message, "Ваш запрос получен и обработан.")

In this example, we imported our logging class BotLoggercreated a logger instance logger and started using it in the message handler. Each incoming message will now be logged with the message text and logging level INFO.

When developing a bot, it is important to log both incoming messages and other events, such as processing commands, sending responses, and internal bot actions. Examples of logging these events:

@bot.message_handler(commands=['start'])
def handle_start(message):
    # Обработка команды /start
    user_id = message.from_user.id
    logger.log_info(f"Пользователь {user_id} запустил бота с командой /start")
    bot.send_message(message.chat.id, "Добро пожаловать в нашего бота!")

@bot.message_handler(func=lambda message: True)
def handle_message(message):
    # Обработка обычных сообщений
    user_message = message.text
    user_id = message.from_user.id
    logger.log_info(f"Пользователь {user_id} отправил сообщение: {user_message}")
    
    # Ваш код для обработки сообщения
    
    bot.reply_to(message, "Ваш запрос получен и обработан.")

In this code, we log events related to the command /start and regular messages. This allows us to track user activity and provide more effective interaction with them.

After logging your bot’s events, it’s important to pay attention processing and storage of log data. You can implement automatic cleaning of old logs or their archiving in order not to overload the disk. Additionally, you can use log analysis tools like ELK to get a more detailed view of your bot’s performance and identify problem areas.

An example of storing logs in a file (by rotation):

import logging
from logging.handlers import RotatingFileHandler

log_handler = RotatingFileHandler('bot.log', maxBytes=1e6, backupCount=5)
log_handler.setLevel(logging.INFO)
formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
log_handler.setFormatter(formatter)

logger.addHandler(log_handler)

Here we used RotatingFileHandler to store leagues in a file with rotation when a certain size is reached. This will help avoid overflowing the disk with logs.

Monitoring and alerts

  1. Monitoring of robot activity: You can track how many requests and messages your bot processes per unit of time. This can help you understand which parts of your bot’s functionality are most active.

    import telebot
    
    bot = telebot.TeleBot('YOUR_BOT_TOKEN')
    
    # Логирование входящих сообщений
    @bot.message_handler(func=lambda message: True)
    def log_message(message):
        # Логируем информацию о сообщении
        log_info(f"Received message: {message.text}")
    
    # ... остальной код бота
    
    bot.polling()
  2. Error and exception tracking: Log monitoring allows you to detect and respond to errors that may occur in the bot’s operation. For example, you can notify about errors through alerts.

    import telebot
    
    bot = telebot.TeleBot('YOUR_BOT_TOKEN')
    
    # Логирование ошибок
    @bot.message_handler(func=lambda message: True)
    def process_message(message):
        try:
            # Обработка сообщения
            pass
        except Exception as e:
            log_error(f"Error processing message: {str(e)}")
    
    # ... остальной код бота
    
    bot.polling()
  3. Performance monitoring: You can log information about the execution time of various operations in your bot. This will help you identify bottlenecks and optimize them.

    import telebot
    import time
    
    bot = telebot.TeleBot('YOUR_BOT_TOKEN')
    
    # Логирование времени выполнения операции
    def some_time_consuming_operation():
        start_time = time.time()
        # Выполняем долгую операцию
        end_time = time.time()
        log_info(f"Operation took {end_time - start_time} seconds")
    
    # ... остальной код бота
    
    bot.polling()

Setting up notifications in case of problems

When monitoring logs reveals a problem, it’s important to react quickly. To do this, configure the notification system. You can set up notifications via Telegram and email:

  1. Alerts via Telegram:

    import telebot
    
    bot = telebot.TeleBot('YOUR_BOT_TOKEN')
    chat_id = 'YOUR_CHAT_ID'
    
    def send_telegram_notification(message):
        bot.send_message(chat_id, message)
    
    # ... остальной код бота
    
    # Где-то в обработчике ошибки или мониторинге
    send_telegram_notification("Произошла ошибка в боте!")
    
    bot.polling()
  2. Email notification:

    import smtplib
    from email.mime.text import MIMEText
    
    def send_email_notification(subject, message):
        # Настройте SMTP сервер и учетные данные
        smtp_server="smtp.example.com"
        smtp_port = 587
        smtp_username="your_username"
        smtp_password = 'your_password'
    
        sender="[email protected]"
        recipient="[email protected]"
    
        msg = MIMEText(message)
        msg['Subject'] = subject
        msg['From'] = sender
        msg['To'] = recipient
    
        try:
            with smtplib.SMTP(smtp_server, smtp_port) as server:
                server.login(smtp_username, smtp_password)
                server.sendmail(sender, recipient, msg.as_string())
        except Exception as e:
            log_error(f"Error sending email notification: {str(e)}")
    
    # ... остальной код бота
    
    # Где-то в обработчике ошибки или мониторинге
    send_email_notification("Ошибка в боте", "Произошла ошибка в работе бота!")

It’s important to customize these alerts based on your needs and infrastructure.

Login security

Login security is very important when developing bots, especially if they handle sensitive data.

Data protection in logs (masking, encryption)

Data protection in logs can be achieved by masking sensitive information such as passwords or keys, and by encrypting logs.

Masking of confidential data:

import logging

logger = logging.getLogger(__name__)

def log_sensitive_data(data):
    # Маскируем конфиденциальные данные, например, заменяем пароль на звездочки
    sanitized_data = data.replace("password=SECRET", "password=******")
    logger.info("Отправлены данные: %s", sanitized_data)

Log encryption:

import logging
import cryptography

# Используем библиотеку cryptography для шифрования
from cryptography.fernet import Fernet

logger = logging.getLogger(__name__)

# Генерируем ключ для шифрования (ключ должен быть безопасно сохранен)
key = Fernet.generate_key()
cipher_suite = Fernet(key)

def encrypt_and_log_data(data):
    encrypted_data = cipher_suite.encrypt(data.encode())
    logger.info("Зашифрованные данные: %s", encrypted_data)

def decrypt_data(encrypted_data):
    decrypted_data = cipher_suite.decrypt(encrypted_data)
    return decrypted_data.decode()

Limiting access to logs

Limiting access to logs is important to prevent unauthorized access to sensitive data.

Restriction of access rights to log files:

import logging

logger = logging.getLogger(__name__)

def main():
    # ... инициализация бота ...

    # Создаем файловый обработчик логов
    log_file = "bot.log"
    file_handler = logging.FileHandler(log_file)

    # Ограничиваем права доступа к файлу
    file_handler.setLevel(logging.INFO)
    file_handler.setFormatter(logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s'))

    logger.addHandler(file_handler)

    # ... остальной код бота ...

if __name__ == "__main__":
    main()

This will restrict access to the log file to only users with read access rights.

Restrict access to logs using a password:

from flask import Flask, request, send_from_directory, Response

app = Flask(__name)

# Устанавливаем пароль для доступа к логам
password = "your_password"

@app.route('/logs')
def logs():
    if request.args.get('password') == password:
        log_file = "bot.log"
        return send_from_directory('.', log_file)
    else:
        return Response("Недостаточно прав доступа", 401)

if __name__ == "__main__":
    app.run()

In this example, access to the log file can only be obtained with the correct password.

Compliance with regulations and laws on log storage

Compliance with league storage regulations and laws depends on the location of your bot and regulatory authorities. The main thing is to store logs in a safe place, limit access to them and destroy outdated data in accordance with the law.

Log audit:

Auditing logs is another important aspect of security. Your code should support auditing of those events that may be important to identify security breaches.

import logging

logger = logging.getLogger(__name__)

def audit_log(event_type, description, user_id=None):
    logger.info("Событие: %s, Описание: %s, Пользователь: %s", event_type, description, user_id)

This allows you to track important actions and events in the work, which can be useful for detecting anomalies or security breaches.

Regular updates of libraries and tools:

To ensure login security, remember to regularly update the libraries you use. Updates may include fixes for vulnerabilities and security improvements.

Training people who have access to the bot:

And of course, an equally important aspect is the training of staff on the rules of secure logging and log management. This will help to avoid human errors that can lead to leakage of confidential information.

Conclusion

We hope you find this guide useful and inspired to apply this knowledge to your future projects. Event logging and unique monitoring will help make your bots more reliable and resilient, which in turn will provide a more positive experience for your users and business as a whole.

And not only experts from OTUS tell more about architectural solutions in the framework online courses. Go to the catalog and choose the direction that interests you.

Related posts