Automatic Generation of Configurations for Make Assemblies / Habr

Automatic Generation of Configurations for Make Assemblies / Habr

Prologue

When developing software (especially for microcontrollers), sooner or later you will have to face the fact that you need to transfer configurations for a given software project.

In my experience, I’ve found that in terms of scaling the codebase, it’s easiest to pass configs via environment variables. Yes.. The plus is that environment variables can be defined by writing them directly in scripts (Make, CMake, etc.).

It looks like this. Each assembly has a file config.mk which lists the software components from which this particular assembly should be built. The contents of the file usually look like this.


ADC=Y
ADC_ISR=Y
ADT=Y
ALLOCATOR=Y
ARRAY=Y
ASICS=Y

....

TASK=Y
TBFP=Y
TERMINAL=Y
TIME=Y
UART0=Y
UART2=Y
UNIT_TEST=Y
UTILS=Y
UWB=Y

These are just atomic strings. This is where environment variables are defined. It is declaratively enumerated from which to assemble the firmware. Another build has its own declarative config.mk file and its own set of environment variables.

These same environment variables decide which files to include in the compilation and which to exclude. For example, the environment variable SD_CARD=Y will add this make script to the assembly


ifneq ($(SD_CARD_MK_INC),Y)
    SD_CARD_MK_INC=Y

    $(info + SD card SPI driver)

    SD_CARD_DIR = $(ASICS_DIR)/sd_card
    #@echo $(error SD_CARD_DIR= $(SD_CARD_DIR))

    INCDIR += -I$(SD_CARD_DIR)

    OPT += -DHAS_CRC7
    OPT += -DHAS_CRC16
    OPT += -DHAS_SD_CARD
    OPT += -DHAS_SD_CARD_CRC7
    OPT += -DHAS_SD_CARD_CRC16

    SOURCES_C += $(SD_CARD_DIR)/sd_card_drv.c
    SOURCES_C += $(SD_CARD_DIR)/sd_card_crc.c
    SOURCES_C += $(SD_CARD_DIR)/sd_card_crc16.c

    ifeq ($(DIAG),Y)
        ifeq ($(SD_CARD_DIAG),Y)
            $(info + SD card diag)
            OPT += -DHAS_SD_CARD_DIAG
            SOURCES_C += $(SD_CARD_DIR)/sd_card_diag.c
        endif
    endif

    ifeq ($(CLI),Y)
        ifeq ($(SD_CARD_COMMANDS),Y)
            $(info + SD card commands)
            OPT += -DHAS_SD_CARD_COMMANDS
            SOURCES_C += $(SD_CARD_DIR)/sd_card_commands.c
        endif
    endif
endif

Enough inside config.mk register SD_CARD=Y and the scripts themselves will add everything for compiling C code: source code, paths to source code and preprocessor directives. You replaced the single c character “Y” with “N” in the scripts and the make scripts automatically excluded the source code, source code paths, and preprocessor directives for that particular software component from the build. Easy!

Environment variables inside config.mk, attention, sorted! And it is very convenient to compare the configs of two different assemblies in the WinMerge utility.

As you know, computer programs are built in a hierarchical manner. Here, for example, the software component of the CLI command line interface requires that such software components as UART, TIMER, recognizer of numbers from a string, and other MISC trifles are already implemented in the program.

What is the problem?
The problem is that there are many environment variables (configs). If you build via make scripts, you will have many environment variables for each specific build. Each config.mk will have 100 … 200 lines and you can accidentally forget to register some important environment variable that activates some config. And without which the firmware will not work properly in run-time. Here is an example of a typical config.mk

ADC=N
ADC_ISR=Y
ALLOCATOR=Y
ARRAY=Y
ASICS=Y
AUDIO=Y
BIN_2_STR=Y
BOARD=Y
BOARD_INFO=Y
BOARD_UTILS=Y
BUTTON=Y
CLI=Y
CLOCK=Y
CMSIS=Y
COMMON=Y
COMPLEX=Y
COMPONENTS=Y
CONNECTIVITY=Y
CORE=Y
CORE_APP=Y
CORE_EXT=Y
CORTEX_M33=Y
CRC16=Y
CRC8=Y
CRC=Y
CSV=Y
CUSTOM_PRINTF=Y
DATA_POC=Y
DEBUG=Y
DEBUGGER=Y
DFT=Y
DIAG=Y
DRIVERS=Y
DSP=Y
DYNAMIC_SAMPLES=Y
FIFO=Y
FIFO_CHAR=Y
FIFO_INDEX=Y
FLASH=Y
FLASH_EX=Y
FLASH_FS=Y
FLASH_FS_WRITE=Y
FLASH_WRITE=Y
GENERIC=Y
GPIO=Y
HEALTH_MONITOR=Y
I2C1=Y
I2C=Y
I2S0=Y
I2S0_MASTER=Y
I2S=Y
I2S_ISR=Y
INDICATION=Y
INTERFACES=Y
LED=Y
LED_MONO=Y
LED_VERIFY=Y
LIMITER=Y
LOG=Y
LOG_COLOR=Y
LOG_DIAG=Y
LOG_TIME_STAMP=Y
LOG_UTILS=Y
MATH=Y
MATH_VECTOR=Y
MCAL=Y
MCAL_NRF5340=Y
MICROCONTROLLER=Y
MISCELLANEOUS=Y
MULTIMEDIA=Y
NORTOS=Y
NRF5340=Y
NRF5340_APP=Y
NRF5340_DK=Y
NRFX=Y
NVIC_COMMANDS=Y
NVS=Y
NVS_WRITE=Y
PARAM=Y
PARAM_SET=Y
PCM_16_BIT=Y
PINS=Y
PROTOCOLS=Y
REAL_SAMPLE_ARRAY=N
SENSITIVITY=Y
SOFTWARE_TIMER=Y
STORAGE=Y
STR2_DOUBLE=Y
STREAM=Y
STRING=Y
STRING_PARSER=Y
SUPER_CYCLE=Y
SW_DAC=Y
SW_DAC_STATIC_SAMPLES=Y
SYSTEM=Y
SYSTICK=Y
SYS_INIT=Y
TABLE_UTILS=Y
TASK=Y
TERMINAL=Y
TEST_HW=Y
TEST_SW=Y
THIRD_PARTY=Y
TIME=Y
TIMER0=Y
TIMER1=Y
TIMER2=Y
TIMER=Y
UART0=Y
UART2=Y
UART=Y
UART_INTERRUPT=Y
UART_ISR=Y
UNIT_TEST=Y
WM8731=Y
WM8731_I2S_SLAVE=Y
WM8731_USB_MODE=Y
WM8731_VERIFY=Y
WRITE_ADDR=Y

In the Zephyr Project, the problem of forgotten configs is partially solved by such a mechanism as KConfig. If you have not registered the config, then KConfig will issue a build error or will automatically silently add the required config and continue building. Unfortunately, however, there is no stand alone KConfig.exe utility that can be used in any build on Windows unrelated to the Zephyr Project. Like this…

Decision

It is obvious that it is necessary to make sure that, at the stage of working out the make scripts, the forgotten configs for the dependencies of those software components that we initially selected in the file are somehow automatically and magically registered config.mk.

It is necessary to make sure that the configs are registered automatically.

First of all, you need to create a separate file for each software component, which will contain information about its dependencies. Otherwise, how does the assembly system guess what else needs to be connected? You can name this file xxx_preconfig.mk. For example, the nvram_preconfig.mk file. It is obvious that such software components as CRC8 and MCAL for Flash peripherals are necessary for the operation of the on-chip NVRAM code. In this connection, the necessary environmental variables are determined.

ifneq ($(NVRAM_PRECONFIG_INC),Y)
    NVRAM_PRECONFIG_INC=Y

    NVRAM=Y
    FLASH=Y
    NVS=Y
    NVRAM_PROC=Y
    CRC=Y
    CRC8=Y
endif

If the nvram_preconfig.mk script works in the assembly scripts, then you will not have to write CRC8=Y, FLASH=Y, NVS=Y, etc. in the config.mk file. You will only need to register NVRAM=Y. Then everything else will be displayed by itself.

And this is the preconfig for SD card as SPI.


ifneq ($(SD_CARD_PRECONFIG_MK_INC),Y)
    SD_CARD_PRECONFIG_MK_INC=Y

    CRC7=Y
    SPI=Y
    GPIO=Y
    CRC16=Y
    SD_CARD=Y
    SD_CARD_CRC7=Y
    SD_CARD_CRC16=Y
endif

In fact, the basic idea behind this trick is taken from the ideology of CMake. CMake compiles the config, then the build system (make, ninja or IDE) compiles the project itself. Only in this case, everything is easier. Both the config and the project are collected by the collection system itself! Here, the comprehensive make utility does it.

Here is a simplified root file code_base.mk compilation of the used repository

ifneq ($(CODE_BASE_MK),Y)
    CODE_BASE_MK=Y

    include $(WORKSPACE_LOC)/code_base_preconfig.mk

    #preconfig/presets done!

    ifeq ($(CORE),Y)
        include $(WORKSPACE_LOC)/core/core.mk
    endif

    ifeq ($(MICROCONTROLLER),Y)
        include $(WORKSPACE_LOC)/microcontroller/microcontroller.mk
    endif

    ifeq ($(BOARD),Y)
        include $(WORKSPACE_LOC)/boards/boards.mk
    endif

    ifeq ($(THIRD_PARTY),Y)
        include $(WORKSPACE_LOC)/third_party/third_party.mk
    endif

    ifeq ($(APPLICATIONS),Y)
        include $(WORKSPACE_LOC)/applications/applications.mk
    endif

    ifeq ($(MCAL),Y)
        include $(WORKSPACE_LOC)/mcal/mcal.mk
    endif

    ifeq ($(ADT),Y)
        include $(WORKSPACE_LOC)/adt/adt.mk
    endif

    ifeq ($(CONNECTIVITY),Y)
        include $(WORKSPACE_LOC)/connectivity/connectivity.mk
    endif

    ifeq ($(CONTROL),Y)
        include $(WORKSPACE_LOC)/control/control.mk
    endif
    
    ifeq ($(COMPONENTS),Y)
        include $(WORKSPACE_LOC)/components/components.mk
    endif

    ifeq ($(COMPUTING),Y)
        include $(WORKSPACE_LOC)/computing/computing.mk
    endif

    ifeq ($(SENSITIVITY),Y)
        include $(WORKSPACE_LOC)/sensitivity/sensitivity.mk
    endif

    ifeq ($(STORAGE),Y)
        include $(WORKSPACE_LOC)/storage/storage.mk
    endif

    ifeq ($(SECURITY),Y)
        include $(WORKSPACE_LOC)/security/security.mk
    endif

    ifeq ($(ASICS),Y)
        include $(WORKSPACE_LOC)/asics/asics.mk
    endif

    ifeq ($(UNIT_TEST),Y)  
        include $(WORKSPACE_LOC)/unit_tests/unit_test.mk
    endif

    ifeq ($(MISCELLANEOUS),Y)
        include $(WORKSPACE_LOC)/miscellaneous/miscellaneous.mk
    endif
endif

Please note that code_base_preconfig.mk

include $(WORKSPACE_LOC)/code_base_preconfig.mk

is called before the drafting of the project itself. What is code_base_preconfig.mk? This is just a script for automatic placement of configs that we forgot about when compiling config.mk.

ifneq ($(CODE_BASE_PRECONFIG_MK),Y)
    CODE_BASE_PRECONFIG_MK=Y

    ifeq ($(BOARD),Y)
        include $(WORKSPACE_LOC)/boards/boards_preconfig.mk
    endif

    ifeq ($(MICROCONTROLLER),Y)
        include $(WORKSPACE_LOC)/microcontroller/microcontroller_preconfig.mk
    endif

    ifeq ($(CORE),Y)
        include $(WORKSPACE_LOC)/core/core_preconfig.mk
    endif

    ifeq ($(MCAL),Y)
        include $(WORKSPACE_LOC)/mcal/mcal_preconfig.mk
    endif

    ifeq ($(ADT),Y)
        include $(WORKSPACE_LOC)/adt/adt_preconfig.mk
    endif

    ifeq ($(CONNECTIVITY),Y)
        include $(WORKSPACE_LOC)/connectivity/connectivity_preconfig.mk
    endif

    ifeq ($(CONTROL),Y)
        include $(WORKSPACE_LOC)/control/control_preconfig.mk
    endif
    
    ifeq ($(COMPONENTS),Y)
        include $(WORKSPACE_LOC)/components/components_preconfig.mk
    endif

    ifeq ($(COMPUTING),Y)
        include $(WORKSPACE_LOC)/computing/computing_preconfig.mk
    endif

    ifeq ($(SENSITIVITY),Y)
        include $(WORKSPACE_LOC)/sensitivity/sensitivity_preconfig.mk
    endif

    ifeq ($(STORAGE),Y)
        include $(WORKSPACE_LOC)/storage/storage_preconfig.mk
    endif

    ifeq ($(SECURITY),Y)
        include $(WORKSPACE_LOC)/security/security_preconfig.mk
    endif

    include $(WORKSPACE_LOC)/asics/asics_preconfig.mk
    
endif

The same nvram_preconfig.mk and sd_card_preconfig.mk scripts will be called somewhere inside storage_preconfig.mk. Etc.

This way your initial config.mk can be simplified to look like this

CLI=Y
COMPLEX=Y
CSV=Y
DEBUG=Y
NVRAM=Y
DEBUGGER=Y
DFT=Y
DIAG=Y
DSP=Y
DYNAMIC_SAMPLES=Y
GENERIC=Y
I2C1=Y
I2S0_MASTER=Y
NRF5340_DK=Y
NORTOS=Y
TASK=Y
TIMER0=Y
TIMER1=Y
TIMER2=Y
UART0=Y
UART2=Y
UNIT_TEST=Y
WM8731_I2S_SLAVE=Y
WM8731_USB_MODE=Y

The rest will be arranged by preconfig automatically! The root config.mk has become 10 times easier! Success!

Result
As you can see, assembly from scripts gives such bonuses as the ability to automatically arrange configurations!

A technology has been developed for simple automatic registration of configs of dependencies of software components based on the dependencies specified in files xxx_preconfig.mk

I hope this text will help other programmers in developing their programs.

Links

Why it is Important to Collect Code from Scriptshttps://habr.com/ua/articles/723054/

Sorting Configs for Make Compilation https://habr.com/ua/articles/745244/

Setting up ToolChain for Win10+GCC+С+Makefile+ARM Cortex-Mx+GDBhttps://habr.com/ua/articles/673522/

Compiling firmware for CC2652 with Makefilehttps://habr.com/ua/articles/726352/

Generation of dependencies within the programhttps://habr.com/ru/articles/765424/

ToolChain: Configuring firmware assembly for Artery microcontrollers with Makefile https://habr.com/ua/articles/792590/

Automatic Firmware Version Update https://habr.com/ua/articles/791768/

Related posts