Multicriteria optimization for ranking and selection of trading systems
Contents
Selection of trading systems: how to choose the best from any number of available ones
The article will discuss the selection mechanism or calculation of the coefficient for ranking the pool of trading systems. We will consider one of the ways, using which you can determine the best options.
It is important to understand that the results of the analysis of the efficiency and effectiveness of trading systems can be complex and ambiguous. For example, a trading system with high profit may have a large drawdown, but it can also generate stable income for a long time. Therefore, when choosing a trading system, it is important to consider a set of characteristics and indicators. This is the problem addressed in the article.
Formulation of the problem
So, let’s say we have several trading systems, and some of them are better and some are worse. Based on the set of metrics, it is difficult to determine whether one or another system is better or worse. One system can have a high yield, and at the same time, a high drawdown and a lower recovery factor. The other has a minimal drawdown, but the profit seems pretty average. And these are only three metrics, and, of course, much more can be calculated. Example:

the ratio of annualized profit on the test and on the inspection

the proportion of used trading time to the total, for example, the proportion of days on which trades took place in relation to the total number of trading days

share of the time of the trading system on the market – the time during which the capital was occupied by the position in relation to the total trading market time

maximum, average, and minimum contract lengths — in bars, minutes, hours, or days

the average number of transactions per unit of time, for example, per hour or per day or per week

average number of trades per trading day and average number of trades per calendar day

the maximum length of the series of profitable trades and the same for losing trades

annualized return

the share of commissions in the transaction, the share of commissions in the final profit, the size of the commission per item of income

maximum realized profit achieved

average profit per hour, day, month

recovery factor

all kinds of MAE, MAPE, SMAPE, RMSE, MSPE and more

the square of deviations and the square of negative deviations, and by analogy –

the Sharpe coefficient and the Sortine coefficient

mathematical expectation

maximum realized drawdown, relative realized drawdown, maximum unrealized drawdown

maximum unrealized potential profit

the number of profitable / unprofitable transactions, the total profit and loss of profitable and unprofitable transactions

financial result for long and short contracts

the correlation between the yield curve and the market price of the asset

benchmark profit / drawdown for the case of “buy and hold” for the given asset for the same period in relative units to the corresponding indicators of the system itself

the benchmark of profit in the optimal way — the maximum theoretical possible profit, taking into account commissions from this asset
Let you have a matrix, where each trading system corresponds to a row, and to each of the indicators above a column. I would like to rank the trading systems in some way to select the best ones, but it is desirable to have a continuous range of values for the rank, which can already be broken down into quanta, quart or use other techniques. And here we have a set of motley metrics, and sorting by several of them at the same time will not give anything.
Search history

At first I did it by hand – literally applying a filter to something like the matrix described above, and the technique used was “I think so”.

Then I automated it, but the bottom line didn’t change, it was a “probably it should be like this” decision, but no certainty.

Next, I enlisted the help of a regression neural network whose input was this normalized matrix, and the results were pretty good.

Now I want to approach this process from a more formal point of view: essentially, this is what the neural network I created seems to have done, and now there will be a mathematical method.
Formulation of the problem
It is up to us to find one or some subset of the best, using a variety of criteria – metrics for testing trading systems. Such tasks are tasks of the class of multicriteria optimization. This will be discussed below. In essence, the task is as follows: we need to calculate (or construct) the metrics of an ideal trading system and find the distance to such an ideal system from all others. This distance will be the value for the selection of systems – a certain value by which you can sort the systems or rank their list.
Found solutions
So, after asking what tools exist for this, I dug up the following.
First, method classes:

methods based on artificial intelligence, such as neural networks, genetic algorithms, fuzzy logic, etc. These methods allow you to create complex and flexible models that are able to learn from data, identify patterns and predict market behavior;

methods based on optimization, such as the simplex method, gradient descent, Newton’s method, etc. These methods allow finding the optimal values of strategy parameters that maximize or minimize the objective function, such as profit, risk, Sharpe ratio, etc.

Methods based on statistics, such as regression analysis, hypothesis testing, time series analysis, etc. These methods allow you to check the statistical significance and reliability of strategies, as well as to evaluate their risks and returns.
And here are the methods, in particular, that I came across during my searches (and seemed applicable to the solution of the contextual task):

one of the popular methods of multicriteria selection of trading strategies is the ELECTRE (Elimination Et Choix Traduisant la REalité) method, which is based on a pairwise comparison of strategies and the exclusion of those that dominate others by one or more criteria. This method allows taking into account not only quantitative, but also qualitative criteria, as well as criteria weights and decisionmaking thresholds;

Another widespread method of multicriteria selection of trading strategies is the TOPSIS method (Technique for Order of Preferences by Similarity to Ideal Solution), which is based on the calculation of distances between strategies and ideal points corresponding to the maximum and minimum value of each criterion. This method allows you to determine the best and worst strategy, as well as calculate the relative proximity of each strategy to the ideal solution;

Another interesting method of multicriteria selection of trading strategies is the MLLDM (MultiLingual Linguistic Decision Making) method, which is based on the use of linguistic variables to describe the criteria and evaluation of strategies. This method allows you to take into account uncertainty and vagueness of data, as well as different areas of expertise and preferences of participants in the decisionmaking process;

the next will be the AHP (Analytic Hierarchy Process) method, which is based on the construction of a hierarchy of criteria and alternatives, as well as on the determination of priorities using a pairwise comparison of the elements of the hierarchy;

the PROMETHEE method (Preference Ranking Organization METHod for Enrichment Evaluations), which is based on the calculation of the degree of preference of one alternative over another according to each criterion, as well as on the aggregation of these degrees taking into account the weights of the criteria;

The VIKOR (VIseKriterijumska Optimizacija I Kompromisno Resenje) method, which is based on determining the best and worst alternative for each criterion, as well as calculating the distances between the alternatives and these reference points.
And that’s not all.
Selected solution
I liked the TOPSIS method that was mentioned above. I will use it. Graphically, the solution to the task can be imagined, in some approximation in the form of a “Radar” type diagram, where the metrics of trading systems are plotted on the circle, and the values of these metrics are plotted on the axes from them to the center. And in fact, we will first look for an ideal system (which does not exist), it can be called an ideal point or even a utopian trading system. And then the distance to her from all others.
The relative closeness of an algorithmic or any other trading strategy to the ideal solution is a measure of how close that strategy is to the best possible strategy by all criteria. One way to calculate relative proximity is to use the TOPSIS method.
Here is the sequence of steps to solve the problem of the article task:

form a decision matrix in which the rows correspond to alternative strategies and the columns correspond to evaluation criteria (for example, profit, risk, stability – as shown above);

normalize the decision matrix to bring all values to the same scale and eliminate the influence of units of measurement (remember that we have somewhere fractions, somewhere percentages, somewhere quantities and sums);

multiply the normalized matrix by the vector of criteria weights to account for their relative importance. This is the weakest point – you need to determine the weight of each criterion, but you can go iteratively, adjusting them in the process of research;

define the ideal solution as a vector consisting of the best values for each criterion (metric), and the antiideal solution as a vector consisting of the worst values of each metric. This will be our opportunity space;

calculate the Euclidean distance between each alternative and the ideal solution, and between each alternative and the antiideal solution. You can, by the way, not necessarily use Euclidean space, but, for example, Chebyshev, Manhattan or other, but this will complicate the task. Although for uncorrelated metrics they would be better;

the distance is easy to calculate: as the sum of the squares of the differences between the ideal value of each metric and one of the existing ones. Thus, we get some number that characterizes the distance of this vector in relation to the ideal solution (vector). Once again: we consider the difference between the ideal value of the metric and the value of this metric for this trading system, square it, sum all the squares;

calculate the relative proximity of each alternative to the ideal solution as the ratio of the distance to the antiideal solution to the sum of the distances to the ideal and antiideal solutions. This value will lie in the interval from 0 to 1, where 0 means complete noncompliance with the ideal solution (antiideal solution), and 1 – complete compliance (ideal solution); you can also do the opposite – expand this continuum as you like;

rank the alternatives in terms of decreasing relative closeness and choose the one with the greatest value as the best (or top best) strategy.
It sounds complicated, but in fact the formulas are quite simple – the sum of squares and no more. What is your idea?
Conclusion
Finally, a bit of cold water: the ideal solution in algorithmic trading is a strategy that maximizes profit and minimizes risk and other undesirable factors. However, in reality, such a solution may be unattainable or nonexistent, as there are many uncertainties and constraints in the market. Therefore, instead of looking for an ideal solution, it is more appropriate to look for the most acceptable or optimal solution that satisfies the given criteria and constraints. To do this, you can use various optimization methods, such as the simplex method, gradient descent, Newton’s method, etc. These methods allow finding the optimal values of strategy parameters that maximize or minimize the objective function, such as profit, risk, Sharpe ratio, etc.
It is good when a person has a goal, it is formalized, “we have a plan”, we move in this direction and burn with our idea. This is one of the main factors that make us happy 🙂