typical errors in e-commerce analytics
Every day we collect more than 200 GB of data about Lamoda customers who browse the site and place orders. But having good numbers about our storage is one thing, getting the right conclusions from the data is quite another.
My name is Sasha Aivaz, I lead the Data & Analytics team at Lamoda Tech. And today I will talk about mistakes that are easy to make when trying to analyze data in retail and e-commerce. I am sure that the article will help someone to look at their own product in a different way – or give an idea of what issues analysts in our field are working with.
Of course, we ourselves have made many of these mistakes. In the article, I will try not to delve into the numbers, but will talk about the reasons using my own example.
- 1 1. Deaveraging. Why there is no such thing as an “average” customer
- 2 2. Survivor’s mistake. Dolphins did not save everyone
- 3 3. Causation and correlation: the main thing is not to confuse
- 4 4. Obvious does not mean successful
- 5 5. Ramp up, ramp down: wait for real results
- 6 6. We are not deciding anything, we are waiting for the analysts
1. Deaveraging. Why there is no such thing as an “average” customer
We have long believed that Lamoda customers have one quality in common: they shop to get a shot of endorphins. New clothes for them are inspiration. With it, they become more fashionable and interesting, unlike others.
We lived in this paradigm: we aimed at such customers, looked for them, thought about how to attract advertising. Tested on them. In ranking the catalog, preference was given to the latest collections and new arrivals. Of course, this led to the fact that we found such customers. But a huge number of people were missed along the way.
The problem is that we looked at the picture on average, saw the general figures for a month or a day in the analytics. They saw that they sold so many baskets with such an average check that customers buy with a frequency of, for example, once every three months. In general, we looked at the average characteristics of our average customer.
But we only saw a small percentage of our customer base. If you dive deeper and cluster it, it becomes clear that there are customers who make three orders a day, and there are those who make an order once every six months. And on average, a figure was obtained that did not tell us anything about real buyers.
We currently identify 13 major customer segments. They are all different: they buy different things with different frequency, they have different requirements for the product. For some, the size of the discount is important, for others – next day delivery. Now we offer each client what he needs, not the “average” user.
2. Survivor’s mistake. Dolphins did not save everyone
Ok, the average users have been dealt with. Now we do everything properly: we study different customer segments. We study deeply, but we still make mistakes – we must also look broadly. We need to look at buyers in general, and not just at our customers, otherwise we make a survivor’s mistake.
A textbook example of such a mistake is the story of dolphins. You have probably heard about cases when dolphins help drowning people and push them to the surface with their noses. Dolphins generally like to play by pushing objects with their noses. Presumably, there are people whom the dolphins did not push to the surface, but, on the contrary, pulled deep, under the water. But we cannot find out about this, because we only hear the words of grateful survivors. And the number of deaths from dolphins is unknown to us.
In business, we also research only our customers who “survived” and draw conclusions from these statistics. And perhaps we are missing out on a huge piece of business: we do not offer products or services that are not represented by us or are poorly represented.
Let’s assume that most customers buy clothes and shoes on Lamoda. From this we can conclude that our customer base is not interested in sports equipment, and therefore, this category does not need to be developed, because now the profitability of clothes and shoes is much higher. And if we fill the warehouse 100% with clothes, we will be able to earn more money.
But the truth is that dumbbells and yoga mats are not bought from us as often as clothing because we have never developed this segment. It is unlikely that anyone will come to Lamoda to choose exercise equipment: people are used to buying them from other sellers.
But in fact, this market is bigger than we think. It’s just that we never invested our time in it, did not analyze consumer demand and behavior. They did not look at the elasticity of demand, depending on the price. There is a different unit-economy: dumbbells are not taken 10 pieces at once to return inappropriate ones, as customers do with clothes.
Customer research helps us spot such errors and distortions. If we conduct surveys or focus groups, we not only attract our current audience, but also those who have never shopped on Lamoda. This gives more objective data about the market in general.
3. Causation and correlation: the main thing is not to confuse
When we observe two phenomena at the same time, it means that there is a cause and effect relationship between them. For example, ice cream sales increase every summer. But at the same time, the number of sunstrokes is also increasing. From this, it is possible to make a false conclusion that if ice cream is banned, sunstrokes will also stop.
We often make similar mistakes at work. We see that customers are buying more, and at the same time some metric is increasing. Then this metric drops and customers buy less. At this point, it’s easy to fall into the trap and decide that there is a direct connection here, and that’s how you can influence customer behavior.
I will give an example from our practice. Lamoda has the possibility of shipping with a fitting: people order several sizes and models to choose the right thing. Anything that does not fit is returned to the courier or delivery point.
But there is also delivery without fitting. People refuse such orders much less often.
So maybe it’s worth just eliminating the possibility of trying on? Then the percentage of rejections and returns will fall, and our profit will increase.
No. Delivery without trying on was chosen by buyers who were confident in their choice: for example, they ordered hygiene products or accessories. But if we deprive all users of fitting, many will stop buying from us. And the profit will fall as a result.
4. Obvious does not mean successful
Everything needs to be tested and criticized. Even hypotheses that at first glance seem logical and useful.
Dynamic filters periodically appear on Lamoda. These are additional buttons with selections: “Clothes for a picnic” or “For a trip to the sea.” These seasonal filters seem logical, right? But we have been testing them for many years, on different topics – they do not fly.
Fortunately, these filters do not spoil the metrics, so sometimes at the request of colleagues we return to the idea and continue the experiments. Another thing is when time, resources, money are spent on a hypothesis, perhaps some other benefit is lost. And the metric has a negative indicator.
A good example of how an obvious idea can lose revenue is promoting seasonal products. For example, sales of T-shirts, shorts, and flip-flops increase in the summer. So can start showing them first? Give them priority in the ranking of the catalog and show them first – that’s how sales should increase?
No. If people go for these goods themselves, they will still buy them. But if at the same time we stop showing pants, bags and so on, they will be bought less: customers simply will not remember that it is not in season.
5. Ramp up, ramp down: wait for real results
Sometimes you have to wait a really long time for changes in business metrics.
In new cities, we open points of delivery of orders according to a certain scheme. Courier delivery appears first – with the help of partners or our own services. It’s not cheap for the company, but we can gauge the demand. When a critical mass of orders accumulates, we open a delivery point.
Clients in the city are already used to courier delivery, they do not know about the drop-off point for some time. It also takes time before they decide on a new order. But we start to bear the expenses for the work of the new point immediately after its opening.
If we had not known that there would be a drop in profits after the opening, we would not have been able to appear in many regions.
This effect also works in testing some features. We have changes that affect repeat purchases or user returns. To see how the changes will affect customers who buy once every six months, you have to run really long tests.
In addition, some changes can shoot in the short term – and then only miss the revenue. Or vice versa. We are constantly learning to anticipate and analyze such situations.
6. We are not deciding anything, we are waiting for the analysts
Lamoda Tech calls itself a data-driven company for a reason. We have our own A/B platform, several analytics teams, a team of user experience researchers. All product solutions are supported by experiments. But sometimes collecting data and analyzing it is the biggest mistake.
Imagine: the conversion to purchases suddenly drops on the site. We have several hypotheses about the causes of what happened, but there is no way to collect all available information and conduct research, because the problem needs to be solved now.
That’s why we do quick research — and make decisions based only on our knowledge of the site and users. We make changes based on hypotheses without waiting for the full picture. If the changes have a positive effect on conversions, we will implement them. If not, let’s look at the results and try other steps.
One can endlessly analyze the causes and effects of a drop in conversion. Or you can put forward hypotheses, choose the most valid one — based on your own experience and communication with a limited sample of users — and run it in an A/B test.
Sometimes we do not have the time and opportunity to collect data, assess risks and calculate profits. You have to rely only on your experience and knowledge. In addition, at a certain point, further gathering information about decision options is more expensive than making the wrong choice. And then it is better to make a mistake than to do nothing.
By the way, that is why one of the main principles at Lamoda Tech is not to be afraid of mistakes. Constant communication in the team and exchange of experience helps us not to step on the same rake.
I hope that the examples of the article will be useful not only to us. If you have something to add to them, share in the comments!