Digital gemba separation, anomalies and Visual Mining

Digital gemba separation, anomalies and Visual Mining

In the bank’s offices, there is a “Kraken” – a standard place for customer service, a large corner table that takes up a lot of space. It turned out that it is used much less often than standard small tables, at which customers are served more often. This unexpected discovery appeared after we tested the Visual Mining technology in one of the offices. Below we will talk about Visual Mining for building a digital map and collecting business metrics, and some other unexpected discoveries. There will not be many details of the technology, we will tell more about the processes.

About the author.

Visual Mining: Why Visualizations Are Necessary

Visual Mining (VM) is a technology for extracting data from an image or video. VM-analytics allows you to analyze the flow of data and discover metrics that are not available within the framework of standard analytics. Let’s show with an example (a little ahead of time).

In the bank’s offices, there is a “Kraken” – a standard place for customer service, a large corner table. Next to it is a little smaller, but it still takes up a lot of space. And with the help of analytics, we saw that Kraken is used much less often than standard small tables, which are served much more and more often.

This is how Visual Mining technology works, it can show what we ourselves do not expect, reveal “gray areas”. After all, it is intuitively obvious that the small Kraken is not used much. Yes, you can conduct analytics in the office by conducting a gemba, but we have several hundred offices of various formats and it is scary to imagine how many man-hours are needed for this. Therefore, it is possible to collect a single picture only by using digital technologies – looking at the movement map, you can see that this is so, even if there was no such hypothesis in the head.

That is why we were interested in this technology.

“More metrics needed”

But before putting the technology into production, you need to understand its application area.

  • Is it possible to find out with the help of VM how much time passes from the moment of entry to the moment of first contact, how often the office is visited during working hours, which hours are the busiest and which employees, which locations are in demand with customers?

  • Is it possible to collect data and visualize it in the form of graphs, dashboards of customer and employee actions?

  • Can it be used to prevent negative scenarios, for example, knowing exactly what time more customers come and whether there are enough employees?

  • Can the office experience be improved, such as by adding new areas, reducing voids, and rebuilding those that create queues, by studying the “timeline” of how busy the office locations are throughout the day?

  • Can this technology detect anomalies in customer or employee behavior?

Heat maps. The most popular locations are marked in red. The table in the close-up turned out to be the busiest.

Again, running ahead, we will say that it is possible. For example, with the help of VM, we found a congested area – this is a desk that is on the road in the office from one location to another. It turned out that there are two doors that open at the same time, which hinder both the passage and the finding of the client at this point.

  • The first door is to the cash register, it is constantly opened and people try to pass between the service area and the cash register.

  • The second is a printer, where there is a constant flow of people.

And this is all one location, where the client has to move around the table to let others pass. A good reason to think about building offices according to the ways of employees and customers.

Of course, we did not know the answer in advance, so we decided to pilot the technology. Alfa-Bank has such a good practice when new and useful things are tested in real conditions, for example, in offices, and then scaled to everyone. Thus, over the past two years, several hundred innovations have been implemented in various divisions and offices of Alfa-Bank. A few examples.

If you make such innovations internally, then, as you know, it will take a very long time. For example, it is clearly not possible to assemble a team for the application, write down the requirements, develop, test and go into production in a few weeks. But in a few weeks, finding a start-up that already has such a solution, offering them cooperation with the bank, support and testing of their real base is quite. Innovation is triggered by the Fast Track system. It is a process that:

  • It starts with a business request for a super product or service.

  • It continues with scouting – the search for companies that can help implement it in practice (we look for it ourselves and/or study applications for cooperation, which can be left for landing). The selection process is based on a long list of criteria, the main of which are: availability of technologies, development backlog, implemented projects, scalability.

  • We invite startups “to us” – we test their services and technologies.

  • We quickly launch and scale the most successful ones.

Everything goes quickly, we don’t reinvent the wheel, we don’t waste time on development, if someone on the market has already done it cool.

But why is this for a startup? To present your product and wedge yourself into the processes of the corporation, you need to go a long and long way: pitches, incubators, presentations, and so on. This goes on for months. We come to the startup with a ready-made implementation plan and a ready-made customer, to whom we have already presented hypotheses and shown the usefulness of technologies, and immediately offer to “launch” our solution inside the bank – to become a partner. You don’t have to go through this big and difficult stage of a startup.

That is, the long, long stage of the sale has already been completed — everything has already been done for you.

It is beneficial to the bank because it does not waste time on developing solutions. And since video analytics requires sufficient sizes of servers and equipment, the issue of material resources is closed. If we talk about the terms, we started this pilot at the beginning of the summer, and at the end of June, we concluded the contract.

Test video, outline and very popular table

We would like to clarify that we decided not to use only Visual Mining, but to combine two technologies:

  • The first is the analysis of the video stream using neural networks, the transformation of this data into numbers and logs.

  • The second – with the help of Process Mining (PM, a technology that allows you to understand the essence of the company’s processes in detail) to analyze these numbers, turn them into dashboards, analyze the process itself, build a digital map, identify some metrics that lie in the gray area for business, so they cannot be collected in any other way, as they do not appear in any of the systems.

In other words, we need to train the neural network to recognize objects and events in the video, collect the data, and overlay the data with instrumental analytics, process analysis, to get visualization.

Few people on the market combine technologies like this, but we quickly found the startup we needed (which we also call a “vendor”, because it “supplies” us with technologies). The company Promease Soft (Promease platform) became a partner, it is one of the market leaders.

As part of the presentation, they showed us the possibility of conducting, including similar analytics, and a pilot example for another client, where they identified video imperfections in the customer journey. Below is an example of the work of the Promease platform.

Next is the pilot himself. There are two ways.

  • Install the solution in the circuit and pilot it in the bank circuit.

  • Piloting by contour, which is much faster and with fewer adjustments.

We chose the second way because we were able to anonymize sensitive data.

Next, we agreed with the security department to use a segment of the video from the Olympic office in Moscow for a specific period of time from one camera. On the bank’s side, the video stream was converted into digital logs using appropriate software and transferred to the vendor in this form.

The vendor uploaded the data to its Promease platform, built a digital (heat) map, a process map with metrics, and generated various dashboards and business analytics. At the same time, the video remains with us and we can always switch from the map to the real frames of the source.

In order for the vendor to be able to do something with the model, we first shot a test video from the right camera, where we showed examples of employees moving around the office, told where and what equipment was placed, so that the vendor could make the appropriate markings on the picture.

A frame from a video from a test recording.

In other words, they gave the vendor a test video to teach the neural network to understand what will happen in the real video.

How the office was marked.

As part of the pilot, we did not collect all the metrics that can be collected, this is a pilot. But also with what is, we saw the peculiarities of the location.

Perspectives of Visual Mining

One of the goals of our research is to confirm the hypothesis that with the help of video analytics, it is possible to verify anomalous scenarios. And we succeeded. In addition to the examples above, we saw, for example, that a non-bank customer often approaches the coffee machine. Who was that? It turned out that this was not a client of the bank and not an employee, but an employee who passed and monitored the order. Again, even a careful individual worker in the gemba process would be unlikely to detect this anomaly.

We’ve done the pilot, collected the data, what’s next? How to use such analytics? By learning dashboards, you can add or reduce the number of employees, increase the speed of service and the speed of reaction to the customer.

It is possible to detect customers waiting too long at the table without the presence of an employee.

Duration and frequency of presence in specific locations.

And if you carry out integrations with internal systems, for example, SFA, where banking operations take place, with data from the electronic queue management system, voice analytics, VOC, etc., you can get a 360-degree view of the client. Then there is a “super opportunity” to correlate the customer journey for a specific product. For example, let’s say a customer came in at 6pm with a credit card question, was served for an hour, and a customer with the same question that came in an hour earlier was served 10 times faster. Anomaly. Why so?

We take VM analytics, “glue” all the data about the client and look for the causes of the anomaly: we study VOC, study voice data, transaction data, video analytics metrics on the behavior of everyone present in the office, and if that is not enough, we watch a specific segment of the video and understand what happened trouble And we understand that the client asked a dozen additional questions, then issued a cash loan and made a 10-minute call to the coffee maker.


The technology has a development perspective from the point of view of integration with data from internal systems, which will allow building a real CJM of the client in the office in various sections.

But, most importantly, everything we are talking about would not be possible without two factors – a well-established and fast process for launching pilots and startups, for example, we started work in July and finished at the end of August. The bank is interested in technological solutions, we study the market, are ready to pilot, enter into long-term partnerships, help test your solutions on a live basis with our support and resources, are ready to develop and scale.

Of course, we cannot follow all innovations, and if you have something interesting and you are interested in cooperation with Alfa-Bank, offer a ready-to-launch service. All you need is Send application for piloting. We invite all startups that have a ready-to-deploy fintech product or technology for a large network bank and its customers. We carefully review all applications, and if any of Alfa-Bank’s business lines are interested, we will arrange a meeting. You get the opportunity to test your product together with a large commercial bank and the prospect of a mutually beneficial partnership.

Related posts