How we transferred the service of a large project to colleagues from India

How we transferred the service of a large project to colleagues from India

© The Office

Hello, Habre! Daryna is with you – a second-line platform support engineer or a former member of the second-line UNIX team on an international retail project. I joined the team in January 2021 as a newcomer, a junior sysadmin of UNIX systems. I worked on the project for more than two years, received a promotion, and ended up in the core team. During this time, she not only participated in all activities, but also grew quite well as a system administrator, trained 3 interns, who later also became system administrators on the project.

The scope of tasks was quite large, and at any moment there could be an urgent need – for example, a technician came to the site to change the hard drive, and from our side it was necessary to make sure that everything was done correctly and start the process of reinstalling the unit. But the work itself was so interesting that everyone was in high spirits. *remembered and cried*

One of these projects was the last and largest, probably the big one, which will be discussed in this article. It so happened that, having previously worked as a key business partner of a Japanese IT corporation, whose projects we outsourced, after the well-known events of the beginning of 2022, we had to completely separate from the company and start work completely from scratch. After all, the service had to be transferred to someone in its entirety – and our colleagues from India became such an addressee.

A few words about the project, or what all the fuss is about

The project itself was successfully supported by us for about 10 years. The 2nd and 3rd line of platform support (Unix team), the 2nd and 3rd line of application support, change management team, release management team, developers and testers, architects worked on it. In general, no one was there.

We had 40 countries in support (mostly Europe, but there were also a few countries in Asia – South Korea, Japan, Australia and New Zealand), which is about 3,000 stores and, accordingly, about 30,000 units supported by a team of of 20 boys in Kazan. In addition to us, there were many teams from other countries, for example, Service Desk DBA team, problem management team, etc.

A little about weekdays on support

What did we do during our working days as the second line of support for the platform?

© The Office

  1. Resolution of incidents

    Most of the work was related to solving incidents, we received them from ServiceNow and solved them according to the SLA terms. Naturally, they kept statistics on the number of incidents per day, week and month, by country (to understand what breaks down most often in a specific country or, for example, there was a surge of incidents after a change), there were also statistics on units (we looked at what cash registers break down most often, Computers, scanners, etc.).

    The service was in a 24/7 format, and up to 300-400 Ukrainians were solved in our stack during a working day. Some could take a small amount of time – to refresh the unit or check whether the peripherals are connected correctly, and for incidents related to the reinstallation of units, it took much longer. especially if this unit is a store server.

  2. Implementation of changes – changes

    In addition to daytime activities, we did work at night, after the shops closed. If during the day we drove the changers in test environments, then at night we did it already in production.

    After completing all the assessments, installing the changes in the test and receiving confirmation that everything is working perfectly, we were getting the green light for the changes in the product according to the list of stores and dates. We made changes 4 nights a week, without touching Friday, Saturday and Sunday. And usually 5-6 countries were covered overnight.

    The changes were different – at the level of the country (change of configurations for all stores together) or of specific stores (new releases, peripheral updates, switching time from winter to summer and vice versa). Store-level changers followed the workflow prepared by the third line and, in case of problems during implementation, there was a TOC engineer on duty on the third line who should be called in a bad situation.

  3. Monitoring of mail and calls

    Probably the most standard. In the mail, we received information from technicians who needed our help when they were directly in the store, letters from other teams, or requests to perform changes in test environments or discuss problems after a change in production, invitations to calls and other urgent tasks.

    © The Office

    Also, every day we had calls with representatives of other teams – for example, with the new store opening team, where we discussed future openings, previous steps from our side or any problems with current openings, or calls with the Service Desk team, where questions about specific incidents were resolved .

How the transfer of the service began

Around July 2022, we learned that the transfer process of our project will soon begin, as we have found a team that will take over this service from us. Of course, for obvious reasons, we knew since March that this day would come sooner or later, and one day we would have to start the transfer process and say goodbye to the project. But it turned out that in our hearts we were not ready for this, and everyone became as sad as possible from the realization that the process had already started.

Looking ahead, I will say that the transfer of the service to Indian colleagues began in August 2022 and ended in June 2023. To do this, we divided the entire process into 5 stages: we allocated a core team, which consisted of competent and high-grade engineers of the 2nd line, made a work plan with CT and QA sessions for each topic – then showed how we solve incidents, watched how Indians they did it independently, and at the last stage they were mentors who answered the questions of the Hindus about complex inces or changes. I will tell you in more detail below.

5 steps of service transfer

1. Preparation of the knowledge base

In the first two weeks, we checked our entire knowledge base, updated it as much as possible and added new articles, as well as translated part of the articles into English. As a result, our knowledge base reached 700 pages.

2. CT sessions

For 1.5+ months, every day we held two sessions where we talked about each topic from KB. In advance, we created the core team of the project (we were 6 people), together we distributed the schedule of the sessions among ourselves, everyone chose the topics they would like to talk about and went to prepare each for their part. Every two weeks additionally received invitations and a list of questions from colleagues in India for QA sessions on the topics covered.

For the first two weeks, we covered topics (two calls per day – at least two topics, and if the topics are small, then in one session they could be up to 4) and after 2 weeks we received 25+ topics for which Hindus sent us questions.

Some of the questions were covered by articles from KB, and we simply indicated the required topic that should be read to get an answer to the question, some could be closed by sending Hindus to watch a CT session on the topic, and there remained a part of the questions that we answered on the call.

3. Shadow

Within the framework of this block, everything was simple: we just did our usual work and in parallel searched our screen for Hindus.

Usually in the afternoon, we had such sessions for 3-4 hours, for which we allocated one engineer to the call, who performed the changes, then the scope grew, and we showed the changes already in test environments.

Later, we had such night calls, where we also showed work on incidents and changes in the prod. It was especially important to show how to interact with other teams, the previous steps, the actual implementation of the change, what to do in an emergency situation, if something went wrong, what a rollback is and when it should be done, formats of letters to the customer after activity. The sessions were as productive as possible, because at every step there was a question about actions from our side.

4. Reverse shadow

That’s what we called the block where the Hindus did it, and we watched them do it. At the same time, they answered their questions, looked at KBshki together, corrected them and did their work.

© The Office

The schedule of such sessions was the same as in the previous block. On our side, there was an engineer who sat on the call with them during the day and another who did the same at night. But if at night we watched colleagues from India make changes in the trade, the calls could stretch for more hours, because it was important to bring the activity to the end. All this happened during the call, where they rummaged through the screen, and at the first stage made incidents in selected countries, then moved to the same format of implementing changes.

At this stage, we have started the process of transferring some countries to their support. The transfer process started with three countries, and we transferred all 40 countries after one and a half months. Such calls were necessary and important, because with each new country added to the list of support for India, there were new specifics that the Hindus had not yet encountered in practice.

5. Mentorship

© The Office

From January to June of this year, we remained in the roles of mentors. We no longer had countries to support us and we only helped the Hindus in solving their own problems. We had a chat and a specially prepared template for the appeal (incident number, which articles were used, what had already been done and what exactly the question was for us). Every day there were joint meetings, calls with the customer and other teams, as we acted as experts on some issues.

Instead of imprisonment, or a little about the impressions of working with Hindus

After completing the transfer process, I realized that I had never had such work experience (and not work in general) – because at first everything seemed too strange and incomprehensible.

We take the stress factor, the loss of a project dear to the heart, add the well-known and extremely difficult Indian accent of the English language, which we got used to only after a couple of months, addressing each other through “hey bro!”, the noisy vibe of Indian streets in the background, other approaches to work – and we get serious intercultural batch, Adaptation to which does not occur immediately. For example, it was a novelty for our guys that in response to an unread message, colleagues from India immediately called, and even several timesor they could calmly come out of a run in the forest or from the cash register in a supermarket to make calls.

But we managed – thanks to our team, project and service managers, we managed not only to make the transfer as painless as possible, but also to receive a big thank you from the customer for our efforts.

Despite the fear and tension of the moment, I now remember this time with a smile. And sometimes I can miss days of work on my favorite project. On my first project as a system administrator, which taught me a lot and introduced me to the world of IT.

© The Office

Related posts