How to transfer 3,000 employees to a corporate VPN and not go crazy (well, almost)
Vulnerabilities in Jira and Confluence are an inevitable evil. We all got used to them and learned to deal with them: we posted an additional password, updated to a secure version and live well, drinking juice.
And then it turned out that from the third quarter of 2024 cinema there will be no Atlassian updates. And soon any serious vulnerability would be fatal for us.
What does this mean?
-
Jira and Confluence could lie down. And not in order to rest and return to us with fresh strength, but forever.
Even if we raised them, there would still be a fierce simple business process.
-
Our data could be taken away. In about a year, the confidential file and everything else would appear on the darknet or other unauthorized places.
And that’s why we decided to roll out its – cool corporate VPN.
Contents
- 1 And what kind of kipish?
- 2 It will be humane, it is out of love
- 3 Instructions
- 4 Access granting forms
- 5 Technical support for transferring systems via VPN
- 6 Support: v. 1.0
- 7 Support: v. 2.0. This is Suppooort!
- 8 Stage 1. Assess the employee base using Confluence and Jira
- 9 Stage 2. Distribute access at the network level
- 10 Stage 3. Issue access
- 11 The final chord
- 12 And what happened later?
- 13 Yes, it was scary, but we did it:
- 14 God, how beautiful we are!
And what kind of kipish?
Twice in the last year we have encountered critical vulnerabilities. In an emergency, they hung a window with an additional password and protected the unfortunate Confluence. But if that had happened—for example, with a different type of vulnerability—the rescue mission could have failed.
We decided to remove the leaky magnificence of an enterprise VPN, and the Infra team has very appropriately polished the new version. The program has become safer, faster and more productive, it is easier to scale it to the whole company – not a VPN, but the son of a mother’s friend.
Although it works through infinite two-factor, it’s super cool from a security perspective. And we started thinking about how to provide access.
It will be humane, it is out of love
We had 3,000 employees, a bunch of bots and freelancers… Not that the infosecurity team lacked challenges, but when I started thoughtfully transitioning the company to a corporate VPN, I had to go all the way.
We could send out logins and passwords to everyone, say “Connect” and the next day remove Confluence from Jira behind a VPN. But they decided to be humane and set up systems so that no one panics and business does not stop.
To do this, we deployed an entire event and provided access gradually over two months.
Trap plan
Task: prevent downtime of business processes and quickly help those who have problems (who could not connect).
Decision: to connect so many employees that after moving Jira and Confluence behind the VPN, support will not be drowned in loss requests.
How was the required amount of support determined
We estimated that one in five users log in every day, and calculated what percentage of the total number of Confluence and Jira users this is.
Based on the data, a corridor of users who can be left disconnected was determined. After reaching this corridor, we could “pull the switch” with a calm soul and know that no one will fall off.
30 in people who will be processed by support in a day |
h |
3 every third one turns to support |
h |
5 a fifth of users log into the system every day (actually less, we take it with a margin) |
= 450 people |
Powerful preparation
Instructions
Before connecting the commands, we compiled the most clear user-friendly instructions – such that even a shrimp would understand (spoiler: it helped, but not for everyone).
Access granting forms
We have updated the access request forms in the internal portal so that colleagues do not have to search for information about the old VPN.
Technical support for transferring systems via VPN
Sad but true: our company does not have organized internal support. The guys who were supposed to support the VPN transition were swamped with other work, so the team had to be modified and changed twice.
Support: v. 1.0
Two office support staff – these guys can process up to 40 applications a day.
We understood that the applications could go away shoals by hundreds. Moreover, their complexity will vary:
— from serious problems related to the specifics of specific regions (in this case, engineers should be involved);
– To the questions “Where are the instructions?” etc.
And the link to it is highlighted, pancake, in caps in the message with the instructions! Sorry bomb…
Let’s run forward
Despite all the instructions and pushes, in total, the two agents processed more than 1,000 applications.
So that the boys do not suffocate under the weight of applications, we went to the tops for help.
Support: v. 2.0. This is Suppooort!
The tops gave us another support team. We trained her, and the two heroes were joined by 300 Spartans.
We sent teams with weak technical skills to the new support. As planned, there were more requests – every second one applied. But the support team also increased and could process 120–150 requests.
We calculate according to the formula and get: 120 (150) x 2 x 5 = 1200 (1500).
We are targeting 1,200 users in teams with high turnover.
Corridor of users with support v. 2.0
450 users in the corridor of the first support + 1200 in the second = 1650 (this is 55%).
So we calculated that 45% of connected users is enough to translate Jira and Confluence over VPN.
Provision of VPN accesses
Stage 1. Assess the employee base using Confluence and Jira
At this point, we looked at who in the company had access to Atlassian. We have revoked access for those who have not logged into Confluence and Jira for a long time.
This reduced security risks and the burden on support and information security teams.
Stage 2. Distribute access at the network level
Teams with high turnover do not use most infrastructure services, so they do not need access there only at the network level.
After all, even if the guys do not have access to the terminal programs, it is not a fact that their devices cannot be used to penetrate the network. For example, sports bettors can catch a virus on their work computer and give attackers access to our system.
Therefore, we limited access to such teams.
Stage 3. Issue access
We did it gradually – by teams. We agreed with the managers about support: the team leaders additionally pushed their guys and monitored the tasks.
The logic was as follows:
In order to monitor the dynamics, we started a board in Grafana. In it, they saw the logins of everyone who connected to the VPN at least once, and calculated the percentage of those who connected.
pushy And more guns. And again guns. And again guns
We must have tortured everyone before switching to VPN. Because from every channel they shouted: “Connecting, or your work will stop!”
It was important to communicate the changes to everyone, so we sent reminders in the corporate messenger and the Home portal, arranged support with managers and came to them with lists so that the team leaders gently poked their guys.
Interns vs privacy
We have a lot of interns at our company and they were all working in Confluence and Jira from the same account (yes, that’s wrong).
What’s the trick?
There is always high turnover among interns, so giving them full access to company documents is dangerous.
Creating an account in Confluence for everyone is long and difficult.
It is even more difficult to support the connection of interns to VPN – the technical literacy of the guys is usually low.
Even if we gave interns access to Confluence, we would have to protect them from sensitive information.
We analyzed the knowledge base that the interns work with and realized that there is no sensitive information. Therefore, we decided to take all information for interns to a separate landing – it is available without a VPN.
It was necessary to make the mirror as similar as possible, because:
-
Given the level of technical literacy, trainees might not recognize buttons and sections even in a different color, it is important that everything in the interface matches.
-
Many interns eventually become employees. We wanted to simplify their onboarding and teach them how to work with Confluence.
We have set up synchronization so that all changes in the Atlassian system are immediately reflected on the intern landing page. To get to the landing page, trainees only need to enter their login and password.
The final chord
In two months, enough workers connected to the VPN. We clicked the same button and set up two-factor access to all Atlassian systems.
Everything went smoothly, except for one thing
The guys were going to roll out the corporate VPN at 8pm and I was going to have a drink for them in the meantime.
As a result, they pulled the switch already at one o’clock in the morning, and I had to… drink for them all this time to “Game of Thrones”.
And what happened later?
Of course, after translating Atlassian to VPN, we received many support requests again. New ones came, something went wrong with the old ones. and that’s how the third support team was born.
It was created by the guys who created our corporate VPN. Of course, it was not without a new cycle of training, but we already had experience.
Some problems were solved by hand, but there were no downtimes.
The active phase of granting access has passed late. We responded to all inquiries and scathing comments, performed security checks and ensured that no one else gained access to our systems.
Dynamics of metrics when switching to a corporate VPN:
Yes, it was scary, but we did it:
We invested more than two quarters, but the active phase took only two and a half months.
There was a risk that the prank would fail. But we were not afraid rolled up rolled out a company-wide VPN.
After all, now the system and accesses are already configured.
God, how beautiful we are!
A VPN with two-factor authentication access is much more secure than our Jira and Confluence patches, and access segmentation reduces risk further.
Ask questions in the comments, use our case and get ready to hiccup a lot. Because they will often remember you with kind (but inaccurate) words!