How to create 100 servers in the cloud in a minute? Basic work with the OpenStack client

How to create 100 servers in the cloud in a minute? Basic work with the OpenStack client

For cloud infrastructure management, the industry has accumulated quite a lot of tools: terraform, pulumi, ansible, management via SDK or directly via REST API.

At the same time, CLI tools in connection with shell-automation are often not taken into account. And in vain! They help save time spent on creating servers manually or learning complex tools.

Suppose we need to create a stack of servers for tests or calculations. One can only guess how much time will be spent manually creating servers through the control panel and how many mistakes will be made due to the human factor.

And with the help of the OpenStack client CLI, you can create our conditional hundred servers with one command.

openstack server create --image "Ubuntu 22.04 LTS 64-bit" --network my_network --flavor SL1.1-2048-16  --max 100 my_server

Actually everything. This is the end of the article. But for those who are interested in what is happening here and how to work with the CLI, let’s take a closer look.

Use the navigation if you don’t want to read the entire text:
→ Introduction to OpenStack CLI
→ Practice
→ Work with network objects
→ Work with network drives
→ Results and cheat sheet

Introduction to the OpenStack CLI


Immediately a warning – creating a virtual machine (server, instance) with a local disk is different from creating a VM with a network disk. In the first case, the local disk is part of the flavor, and the disk image is immediately created by the VM. In the second, a volume is created from the disk image, and the volume is already transferred during the creation of the VM.

Let’s imagine the starting situation: you have an empty project in the Selectel cloud platform. And we will work with the OpenStack CLI console tool.

Preparatory stages:

We download the RC file with the environment variables necessary for the authorization of the OpenStack client. Working with the above RC file involves entering a password from the console. You can edit the file and write the password in the file so that you don’t have to enter it every time you initialize the environment variables. But it is not completely safe.

Activate the environment variables:

source rc.sh

Basically, you can create one server with a local disk with the following command:

openstack server create --image 'Ubuntu 22.04 LTS 64-bit' --network my_network --flavor SL1.1-2048-16 my_server

Required arguments:

  • image – the original disk image from which the VM is created,
  • flavor – a flavor object containing the VM configuration,
  • network — the network where the VM is created.

With a network disk, the algorithm will be slightly different – we will consider such an example later, but for now we will simplify our task and work with a local disk.

OpenStack consists of several components that are responsible for their own part of the work and provide their own endpoints. Working with the client, we can think from here, because he will do everything for us. But I would like to highlight the logical blocks of understanding the concepts:

  • Keystone — service for authorization and obtaining endpoints of components and regions,
  • Nova (Compute) – a service for creating VMs (servers, instances),
  • Neutron (Network) – network resource management service,
  • Glance (Images) – a service for storing images (from the OS) from which the VM or Volume itself is created directly,
  • Cinder (Volume) – network drives.

Practice

Let’s move on to practice and work with image, network, and flavor objects through the CLI.

We get a list of images:

openstack image list

An example of use (the conclusion is cropped for a more convenient display):

openstack image list
+--------------------------------------+------------------------------------------+--------+
| ID                                   | Name                                     | Status |
+--------------------------------------+------------------------------------------+--------+
| 41383415-4c61-46e5-8cc4-ead5a6f78cf8 | CentOS 7 64-bit                          | active |
| 159f6bfa-d2fd-4d90-b2cc-e19d8aeafd22 | CentOS 7 Minimal 64-bit                  | active |
| 49653158-53a4-4cfe-95e0-b70509e6c1ac | CentOS 8 64-bit                          | active |
| 0b6fb188-0df5-46cf-9111-2840195df219 | CentOS 8 Stream 64-bit                   | active |
| a1b081d7-9eff-44b0-8d3f-85fabf2c88a6 | CentOS 9 Stream 64-bit                   | active |
| 7a827898-058b-4faf-bf8a-0d2311bef7c3 | Ubuntu 20.04 LTS 64-bit                  | active |
| 1fd341e8-82c2-4421-8a37-912cd5ee3ae3 | Ubuntu 22.04 LTS 64-bit                  | active |
......
+--------------------------------------+------------------------------------------+--------+

We get a list of flavors.

In OpenStack, when creating a VM, you must pass the identifier of the Flavor object. It can be considered the configuration of the server memory, disk, number of CPUs.

See the available flavors.

openstack flavor list

Example command. The conclusion is truncated.

openstack flavor list
+--------------------------------------+------------------------------------+--------+------+-----------+-------+-----------+
| ID                                   | Name                               |    RAM | Disk | Ephemeral | VCPUs | Is Public |
+--------------------------------------+------------------------------------+--------+------+-----------+-------+-----------+
| 1                                    | m1.tiny                            |    512 |    0 |         0 |     1 | True      |
| 1011                                 | SL1.1-1024                         |   1024 |    0 |         0 |     1 | True      |
| 1012                                 | SL1.1-2048                         |   2048 |    0 |         0 |     1 | True      |
| 1013                                 | SL1.2-4096                         |   4096 |    0 |         0 |     2 | True      |
| 1014                                 | SL1.2-8192                         |   8192 |    0 |         0 |     2 | True      |
| 1015                                 | SL1.4-16384                        |  16384 |    0 |         0 |     4 | True      |
| 1016                                 | SL1.6-32768                        |  32768 |    0 |         0 |     6 | True      |
| 1017                                 | SL1.8-32768                        |  32768 |    0 |         0 |     8 | True      |
| 1018                                 | SL1.12-49152                       |  49152 |    0 |         0 |    12 | True      |
| 1019                                 | SL1.16-65536                       |  65536 |    0 |         0 |    16 | True      |
| 1020                                 | SL1.24-98304                       |  98304 |    0 |         0 |    24 | True      |
| 1311                                 | SL1.1-1024-8                       |   1024 |    8 |         0 |     1 | True      |
| 1312                                 | SL1.1-2048-16                      |   2048 |   16 |         0 |     1 | True      |
| 1313                                 | SL1.2-4096-32                      |   4096 |   32 |         0 |     2 | True      |
| 1314                                 | SL1.2-8192-64                      |   8192 |   64 |         0 |     2 | True      |
| 1315                                 | SL1.4-16384-128                    |  16384 |  128 |         0 |     4 | True      |
| 1316                                 | SL1.6-32768-256                    |  32768 |  256 |         0 |     6 | True      |
| 1317                                 | SL1.8-32768-384                    |  32768 |  384 |         0 |     8 | True      |
| 1318                                 | SL1.12-49152-512                   |  49152 |  512 |         0 |    12 | True      |
| 1319                                 | SL1.16-65536-768                   |  65536 |  768 |         0 |    16 | True      |
| 1320                                 | SL1.24-98304-1024                  |  98304 | 1024 |         0 |    24 | True      |

If necessary, you can create a Flavor as a team

openstack flavor create

Note that some flavors have the Disc field null, while others have values. To use flavor with a zero disk, you need to create a volume (network disk) in advance. A non-zero value will use the local drive.

First, let’s create a private network:

openstack network create my_network

Let’s create a subnet resource in the created network:

openstack subnet create --subnet-range 192.168.0.0/24 --network my_network my_subnet

Note – now a router is not created and routing is not configured. Thus, “access” to the Internet will not work. More on that later.

SSH Keypair

If you create a server now, you will only be able to log in via SSH with a password. This can be inconvenient and dangerous, so let’s add the public part of the private ssh key to the repository. Then we can specify this key when creating the VM.

openstack keypair create --public-key ~/.ssh/id_rsa.pub my_keypair

Now we can create our conditional 100 servers:

openstack server create --image "Ubuntu 22.04 LTS 64-bit" --network my_network --flavor SL1.1-2048-16 --key-name my_keypair --max 100  my_server

Let’s check the created servers using the openstack server list command:

openstack server list

The output command example is truncated.

openstack server list
+--------------------------------------+---------------+--------+--------------------------+-------------------------+---------------+
| ID                                   | Name          | Status | Networks                 | Image                   | Flavor        |
+--------------------------------------+---------------+--------+--------------------------+-------------------------+---------------+
| 031080fa-3aa0-4848-8551-fd1e6f6ec65f | my_server-94  | ACTIVE | my_network=192.168.0.58  | Ubuntu 22.04 LTS 64-bit | SL1.1-2048-16 |
| 03864c9b-6b9b-4bea-8679-93367ca52250 | my_server-87  | ACTIVE | my_network=192.168.0.24  | Ubuntu 22.04 LTS 64-bit | SL1.1-2048-16 |
| 0efb6875-c481-4ec0-bc80-3392d84c89d6 | my_server-82  | ACTIVE | my_network=192.168.0.110 | Ubuntu 22.04 LTS 64-bit | SL1.1-2048-16 |
| 14c1677d-ef4c-4369-a64d-d4be73416eb9 | my_server-67  | ACTIVE | my_network=192.168.0.116 | Ubuntu 22.04 LTS 64-bit | SL1.1-2048-16 |
| 1619eaf7-c7ab-4db0-a875-38c39acabdac | my_server-76  | ACTIVE | my_network=192.168.0.52  | Ubuntu 22.04 LTS 64-bit | SL1.1-2048-16 |
| 1eadae93-f394-4301-a26f-b496f1e023ae | my_server-73  | ACTIVE | my_network=192.168.0.81  | Ubuntu 22.04 LTS 64-bit | SL1.1-2048-16 |
| 218bdd45-01a9-4b49-aa26-fb25ce7f7329 | my_server-80  | ACTIVE | my_network=192.168.0.76  | Ubuntu 22.04 LTS 64-bit | SL1.1-2048-16 |
| 24642e7f-b265-41a4-8d73-c29434655b30 | my_server-84  | ACTIVE | my_network=192.168.0.120 | Ubuntu 22.04 LTS 64-bit | SL1.1-2048-16 |
| 2e0c2dd3-8a07-421f-ba18-067e5fa320d9 | my_server-81  | ACTIVE | my_network=192.168.0.194 | Ubuntu 22.04 LTS 64-bit | SL1.1-2048-16 |

And we believe that 100 servers are active:

openstack server list -f value | grep ACTIVE -c
100

Work with network objects

We got 100 servers, but how to enter and work with them? You can use a scheme with an additional one

bastion host

, which will have access to the Internet. At the same time, we will get acquainted with the network part.

You need to add a router to the network scheme, connect it to our private and external networks, create a floating address (floating ip, external address) and bind this address to the bastion VM:

openstack router create my_router
openstack router add subnet my_router my_subnet

In order for the router to allow the VM to “access” the Internet from a private network (in this case, from my_subnet) through NAT (for installing packages, updating software, etc.), you need to connect the router to an external network.

Let’s get the name of the external network:

openstack network list --external -f value -c Name
external-network

Let’s bind the external network to the router:

openstack router set --external-gateway external-network my_router

Let’s create a floating IP, in the conclusion we will get the address:

openstack floating ip create external-network

You can view the created addresses with the command:

openstack floating ip list

In my case, the address is 31.129.42.122, and it will appear in the text below. You will have a different address.

Now let’s apply the approach with a basis in practice. Let’s choose, for example, the first VM as a bastion host (you can create a separate VM with a more appropriate configuration and hostname, but for simplicity we will skip this step). It will be possible to connect to the rest of the VM through the bastion. In my case, I will choose VM 99.

We associate Floating IP and VM 1:

openstack server add floating ip my_server-1 31.129.42.122

We get the internal address (address from the private network 192.168.0.0/24) of VM 99:

openstack server show my_server-99 -f value -c addresses
{'my_network': ['192.168.0.111']}

In this case, the address is 192.168.0.111.

Let’s connect to VM-99 and check that everything works as intended:

ssh -J [email protected] [email protected]
root@my-server-99:~#

Working with network drives

We have already worked with local drives. Let’s make things a little more complicated and create 100 hosts with network drives.

First, delete all current VMs.

Important: When the following command is executed, all VMs in the current project will be deleted. Make sure you are in the right project.

openstack server list -f value -c ID | xargs -i -P 10 bash -c 'openstack server delete {}'

First you need to create a volume (network drive). In this case, there is no tool in the API to create 100 disks, and you need to wrap simple shell commands in loops.

You can perform them in a loop for i in {1..100}; do COMMAND; done. In this case, operations will be performed one at a time. The process will take some time, but it will be clear. Or you can use the xargs utility for parallel execution.

For our example, we use the generated sequence from 1 to 100. And in the 10th thread (-P 10) we execute the volume creation command — {} will be expanded by xargs into the received argument from seq:

seq 1 100 | xargs -i -P 10 bash -c 'openstack volume create --size 10 --image "Ubuntu 22.04 LTS 64-bit" my_volume_{}'

For example, let’s create our flavor:

openstack flavor create --private --disk 0 --vcpus 1 --ram 1024  my_flavor_net_hdd

And in the final, we will create 100 VMs with a network drive:

seq 1 100 | xargs -i -P 10 bash -c 'openstack server create  --network my_network --key-name my_keypair --flavor my_flavor_net_hdd --volume volume_{} my_server_{}'

Results and cheat sheet

We looked at how to manage the infrastructure using CLI tools while working in the terminal. And also, the automation of more complex processes was disassembled into basic “bricks”.

Here’s a cheat sheet with basic commands to help you get started with managing OpenStack resources.

openstack keypair create
openstack keypair delete
openstack keypair list
openstack keypair show
openstack server create
openstack server delete
openstack server list
openstack server show
openstack server add floating ip
openstack server remove floating ip
openstack volume create
openstack volume delete
openstack volume list
openstack volume show
openstack router create
openstack router delete
openstack router list
openstack router show
openstack router add subnet
openstack router remove subnet
openstack router add route
openstack router remove route
openstack flavor list
openstack network create
openstack network list
openstack network show
openstack network delete
openstack subnet create
openstack subnet list
openstack subnet show
openstack subnet delete
openstack network list --external
openstack floating ip create
openstack floating ip list
openstack floating ip delete

Write in the comments questions or alternative ways to create 100 servers.

You may also be interested in these texts:
→ American-style import substitution: deployment of own semiconductor production is not going exactly according to plan
→ Capacitors again: how I repaired the HP Specter X360 13 laptop and what came of it
→ Not washing, but rolling: NVIDIA will start supplying China with powerful video cards that are not subject to US sanctions

Related posts