Have a good day! We all want to live without worries, without product attacks and customer access problems. We don’t want left-wing people and robots to get into our network and have access to endpoints. For this, ancient sysadmins invented a big and terrible firewall. In modern products such as Fortigate, it is possible to conveniently and precisely route traffic, give limited access according to specific protocols, etc.

But the immediate problem is to fill this good from the box above the handles. When you configure a corporate network, it is not so painful, but when you need to let clients into your network, which have many machines with different addresses, they need to get to different components of your product, then it becomes sad. Especially when this duty falls on the hands of an engineer, who could do more interesting things.

But an engineer, especially devops, is an engineer. If you see something moving, automate it. Does it not move? Move and automate!

Move and automate!

Briefly about our case: how does the Fortigate firewall work; requests to add customers through our portal go to Jira as a task in the CHG (Change) project. At first, engineers took on this task, went to fortigate, created a group for the client or found an existing one, added addresses to it, and the group to a special policy that leads to part of the product by a certain protocol (for example, RabbitMQ, TCP, SSH, etc.)

You can immediately notice that the task is routine, not particularly difficult and always repeated, but at the same time, with a larger number of addresses being added, the execution may take time.

An observant engineer could also notice that possible solutions and automation of this routine lie on the surface – fortigate and Jira have APIs. Bingo!

We take the brains, programming skills and API documentation of these two services into our hands.

A little black magic shit code and everything will fly away. Although, why stress? Moreover, our engineer in a vacuum is probably using something more advanced, for example, for the same IaC. We dig into the depths of the Internet and find something else very interesting – there are collections for both Jira and fortigate in Ansible (actually a high-level wrapper over the API).

Well, now we have to collect it!

Let’s discuss workflow of our future fake and let’s start implementing it step by step:

1. Someone (in my case it would be a coordinator, first line of support) receives a customer request that looks like “I want access from my server (address attached) to a part of your product (let’s call it “Buying Cupcakes”)”

2. He creates a task in Jira upon request, with a form that contains all the fields we need for the operation (name of the client’s company, server addresses, the desired product and, in our case, the prod/pre-prod environment)

3. After checking for validity, the task is marked with the “whitelisting” label and its status is translated into “Ready to Dev” (the label and status can be set to any, it will be configured by the Ansible variable)

4. Scheduled task on AWX goes once every 30 minutes to Jira, looks for tasks with the right label and status and does black magic.

5. After unsubscribing from the task and his comment, the Jira automation is triggered, which safely closes the task or barks at the author if something goes wrong

It seems simple and transparent. You can now walk along steps himself Ansible playbook

1. We are looking for the tasks we need

2. We collect the necessary data from them

3. We create address objects in Fortigate

4. We create or find a group for the client. We add addresses there

5. We put the group itself in the product/component group, which was previously put in the desired policy

6. We write a comment about a successful or unsuccessful launch in Jira

We will need it: Python 3, Ansible, ansible collections (community.general , fortinet.fortios), an AWX instance, or any other environment with dependencies that crontab will run the playbook on a schedule

We rotate Ansible, automate

Let’s start to describe our automation step by step in Ansible:

1. We look for tasks to be performed and take their numbers

---
# tasks for fetch whitelisting tasks from Jira
- name: Search for an whitelisting issues
  community.general.jira:
    uri: '{{ jira_server }}'
    username: '{{ jira_user }}'
    password: '{{ jira_api_token }}'
    operation: search
    project: "{{ target_project }}"
    maxresults: 5 #Количество задач, обрабатываемых за запуск
                  #меньшее количество проще логировать
    jql: 'project="{{ target_project }}" AND labels="{{ target_label }}" AND status="{{ target_status }}"'
  args:
    fields:
      lastViewed: null
  register: issues

- name: Search result
  debug:
    msg: "Issues wasn't founded"
  when: issues.meta.total == 0

- name: Start Whitelisting
  include_tasks: whitelisting.yml
  with_items: '{{ issues.meta.issues }}'
  when: issues.meta.total > 0

Here you can see mine code black magic Loops, especially nested ones, are a problematic topic in Ansible, so in this piece we first collect the tasks to be executed, and then throw them as an item into the second task (whitelisting.yaml) of our role, in which whitelisting is already taking place. In this implementation, everything is completely readable and convenient, the main thing is not to get confused in the context of using elements in the next stages of participation.

2. We get information from the task and get the data of the completed form from custom fields

- name: Retrieving payload from issues
      community.general.jira:
        uri: '{{ jira_server }}'
        username: '{{ jira_user }}'
        password: '{{ jira_api_token }}'
        project: '{{ target_project }}'
        operation: fetch
        issue: '{{ item.key }}'
      register: target

    - name: Initiate WhiteList
      set_fact:
        target_data: 
          client: '{{ target.meta.fields.customfield_10863.replace(" ", "") }}'
          ips: '{{ target.meta.fields.customfield_11092.replace(" ", "").split(",") }}'
          enviroments: '{{ target.meta.fields.customfield_10865[0].value }}'
          components: '{{ target.meta.fields.components[0].name }}'
          #Название полей в полученных данных зависит от вашей Jira
          #Проверьте соотвествие заранее с помощью таска на получение таски

3. We check the received addresses with the ipaddr filter and if the data is invalid, we write a comment in Jira about the failure with a request to check the instance or task and fail the playbook

- name: Set wrong ips counter
      set_fact:
        wrong_ips: 0

    - name: Validate IPs with ipaddr filter
      set_fact:
        wrong_ips={{ wrong_ips | int + 1 }}
      with_items: " {{ target_data.ips }} "
      when:  not (item | ipaddr)

    - name: Fail play if we have wrong IPs in payload
      block:
        - name: Comment on issue
          community.general.jira:
            uri: '{{ jira_server }}'
            username: '{{ jira_user }}'
            password: '{{ jira_api_token }}'
            issue: '{{ item.key }}'
            operation: comment
            comment: "Task failed. Maybe IPs invalid format or other causes. Please, check logs and correct payload/fix instance"
        - name: Fail the play
          fail:
            msg: "Stop the play because payload wrong or we have some problems on AWX instance or local laptop"
      when: wrong_ips | int > 0

Jira also has automation installed, which, after a given comment, duplicates it with a mention of the author and he receives a message

4. We create address objects in fortigate. ARI Fortigate requires specifying the address type, so I had to fiddle with ternary operations

- name: Create IP ADDRESSES
      fortios_firewall_address:
        vdom: '{{ vdom }}'
        access_token: '{{ fortigate_token }}'
        firewall_address:
          name: 'IP-{{ item }}'
          type: '{{ (item is search("-"))|ternary("iprange","ipmask") }}'
          subnet: '{{ (item is search("-"))|ternary(omit,item+((item is search("/"))|ternary(omit,"/32"))) }}'
          start_ip: '{{ (item is search("-"))|ternary(item.split("-")[0],omit) }}'
          end_ip: '{{ (item is search("-"))|ternary(item.split("-")[1],omit) }}'
          associated_interface: '{{ interface }}'
          comment: '{{ issue }}'
        state: "present"
      with_items: '{{ target_data.ips }}'
      register: IPs

5. We check whether there is a group for our client, and create one if there is none. fortigate does not allow you to create an empty group, so we put the first address from the list in it (or you can add a Null address in Fortigate, it’s your choice here)

- name: Check if client group already exists
      fortinet.fortios.fortios_configuration_fact:
        vdom: '{{ vdom }}'
        access_token: "{{ fortigate_token }}"
        selector: firewall_addrgrp
        params:
          name: "GROUP_{{ target_data.client | upper }}"
      ignore_errors: true
      register: group_not_exists

    - name: Create CLIENT GROUP
      fortinet.fortios.fortios_firewall_addrgrp:
        vdom:  '{{ vdom }}'
        state: "present"
        access_token: '{{ fortigate_token }}'
        firewall_addrgrp:
          allow_routing: "disable"
          category: "default"
          color: "5"
          comment: '{{ issue }}'
          name: 'GROUP_{{ target_data.client | upper }}'
          type: "default"
          uuid: "GROUP_{{ target_data.client | upper }}"
          visibility: "enable"
          member:
            - name: 'IP-{{ IPs.results[0].item }}'
      when: group_not_exists.failed == true

6. Add addresses to the desired group

- name: Put target IPs in client group
      fortinet.fortios.fortios_firewall_addrgrp:
        vdom:  '{{ vdom }}'
        state: "present"
        access_token: '{{ fortigate_token }}'
        member_path: member:name
        member_state: present
        firewall_addrgrp:
          allow_routing: "disable"
          category: "default"
          color: "5"
          comment: '{{ client_group_info.meta.results[0].comment  + "," + issue }}'
          name: 'GROUP_{{ target_data.client | upper }}'
          type: "default"
          uuid: "GROUP_{{ target_data.client | upper }}"
          visibility: "enable"
          member: 
          - name: 'IP-{{ item.item }}'
      with_items: ' {{ IPs.results }} '
      loop_control:
        label: ' {{ IPs.results }} '

7. We add a group to the group (sorry for the tautology) of the component we need

- name: Put target client group in external COMPONENT GROUP in policy
      fortinet.fortios.fortios_firewall_addrgrp:
        vdom:  '{{ vdom }}'
        state: "present"
        access_token: '{{ fortigate_token }}'
        member_path: member:name
        member_state: present
        firewall_addrgrp:
          allow_routing: "disable"
          category: "default"
          color: "5"
          comment: '{{ component_group_info.meta.results[0].comment  + "," + issue }}'
          name: 'COMPONENT_{{ target_data.components | upper }}'
          type: "default"
          uuid: "COMPONENT_{{ target_data.components | upper }}"
          visibility: "enable"
          member: 
          - name: 'GROUP_{{ target_data.client | upper }}'

8. We set the labels for searching for the task and write a comment about the successful launch, in which we display the result (which addresses were added to which group and for which component)

- name: Set the labels of created group for trace
      community.general.jira:
        uri: '{{ jira_server }}'
        username: '{{ jira_user }}'
        password: '{{ jira_api_token }}'
        issue: '{{ item.key }}'
        operation: edit
      args:
        fields:
            labels:
              - whitelisting
              - 'GROUP_{{ target_data.client | upper }}'
              - autocompleted

    - name: Comment on issue
      community.general.jira:
        uri: '{{ jira_server }}'
        username: '{{ jira_user }}'
        password: '{{ jira_api_token }}'
        issue: '{{ item.key }}'
        operation: comment
        comment: " {{ 'AutoWhitelisted for ' + target_data.components | upper +  ' using, created/updated GROUP_' + 
        target_data.client | upper  + ' group in firewall. Please check the results  \n List of whitelisted IPs:\n' + target_data.ips | join(',') | replace(',', ',\n')}} "

Let’s return from the code to real life for clarity. The form in Jira tasks has a similar appearance in our case

Information about the result in the form of a comment (my account and token were used in the tests):

That’s all!

Voila! Life has been successful, the routine has decreased. All that remains is to run this role and the playbook for it in AWX or any other system with Ansible, set the schedule and be happy. In our company, this wonderwaffle already works in a pre-prod environment and feels quite good, coping with the work that a person can do (with a larger number of addresses) in 20 minutes, in a matter of minutes. Our coordinators just need to create a task, which they already started, but now simply with a form and change the status after verification.

Thanks for your attention, I hope you found it useful or at least interesting. There are a couple of steps in the code not described above, but they are all more in the nature of an additional trace. Below is a link to the repository with the code and how to get started, if you want to see how it works or even use it yourself. Happy automation!

Repo: https://github.com/devops-engineer-gaming/fortigate_whitelisting/tree/master/whitelisting

PS Pictures of cats are taken from the VK group “public for employees 5/2”