a new tool in the fight against bugs. How AI can be used to improve the quality of testing

Short description

A QA Engineer has shared their experience of using the neural network ChatGPT in software testing. The article provides examples of ChatGPT’s uses in generating test cases, creating competency matrices, selecting mobile devices, identifying locators on a webpage, and creating SQL queries and API requests. The article also outlines the advantages and disadvantages of using such networks in testing, such as automation of test data generation and a limitation on its ability to create complex test cases. The author predicts that specialized neural networks for software testing will be released soon, which will offer even greater possibilities for QA Engineers.

a new tool in the fight against bugs. How AI can be used to improve the quality of testing

Hello everybody! I am a QA Engineer and recently discovered ChatGPT and want to open this world to other testers. This article is not a guide to use and there will be no super life hacks here. The purpose of this article is to show the huge possibilities of neural networks in helping us testers in our work. I hope that someone who reads this can make it a little easier to write test cases, write documentation, or write self-tests. Examples with writing code are here at least, since there are already many such materials. A small introduction:

Advantages of using ChatGPT in software testing:

  1. Automation of test data generation: ChatGPT can be used to generate various test data, such as output data and expected results, which can greatly speed up the testing process.

  2. Improving the quality of testing: ChatGPT can be used to create more realistic and complex test cases that can help uncover a wider range of bugs and issues in software.

  3. Reduction of testing time: ChatGPT can speed up the process of creating test cases and testing, reducing the time spent on manual testing.

Disadvantages of using ChatGPT in software testing:

  1. Learning limitation: ChatGPT can only be trained on the available data, which can limit its ability to create more complex test cases.

  2. The need for additional verification: ChatGPT may generate test cases that are not always correct and accurate, so additional validation and manual testing is required.

  3. High difficulty: ChatGPT is a rather complex technology that may require additional efforts for its implementation in software testing.

Below are possible examples of the use of technology in work.

1. Test cases

I asked ChatGPT to write test cases for the authorization form, a favorite task in any QA interview. Of course, he needed conditional requirements for the compilation of cases.

Request:

Answer

The first 10 cases are quite enough for a validation check, I think, but I decided to torture him and asked for more.

Answer

After 21 cases, he already began to fantasize and invent his requirements, because he thought that he had written enough for my requirements. This suggests that the checks it offers are optimal and cover the necessary requirements for the minimum number of cases:

Answer

2. Competence matrix

We have a small startup and there are no ready-made competency matrices for the position, and I want a promotion and I need to know what to strive for in order to objectively evaluate myself. And I can also take this matrix and become the author of grades in my company 🙂

Request: “Compile the competency matrix for the Manual Senior QA Engineer”

Answer

3. Types of testing

I’m writing documentation and I want to describe the types of testing to show the whole team what we do.

Request:

  1. What is the purpose of a load test?

  2. What is the method of load testing?

  3. How to determine that the load test was carried out successfully?

Answer

4. How to determine which cases need to be automated?

Also, by the way, interview questions:

Request: “I came for a new project. How to determine which tests should be automated?”

Answer

5. Selection of mobile devices

Let’s say we have some metric of devices and we need to transfer the task to procurement, choosing the right models optimally, consider an example:

Request: “We have a list of 10 smartphones used by users of our site, write on which permissions and operating systems we need to test the site to make sure everything will work on them, as well as the prices of these smartphones”:

  1. Xiaomi Redmi 9C

  2. Samsung Galaxy A12

  3. OPPO A17

  4. Realme 8

  5. Xiaomi Redmi Note 11 Pro

  6. POCO X4 Pro

  7. Realme GT Master Edition

  8. Infinix HOT 12i

  9. iPhone 11

  10. iPhone 5s”

Answer

You can also ask it to identify devices.

Request: “Write a list of the 10 most popular smartphones on the Internet, write what permissions and operating systems we need to test the site on to make sure everything will work on them, and also the prices of these smartphones”:

Answer

PS, it is better to check such information once again, at least point by point.

6. Search for locators on the page

I don’t want to spend a lot of time writing the test, the search and description of locators can be delegated to a neural network:

Request:

“I will reset the site for you, find on its main page all the elements containing “Security”, “Sign Up”, as well as the search bar and buttons with links to social networks at the bottom of the page, and write me the unique locators of these elements in XPATH format

https://github.com/”

Answer

Of course, he did not find all the locators flawlessly, but he managed something and, in general, it is worth a try:

Checking locators

7. SQL query

I’m not strong in SQL, but I urgently need to make a query and get the necessary information:

Request:

“These are two tables. The first one has columns: id, age, name. The second one is id, salary, vacation_days. Write a query that will display the name with the highest age and its vacation_days”

Answer with explanations

8. API request for Requests

In my opinion, he did the best with this. Of course, you can’t cover the entire project with such tests, but in fact you have generated code for yourself, and then you finish it with your hands, why not.

Request:

“Please write code in Python + Requests that sends a GET request to the URL https://reqres.in/api/users?page=2 and checks the status of the code plus the validity of the JSON schema”

Reply with comments
import requests
import jsonschema

url="https://reqres.in/api/users?page=2"

# отправляем GET запрос на URL
response = requests.get(url)

# проверяем статус код
if response.status_code == 200:
    print('Статус код:', response.status_code)
else:
    print('Ошибка! Статус код:', response.status_code)

# проверяем валидность схемы JSON
schema = {
    "type": "object",
    "properties": {
        "page": {"type": "integer"},
        "per_page": {"type": "integer"},
        "total": {"type": "integer"},
        "total_pages": {"type": "integer"},
        "data": {
            "type": "array",
            "items": {
                "type": "object",
                "properties": {
                    "id": {"type": "integer"},
                    "email": {"type": "string"},
                    "first_name": {"type": "string"},
                    "last_name": {"type": "string"},
                    "avatar": {"type": "string"}
                },
                "required": ["id", "email", "first_name", "last_name", "avatar"]
            }
        }
    },
    "required": ["page", "per_page", "total", "total_pages", "data"]
}

try:
    jsonschema.validate(response.json(), schema)
    print('Схема JSON валидна!')
except jsonschema.exceptions.ValidationError as err:
    print('Ошибка валидации схемы JSON:', err)

Such cases. This is only a small part of what you can think of. I am sure that neural networks specialized in software testing will be released this year, one can only imagine what functions they will perform. Perhaps one of those who read this article will be inspired here and make their own product. If I had known at the university that I might need matrices and math analysis to write my neural network, I would definitely have studied better.

By the way, ChatGPT, Notion AI and MidJourney neural networks also helped me in writing this article.

If you look events in Mykolaiv – https://city-afisha.com/afisha/

Related posts