Website developers can now block OpenAI’s GPTBot from visiting the site

Short description

Summarize this content to 100 words OpenAI said in its blog that third-party web developers can explicitly block GPTBot from visiting a site. In this way, OpenAI will not be able to use the content on the site to train its models.An instruction has appeared on the OpenAI website, which describes how to prevent GPTBot from accessing content on the site. To do this, you need to specify the following in the robots.txt file:User-agent: GPTBot
Disallow: /You can also set up more flexible access. For example, the robot will be able to access only selected site directories:User-agent: GPTBot
Allow: /directory-1/
Disallow: /directory-2/The GPTBot bot is used by OpenAI to collect information and generate datasets. The company may use this data for future models. At the same time, it is not possible to prohibit access to the site retroactively. If GPTBot has already managed to get some data, it cannot be deleted from the OpenAI database.

Website developers can now block OpenAI’s GPTBot from visiting the site

OpenAI said in its blog that third-party web developers can explicitly block GPTBot from visiting a site. In this way, OpenAI will not be able to use the content on the site to train its models.

An instruction has appeared on the OpenAI website, which describes how to prevent GPTBot from accessing content on the site. To do this, you need to specify the following in the robots.txt file:

User-agent: GPTBot
Disallow: /

You can also set up more flexible access. For example, the robot will be able to access only selected site directories:

User-agent: GPTBot
Allow: /directory-1/
Disallow: /directory-2/

The GPTBot bot is used by OpenAI to collect information and generate datasets. The company may use this data for future models. At the same time, it is not possible to prohibit access to the site retroactively. If GPTBot has already managed to get some data, it cannot be deleted from the OpenAI database.

Related posts