OpenAI launches web crawler ‘GPTBot’ amid plans for next model: GPT-5

ChatGPT users have the option to scrap the web crawler by adding a “disallow” command to a standard file on the server.