optimize robots.txt wordpress

When search engines like Google look at your website, they don’t just randomly browse your pages. They use automated bots