Google’s Martin Splitt Shares Tips for Combating Malicious Bots and Enhancing Website Performance

In a recent discussion, Google’s Martin Splitt tackled the growing issue of malicious bots and how they can negatively impact site performance. His advice is essential for SEOs and website owners looking to protect their sites and improve overall functionality.

The Problem: Malicious Bots and SEO

Security issues, including malicious bot traffic, are often overlooked during SEO audits. Many digital marketers focus on improving performance metrics, such as core web vitals, without considering the damaging impact security lapses can have on a website’s crawlability. A compromised site is unlikely to benefit from performance enhancements if bots are overloading its server. Security must be a key part of the conversation around site health.

When malicious bots excessively crawl a site, they can trigger “500 server errors,” preventing pages from loading and stopping Google from crawling the site properly. This can cripple a site’s search engine rankings.

Defending Against Malicious Bots

A site owner reached out for advice regarding scraper bots that were overloading their server and causing performance issues. Despite attempts at blocking IP addresses, the problem persisted.

The site owner asked:

“Our website is suffering from targeted scraping by automated bots, leading to server overload and security concerns. Despite blocking IPs and taking preventive steps, the issue remains. What can we do?”

Martin Splitt responded with a few key strategies:

• Identify the Source: First, try to determine the service or provider hosting the bots. Use tools like WHOIS to identify the owner and report the abuse.
• Use a CDN: Content Delivery Networks (CDNs) like Cloudflare have built-in firewall features that can detect and block malicious bot traffic, helping reduce server strain. CDNs distribute traffic across servers, lessening the load on your own server and improving site performance.

Will This Approach Work?

While identifying and reporting the cloud service hosting the bots can be helpful, it’s not always a reliable solution. Here’s why:

1. Bots Hide Their Origins: Many bots use VPNs or networks like Tor to mask their real locations. Hackers also leverage botnets, a network of compromised computers, to launch attacks, making it difficult to track their origin.
2. Frequent IP Switching: Bots often switch IP addresses as soon as they’re blocked, resuming their attack from a different network. Blocking one address doesn’t stop the problem entirely.
3. Inefficient Time Investment: Contacting infrastructure providers may not always be the best use of time. Even if you manage to block some malicious bots, many others are waiting to take their place. Large-scale bot attacks often involve thousands of compromised computers worldwide, making it nearly impossible to notify every service provider.

A More Effective Solution: Web Application Firewalls (WAF)

Splitt’s advice to use a CDN with a built-in Web Application Firewall (WAF) is spot-on. A CDN like Cloudflare not only improves site performance by distributing traffic but also blocks harmful bots automatically with its firewall.

Another great option is to use WordPress plugins like Wordfence, which offers a WAF that blocks bots based on their behavior. If a bot tries to access too many pages too quickly, it can trigger an automatic IP block. Even if the bot switches IP addresses, Wordfence recognizes the behavior and blocks it again.

For non-WordPress users, Sucuri is an excellent alternative. It provides a combination of WAF and CDN services to boost performance and secure your site against attacks. Both Wordfence and Sucuri offer free versions with limited but effective features.

The Bottom Line: Stop Bots Before They Hurt Your Site

Malicious bots can wreak havoc on website performance and security. Ignoring bot attacks can lead to server overloads and hinder your site’s ability to rank well in search results. By implementing a robust defense strategy—like utilizing a CDN with a WAF—you can significantly reduce the impact of bot traffic while enhancing site performance. Consider using tools like Wordfence or Sucuri to add an extra layer of protection.