Cloudflare announced a new tool to help customers detect and analyze the presence of bots on their website. Called Bot Analytics, the new solution provides more granular detection, informing users of the number of bots present and whether they can be classified as "good bots" or "bad bots." While bots get a bad reputation online, not all bots are malicious. In fact, many useful services that use some level of automation use bots. These are completely above the board and, as long as they are open to practice, they should not be prevented from visiting the sites. On the other hand, "bad bots" can be an unwanted presence. They can accumulate inventory or pull content from a particular site. However, since bots account for about 40% of all online traffic, it is not easy to determine which bots to block and which to allow. This is where Cloudflare's Bot Analytics comes into play.
Bad robots
"Starting today, we'll show you the bots that are coming to your website," says Ben Solomon, product manager for bot management at Cloudflare. “You can see these bots with a new tool called Bot Analytics. It's fast, accurate, and loaded with information. You can query data up to a month in the past without noticeable lag. To do this, we exposed the data with GraphQL and paired it with Adaptive Throughput (ABR) technology to dynamically load content. If you have already added Bot Management to your Cloudflare account, Bot Analytics is included in your service. Boy Analytics can be found in the Firewall tab of the Cloudflare dashboard. Businesses without access to Bot Management will only see a snapshot of their bots' traffic, but that should be enough to determine if bots are swarming a web page. Website admins can also set their own bot threshold to determine the automated traffic they allow in. Of course, there's no foolproof way to keep all bots out, but setting the right limit should ensure more bandwidth is granted to genuine website visitors.