SEO Automation Suite

Choose your display preferences to get started

Theme

Color Vision

SEO Suite

Log Analysis

Demo Data nginx_combined

demo_access.log.gz · 2026-05-29 → 2026-06-04 · Week-over-Week comparison

Total Bot Requests

80656

↑ 2.6% vs prev period

HTML Pages Crawled

30368

↑ 1.6% vs prev period

Active Bot Types

15

across all crawlers

Crawl Errors (4xx+5xx)

8784

↑ 2.6% vs prev period

Crawl Volume Timeline

Daily bot requests — current vs previous period

Current Previous

Most Active Bots

By request count — current period

Googlebot Variants

Desktop · Mobile · Image · Ads · News · Inspection

Resource Type Distribution

What bots are actually fetching

HTTP Status Distribution

2xx / 3xx / 4xx / 5xx — current vs previous

Hourly Crawl Pattern (last 72 h)

Identify crawl windows and budget peaks

Week-over-Week Comparison

Key metrics across periods

Metric Current Previous Change

Bot IP Verification (rDNS)

Forward-confirmed reverse DNS — detect impostor bots

Verified Unverified
IP Address Bot Name Reverse DNS Requests Status
66.249.64.2 Googlebot Mobile crawl-66-249-64-2.googlebot.com 14798 Verified
198.50.128.1 MJ12bot crawl-198-50-128-1.mj12.co.uk 14521 Verified
149.56.123.1 SemrushBot crawl-149-56-123-1.semrush.com 14243 Verified
66.249.64.1 Googlebot Desktop crawl-66-249-64-1.googlebot.com 13743 Verified
66.249.73.2 Google Inspection Tool crawl-66-249-73-2.googlebot.com 13705 Verified
20.191.45.1 DuckDuckBot crawl-20-191-45-1.duckduckgo.com 12695 Verified
185.191.171.1 Unknown Bot unknown-185.191.171.1 10416 Unverified
203.208.60.1 Unknown Bot unknown-203.208.60.1 10301 Unverified
40.77.167.1 Bingbot crawl-40-77-167-1.search.msn.com 9118 Verified
66.249.64.3 Googlebot Image crawl-66-249-64-3.googlebot.com 7598 Verified
66.249.73.1 AdsBot Google crawl-66-249-73-1.googlebot.com 6368 Verified
17.58.98.1 Applebot crawl-17-58-98-1.applebot.apple.com 5691 Verified
180.76.15.1 Baiduspider crawl-180-76-15-1.baidu.com 5434 Verified
54.36.148.1 AhrefsBot crawl-54-36-148-1.ahrefs.com 5177 Verified
5.255.231.1 YandexBot crawl-5-255-231-1.yandex.ru 842 Verified

Top Crawled Pages

Most frequently requested HTML resources

# Path Hits
1 /
13644
2 /blog/
7769
3 /products/
6398
4 /blog/how-to-improve-seo/
5746
5 /blog/technical-seo-guide/
4975
6 /blog/keyword-research-tips/
4010
7 /pricing/
3802
8 /products/enterprise/
3516
9 /about/
3383
10 /blog/link-building-strategies/
3180
11 /products/pro/
2769
12 /contact/
2707
13 /blog/core-web-vitals/
2691
14 /products/starter/
2687
15 /blog/e-e-a-t-guide/
2123
16 /blog/structured-data/
2118
17 /docs/api/
2018
18 /docs/getting-started/
1952
19 /products/compare/
1952
20 /robots.txt
1434