Log File Analyzer
Crawl budget · Bot traffic · Resource types
Parses server access logs (nginx, Apache, W3C/IIS, CloudFront) to map Google crawl activity, detect wasted crawl budget, classify resource types, and surface week-over-week or month-over-month trends across 7, 30, or 90 days.
What this demo shows
- ▸ Auto-detected format + rDNS bot validation for 25+ crawlers
- ▸ Google crawler variant breakdown (Googlebot, AdsBot, Mobile…)
- ▸ Resource type distribution — HTML vs JS/CSS vs image vs API
- ▸ Status code groups, hourly heatmap, WoW/MoM timeline
- ▸ Top crawled pages + weekly drilldown (30-day mode)