- Free, no sign-up
- Logs stay in your browser
- Real vs fake bots
Free Log File Analyzer
Your server logs are the only complete record of what Googlebot and the AI crawlers did on your site. Drop an access log (Apache, Nginx, Cloudflare or IIS, plain or .gz) and see every bot that came, which of them are real, what they crawled day by day and by folder, the status codes they got, the slow URLs, and the sitemap pages Google never asked for. The log is read in your browser and never uploaded. No sign-up.
0.0 0 ratings
How useful was this tool?
Be the first to rate it.
How to use it
Collect, drop, fix
Collect 30 days of logs, every hostname
From the host (cPanel Raw Access, or the web server's log folder on a VPS) and from the CDN, since behind Cloudflare many requests never reach your server. A week is too short to see how often deep pages get crawled. Plain or .gz files, several at once.
Drop them in
The file is read in your browser, line by line, so a 500 MB log works and nothing is uploaded. Every crawler's IP is checked against the ranges its maker publishes, so fake Googlebots are left out. Add your sitemap to see the URLs Googlebot never asked for.
Fix, then compare next month
Blocked AI bots, crawl spent on filters and search pages, server errors at night, old URLs answering 404, sitemap pages nobody crawls: each with the fix. Export the report and run the same check in 30 days.
Built forLarge shopsPublishersMigrationsAI visibility
Quick answer
What does a log file analyzer do?
A log file analyzer reads your web server's access log, where every request is recorded with the IP, time, URL, status code, size and user agent, and pulls out the search engine and AI crawlers. It verifies each one against the IP ranges Google, Bing, OpenAI, Anthropic and Perplexity publish, so fake Googlebots are not counted, then shows what the real ones crawled, how often, which status codes they got and where they spent their time. Search Console's Crawl stats is a sample; the log is the complete record.
Where to find your logs
Host, server and CDN
| Where the site runs | Where the access log is |
|---|---|
| Shared hosting with cPanel | Metrics, Raw Access: download each domain's log (.gz). Turn on "archive logs" so a month is kept. |
| Hostinger hPanel | Look in the site's Advanced settings for access logs, or ask support for the raw files. |
| A VPS with Nginx or Apache | /var/log/nginx/access.log or /var/log/apache2 (httpd) access_log, with rotated .gz files beside them. |
| Behind Cloudflare | Logpush (every plan, Free included, 25 GB a month) to a storage bucket, as JSON with ClientIP, ClientRequestURI, EdgeResponseStatus and ClientRequestUserAgent. |
| Windows server with IIS | C:\inetpub\logs\LogFiles, W3C format. |
A log holds your visitors' IP addresses. That is why this tool reads it in your browser: the file never reaches our server. Only the crawlers' IP lists come from us, and, if you ask, a DNS check of up to 20 bot IPs.
Real or fake
How bots are verified
Anyone can put "Googlebot" in a user agent. Google documents two checks: a reverse DNS lookup that ends in googlebot.com, google.com or googleusercontent.com, then a forward lookup back to the same IP; or matching the IP against the range files Google publishes at developers.google.com/static/crawling/ipranges/ and refreshes daily. This tool matches every line against those files and the ones Bing, Apple, OpenAI (GPTBot, OAI-SearchBot, ChatGPT-User), Anthropic, Perplexity and Common Crawl publish, and runs the DNS check on request for the leftovers. Bots that publish no list (Amazonbot, Bytespider, Meta, SEO tools) are counted but marked not checked.
AI crawlers in your logs
Training, search and user fetches
AI companies run separate bots for training, for search answers and for fetching a page a person asked about. In July 2025, 79% of AI crawling was for training, 17% for search and 3.2% for user actions (Cloudflare). The report groups them that way and flags a search or user bot that gets 403s: if robots.txt allows it, the block is a firewall, CDN or security plugin setting nobody decided on.
FAQ
Questions people ask
Is my log uploaded?
No. It is read in your browser by a background worker. Only the crawlers' public IP lists come from our server, and the sitemap if you add one.
What is log file analysis in SEO?
Reading the server's access logs to see what crawlers really requested and the status codes they got, rather than what tools estimate.
How do I verify Googlebot?
Reverse DNS plus forward DNS, or Google's published IP ranges. The tool checks every line against the ranges and runs the DNS check on request.
How often does Google crawl my site?
Count verified Googlebot requests per day in a month of logs; the chart shows it. Search Console's Crawl stats gives a sampled view of the same thing.
Why does a fake Googlebot visit my site?
Scrapers and some SEO tools borrow the name to get past rules. They are left out of every number here.
Which AI bots crawl my site?
The report lists each one it finds, verified against its maker's IP list, with what it is for: training, search or user fetches.
What does a 403 for an AI bot mean?
Something refused it: usually a firewall, CDN or plugin bot setting. If you want to be in ChatGPT or Perplexity answers, let their search bots through.
How long a log do I need?
30 days or more, for every hostname. A week is too short for deep pages.
Is log analysis useful for a small site?
On a small WordPress site it can wait; it pays on large shops, publishers and migrations, and for checking AI bots on any site.
Which formats work?
Apache and Nginx combined (with or without a virtual host first, and a response time at the end if present), Cloudflare Logpush JSON and IIS W3C, plain or gzip.
Keep going
More free tools and guides
- AI Crawler Checker: what robots.txt lets each AI bot do.
- Robots.txt Tester: test the rule for one bot and one URL.
- XML Sitemap Validator: the sitemap you compare against.
- Redirect Checker: the old URLs Googlebot still asks for.
- Crawling at scale and log files (Advanced, chapter 1) and AI crawlers (chapter 8).
Did this tool help you?
Tap a star. It takes a second, no sign-up.
–
0.0 0 ratingsNo ratings yet.
See what the bots really did
Verified Googlebot and AI bots, folders, errors and gaps. Free, your log stays with you.