Home › Guides › Find errors in an Nginx access log
Find errors in an Nginx access log
Direct answer: filter to 4xx/5xx responses, then group by path with numeric IDs masked — /products/123 and /products/456 are one route problem, not two. One pattern with many errors means a broken route; many patterns with few errors each means something systemic or a scanner. The tool below does it on your file, locally.
Triage this access log
Error paths, grouped (IDs masked)
| Path pattern | Errors | Total hits | Statuses |
|---|
Status distribution
| Status | Responses |
|---|
Top paths
| Path pattern | Hits |
|---|
Busiest hours
The two shapes of an error burst
Concentrated: one path pattern holds most errors. Think deleted product range still linked from the homepage, a route renamed in the last deploy, an API version retired while a client still calls it. The fix is specific and usually quick.
Diffuse: errors spread thinly across hundreds of paths. Think scanner wordlists
(/wp-login.php, /.env on every site on the internet), a broken sitemap submitting
every old URL, or a site-wide 500 during an outage — distinguishable by timestamp concentration and by who
the user-agents are. The bot checker answers the "who" from the same file.
The awk version (and its limit)
awk '$9 >= 400 {print $9, $7}' access.log | sort | uniq -c | sort -rn gets you counts by raw
path in seconds. Its limit is the splintering: every distinct ID is its own row, so real route problems hide
below one-off junk. Mask the segments — replace pure-number, UUID and long-ID path segments with a placeholder
before counting — and the table becomes the diagnosis. That masking step is exactly what the tool above
automates, alongside the status distribution and busiest hours.
Don't forget the clock
An error burst aligned with a traffic peak suggests capacity; aligned with a deploy timestamp suggests the release; aligned with one crawler's visit suggests a sitemap or robots problem. The triage report's hour table plus first/last seen per route pattern is usually enough to pick between those three. For application errors behind a 500, the access log can only point — the stack traces live in the app log, which is Error Grouper territory.
Frequently asked questions
What is the fastest way to find errors in an access log?
Filter to status >= 400, then group by path with IDs masked. awk '$9 >= 400' is the classic start, but raw paths splinter counts across every product/user ID. Grouping /products/123 and /products/456 into /products/<n> is what turns "900 scattered 404s" into "one route pattern is broken". The tool below does the grouping step for you.
A 404 spike: hacked, or something dumber?
Usually dumber. In rough order: a deploy removed or renamed pages; a sitemap or internal link points at dead URLs; a marketing link was shared with a typo; a scanner is walking a wordlist (check the user-agents and whether the paths look like /wp-login.php on a site that is not WordPress). A genuine attack probe tends to mix path shapes and methods. The bot checker on this site separates the scanner case.
Where are POST errors and API failures?
Same file, same method: filter status >= 500 and group by path. API routes mask cleanly (/api/v1/users/<n>) because the IDs are path segments. If your API errors are mostly one pattern with one status, the bug is in that handler; if they spread across patterns at one timestamp, look at the deploy and the database, not the handlers.