Logrotate not resetting counter

[kevinkk525]

I'm having a bit of trouble with using grok_exporter with a copytruncate logrotate.

I'm using a counter to count e.g. the amount of ERROR lines in a logfile. That file gets rotated every day at 0:00am.

Expected behaviour
So I expect grok_exporter to read that file (readall=true btw) and count the ERROR lines, meaning after log rotation when the file is empty, I would expect that counter to be 0.

Actual behaviour
Instead, the counter continues to rise, grok_exporter just adds the new lines to the counter of the old lines. And when I restart grok_exporter, it falls back to what is actually in the current file.

Sadly this is a not helpful behaviour if you just want to count all ERROR messages within a day (within a single file that gets rotated every day).

System
Ubuntu 20.04
Grok built from source in docker container using dockerfile from #62

[fstab]

The typical approach is to use Prometheus queries to achieve what you want. For example

rate(errors_total[24h])

gives you the per second increase rate of the errors_total counter looking at the last 24h. It doesn't matter how often you run logrotate (as grok_exporter doesn't reset the counter on logrotate), and it doesn't matter if you restart grok_exporter (the rate() function adjusts for counter resets).

[kevinkk525]

Looks like you're right, at least if I use readall=false otherwise it messes with the rate. Thanks.
I kind of wanted to make the data stored in prometheus as "easily understandable" as possible but I guess that is just not how it should be done. Processing the data with rate() before understanding/looking at them should be good enough.
The only possible disadvantage is that with readall=false some lines could be lost when the grok_exporter gets restarted.