Log parsing clarification

[tomswartz07]

Hi there!

Love the project, it's super useful and provides very interesting insights.

I note that you have documented that you can collect from the log files and display detailed information from these logs, but it's not super clear in which situations it applies.

Specifically, there are two separate use cases where I see issues (or confusion on my part) in getting log parsing to work:

1. Remote server, even when specifying --log-file paths
2. Local server, when the log destination is syslog/journald

Can you clarify if it's possible to enable the log parsing (and seemingly associated info in the output, like Slow Queries) when using pgmetrics with either of these configurations?

If it's not possible to do so in either of these situations, can the specific configurations/environment necessary for this be documented?

Thanks very much!

[mdevan]

Currently, pgmetrics can only parse local log files that are actually present as files in the filesystem.

In case of remote servers, you'll have to first fetch the log files (over scp/rsync or similar) to the server where pgmetrics is running, and then invoke pgmetrics. In future, we plan to support an scp-like syntax for the --log-file/--log-dir options.

Syslog/journald also will have to be pulled out into a file somehow before invoking pgmetrics. Not sure if we'll be adding support for this in the future though. Note that the extracted file should have lines that match the "log_line_prefix" configuration setting.

and seemingly associated info in the output, like Slow Queries

pgmetrics currently collects (1) autovacuum runs, (2) deadlocks and (3) auto_explain query plans from log files. Slow queries output comes from pg_stat_statements.

[tomswartz07]

you'll have to first fetch the log files (over scp/rsync or similar) to the server where pgmetrics is running

That's a reasonable workaround for the time being. Thanks for the info.

pgmetrics currently collects (1) autovacuum runs, (2) deadlocks and (3) auto_explain query plans from log files. Slow queries output comes from pg_stat_statements.

Ah, thanks for clarifying. The docs were not super clear on this front, it seemed to imply that there were some magic that grabbed log lines from log_min_duration_statement no explicit mention of pg_stat_statements for that.

Would it be possible to add a small section to the docs highlighting what extensions are needed for what feature? Perhaps even showing the ideal/expected settings associated for each.

Thanks again for the very nice project!

[mdevan]

Would it be possible to add a small section to the docs highlighting what extensions are needed for what feature? Perhaps even showing the ideal/expected settings associated for each.

That sounds useful, will add. Thanks for the suggestion!

[tomswartz07]

Thanks very much @mdevan

Feel free to close this now, or if you prefer to keep it opened until such docs have been added, that's fine too.

Cheers