Info

License: Free — included in the free tier alongside CPU, memory, network, and logical disk. Same posture as the other host probes: local-OS observability, no remote API call, no vendor SDK.

Warning

Linux only. The probe spawns journalctl --output=json --follow as a subprocess to stream entries from the local systemd journal. The agent compiles for Windows and macOS as well, but the probe returns a clear not supported on <GOOS> error on those platforms — operators can keep a single configuration file across mixed fleets without conditional logic.

Linux Logs Probe¶

The linux_logs probe collects entries from the local systemd journal and publishes them as structured OTel log records. Combined with the OTLP storage, this gives a fully self-contained agent → OTel collector pipeline for OS logs — no Filebeat / journald-exporter sidecar to deploy.

Quick start¶

# probes.d/10-linux_logs.yaml — each file under probes.d/ is a YAML array of probes
- name: system-logs
  type: linux_logs
  params:
    priority: 6                          # max severity (0..7)

This emits every entry with priority ≤ 6 (i.e. notice and below; drops debug). The records flow through the agent log channel identical to records produced by the syslog and event probes — any storage that consumes logs (today: OTLP) ships them.

Parameters¶

Parameter	Must set	Default	Description
`units`	No	-	systemd units to follow; empty means every unit. Example: `nginx.service`
`identifiers`	No	-	Program names (SYSLOG_IDENTIFIER) to follow. Example: `sshd`
`priority`	No	`7`	Highest syslog priority to include, 0 (emergency) to 7 (debug)
`include_boot`	No	`false`	Replay entries since the current boot instead of streaming only new ones

Each units entry becomes a --unit= filter and each identifiers entry a --identifier= filter on the journalctl command line, so several entries widen the selection. Without include_boot, the stream starts at the moment the probe starts (--since=now).

Examples:

# probes.d/10-linux_logs.yaml
# Only ssh + kernel, error severity and above
- name: critical-only
  type: linux_logs
  params:
    identifiers: [sshd, kernel]
    priority: 3

# probes.d/10-linux_logs.yaml
# Multiple service filters
- name: app-services
  type: linux_logs
  params:
    units: [nginx.service, postgresql.service, my-app.service]
    priority: 6

Output¶

Each journal entry becomes an OTel log record with these attributes (mapped per the OTel semantic conventions):

Attribute	Source
Severity (number + text)	`PRIORITY` field, mapped with the RFC 5424 to OTel table
Body	`MESSAGE` field
`systemd.hostname`	`_HOSTNAME`
`systemd.unit`	`_SYSTEMD_UNIT`
`syslog.appname`	`SYSLOG_IDENTIFIER`
`process.pid`	`_PID`
`process.owner.uid`	`_UID`
`process.executable.name`	`_COMM`
`systemd.transport`	`_TRANSPORT`
`senhub.probe.name`	The probe instance name (from `probes[].name`)
`senhub.probe.type`	`"linux_logs"`

The host a record came from is the one on the resource, resolved as an FQDN, the same value the metrics carry. _HOSTNAME is the short kernel name of that same machine, so it is emitted as systemd.hostname and never as a record-level host.name — two spellings of one key would split the host in two on the consumer side.

Severity mapping (RFC 5424 → OTel):

Syslog	Name	OTel SeverityNumber	OTel SeverityText
0	emergency	24	FATAL4
1	alert	23	FATAL3
2	critical	22	FATAL2
3	error	17	ERROR
4	warning	13	WARN
5	notice	10	INFO2
6	info	9	INFO
7	debug	5	DEBUG

Operational notes¶

No periodic Collect. The probe is event-driven: the subprocess pushes records as they appear. The poller's tick is a no-op.
Subprocess lifecycle. On probe start, journalctl is spawned with Setpgid so signals reach only it, not the agent process group. On stop, SIGTERM is sent, then SIGKILL escalates if the process doesn't exit within the shutdown deadline.
Resilience. Each malformed JSON line is logged at DEBUG and skipped — a single garbled entry never breaks the stream.
Cardinality. The default priority: 7 keeps debug entries and can produce thousands of records per minute on a busy host. Set 6 to drop debug, or 4 (warnings and errors only) on chatty hosts where log storage is a concern.

Cross-platform notes¶

Linux: uses the standard journalctl from systemd. Works with systemd 230+ (universal across modern distros).
Windows + macOS: the probe registers but its OnStart returns "linux_logs probe is not supported on <GOOS> (requires systemd-journald)". The agent keeps running with other probes; only this probe is inert.
No CGO. Uses subprocess + JSON parsing, not libsystemd — preserves the agent's pure-Go build profile.

Metric reference¶

Every metric this probe can emit. Metric is the OpenTelemetry name the OTLP, Prometheus and Zabbix outputs derive theirs from. Name is what a Nagios check and the API metrics= filter match. PRTG channel is the label PRTG shows, placeholders filled from the series' tags.

Metric	Name	PRTG channel	Unit	Description
`senhub.linux_logs.records_emitted`	`senhub.linux_logs.records_emitted`	Linux Logs Records Emitted	#	Cumulative count of journal records this probe has published to the log rail