ops.foreversorted.com

Operations Watchdog

Hybrid watchdog foundation for Northbridge-first operational monitoring.

Watchdog Help

Critical

Current watchdog summary across generated artifacts: 8 healthy, 0 warning, 5 critical, 0 unknown.

Generated
2026-05-06 21:09
Watchdog Path
/home/shared/webprojects/operations/watchdog/output
Output Path Exists
Yes

Events / Raw

Recent alerts, collector stderr, and raw drill-down output.

CRITICAL
Latest Artifact
collector.json
Last Updated
2026-05-06 21:00
Artifacts
1

Live Check Summary

Artifact Healthy
0
Artifact Warning
0
Artifact Critical
1
Artifact Unknown
0
Maintained Artifacts
0
Maintained Criticals
0
Checks Healthy
0
Checks Warning
0
Checks Critical
0
Checks Unknown
0
Maintained Checks
0

Operator Notes

  • Store recent alert history and collector failures here for operator drill-down.
  • Use this tab for raw outputs rather than overloading the overview tab.

Collected Artifacts

Latest artifact details and the individual checks reported for this section.

Collector

Latest watchdog collector run metadata.

CRITICAL
Artifact File
collector.json
Updated
2026-05-06 21:00
Checks
0
Raw File
Open raw file
Maintenance
Off
Enter Maintenance Mode

This writes a maintenance rule into the watchdog config so the issue remains visible but no longer pages as an active critical.

Generated Files

  • hosts/main-host.json
  • services/apache.json
  • services/mariadb.json
  • services/exim.json
  • services/openvpn.json
  • services/backuppc.json
  • storage/capacity-and-storage.json
  • storage/backuppc.json
  • network/access.json
  • devices/arduino-pool.json
  • devices/solar-inverter.json
  • overview/summary.json
  • summary.json
  • events/collector.json

Embedded Summary

{
    "label": "Critical",
    "tone": "critical",
    "detail": "Watchdog collector completed with 11 artifacts, including 0 warning, 3 critical, 0 maintained critical, and 0 unknown states.",
    "updated_at": "2026-05-06T11:00:01+00:00",
    "counts": {
        "healthy": 8,
        "warning": 0,
        "critical": 3,
        "unknown": 0,
        "suppressed_critical": 0,
        "maintained": 0
    },
    "alert_items": [
        {
            "kind": "check",
            "id": "backuppc_last_success",
            "label": "BackupPC / BackupPC host homefile-data",
            "detail": "Latest completed successful backup for BackupPC host homefile-data is full #2685 from 2026-04-30 22:02 (age 6.0 days). Recent successful backups: #2685 full 2026-04-30 22:02 (6.0 days old); #2683 incr 2026-03-16 22:17 (51.0 days old); #2682 incr 2026-03-15 22:18 (52.0 days old)."
        },
        {
            "kind": "check",
            "id": "arduino-pool_freshness",
            "label": "Arduino Pool Controller / Arduino Pool Controller",
            "detail": "Freshness file not found: /home/shared/webprojects/operations/watchdog/heartbeats/arduino-pool.heartbeat"
        },
        {
            "kind": "check",
            "id": "solar-inverter_freshness",
            "label": "Solar Inverter / Solar Inverter",
            "detail": "Freshness age is 76954 minutes for /home/shared/webprojects/operations/watchdog/heartbeats/solar-inverter.heartbeat."
        }
    ],
    "maintained_items": []
}

Summary Artifact

{
    "label": "Critical",
    "tone": "critical",
    "detail": "Watchdog collector completed with 11 artifacts, including 0 warning, 3 critical, 0 maintained critical, and 0 unknown states.",
    "updated_at": "2026-05-06T11:00:01+00:00",
    "counts": {
        "healthy": 8,
        "warning": 0,
        "critical": 3,
        "unknown": 0,
        "suppressed_critical": 0,
        "maintained": 0
    },
    "alert_items": [
        {
            "kind": "check",
            "id": "backuppc_last_success",
            "label": "BackupPC / BackupPC host homefile-data",
            "detail": "Latest completed successful backup for BackupPC host homefile-data is full #2685 from 2026-04-30 22:02 (age 6.0 days). Recent successful backups: #2685 full 2026-04-30 22:02 (6.0 days old); #2683 incr 2026-03-16 22:17 (51.0 days old); #2682 incr 2026-03-15 22:18 (52.0 days old)."
        },
        {
            "kind": "check",
            "id": "arduino-pool_freshness",
            "label": "Arduino Pool Controller / Arduino Pool Controller",
            "detail": "Freshness file not found: /home/shared/webprojects/operations/watchdog/heartbeats/arduino-pool.heartbeat"
        },
        {
            "kind": "check",
            "id": "solar-inverter_freshness",
            "label": "Solar Inverter / Solar Inverter",
            "detail": "Freshness age is 76954 minutes for /home/shared/webprojects/operations/watchdog/heartbeats/solar-inverter.heartbeat."
        }
    ],
    "maintained_items": []
}