//nbkelley /homelab

AI-Driven Monitoring Pipeline

AI-Driven Monitoring Pipeline#

What Was Established#

The monitoring pipeline is fully operational and running hourly. It collects rich structured data from four sources (Prometheus — 7 metrics, Uptime Kuma, UniFi, Synology), runs 4 parallel Ollama summarization calls, synthesises a final status report, and writes everything to Postgres. Hourly snapshots of raw UniFi and Prometheus data are stored in dedicated tables for delta computation. End-to-end runtime is ~13 minutes using gemma4:e4b CPU-only on the Pavilion — accepted as-is pending Mac Studio.

n8n

n8n#

What Was Established#

n8n is the automation and orchestration hub for the homelab. It runs as an LXC on Proxmox, connected to PostgreSQL for persistent workflow state and execution history. Community edition is sufficient for current use.

Deployment#

Detail Value
LXC host n8n
IP 192.168.1.169
Port 5678
URL http://192.168.1.169:5678
Version 2.15.1 (Self Hosted)
Installed via tteck Proxmox helper scripts
OS Debian 13 (unprivileged LXC)
Config file /opt/n8n.env

Configuration (/opt/n8n.env)#

N8N_SECURE_COOKIE=false
N8N_PORT=5678
N8N_PROTOCOL=http
N8N_HOST=192.168.1.169
DB_TYPE=postgresdb
DB_POSTGRESDB_HOST=192.168.1.57
DB_POSTGRESDB_PORT=5432
DB_POSTGRESDB_DATABASE=homelab
DB_POSTGRESDB_USER=homelab
DB_POSTGRESDB_PASSWORD=<password>

After editing: systemctl restart n8n

Book Discovery Pipeline

Book Discovery Pipeline#

What Was Established#

A multi-agent, multi-node pipeline designed to identify high-prestige, upcoming literary works by analyzing critical reviews before they hit the mainstream.

Key Decisions#

  • Architecture: Two-tier agent system. Lightweight models (E4B) on the Orchestrator (T480) handle routing/filtering; heavier models (26B) on the Inference node (Pavilion) handle deep analysis.
  • Tech Stack: n8n (Orchestration), PostgreSQL (Data Storage), Hugo (Static Site Generation), Python/JS (Custom Logic).
  • Data Sources: RSS feeds (Literary Hub, etc.), Web Scraping (for indie blogs), and Goodreads API/Scraping for popularity comparison.

Current Configuration#

The Pipeline Chain#

  1. Ingestion: n8n fetches RSS feeds and scrapes blogs.
  2. Filtering (E4B): Classifies content (Review vs. News). Discards non-reviews.
  3. Extraction (26B): Extracts title, author, publisher, and critical language.
  4. Scoring (26B): Analyates “prestige signals” (e.g., phrases like “formally ambitious”) and compares against Goodreads popularity.
  5. Aggregation: Aggregates data into PostgreSQL.
  6. Publication: n8n generates a Markdown file and commits it to a Hugo repository.

Data Schema (PostgreSQL)#

  • sources: RSS metadata.
  • articles: Raw ingested content.
  • books: Normalized book records.
  • reviews: Links articles to books + extracted critical language.
  • prestige_scores: Historical scoring for trend tracking.

Open Questions#

  • How to effectively scrape Substack/paywalled content without high costs.
  • Determining the optimal frequency for the pipeline run (Weekly vs. Bi-weekly).

Open WebUI Deployment, Ollama Configuration, AI-Driven Monitoring Pipeline