Wiki Pipeline Scripts#

What Was Established#

Eight Python scripts in /opt/wiki/homelab/scripts/ implement the full wiki pipeline: file conversion, document ingestion, conversation crystallization (standard, DeepSeek, and Claude formats), shared LLM infrastructure, wiki health-checking, and knowledge-graph integration. All scripts were ported from the work wiki pipeline (itself developed 2026-04-21 → 2026-04-26) with homelab-specific infrastructure baked in.

crystallize.py (Claude format) uses a two-step LLM approach: gemma4:e2b cleans, qwen3.6:35b crystallizes. crystallize_deepseek.py skips gemma — JSON parsing is handled deterministically in Python (load_conversation + _clean_text), so only qwen is needed.

Wiki System - Architecture#

What Was Established#

The wiki system is designed around the LLM wiki pattern (Karpathy): raw sources (chat transcripts, notes, docs) are crystallized into structured markdown pages, embedded into pgvector, and retrieved semantically by agents in future sessions. A dedicated LXC (nk-wiki) will host the wiki VM, separating wiki infrastructure from other services.

Multi-Wiki Namespace Design#

Three wikis are planned, each with its own namespace in pgvector:

PostgreSQL#

What Was Established#

PostgreSQL 16 runs as an LXC on Proxmox, serving as the central database for n8n, the monitoring pipeline, and pgvector embeddings for the wiki.

Deployment#

Databases and Users#

Setup commands (run as postgres user)#

CREATE DATABASE homelab;
CREATE USER homelab WITH PASSWORD '<password>';
GRANT ALL PRIVILEGES ON DATABASE homelab TO homelab;
GRANT ALL ON SCHEMA public TO homelab;  -- required for n8n
\q

The GRANT ALL ON SCHEMA public step is required — without it n8n fails to start with a permissions error even though the database and user exist.