memory-lancedb-pro

Production-grade long-term memory system (v1.1.0-beta.8) for OpenClaw AI agents. Provides persistent, intelligent memory storage using LanceDB with hybrid vector + BM25 retrieval, LLM-powered Smart Extraction, Weibull decay lifecycle, and multi-scope isolation.

For full technical details (thresholds, formulas, database schema, source file map), see references/full-reference.md.

Applying the Optimal Config (Step-by-Step Workflow)

When the user says "help me enable the best config", "apply optimal configuration", or similar, follow this exact procedure:

Step 1 — Present configuration plans and let user choose

Present these three plans in a clear comparison, then ask the user to pick one:

Plan A — 🏆 Full Power (Best Quality)

• Embedding: Jina jina-embeddings-v5-text-small (task-aware, 1024-dim)
• Reranker: Jina jina-reranker-v3 (cross-encoder, same key)
• LLM: OpenAI gpt-4o-mini (Smart Extraction)
• Keys needed: JINA_API_KEY + OPENAI_API_KEY
• Get keys: Jina → https://jina.ai/api-key · OpenAI → https://platform.openai.com/api-keys
• Cost: Both paid (Jina has free tier with limited quota)
• Best for: Production deployments, highest retrieval quality

Plan B — 💰 Budget (Free Reranker)

• Embedding: Jina jina-embeddings-v5-text-small
• Reranker: SiliconFlow BAAI/bge-reranker-v2-m3 (free tier available)
• LLM: OpenAI gpt-4o-mini
• Keys needed: JINA_API_KEY + SILICONFLOW_API_KEY + OPENAI_API_KEY
• Get keys: Jina → https://jina.ai/api-key · SiliconFlow → https://cloud.siliconflow.cn/account/ak · OpenAI → https://platform.openai.com/api-keys
• Cost: Jina embedding paid, SiliconFlow reranker free tier, OpenAI paid
• Best for: Cost-sensitive deployments that still want reranking

Plan C — 🟢 Simple (OpenAI Only)

• Embedding: OpenAI text-embedding-3-small
• Reranker: None (vector+BM25 fusion only, no cross-encoder)
• LLM: OpenAI gpt-4o-mini
• Keys needed: OPENAI_API_KEY only
• Get key: https://platform.openai.com/api-keys
• Cost: OpenAI paid only
• Best for: Users who already have OpenAI and want minimal setup

Plan D — 🖥️ Fully Local (Ollama, No API Keys)

• Embedding: Ollama mxbai-embed-large (1024-dim, recommended) or nomic-embed-text:v1.5 (768-dim, lighter)
• Reranker: None — Ollama has no cross-encoder reranker; retrieval uses vector+BM25 fusion only
• LLM: Ollama via OpenAI-compatible endpoint — recommended models with reliable JSON/structured output:
  • qwen3:8b (recommended — best JSON output, native structured output, ~5.2GB)
  • qwen3:14b (better quality, ~9GB, needs 16GB VRAM)
  • llama4:scout (multimodal MoE, 10M ctx, ~12GB)
  • mistral-small3.2 (24B, 128K ctx, excellent instruction following, ~15GB)
  • mistral-nemo (12B, 128K ctx, efficient, ~7GB)
• Keys needed: None — fully local, no external API calls
• Prerequisites:
  • Ollama installed: https://ollama.com/download
  • Models pulled (see Step 5 below)
  • Ollama running: macOS = launch the app from Applications; Linux = systemctl start ollama or ollama serve
• Cost: Free (hardware only)
• RAM requirements: mxbai-embed-large ~670MB; qwen3:8b ~5.2GB; qwen3:14b ~9GB; llama4:scout ~12GB; mistral-small3.2 ~15GB
• Trade-offs: No cross-encoder reranking = lower retrieval precision than Plans A/B; Smart Extraction quality depends on local LLM — if extraction produces garbage, set "smartExtraction": false
• Best for: Privacy-sensitive deployments, air-gapped environments, zero API cost

After user selects a plan, ask in one message:

1. Please provide the required API key(s) for your chosen plan (paste directly, or say "already set as env vars")
2. Are the env vars already set in your OpenClaw Gateway process? (If unsure, answer No)
3. Where is your openclaw.json? (Skip if you want me to find it automatically)

If the user already stated their provider/keys in context, skip asking and proceed.

Do NOT proceed to Step 2 until API keys have been collected and verified (Step 2 below).

Step 2 — Verify API Keys (MANDATORY — do not skip)

Run ALL key checks for the chosen plan before touching any config. If any check fails, STOP and tell the user which key failed and why. Do not proceed to Step 3.

Plan A / Plan B — Jina embedding check:

curl -s -o /dev/null -w "%{http_code}" \
  https://api.jina.ai/v1/embeddings \
  -H "Authorization: Bearer <JINA_API_KEY>" \
  -H "Content-Type: application/json" \
  -d '{"model":"jina-embeddings-v5-text-small","input":["test"]}'

Plan A / B / C — OpenAI check:

curl -s -o /dev/null -w "%{http_code}" \
  https://api.openai.com/v1/models \
  -H "Authorization: Bearer <OPENAI_API_KEY>"

Plan B — SiliconFlow reranker check:

curl -s -o /dev/null -w "%{http_code}" \
  https://api.siliconflow.com/v1/rerank \
  -H "Authorization: Bearer <SILICONFLOW_API_KEY>" \
  -H "Content-Type: application/json" \
  -d '{"model":"BAAI/bge-reranker-v2-m3","query":"test","documents":["test doc"]}'

Plan D — Ollama check:

curl -s -o /dev/null -w "%{http_code}" http://localhost:11434/api/tags

Interpret results:

If any check fails: Tell the user exactly which provider failed, the HTTP code received, and what to fix. Do not proceed with installation until all required keys pass their checks.

If the user says keys are set as env vars in the gateway process, run checks using ${VAR_NAME} substituted inline or ask them to paste the key temporarily for verification.

Step 3 — Find openclaw.json

Check these locations in order:

# Most common locations
ls ~/.openclaw/openclaw.json
ls ~/openclaw.json
# Ask the gateway where it's reading config from
openclaw config get --show-path 2>/dev/null || echo "not found"

If not found, ask the user for the path.

Step 4 — Read current config

# Read and display current plugins config before changing anything
openclaw config get plugins.entries.memory-lancedb-pro 2>/dev/null
openclaw config get plugins.slots.memory 2>/dev/null

Check what already exists — never blindly overwrite existing settings.

Step 5 — Build the merged config based on chosen plan

Use the config block for the chosen plan. Substitute actual API keys inline if the user provided them directly; keep ${ENV_VAR} syntax if they confirmed env vars are set in the gateway process.

Plan A config (plugins.entries.memory-lancedb-pro.config):

{
  "embedding": {
    "apiKey": "${JINA_API_KEY}",
    "model": "jina-embeddings-v5-text-small",
    "baseURL": "https://api.jina.ai/v1",
    "dimensions": 1024,
    "taskQuery": "retrieval.query",
    "taskPassage": "retrieval.passage",
    "normalized": true
  },
  "autoCapture": true,
  "autoRecall": true,
  "captureAssistant": false,
  "smartExtraction": true,
  "extractMinMessages": 2,
  "extractMaxChars": 8000,
  "llm": {
    "apiKey": "${OPENAI_API_KEY}",
    "model": "gpt-4o-mini",
    "baseURL": "https://api.openai.com/v1"
  },
  "retrieval": {
    "mode": "hybrid",
    "vectorWeight": 0.7,
    "bm25Weight": 0.3,
    "rerank": "cross-encoder",
    "rerankProvider": "jina",
    "rerankModel": "jina-reranker-v3",
    "rerankEndpoint": "https://api.jina.ai/v1/rerank",
    "rerankApiKey": "${JINA_API_KEY}",
    "candidatePoolSize": 12,
    "minScore": 0.6,
    "hardMinScore": 0.62,
    "filterNoise": true
  },
  "sessionMemory": { "enabled": false }
}

Plan B config:

{
  "embedding": {
    "apiKey": "${JINA_API_KEY}",
    "model": "jina-embeddings-v5-text-small",
    "baseURL": "https://api.jina.ai/v1",
    "dimensions": 1024,
    "taskQuery": "retrieval.query",
    "taskPassage": "retrieval.passage",
    "normalized": true
  },
  "autoCapture": true,
  "autoRecall": true,
  "captureAssistant": false,
  "smartExtraction": true,
  "extractMinMessages": 2,
  "extractMaxChars": 8000,
  "llm": {
    "apiKey": "${OPENAI_API_KEY}",
    "model": "gpt-4o-mini",
    "baseURL": "https://api.openai.com/v1"
  },
  "retrieval": {
    "mode": "hybrid",
    "vectorWeight": 0.7,