Ternlight: 7 MB Embedding Model That Runs in the Browser (WASM SIMD Guide)

Semantic search usually means: embed documents on a server, embed the query via an API, pay per token, pray your RAG pipeline stays up. Ternlight flips the default — a 7 MB sentence encoder that runs in the browser via Rust → WASM SIMD, returning 384-dimensional vectors in ~5 ms with zero network after load.

The live demo indexes 2,000 React documentation chunks entirely on-device. Type a question; cosine similarity ranks matches even when wording diverges ("reset my password" ↔ "I forgot my password" → ~0.88). For anyone building FAQ bots, support search, or privacy-first apps, Ternlight is the kind of small model that makes embeddings feel like infrastructure, not a SaaS line item.

TL;DR — Ternlight at a glance

What problem Ternlight solves

Embeddings encode meaning as dense vectors. Classic workflow:

User query → API embed → vector DB ANN search → top-k chunks → LLM

Pain points:

• Latency — network + cold starts on every keystroke in search-as-you-type
• Cost — embedding APIs charge per token at scale
• Privacy — support tickets and internal docs leave the device
• Offline — mobile and air-gapped environments

Ternlight targets small-to-medium corpora (hundreds to low thousands of chunks) where client-side embed + brute-force cosine is fast enough after one-time indexing.

Architecture — distill, quantize, WASM

Per the GitHub README:

1. Distillation from MiniLM sentence encoder (smaller student, teacher supervision)
2. Ternary quantization-aware training — weights constrained for extreme compression
3. Custom inference engine in Rust, compiled to WASM SIMD for browsers and Node
4. Bundled weights inside npm — no separate model download step

┌─────────────┐     ┌──────────────────┐     ┌─────────────────┐
│ Input text  │ ──► │ WASM SIMD engine │ ──► │ 384-dim vector  │
└─────────────┘     │ + 7 MB weights   │     └────────┬────────┘
                    └──────────────────┘              │
                                                      ▼
                                            cosine vs corpus vectors

Why not transformers.js or ONNX in browser? Ternlight optimizes for one job — short sentence embeddings — at single-digit megabytes. General runtimes ship broader ops at higher weight cost.

Quick start — three lines

npm install @ternlight/base

import { embed, similar } from '@ternlight/base';

const recipes = [
  'quick pasta with garlic and olive oil',
  'slow cooker beef stew for winter',
  '15-minute stir fry with rice noodles',
];

similar('easy weeknight dinner ideas', recipes, { topK: 3 });
// → ranked matches · ~5 ms · zero network

@ternlight/mini — smaller wire size, ~2.5 ms/embed — trade accuracy for speed in tight UI loops.

Demo walkthrough — React docs on your CPU

Open ternlight-demo.vercel.app:

HN insight: Precompute embeddings server-side once, ship vectors JSON to the client, run query embed + cosine locally — best of both worlds for larger corpora.

wazzup.im integration (HN comment): offline search at app.wazzup.im/search — first visit downloads model; later runs from cache.

Use cases that fit (and cases that don't)

Good fits

Poor fits

vs cloud APIs and gte-small

Creator soycaporal on HN (July 2026):

• gte-small scores higher on MTEB (~61 vs MiniLM ~56)
• Ternlight holds 0.84 Spearman fidelity to MiniLM teacher
• Head-to-head STS-B/MTEB — on roadmap
• gte-small distillation — also on roadmap

explainx.ai read: Ternlight is a wedge product — prove useful embeddings without ops. Pair with a small local LLM workflow for full on-device Q&A.

WASM, SIMD, and the W3C LLM API question

Ternlight rides WebAssembly SIMD — near-native CPU math in Chrome, Firefox, Safari (modern versions). HN tangents asked for a W3C LLM API (Language.complete(...)). Embeddings are simpler: deterministic forward pass, no sampling, no streaming tokens — ideal first citizen for standardized in-browser inference.

Browser matrix (July 2026): SIMD WASM is widely available on desktop; iOS Safari supports WASM but test memory limits on older iPads when loading 7 MB weights + 2k×384 float32 vectors (~3 MB vectors alone).

Practical 2026 pattern:

Static site / SPA
  ├── ternlight.wasm + weights (7 MB)
  ├── precomputed embeddings.json (server build step)
  └── query: embed locally → rank → optional LLM API for answer synthesis

Separation of concerns: cheap local retrieval, expensive cloud generation only on the top-k.

Training pipeline — why open weights matter

Unlike black-box API embedders, Ternlight ships:

• MIT license on code + weights
• Training pipeline in repo — reproduce or fine-tune on domain data
• Distillation recipe documented for ML engineers auditing compression

For regulated industries, on-device + auditable weights beats "trust our API" — aligns with PII tooling patterns (keep text local, embed local, redact before any cloud LLM).

Limitations and roadmap (honest)

From GitHub issues and HN:

• Benchmarks — public STS-B/MTEB table still pending
• gte-small teacher — planned upgrade path
• Initial index time — ~30s for 2k chunks surprises users; document pre-indexing
• Fan noise — demo maxes CPU; HN joked about laptop fans — use Web Workers in production
• Not generative — needs separate LLM for answer composition

When to choose Ternlight in your stack

Choose Ternlight if:

• Corpus under ~10k chunks and brute-force cosine is OK
• Privacy/offline is non-negotiable
• You want search-as-you-type without embedding API bills
• You are prototyping before committing to Zvec or Pinecone

Skip Ternlight if:

• You need multilingual SOTA on long-tail languages (verify first — not marketed for 100+ langs)
• Your corpus is massive — invest in ANN + server embedders
• Sub-1% retrieval recall matters for legal/medical — benchmark against cloud baselines first

Benchmarks and evaluation — what to measure before shipping

Before swapping OpenAI embed calls for Ternlight in production, run a domain eval (same method as embeddings guide):

1. Sample 100–200 real user queries from support logs or search analytics
2. Label 3–5 relevant docs per query from your corpus
3. Embed corpus once (server or client — store vectors in IndexedDB)
4. Measure Recall@5 and NDCG@10 vs your OpenAI baseline

Expectation setting: Ternlight targets ~0.84 Spearman fidelity to MiniLM teacher — not MTEB leaderboard wins. For internal docs in English, that is often good enough for search-as-you-type; for legal retrieval, run side-by-side.

Pre-indexing pattern (HN-recommended):

CI job: embed all docs with @ternlight/base (Node)
        → upload embeddings.json + manifest to CDN
SPA:    fetch vectors on first visit (cache)
        → embed query in WASM on each keystroke (debounced)
        → rank top-k locally

This splits one-time O(n) indexing from interactive O(n) query — acceptable until ~10k chunks, then shard or move ANN server-side.

Web Workers and production hygiene

The demo maxes a CPU core — users on HN noted laptop fans spinning on page open. Production checklist:

• Run embed() inside a Web Worker — keep UI thread free
• Debounce keystrokes (150–300 ms) before re-ranking
• Cache model in CacheStorage or IndexedDB after first load
• Show indexing progress — 30s for 2k chunks surprises users who expect instant search
• Fall back to keyword search while WASM loads

What people are asking (from HN)

Integration with RAG and agent stacks

Ternlight slots into the retrieval half of a RAG loop without touching generation:

Ingest:  docs → chunk → embed (Ternlight, CI) → store vectors
Query:   user text → embed (browser WASM) → top-k chunks
Answer:  top-k → cloud LLM (Claude, GPT, local) → streamed reply

This mirrors explainx.ai's local AI workflow — cheap retrieval local, expensive reasoning remote. For Langflow prototypes, swap the embed node with a small custom component calling @ternlight/base before you commit to Pinecone bills.

Agent memory angle: episodic memory systems often embed conversation summaries. A 7 MB embedder in the browser could rank which past sessions to inject into context — same primitive as vector search fundamentals, different packaging.

Mobile PWA note: 7 MB is viable on mid-tier phones after Wi‑Fi first load; test thermal throttling on sustained indexing — same fan issue as laptop demo.

Licensing and forkability

MIT license covers code + weights + training scripts — rare for embedding products. Teams can:

• Fine-tune on domain corpora (support macros, legal clauses, medical intake forms)
• Ship white-label search widgets without API rev-share
• Audit quantization impact on compliance-sensitive vocab

Contrast with OpenAI embed APIs where weights are inaccessible — Ternlight is an inspectable primitive for security reviews.

Roadmap watch: creator noted gte-small distillation and public STS-B/MTEB tables on GitHub — when those land, re-run your domain eval before assuming MiniLM-era fidelity caps.

FAQ — quick answers for engineering leads

Can we drop Pinecone for internal docs? For under ~5k chunks with privacy requirements, yes — pair Ternlight with IndexedDB vectors. Keep Pinecone when you need hybrid BM25, multi-tenant isolation, or billion-scale ANN.

Does this replace OpenAI embed in production RAG? Only after your Recall@5 eval passes on real queries. Many teams run Ternlight in the browser for search UI while keeping server embedders for batch ingest until pipelines are unified.

Security review blocker? MIT weights + WASM sandbox is easier to approve than sending all doc text to a third-party embed API — document the WASM SIMD CSP requirements for your InfoSec team.

Related on explainx.ai

• What are embeddings? Vector search complete guide
• Zvec — in-process vector database
• Build a personal local AI system
• Langflow document Q&A tutorial
• Multimodal AI guide
• reMarkable Riddle — vision LLM on e-ink

External: ternlight GitHub · Demo

Model sizes, latency, and fidelity figures reflect the July 2026 release. Run your own STS-B eval on domain data before replacing production embedders.