explainx / blog
Zvec is Alibaba's battle-tested, open-source in-process vector database. Searches billions of vectors in milliseconds, with no server setup, WAL persistence, hybrid search, and Python/Node.js/Flutter SDKs.
Jun 26, 2026
OpenDataLab's MinerU turns PDFs and Office docs into LLM-ready Markdown and JSON. Version 3.4 ships PP-OCRv6, ~100% faster OCR, auto model-source selection, and 95%+ accuracy on hybrid backends — the default doc stack for RAG.
Jun 21, 2026
PixelRAG skips HTML parsing entirely. Instead it renders web pages and PDFs to screenshot tiles and retrieves over the images using a Qwen3-VL-Embedding model LoRA-fine-tuned on screenshot data. Tables, charts, and visual layout survive. Accuracy improves up to 18% over text-based RAG on SimpleQA benchmarks. There is a hosted API at pixelrag.ai/api backed by 8.28M Wikipedia pages, a CLI install in one pip command, and a Claude Code plugin that lets Claude screenshot any URL and read it like a human.
Jun 21, 2026
Turso rewrites SQLite in Rust — keeping full SQL and file format compatibility while adding MVCC concurrent writes, async io_uring I/O, CDC, vector search, full-text search, and an MCP server mode. 20,000+ stars, 253 contributors, in beta. Here is the full breakdown and why it matters for the agentic era.
Alibaba just open-sourced the vector database they've been running in production. Zvec — an in-process, embedded vector database — landed on GitHub in late 2025 and has already reached 9.9k stars. The v0.5.0 release dropped in June 2026, adding WAL guarantees, libaio support, and prefetch configuration for tuning search latency.
The pitch is straightforward: the speed of a native library, the durability of a real database, and zero server setup. If you've ever added Chroma or Qdrant to a project just to get local vector search and felt like it was too much infrastructure for what you needed — Zvec is the answer.
Zvec is an in-process vector database — a library that runs inside your application rather than as a separate server. You import it, open a collection (a directory on disk), insert vectors, and query them. There is no daemon to start, no port to configure, no connection pooling to manage.
This puts it in the same category as SQLite, not Postgres. The analogy is intentional: just as SQLite gave developers a production-grade relational database they could embed in any app, Zvec gives developers a production-grade vector database with the same zero-infrastructure profile.
Alibaba has been running it internally across multiple production workloads before open-sourcing it, which means the battle-testing has already happened at serious scale.
Most popular vector databases (Qdrant, Weaviate, Chroma in server mode, Milvus) require running a separate process. That's fine for teams with dedicated infra, but it creates real friction for:
Zvec eliminates all of these friction points. Since it's just a library, it starts the instant your process starts and shuts down cleanly when your process exits.
For context on where in-process vector search fits in the broader RAG landscape, see our comparison of RAG vs MCP for context-aware AI systems.
Zvec handles both dense vectors (typical embeddings from models like text-embedding-3) and sparse vectors (BM25-style term weights from models like SPLADE). Both can be queried in a single call, enabling true hybrid retrieval without a separate keyword search layer.
Combine semantic similarity with structured filters in one query. This matters for most real-world retrieval tasks where you want "most semantically similar to X" combined with "must have field Y = value Z."
Write-ahead logging means every insert is durable before it returns. Zvec survives process crashes and power failures without data loss — which separates it from pure in-memory libraries like FAISS.
Multiple processes can open the same Zvec collection simultaneously for reads. Only writes require exclusive single-process access. This makes it viable for multi-worker serving scenarios.
Zvec supports HNSW (the standard graph-based ANN algorithm) and DiskANN (Microsoft's disk-based index that enables billion-scale search from commodity hardware with limited RAM). DiskANN support was added in the v0.4.x cycle.
import zvec
# Define schema
schema = zvec.CollectionSchema(
name="docs",
vectors=zvec.VectorSchema("embedding", zvec.DataType.VECTOR_FP32, 1536),
)
# Open (creates if not exists)
collection = zvec.create_and_open(path="./my_collection", schema=schema)
# Insert
collection.insert([
zvec.Doc(id="doc_1", vectors={"embedding": embedding_1}, fields={"category": "tech"}),
zvec.Doc(id="doc_2", vectors={"embedding": embedding_2}, fields={"category": "finance"}),
])
# Query with filter
results = collection.query(
zvec.VectorQuery("embedding", vector=query_embedding),
topk=10,
filters={"category": "tech"},
)
print(results) # [{'id': 'doc_1', 'score': 0.94, ...}, ...]
Node.js is also supported via npm install @zvec/zvec — the API mirrors the Python SDK.
PO and PL) for tuning I/O prefetch depth during ANN search-Werror across all CI platformsarm64-v8a) and iOS (arm64) — no manual native compilation required| Zvec | FAISS | Chroma | Qdrant | |
|---|---|---|---|---|
| In-process | Yes | Yes | Optional | No |
| Persistence (WAL) | Yes | No | Yes | Yes |
| Hybrid search | Yes | No | Limited | Yes |
| Sparse vectors | Yes | No | No | Yes |
| Mobile support | Yes (Flutter) | No | No | No |
| Multi-process reads | Yes | No | No | N/A |
| DiskANN | Yes | No | No | No |
| Setup required | None | None | Server optional | Server required |
For a deeper dive into where vector search fits vs. keyword retrieval in code contexts, see RAG vs Agentic RAG: why search beats embeddings for code retrieval.
Alibaba benchmarks Zvec against billion-scale datasets. The headline claims: searches across billions of vectors in milliseconds with the HNSW index, and larger-scale recall with DiskANN. The project publishes full benchmark methodology, configurations, and results at their docs site.
For comparison, Google's own vector compression work (TurboVec/TurboQuant) recently demonstrated compressing 10M vectors from 31GB to 4GB — a different angle on the same infrastructure problem of making vector search practical at scale. See our coverage of Google TurboVec and TurboQuant for how the tradeoffs compare.
Good fit:
Less ideal:
# Python
pip install zvec
# Node.js
npm install @zvec/zvec
Source is at github.com/alibaba/zvec under Apache 2.0. The project has 27 contributors, 8 releases, and active CI across Linux, macOS, Windows, Android, and iOS.
Zvec is the kind of infrastructure release that quietly becomes load-bearing in a lot of projects. In-process vector search with WAL durability, hybrid filtering, and mobile support fills a real gap — and the Alibaba production provenance means you're not betting on an untested project.