explainx.ai350K+ learn from us
TrendingAI News TodayExplainx for Business
Pricing
explainx.ai

Upskill in AI — 16 free pathways, live workshops & bootcamps, and 50+ courses from practitioners. Plus the skills, tools, and MCP servers to practice on.

follow us

follow on google

Add explainx.ai as a preferred source

corporate training

support@explainx.ai

get started

Find your pathTake Free Evaluation

community

Join the community

learn

mind: share how you thinkpathways — start freeworkshopsbootcampscoursescompare Explainxcertificationsmock testsexplainx universitycorporate traininglearn skills & mcp

discover

skillsmcp serversexplainx mcpestacktoolsmdx readeragentsllmsdesignsdictionarypeopleagi trackerfelony benchranks

company

aboutvisionmissionteaminstructorsteach on explainxpartnershipscommunityhackathonscareers

content

daily AI newsstate of AI — live resultsblogreleasespromptsgeneratorsresource libraryfor LLMsexplainx.ai kids

solutions

all solutionsdeveloper upskillingmarketing upskillingproduct manager upskillingleadership upskilling

newsletter · weekly

Get AI news, tools, and insights in your inbox.

supportcontactprivacytermsdata rightshow we create contentsubmission guidelines

© 2026 AISOLO Technologies Pvt Ltd

explainx.ai

On this page

  • Quick reference
  • Why compression measures language models
  • How tufazip works
  • What the benchmark page confirms
  • What is verified and what is only claimed
  • Why it costs 7.5 days
  • How tufazip compares with other neural compressors
  • How to try it
  • What people are asking
  • Related reading
← Back to blog

explainx / blog

tufazip: Tufa Labs Squeezes 1 GB of Wikipedia into 105.39 MB With a Transformer

AI Research, Open Source, Compression, Transformers, Tufa Labs

Part of AI Research

Tufa Labs' open-source tufazip compresses enwik9 to 105,388,353 bytes using a 163M-parameter transformer alone. How it works, what it costs, and why it matters.

Nov 20, 2026·9 min read·Yash Thakker
add explainx.ai
go deep
tufazip: Tufa Labs Squeezes 1 GB of Wikipedia into 105.39 MB With a Transformer

Tufa Labs says it shrank 1 GB of Wikipedia to 105.39 MB, losslessly, with a single neural network. The lab's announcement on X calls tufazip "the best entry on the Large Text Compression Benchmark using a neural network as its sole predictor," and adds that everything above it "mixes a transformer with hand-built models."

The size figure holds up. Matt Mahoney's Large Text Compression Benchmark lists tufazip v2 at 105,388,353 bytes total. The cost is the other half of the story: about 7.5 days of compute on two Nvidia B200 GPUs for each direction. This post explains how it works, what is verified, and why a text compressor is a useful lens on language models.

Quick reference

table · 2 cols
QuestionAnswer
What is it?A lossless text compressor driven by a transformer's next-token predictions
Resultenwik9 (10^9 bytes of Wikipedia) to 105,388,353 bytes, decoder and weights included
Bits per byteAbout 0.843 (the best online-only run reached 0.848)
Model12-layer decoder-only transformer, 163M parameters
Hardware and timeTwo Xeon Platinum 8570 CPUs, two B200 GPUs; about 7.5 days each way, per the benchmark page
LicenseMIT for tufazip 0.1.0, per the benchmark page
AuthorTommy He, Tufa Labs
CaveatTuned to one file; the author raises a Goodhart concern
Daily briefFrom Yash · 350K+ students taught

Catch up on AI

One AI story explained in plain English, plus one thing to try. Every day, free. Unsubscribe anytime.

Why compression measures language models

A compressor's job is to predict what comes next. Arithmetic coding turns a probability for the next symbol into a number of bits: the more confident and correct the prediction, the fewer bits needed. The ideal code length equals the summed cross-entropy of the model's predictions, which is the same quantity used as the training loss of a language model.

So a language model is a compressor, and a better one compresses more. That is why the Large Text Compression Benchmark exists. It ranks programs by the compressed size, including the decompressor, of enwik9, the first 10^9 bytes of an English Wikipedia XML dump. Counting the decoder stops a program from hiding the data inside its own weights.

For readers who want the vocabulary, we cover tokens, the context window and quantization in separate guides. The same ideas appear here: a model predicts tokens, a window limits what it sees, and quantization shrinks stored weights.

How tufazip works

The Tufa Labs write-up, dated October 14, 2026, describes the design.

The predictor. A 12-layer, decoder-only transformer with 163M parameters. It uses rotary position embeddings, SwiGLU MLPs and RMSNorm. Tokenization is reversible with a 4K word vocabulary. An arithmetic coder turns the predicted probabilities into the output stream.

Online learning. The encoder and decoder start from identical weights and run the same loop: predict, encode or decode, update. Because both sides learn the same way from the same decoded text, the learned weights never need to be transmitted. The cost is that updates must be fully deterministic, or the decoder will drift and fail to reproduce the file.

Replay. Periodically the model retrains on a window of text it has already coded. The best online run used a 35M-token window and 0.25 replay dropout with a 192-token context. Growing the window from 15M to 35M tokens cut the payload from 107.22 to 105.92 MB, a 1.21% gain, for about 40% more estimated FLOPs.

An offline prior. The final model starts from a compact, offline-trained prior. The page says a 34M-parameter offline model was quantized to 4.33 MB. The "Cycle2" layout trains two shared blocks repeated across twelve steps, stores each once, and then copies them into twelve independent layers that specialize during online learning.

The trade is explicit in the page's results table:

table · 4 cols
ConfigurationPayload (MB)Decoder and weights (MB)Total (MB)
Online v1105.9240.055105.979
L6 (six-layer prior)99.17310.945110.118
C2 (final)100.9584.430105.388

The C2 prior saved 4.97 MB of payload, cost 4.38 MB to store, and netted 0.59 MB, a 0.56% gain over the online v1 run. The stronger L6 prior predicted better but cost too much to ship.

What the benchmark page confirms

The benchmark's own entry for tufazip, as of this writing, adds independent detail:

  • tufazip 0.1.0, submitted September 7, 2026, totals 105,979,069 bytes.
  • tufazip v2, submitted September 30, 2026, totals 105,388,353 bytes, using transformer weights trained offline through cycle 2 and quantized. Compression time does not include offline training time.
  • Both entries use code in C, Python and CUDA, with preprocessor code from Fabrice Bellard, and are tagged as transformer-based.
  • Compression or decompression takes 7.5 days on 2 Intel Xeon Platinum 8570 CPUs with 28 GiB of memory and 2 Nvidia B200 GPUs with 43 GiB of memory.
  • The benchmark lists tufazip 0.1.0 as free, open source (MIT); it states no license for the v2 entry.
  • The v2 notes give about 7.3 days to compress and 7.5 days to decompress, and the main-table timing columns for v2 repeat the 0.1.0 values, so treat "about 7.5 days" as approximate.

One wrinkle worth flagging: the X announcement is from November 19, but the benchmark entries date from September, and the write-up from October 14. The November post is a public push of an existing result, not a new submission. The write-up says the entry ranked #2 at submission. On the benchmark's main table, sorted by total size, tufazip v2 now shows at 16th, behind cmix-family and other hybrid entries, so the "best neural-only" claim and the overall rank are different things.

What is verified and what is only claimed

table · 2 cols
ClaimStatus
enwik9 compresses to 105,388,353 bytesMatches both the write-up and the benchmark table
Best neural-only entryTufa Labs' claim; consistent with the benchmark's own algorithm tags but we did not audit every entry
Everything above it mixes a transformer with hand-built modelsTufa Labs' claim
7.5 days per direction on two B200 GPUsStated on the benchmark page
Beats NNCP v3.2 (107.26 MB)Stated in the write-up
Hardware and wall-clock in the write-upNot reported there; the benchmark page supplies them

The write-up is open about limits: results apply to one fixed file, so the score may not reflect general ability, and the author raises a Goodhart concern about optimizing this proxy. Larger models did not help in the tested settings: on enwik9 a 267M model was slightly worse than a 188M one, and on enwik8 gains stopped around 77M parameters.

Why it costs 7.5 days

Decompression mirrors compression. The decoder repeats the same online learning and replay, so it pays the same compute. A normal compressor such as gzip or zstd decompresses in seconds because decoding is a lookup. Here decoding means retraining a transformer as it goes. That is the nature of an online-learning compressor, and it is why this is a benchmark result and not a tool.

If you want a practical angle on compression for LLM workflows, our explainer on gzip and faster first token in Claude Code covers a case where simple compression helps real latency, and the KV cache guide covers where inference memory goes.

How tufazip compares with other neural compressors

The write-up compares only with NNCP v3.2 from 2023, at 107.26 MB. Fabrice Bellard's NNCP and his ts_zip are the best-known neural compressors, and the benchmark page names transformer and LSTM models among the leading approaches, along with context mixing. We found no ts_zip figure on the Tufa Labs page, so we do not compare against it here.

The other neural-compression stories we have covered point the same way: models shrink things when they are allowed to do heavy work. See PrismML's near-lossless compression of a 27B model for compressing model weights instead of text.

How to try it

The repository is Tufalabs/tufazip. The v2 release notes call for Linux x86-64, exactly two Nvidia B200 GPUs and a pinned Python 3.12.3 environment. Treat that as the intended setup. There is no sensible way to run a full enwik9 pass on a laptop.

What you can do without the hardware:

  1. Read the write-up. The replay and prior-cost trade-offs are a good lesson in why parameter count is not the only lever.
  2. Check the benchmark table. It lists every entry with its sizes and timings.
  3. Try small files. Compressing a small corpus will show the idea, though the setup is tuned to enwik9.
  4. Use the idea elsewhere. Online adaptation plus deterministic updates is a pattern for any system where both ends must stay in sync.

What people are asking

Is this the same as ChatGPT compressing text?

It uses the same family of model, a transformer predicting the next token. The difference is that the predictions drive an arithmetic coder, and everything must be exactly reproducible.

Could this make my downloads smaller?

Not today. At 7.5 days per gigabyte it is a research result.

Why Wikipedia?

enwik9 is the fixed test file of the benchmark, so results are comparable across years.

Does a better compressor mean a smarter model?

Better prediction on text means a better model of that text. The author cautions that optimizing one file is a proxy and can be gamed, so it is a signal, not a verdict.

Related reading

  • What is AI model quantization: a complete guide
  • What are LLM tokens
  • LLM context windows explained
  • What is gzip and why it speeds up Claude Code
  • What is the KV cache: LLM inference explained
  • PrismML Bonsai 2: near-lossless compression of a 27B model

Figures are accurate as of November 20, 2026. The benchmark table and the repository may change as new entries and releases arrive.

Spotted something out of date? Let us know.
Daily briefFrom Yash · 350K+ students taught

Catch up on AI

One AI story explained in plain English, plus one thing to try. Every day, free. Unsubscribe anytime.

  • ✓ Free
  • ✓ One email a day, no spam
  • ✓ Unsubscribe anytime
Yash Thakker

Written by

Yash Thakker

Yash is an AI expert with over 300K learners. Join his workshops →

View Yash Thakker in People in AI →

Related posts

Nov 18, 2026

Multi-Branch Transformer (MBT): Parallel Blocks, Joint-KV, and 51% Faster Decoding

Researchers from UIUC, Cornell, Rice, and Michigan introduce Multi-Branch Transformers that run parallel block stacks, fuse them, and share KV caches. On MoE models up to 2.1B parameters, Balanced MBTs beat matched sequential Transformers and decode up to 51.4% faster on an H200.

Nov 10, 2026

Hugging Face Decision Models: Jev-Style Routing Without JSON Spam

Large language models are great at generating paragraphs and terrible at guaranteeing schema-shaped routing decisions. Hugging Face formalized decision models with a shared API across Transformers and llama.cpp so agents can pick billing vs support without decoding JSON.

Nov 10, 2026

WorldFoundry: Open Infrastructure for World Models and Benchmarks

Researchers released WorldFoundry on arXiv 2611.10771 and GitHub: one stack for world-model inference and evidence-aware evaluation across video, 3D/4D, and embodied policies. This guide explains what ships in the repo and how builders run a first job.