RWKV is an architecture combining the parallelizable training of transformers with the linear-time inference of RNNs, using a novel token-shift and time-mix mechanism. It can be trained like a transformer using matrix operations but runs like an RNN at inference, offering constant memory per token — making it attractive for deployment on resource-constrained devices.