Hyena is an architecture that replaces attention with long convolutions and element-wise gating, achieving sub-quadratic complexity while maintaining competitive quality on language and genomics tasks. It uses implicitly parameterized convolution filters that can model long-range dependencies without the quadratic cost of standard attention.