TurboQuant: Efficient Vector Search with Rust and Python Integration

TurboQuant is a cutting-edge vector search tool implemented in Rust with Python bindings via PyO3, designed to efficiently compress high-dimensional vectors into 2-4 bits per coordinate without requiring training. This unofficial implementation of Google's TurboQuant offers a significant advantage by eliminating the need for training, thus achieving zero indexing time. By employing a unique method of random rotation and Lloyd-Max scalar quantization, TurboQuant achieves near-optimal distortion rates, compressing vectors significantly while maintaining high recall rates.

## Performance Comparison

TurboQuant outperforms traditional methods like FAISS in various scenarios. On ARM architecture, it matches or surpasses FAISS in speed without a training step, and at 4-bit compression, it achieves higher recall. On x86 systems, although slightly slower, TurboQuant maintains a competitive edge in recall accuracy. The speed differences are attributed to TurboQuant's rotation step and AVX2 code generation.

## Compression and Efficiency

TurboQuant excels in compressing vector data, achieving up to 15.8x compression compared to FP32. This is achieved through a process that normalizes vectors, applies a random rotation, and uses precomputed quantization buckets to minimize error. This approach allows TurboQuant to handle new data dynamically without needing to rebuild the index, unlike traditional methods that require offline training.

## Technical Architecture

The tool is structured as a Cargo workspace with two main components: a Rust crate containing the core logic and a Python wrapper for accessibility. The search pipeline utilizes advanced SIMD intrinsics for efficient scoring, with different optimizations for ARM and x86 architectures. TurboQuant's architecture allows for seamless integration and high-performance vector search across platforms.

## Practical Use

TurboQuant is particularly suited for applications requiring fast and efficient vector search, such as large-scale data retrieval and machine learning tasks. Its ability to handle new data online makes it ideal for dynamic environments where data is continuously updated.