Pinterest's Manas Platform: Transforming Search with Quantization

Pinterest's engineering team has recently made strides in their distributed search platform, Manas, by integrating quantization techniques and utilizing SSD-based serving through SPANN. This development tackles the challenges of handling tens of billions of embeddings that drive core discovery experiences like Home Feed, Search, Related Pins, Ads, and Notifications. As their corpus scaled to billions of items, traditional vector search algorithms like HNSW struggled with cost, hardware allocation, and infrastructure flexibility issues.

Why Quantization Matters for Manas

To address these scaling challenges and cut down on memory usage, Pinterest implemented Scalar Quantization (SQ) and Product Quantization (PQ) on a 100-million-embedding GraphSage dataset. PQ compresses floating-point vector representations into compact byte codes, achieving a 74% reduction in HNSW indices and a 93% reduction in Inverted File (IVF) indices, while maintaining a recall rate of 70–80%. On the flip side, SQ compresses vector components into lower-bit integers, reducing HNSW indices by 59% and IVF indices by 75%, with over 90% recall consistency across workloads.

Product Quantization (PQ) works by dividing vectors into smaller subvectors and quantizing each subvector into a compact representation. This approach drastically reduces the size of the indices but comes at the cost of some recall accuracy. Scalar Quantization (SQ), however, reduces the precision of each component of the vectors by converting them into lower-bit integers, maintaining higher recall rates but requiring more memory than PQ.

Performance Trade-offs and Benchmarks

The offline benchmarks brought to light distinct trade-offs across configurations and index sizes. For example, the baseline HNSW index was at 121 GB with a Recall@100 of 93.72% at 302.5 QPS. By applying PQ, the index size shrank to 32 GB, though the Recall@100 dropped to 77.25% at 276.4 QPS. Similarly, integrating SQ resulted in a 50 GB index size with a Recall@100 of 92.92% at 305.2 QPS.

The baseline IVF index stood at 97 GB with a Recall@100 of 91.69% at 1659.8 QPS. Using PQ, they compressed it to 6.8 GB, albeit with a reduced Recall@100 of 76.00% at 1747.9 QPS. However, IVF plus SQ produced a 25 GB index size with a superior Recall@100 of 95.71% at 1588.8 QPS. This highlights the trade-offs between memory usage, recall accuracy, and query performance.

The choice between PQ and SQ hinges on the specific needs of the application—whether minimizing memory usage or maximizing recall accuracy is the priority. In practice, PQ might be selected for applications where resource savings are crucial, while SQ is preferred when high recall is more important.

Implementing SPANN and Its Impact

To further refine their search infrastructure, Pinterest evaluated DiskANN and SPANN for SSD serving. They discovered that SPANN, paired with PQ, achieved three times the QPS of DiskANN, with only a modest 5% recall drop and one-third of the latency. This was a critical finding, enabling Pinterest to slash RAM costs by shifting index storage to high-throughput SSDs. By maintaining a small, fast centroid index in memory, SPANN efficiently located relevant partitions while housing large posting lists on SSDs, optimizing IOPS and ensuring efficient search operations.

SPANN's architecture uses a hybrid approach where a small part of the index, the centroid index, is kept in memory to facilitate quick access to relevant data. This design ensures that the large posting lists are stored on SSDs, significantly cutting down the memory footprint while maintaining high query performance.

In a recommendation evaluation indexing over 5 billion embeddings, SPANN saved over 40% of CPU time for production queries compared to a full in-memory HNSW approach. This showcases the effectiveness of combining SPANN with quantization techniques in scaling search capabilities while reducing operational costs.

Transitioning to Late Interaction Models

Beyond optimizing search with quantization, Pinterest is moving past the expressiveness limits of single-vector two-tower models by adopting multi-vector Late Interaction models like ColBERT. This approach uses Sum of MaxSim scoring to refine relevance matching across tokens. Implementing this required updating the query parser to handle multi-token queries and execute simultaneous ANN searches across indices.

The shift to multi-vector models allows Pinterest to attain finer-grained relevance matching, essential as they tackle increasingly complex and diverse search queries. A successful pilot launch is currently underway with an internal client team to test these advanced multi-embedding query capabilities in live production environments.

Practical Steps for Implementing Quantization and SPANN

For engineering teams considering similar optimizations, here are practical steps to implement quantization and SPANN in a distributed search system:

• Evaluate Data Characteristics: Assess the nature and size of your dataset to determine whether PQ or SQ is more suitable based on compression needs and recall goals.
• Benchmark Configurations: Conduct thorough offline benchmarks to identify trade-offs between index size, recall accuracy, and query performance. This will inform your decision on the appropriate quantization technique.
• Integrate SIMD Intrinsics: Implement Linear Scaling SQ using SIMD intrinsics to mitigate CPU bottlenecks during the decoding step, reducing query compute resources by 10–15%.
• Adopt SPANN for SSD Storage: Consider shifting index storage to SSDs using SPANN to reduce RAM costs. Ensure a fast centroid index is kept in memory for efficient search partitioning.
• Transition to Late Interaction Models: Update query parsers to handle multi-token queries and implement multi-vector models like ColBERT for improved relevance matching.

Limitations and Considerations

While quantization and SPANN offer significant benefits, there are inherent limitations and considerations:

• Recall Trade-offs: Compressed indices often lead to reduced recall accuracy. Engineering teams must weigh the importance of recall versus resource savings.
• Complexity of Implementation: Switching to multi-vector models and integrating SIMD intrinsics can increase system complexity and demand specialized expertise.
• Hardware Dependence: The effectiveness of SPANN and quantization techniques can vary with the underlying hardware, particularly the throughput of SSDs and the performance of SIMD instructions.

In Implementing Error Tracking in React Frontends with Node.js, we explored the importance of optimizing resources in web applications. Similarly, the advancements in Pinterest's Manas platform showcase the critical balance between performance optimization and maintaining system scalability. These innovations illustrate the ongoing evolution of distributed systems and search technologies.

For more insights into optimizing search and distributed systems, you might find Agoda's migration to DragonflyDB relevant, as it also tackled scaling and performance challenges in database systems.