Tantivy 0.20: Schemaless Search Engine Library with New Features

Tantivy, a high-performance full-text search engine library written in Rust, has released its latest version, Tantivy 0.20. This release comes packed with new features, including schemaless columnar storage, faster and more aggregations, performance improvements, phrase prefix queries, percentiles, and more.

Tantivy is inspired by Apache Lucene and serves as a foundation to build a search engine. It is used to build distributed search engines like Quickwit. The latest release of Tantivy focuses on schemaless data, without compromising on indexing performance, query performance, or compression.

Schemaless Fastfields

The new version of Tantivy introduces schemaless columnar storage, which is handled via the tantivy-columnar crate. This feature is a significant step towards full support for schemaless data. It offers unified storage for typed and untyped fields, support for fields with mixed types, a sparse codec for optional values, and cardinality detection. With this feature, there is no need to specify the cardinality of a field upfront anymore.

Aggregations

Aggregations are becoming an essential part of Tantivy, and the latest release has made them even better. Tantivy 0.20 introduces new aggregation types, including date_histogram, percentiles, count, min, max, and sum. It also supports u64, i64, and f64 fields in term aggregation, lower memory consumption for aggregations, and setting memory limits for aggregation.

Phrase Prefix Queries

Tantivy 0.20 also introduces phrase prefix queries, which allow developers to search for phrases that begin with a specific prefix. This feature is useful for autocomplete or search-as-you-type functionality.

Performance Improvements

The latest release of Tantivy also comes with performance improvements, making it faster and more efficient. The library is now more optimized for faster and more efficient search queries, making it an even more powerful tool for developers.

Conclusion

Tantivy 0.20 is a significant release that introduces several new features and improvements to the high-performance full-text search engine library. With schemaless columnar storage, faster and more aggregations, phrase prefix queries, percentiles, and more, Tantivy is now even more powerful and efficient. Developers can use Tantivy to build search engines that are fast, efficient, and easy to use. Stay tuned for upcoming blog posts that will provide a deep dive into the new features of Tantivy 0.20.