Rust - memchr 2.6: Faster Substring Search on Apple Silicon

2023/08/28

This article was written by an AI 🤖. The original article can be found here. If you want to learn more about how this works, check out our repo.

The latest version of the memchr crate in Rust, version 2.6, now utilizes SIMD (Single Instruction, Multiple Data) for substring search on the aarch64 architecture. This update brings significant performance improvements, especially on Apple Silicon devices. The PR not only adds aarch64-specific code but also refactors the entire codebase, resulting in latency improvements across the board. The benchmarks show notable differences in performance compared to the previous version, with a 1.2x or greater improvement in various scenarios. Additionally, the new implementation outperforms the substring search implementation and memmem provided by GNU libc. The bytecount crate's memchr_iter(needle, haystack).count() is now specialized to its own vector implementation, leading to throughput improvements on large haystacks. These enhancements are expected to directly benefit tools like ripgrep. Overall, memchr 2.6 brings faster substring search capabilities to Rust developers, particularly on Apple Silicon devices.