v0.24.0

What's Changed

Breaking Changes 🛠

• feat!: update DataFusion to 45.0 and Arrow to 54.1 by @timsaucer in #3503

New Features 🎉

• feat: add support for explain analyze by @wkalt in #3484
• feat(java): support delete rows from the dataset by @yanghua in #3498
• feat: add support for empty structs to the 2.0 format by @westonpace in #3499
• feat: cache v3 index partitions in dataset session by @BubbleCal in #3467
• feat: add support for pickling fragment metadata by @westonpace in #3497
• feat: support add all null column as metadata-only operation via sql by @albertlockett in #3504
• feat: ConditionalPutCommitHandler for concurrency on S3, faster commit by @wjones127 in #3483
• feat: add gcp token-based auth support by @alex766 in #3511
• feat: emit a trace event when a significant user file is created or deleted by @westonpace in #3519

Bug Fixes 🐛

• fix: scalar quantization can't work with NaNs by @BubbleCal in #3476
• fix: ngram bench target not correct by @BubbleCal in #3490
• fix: bypass the arrow take for struct array by @BubbleCal in #3500
• fix: prevent despecialization of object store methods by @wjones127 in #3506
• fix: the IVF/PQ centroids/codebook is with wrong data type if training on GPU by @BubbleCal in #3502
• fix: pass down correct types when creating indices and items scheduler by @westonpace in #3520

Documentation 📚

• docs: fix typo in read_and_write.rst by @ascillitoe in #3479
• docs: add README.md for java module by @yanghua in #3302
• docs: include create scalar index and drop index to the top level of Python API doc by @eddyxu in #3509

Performance Improvements 🚀

• perf: parallelize ngram indexing by @BubbleCal in #3501
• perf: implement XTR for retrieving multivector by @BubbleCal in #3437

New Contributors

• @ascillitoe made their first contribution in #3479
• @lyang24 made their first contribution in #3453
• @alex766 made their first contribution in #3511
• @timsaucer made their first contribution in #3503

Full Changelog: v0.23.2...v0.24.0