Dataframe API v2 #0: Chunk support for dedupe-latest semantics (rerun…

…-io#7558) Implements support for the new `dedupe-latest` ™️ semantics on `Chunk`. This is one of the fundamental primitives required for the new upcoming dataframe APIs. This requires the use of the `take` Arrow kernel. Unfortunately I made the mistake of testing that new kernel, which revealed that it is allocating a lot of data when it shouldn't, so I'll have to fix it at some point in a future PR. * Part of rerun-io#7495 --------- Co-authored-by: Jeremy Leibs <[email protected]>
TeoNikolov · Oct 2, 2024 · e5ae198 · e5ae198
1 parent 45add8d
commit e5ae198
Show file tree

Hide file tree

Showing 4 changed files with 592 additions and 3 deletions.
diff --git a/crates/store/re_chunk/Cargo.toml b/crates/store/re_chunk/Cargo.toml
@@ -52,6 +52,7 @@ anyhow.workspace = true
 arrow2 = { workspace = true, features = [
   "compute_concatenate",
   "compute_filter",
+  "compute_take",
 ] }
 bytemuck.workspace = true
 document-features.workspace = true