Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(api-gateway): Async native query results transformations #8961

Open
wants to merge 127 commits into
base: master
Choose a base branch
from

Conversation

KSDaemon
Copy link
Member

@KSDaemon KSDaemon commented Nov 15, 2024

This PR brings some performance improvements to Cube:

  • Move cubestore websocket response parsing to native (rust)
    • Now Cubestore results processing in some cases may be faster by up to 50%
  • All query results are now transformed on the native side
  • All query results are now transformed asynchronously, thus offloading the main node.js event loop
  • In case of queries from SQL API, the driver's query results (from cubestore/native or js) are transparently delivered back to native invoker.

Copy link

vercel bot commented Nov 15, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

8 Skipped Deployments
Name Status Preview Comments Updated (UTC)
examples-angular-dashboard ⬜️ Ignored (Inspect) Visit Preview Nov 29, 2024 6:48pm
examples-react-d3 ⬜️ Ignored (Inspect) Visit Preview Nov 29, 2024 6:48pm
examples-react-dashboard ⬜️ Ignored (Inspect) Visit Preview Nov 29, 2024 6:48pm
examples-react-data-table ⬜️ Ignored (Inspect) Visit Preview Nov 29, 2024 6:48pm
examples-react-highcharts ⬜️ Ignored (Inspect) Visit Preview Nov 29, 2024 6:48pm
examples-react-material-ui ⬜️ Ignored (Inspect) Visit Preview Nov 29, 2024 6:48pm
examples-react-pivot-table ⬜️ Ignored (Inspect) Visit Preview Nov 29, 2024 6:48pm
examples-vue-query-builder ⬜️ Ignored (Inspect) Visit Preview Nov 29, 2024 6:48pm

Copy link

codecov bot commented Nov 15, 2024

Codecov Report

Attention: Patch coverage is 91.25000% with 7 lines in your changes missing coverage. Please review.

Project coverage is 82.86%. Comparing base (e55beb1) to head (e878b68).
Report is 5 commits behind head on master.

Files with missing lines Patch % Lines
rust/cubesql/cubesql/src/compile/engine/df/scan.rs 89.39% 7 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #8961      +/-   ##
==========================================
+ Coverage   82.85%   82.86%   +0.01%     
==========================================
  Files         222      222              
  Lines       78828    78821       -7     
==========================================
+ Hits        65309    65317       +8     
+ Misses      13519    13504      -15     
Flag Coverage Δ
cubesql 82.86% <91.25%> (+0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@KSDaemon KSDaemon force-pushed the feat/cubestore-driver-rust-parsing branch from 3f3e17d to 3ef1192 Compare November 20, 2024 17:12
@KSDaemon KSDaemon force-pushed the feat/cubestore-driver-rust-parsing branch 5 times, most recently from 9c18870 to bec29e9 Compare November 29, 2024 18:04
@KSDaemon KSDaemon force-pushed the feat/cubestore-driver-rust-parsing branch 2 times, most recently from e3179ce to be5e22c Compare December 9, 2024 18:09
@KSDaemon KSDaemon force-pushed the feat/cubestore-driver-rust-parsing branch 11 times, most recently from 52ce278 to 5f5cc6c Compare December 17, 2024 15:41
@KSDaemon KSDaemon changed the title feat(cubestore-driver): parse cubestore response in rust feat(api-gateway): async native query results transformations Dec 17, 2024
@KSDaemon KSDaemon force-pushed the feat/cubestore-driver-rust-parsing branch 7 times, most recently from 1a39a49 to f5425ff Compare December 18, 2024 15:58
@KSDaemon KSDaemon force-pushed the feat/cubestore-driver-rust-parsing branch from ef94071 to 2947f5f Compare January 13, 2025 22:29
@KSDaemon KSDaemon marked this pull request as ready for review January 14, 2025 10:32
@KSDaemon KSDaemon requested review from a team as code owners January 14, 2025 10:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant