test: Introduce datafusion engine for executing sqllogictest. #895
Conversation
Signed-off-by: liurenjie1024 <[email protected]>
There was a problem hiding this comment.
Generally make sense to me, i'll defer to someone with more experience in rust.
This follows the datafusion/datafusion/sqllogictest tests suite.
Do you think its worth mentioning that conversion.rs and normalize.rs are both copied over from datafusion/datafusion/sqllogictest?
Is it possible to take datafusion_sqllogictest as a dependency instead of copying over the code?
I've synced with @Fokko and I'll add some notice in LICENSE as what we did in pyiceberg.
Currently impossible because it's not published as crates. |
There was a problem hiding this comment.
Is it possible to take datafusion_sqllogictest as a dependency instead of copying over the code?
+1 on this. It seems we don't need any customization (except for ctx.register_catalog) on the test driver, so it's not economical to maintain a copy. cc @alamb for opinions. We could also consider use a git dependency for now.
I have concerns as using a git dependency, with this approach datafusion community is not treating it as a library. I'm quite interested if datafusion community could consider exposing it as a crate. I've raised discussion, welcome to discuss there! |
|
I published datafusion-sqllogictest to crates.io (details here apache/datafusion#14229 (comment)) . Let us know if you have any problems! |
It looks great! I'll refator the pr and have a try, thanks! |
|
Thank you @liurenjie1024 for working on this. I'm going to convert this PR to draft instead. |
Second part of #581.
Introducing datafusion engine for executing sqllogictest.