Skip to content

Commit

Permalink
rename to suntan; map all the easy fields. hierarchical facets and da…
Browse files Browse the repository at this point in the history
…tes are left as todos
  • Loading branch information
mooreniemi committed Jun 13, 2021
1 parent 66ba517 commit c39f873
Show file tree
Hide file tree
Showing 16 changed files with 74 additions and 320 deletions.
22 changes: 11 additions & 11 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 2 additions & 2 deletions Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[package]
name = "lucky"
version = "0.1.1"
name = "suntan"
version = "0.1.2"
authors = ["alex <[email protected]>"]
edition = "2018"

Expand Down
31 changes: 17 additions & 14 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,28 +1,29 @@
```
.-.-.
__ \ / __
( ` \.'.' )
(__.', \ .__)
/ \`===,
`-^-'
niki
.
\ | /
'-.;;;.-'
-==;;;;;==-
.-';;;'-.
/ | \
jgs '
```
# lucky
# suntan

This is just a proof-of-concept tool to dump Elasticsearch Lucene shards into Tantivy. There's also a couple `examples` of calling Lucene through Rust for querying. You provide input, output, and the Tantivy output schema and the tool dumps into it. Your Tantivy schema must be just like or a subset of the Elasticsearch schema. Not all types are supported yet.

## cli

If run without arguments the cli will attempt to use test resources.

```
./target/debug/lucky --help
lucky 0.1.1
./target/debug/suntan --help
suntan 0.1.2
Alex MN. <[email protected]>
This is a tool for dumping Elasticsearch Lucene shards into Tantivy indices. Elasticsearch stores
fields in a particular way which is why it's not "just" Lucene
USAGE:
lucky [OPTIONS]
suntan [OPTIONS]
FLAGS:
-h, --help Prints help information
Expand All @@ -32,7 +33,7 @@ OPTIONS:
-i, --input <input> The location of the Elasticsearch Lucene index [default:
tests/resources/es-idx]
-o, --output <output> The location of the Tantivy output index [default:
/tmp/lucky/tantivy-idx]
/tmp/suntan/tantivy-idx]
-s, --schema-path <schema-path> The location of the Tantivy schema [default:
tests/resources/tantivy-schema.json]
-t, --test-query <test-query> To test that the docs still match, this is sent in as test
Expand Down Expand Up @@ -78,5 +79,7 @@ rsync -r /var/lib/elasticsearch/nodes/0/indices/TvG2djXSQgqg4PWZSrv2wQ/0/index/

## high level todos

- More types in the schema to schema mapping to make this a fully parameterized CLI tool. Right now only text is supported.
- Properly package the jar with the bin.
- `HierarchicalFacet` and `DateTime` support in the schema mapping.
- Remapping field names on export.
- java_wrapper should probably be made into a git submodule. Right now I `rsync` from another repo.
4 changes: 2 additions & 2 deletions examples/basic_calls.rs
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ use serde_json::Value;
use std::convert::TryFrom;

fn main() {
let entry = ClasspathEntry::new("./java_wrapper/target/lucky-java-1.0-SNAPSHOT.jar");
let entry = ClasspathEntry::new("./java_wrapper/target/suntan-1.0-SNAPSHOT.jar");
let jvm: Jvm = JvmBuilder::new()
.classpath_entry(entry)
.build()
Expand All @@ -16,7 +16,7 @@ fn main() {
// this example shard was generated with some faker data in Latin
let instantiation_args = vec![InvocationArg::try_from("tests/resources/").unwrap()];
let instance = jvm
.create_instance("org.lucky.ShardReader", instantiation_args.as_ref())
.create_instance("org.suntan.ShardReader", instantiation_args.as_ref())
.unwrap();

let field_and_value = vec![
Expand Down
6 changes: 3 additions & 3 deletions examples/curse.rs
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ fn main() {
// Its size expand automatically with the content.
siv.add_layer(
Dialog::new()
.title("lucky")
.title("suntan")
.padding_lrtb(1, 1, 1, 0)
.content(TextArea::new().with_name("text"))
.button("quit", Cursive::quit),
Expand Down Expand Up @@ -57,13 +57,13 @@ fn find(siv: &mut Cursive, text: &str) {
// First, remove the find popup
siv.pop_layer();

let entry = ClasspathEntry::new("./java_wrapper/target/lucky-java-1.0-SNAPSHOT.jar");
let entry = ClasspathEntry::new("./java_wrapper/target/suntan-1.0-SNAPSHOT.jar");
let jvm: Jvm = JvmBuilder::new().classpath_entry(entry).build().unwrap();

// this example shard was generated with some faker data in Latin
let instantiation_args = vec![InvocationArg::try_from("tests/resources/").unwrap()];
let shard_reader = jvm
.create_instance("org.lucky.ShardReader", instantiation_args.as_ref())
.create_instance("org.suntan.ShardReader", instantiation_args.as_ref())
.unwrap();

let res = siv.call_on_name("text", |v: &mut TextArea| {
Expand Down
3 changes: 0 additions & 3 deletions java_wrapper/.idea/.gitignore

This file was deleted.

5 changes: 0 additions & 5 deletions java_wrapper/.idea/codeStyles/codeStyleConfig.xml

This file was deleted.

16 changes: 0 additions & 16 deletions java_wrapper/.idea/compiler.xml

This file was deleted.

20 changes: 0 additions & 20 deletions java_wrapper/.idea/jarRepositories.xml

This file was deleted.

98 changes: 0 additions & 98 deletions java_wrapper/.idea/misc.xml

This file was deleted.

6 changes: 0 additions & 6 deletions java_wrapper/.idea/sbt.xml

This file was deleted.

Loading

0 comments on commit c39f873

Please sign in to comment.