Skip to content

Commit

Permalink
update docs
Browse files Browse the repository at this point in the history
  • Loading branch information
diptanu committed Jul 5, 2024
1 parent 1b82fbb commit f520e33
Show file tree
Hide file tree
Showing 3 changed files with 11 additions and 2 deletions.
2 changes: 1 addition & 1 deletion Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

10 changes: 9 additions & 1 deletion docs/docs/apis/develop_extractors.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
# Developing a new Extractor

When you are working on a use case for which we might not have an extractor already you can write a new extractor, and deploy in your cluster and add new capabilities to Indexify.
Custom Extractors allows you to plug in any data processing code you want into an Indexify pipeline. Examples - Custom chunking algorithms for your data, or using your own model in a pipeline.


## Concepts

Expand Down Expand Up @@ -91,6 +92,13 @@ indexify-extractor describe custom_extractor:MyExtractor
indexify-extractor run-local custom_extractor:MyExtractor --text "hello world"
```

#### Install the extractor locally
```shell
indexify-extractor install-local custom_extractor:MyExtractor
```

This makes the extractor available locally to Indexify server

#### Join with Control Plane
You can join the extractor with the Indexfy server for it to receive streams of content to extract from
```shell
Expand Down
1 change: 1 addition & 0 deletions docs/mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ nav:
- Architecture: 'architecture.md'
- Comparisons: 'comparisons.md'
- Examples: 'examples_index.md'
- Custom Extractors: 'apis/develop_extractors.md'
- CLI and UI:
- User Interface: 'ui.md'
- Extractor CLI: 'extractor_cli.md'
Expand Down

0 comments on commit f520e33

Please sign in to comment.