TODO

(1) Building disk index by merging shards currently needs a lot of working disk space. For each shard, we need to store the index; then we merge them all using merge_shards program to get a merged memory index, and then we create a disk index using this merged memory index. We should have a direct program which adds a new shard to the final disk index (and also perhaps parallelize this code to speed it up).

(2) Add the uint8 distance AVX2.

(3) Do a thorough walk through of code to detect possible crashes due to corner cases

(4) Connect up index if disconnected by just linking start points or rebuilding an index on excluded points

(5) Eliminate parameters all together? It is giving warnings in linux compiler.

(6) Add indexing capability with cosine similarity