Merge of hdtCat

[D063520]

This branch contains an implementation of hdtCat, an algorithm and command line tool to merge 2 hdt files without decompressing them. This especially allows to merge HDT files and serialize big RDF file to HDT with low memory footprint. On a 16Gb machine we were able to generate an HDT file with 5 billion triples. The code is not working under Windows. This issue is known to us.

[dinikolop]

Noticed that when two identical files are merged with hdtCat then the triples are duplicated. This leads to a larger output file where hdtSearch returns the double number of results in comparison to the original file.

[D063520]

Hi,
thank you very much for noting this!!!!! and particularly for giving a look to the pull request. I lost a hash and equal function while refactoring the code. Now it works. I also added a new test checking this.

Salut
D063520

[mielvds]

Guys, love this work. Could you list the major changes to the original code/API? are there any breaking changes?

[D063520]

There are basically no breaking changes, only things beside .... enjoy

[mielvds]

alright, cool, merged! Would you mind extending the README to document the new feature a bit?

[D063520]

cool thank you! Somehow there where two README files. I changed that ..... Also the description of hdtCat is in the hdt-java-cli README, that I already updated!