SOLR-17537: Manage ZK Compression through Curator

[HoustonPutman]

https://issues.apache.org/jira/browse/SOLR-17537

Unfortunately, Curator currently requires that users explicitly use (.compress()) in order to enable compression for a zNode. We need to add a feature in Curator that allows for a "CompressionFilter" or something like that.

We will need to wait on a Curator release with that in it in order to merge this.

[dsmiley]

Could we call compress() universally and then our compressor chooses to only compress based on our criteria (state.json)? Just an idea off the top of my head; I don't know where compress() is called.

[epugh]

Could we call compress() universally and then our compressor chooses to only compress based on our criteria (state.json)? Just an idea off the top of my head; I don't know where compress() is called.

are you suggesting that it just be a default? And so any data, on it's way in is compressed, and on it's way back out is uncompressed? Interesting idea!

[dsmiley]

are you suggesting that it just be a default? And so any data, on it's way in is compressed, and on it's way back out is uncompressed? Interesting idea!

No, but it's confusing because I did say call compress() always. But I also said:

then our compressor chooses to only compress based on our criteria (state.json)?

Thus no effective change from today.

[HoustonPutman]

The idea is that with this we can eventually use the Curator recipes, which will require that our interactions with the CuratorFramework and SolrZkClient behave similarly. So yes, we could in the short term, just use .compress() in the solrZkClient calls. But in the future we need the auto-compression to work without .compress() so that the recipes, which don't know to always use .compress(), work in the same way.

[HoustonPutman]

With the Curator Global Compression stuff merged, we should be good to go here.

[epugh]

This looks much nicer. I can't find it, but wasn't there one path that would need to check solr.xml to decide if compression was happening or not? I guess that still happens in this code path, it's just that Curator manages things.

[HoustonPutman]

Yeah, I'm not changing how the configuration is done, just how it's handled in the backend.