Reduce number of GetSize query to database

[kaloyan-raev]

The IPFS node is bombarded with GetSize requests from the IPFS network. These calls come from the IPFS discovery process when someone is looking for a specific IPFS hash.

We could reduce the pressure on the DB by enabling the bloom filter feature of IPFS. This can be done by setting the BloomFilterSize config to a positive value.

[kaloyan-raev]

PR #17

BloomFilterSize can now be configured with env var.

[kaloyan-raev]

We have an issue with the bloom filter when running more than one IPFS node connected to the same Storj datastore.

When a new file is uploaded to one of the IPFS nodes then its bloom filter is updated with the new content, but the bloom filter of the rest of the nodes is not updated. So when a Get request comes to one of the other IPFS nodes, the request first hits its not-yet-updated bloom filter, and it returns "not found" for the content that has just been uploaded..

[kaloyan-raev]

New solution for CockroachDB: #45

[kaloyan-raev]

It seems we also need to either disable or update the Arc cache.

Others are also trying to disable it: ipfs/kubo#6586

[kaloyan-raev]

Patch to disable the ARC cache: #46

[kaloyan-raev]

Patch for properly decoding the CID before adding it to the bloom filter: #48