Keep an eye on indexes size (geo)

[mdutoo]

Right now, geo areas replicated by the portal have 1,2GB indexes:

rs0:SECONDARY> db.geographical_area.totalIndexSize()
1266200768

which verges on the side of too much for our 4GB RAM, that also has to contain other (esp. DC but also Kernel) indexes, but also java processes (the portal and its memory caches, or the Datacore).

=> at some point (or as soon as it seems to trigger performance problems), we'll have to look into optimizing those indexes, increasing RAM...

[bobeal]

=> at some point (or as soon as it seems to trigger performance problems), we'll have to look into optimizing those indexes, increasing RAM...

Or into removing the geo cache from the portal. Especially since the DC now permits fulltext searches.

[mdutoo]

@bobeal I understand why you're saying that, but let's NOT remove the geo cache, because we really don't want the main (or at least first used) feature of the front application of the whole Ozwillo ecosystem, that can be accessed even by unlogged users, hit on the Datacore. Otherwise all other Datacore use cases will become an asterisk besides this one in termes of performance.The Datacore's goal is to share data, not to power a portal's unlogged autocompletion(!) search field.

(and even if we used dc fulltext search instead, we'd need the right indexes. So either they would take the same amount of disk, or we'd find a way to optimize them for this use case - but then the same optimization would allow to reduce portal geo indexes' size)

Anyway, there are a lot of ways to optimize the portal's geo cache's indexes: only use one instead of 3 (anyway, only 1 at a time can be used by mongo), remove country or even type from compound indexes... BUT this issue is not about "let's redefine indexes", before that it's about "let's truly profile and evaluate performance-wise current indexes for the use case's common queries, and do the same for various alternatives".