JMX Stuff

We use the JMX connector, as documented here.

The main schema is jmx.current, which has around 260 tables. This represents a snapshot of the current system. For particular tables, we could keep a history of them in jmx.history, but only java.lang:type=runtime is kept by default.

Of interest are:

• select * from jmx.current."java.lang:type=operatingsystem";. This shows free memory, open file count, CPU load, etc on a per node basis.
• select * from jmx.current."com.facebook.presto.memory:name=clustermemorymanager";. This shows total cluster memory and queries killed due to out of memory errors.
• select * from jmx.current."java.lang:type=memory";. This shows java memory stats on a per node basis, but it's hard to parse.
• select * from jmx.current."com.facebook.presto.failuredetector:name=heartbeatfailuredetector";

We use this Uhura job https://github.com/stitchfix/aa-infra/tree/master/jobs/jmx-to-cloudwatch to forward some metrics to Cloudwatch.