Let model go on while AA is clustering flows for visuals

[bettinardi]

jeabraham commented on Mar 24, 2016

I would like to turn on PECAS's flow visual clustering. However it's a bit CPU intensive, and there's no reason for the rest of the model to stop running while we're waiting of the outputs to be clustered for visualization. I see three possibilities to avoid slowing down the model run much:

Find a way for AA to explain to the rest of the system that it's finished modelling, but still running in the background processing visuals, so the other modules can start in parallel;
build a separate main() method inside of AA that runs AA for a single iteration from the converged solution, to re-generate flows internally in memory, then only writes out the compressed visuals (no other outputs written); or
write a standalone program that reads in the flow matrices that were written by AA, clusters them, and writes out flow clusters.
If we choose 2) or 3), I would like to be able to specify one of three options for each year of the run: Option A: Flow clustering occurs during the year run (e.g. starts immediately after AA is finished, or in parallel with some other task that doesn't use all the CPU cores), Option B: occurs after the entire run is finished, e.g. after the SwimVIZ database has been generated, Option C: doesn't occur at all (skipped for that year).

jeabraham commented on Apr 1, 2016

I've posted a new .jar for implementing 2). After testing and when Graham agrees, can we put into the system a program to rerun AA in each year after the Swimviz database is collated? I.e., after the entire existing run cycle is complete, the system should rerun AA in each year (parameter on command line to ensure a 0 iteration rerun) to write out extra files not needed by other modules during the run itself.