batch

Purpose: describe usage of the Run Batch Tool

Audience: scientists

Introduction

The Run Batch tool generates inputs to a script, and runs the script with each value of the inputs. For example, suppose you have a process you need to apply to each of the files of a two-spacecraft mission. It would be nice to be able to run a script for each day and each spacecraft of the mission, and this is what the "Run Batch" tool does. It lets you select a list of inputs for a parameter, and the script is run with each of those inputs as one long process. A second parameter can be set as well, and each combination is then run (for example, each day, each spacecraft).

To show an example suppose we have a pile of plot pngs and we want to find the one with a black background. (Of course, the pngwalk tool could be used for the task as well.) Here's a script that counts the number of black or white pixels in an image:

tr= getParam( 'timerange', '2008-01-01', 'timerange to load' )
wb= getParam( 'color', 'white', 'the color to count', ['white','black'] )
ds= getDataSet( 'http://autoplot.org/autoplot/autoplot/autoplot/data/pngwalk/product_$Y$m$d.png?channel=greyscale', tr )
if wb=='white': iwb=255 
else: iwb=0
r= where( ds.eq( iwb ) )
print r.length()

The batch tool can be used to used to run this on each image in a set of files. In Autoplot, select [menubar]->Tools->Run Batch..., and then in the GUI select the script "http://autoplot.org/data/script/examples/countColors.jy" and hit the green play button. On the left window, the parameters of the script are listed in the droplist. Select timerange, and then in the area below, enter each timerange. Typically this would be copy-and-pasted, and there is a context popup for generating lists. Right-click (or context click on a Mac), and press "Generate". "Jan 2008" and "$Y-$m-$d" will enter in the 31 intervals. Click the go button, and see that the console output contains the pixel counts.

The following script parameter types are supported:

Parameter Type	Supported	How
resourceURI	yes	File Chooser Dialog allows selection of files.
enumeration	yes	Checkbox list allows selection of values to use.
floats	yes	Dialog for creating linspace list (0.1,0.2,0.3,...,99.9)

Write PNG

The "Write" checkbox will write the canvas out to a PNG or PDF file. The name of the file is controlled with a template string, where each $x wildcard is replaced with the field value. The number of "wildcards" must match the number of parameters. Time strings have slashes (/), spaces, and colons (:) replaced with underscores.

Some simple formatting controls are available as well. These include:

• %d format the integer value
• %04d integer value is formatted into four characters with leading zeros.
• %06.3f floats are formatted with three decimal places.

Other Use Cases

• rename weblogs by the last date in the file
• reformatting files
• run IDL script for each day, using adapter.

Export Results File

The results can be saved to a file, see File->"Export Results". Once the file name FILE is set, the results will be written to FILE.pending as the batch is run.

multi argument

Since v2022a_2 Autoplot allows either parameter to be a semicolon-separated list of parameters, so that you can just run through a list of values.

For example, using the script https://github.com/autoplot/dev/blob/master/rfe/sf/772/script.jy,

set the left box (param1) to "seed;red;green;blue" and then give it a list of seed, red, green, and blue values, split by a semicolon:

0;200;200;0
1;0;200;200
2;0;100;200
3;200;100;0
4;100;200;0
0;100;200;100
1;100;0;100
2;100;0;200

This will set seed, red, green, and blue parameters for each step.

What's To-Be-Done

This is a long-overdue facility written to solve a problem one day, which turns out to be quite useful. As such, the feature set is still incomplete and needs to be flushed out. New features include:

• A GUI for writing the output file template.

Let us know (1) if you have ideas for how to make this more effective!

See Also

HTML Tutorial: http://autoplot.org/data/tutorials/20170223_runBatch/20170223_1231.html