This directory tree contains the WorkBench benchmark suite, industrially
hardened by BitRaker, and ported to Linux/x86 and ARM ADS (advanced
development system).  This brief document describes the contents of the 
WorkBench, the configuration of the WorkBench, how to run WorkBench 
experiments, and how to use Bitraker tools in conjunction with the WorkBench. 

WorkBench DIRECTORY STRUCTURE:
------------------------------

The WorkBench benchmark directories are arranged as follows:

WorkBench-X.X/		- top level directory

  README		- This file.
  Makefile.arm-ads	- ARM ADS build configuration, emulation with ARMulator
  Makefile.x86		- Linux/x86 GNU GCC build configuration, native x86 
                          execution
  Makefile.simforge	- ARM ADS build configuration, emulation with SimForge

  automotive/		- automotive/industrial benchmarks

    basicmath/		- basic math
    bitc/		- bit counting
    ellf/		- digital filters
    qsort/		- sorting routines
    susan/		- industrial imaging application

  common/		- WorkBench-wide source files (for portability)

  consumer/		- consumer-oriented applications

    forth/		- forth interpreter
    jpeg/		- jpeg graphics encoding/decoding
    lout/		- html typesetting
    mad/		- MP3 decoding
    tiff/		- digital image processing (b/w, quant, dither, etc)

  network/		- networking applications

    dijkstra/		- Dijkstra's packet routing algorithm
    patricia/		- Patricia tree based routing lookup

  office/		- office appliance applications

    grep/		- text searches using regular expressions
    ispell/		- spell checking

  security/		- security applications

    blowfish/		- blowfish bulk cipher encode/decode
    rijndael/		- Rijndael (AES) standard bulk encode/decode
    rsa/		- RSA public key cipher sign/seal
    sha/		- SHA data digest algorithm

  telecom/		- telecommunication oriented applications

    adpcm/		- adaptive differential pulse code modulation codec
    crc32/		- 32-bit cyclic redundancy checksum algorithm
    fft/		- fast fourier transform algorithm
    gsm/		- GSM lossy speech compression encode/decode


WorkBench INDIVIDUAL BENCHMARK DIRECTORY STRUCTURE:
----------------------------------------

Individual benchmark directories are arranged as follows:

Makefile.<variant>	- Makefile, one per unit benchmark "experiment"
src/			- benchmark source code
tests/			- test inputs, reference outputs
dist/			- original benchmark copyrights, distribution files


CONFIGURING THE WorkBench TARGET MACHINE:
---------------------------------

To start using WorkBench, you need to configure it for the type of machine 
you will be building for and running on.  This is accomplished by issuing a 
single command to create a symbolic link file called Makefile.config that
refers to Makefile.<yourTarget>.  The currently supported targets and their 
configuration commands are as follows:

Linux/x86:
  ln -sf Makefile.x86 Makefile.config

ARM ADS (advanced development system with semi-hosting)
  ln -sf Makefile.arm-ads Makefile.config

SimForge (advanced development system with semi-hosting)
  ln -sf Makefile.simforge Makefile.config

Makefile.arm-ads is a top-level configuration makefile for compilation
on ARM ADS tools and emulation with ARMulator.  Makefile.x86 is a
top-level configuration makefile for compilation with GNU GCC on
Linux/x86 and native execution.  Makefile.simforge is a top-level
makefile for compilation on ARM ADS tools and emulation on SimForge.

WorkBench currently ships with all of the Benchmark executabls compiled 
on ARM ADS tools.  If you plan on running WorkBench Experiments on Windows
or Linux, you will probably be using Simforge as your ARM emulator, so 
'ln -sf Makefile.simforge Makefile.config' 
would be the configuration command to use. Makefile.simforge also includes 
the targets used to drive Anvil testing.  The use of Anvil and Simforge 
is described later in this document.


RUNNING INDIVIDUAL WorkBench EXPERIMENTS:
-----------------------------------------

WorkBench supports a large number of programs, and even a larger number
of "experiments".  Experiments are a unique combination of program,
inputs, and validated results.  To see the experiments supported,
run the following command:

  ./runexps.sh

Specific experiments can be invoked by specifying experiment
"tag", followed by a sequence of Makefile targets.  For example, to
run the large experiments for the benchmark "susan":

  ./runexps.sh susan:large run-tests

Experiment tags are a sequence of tokens separated by ":"'s.  Commonly
supported tags are "all", "large",  and "small"; and also program
class tags "automotive", "consumer", "network", "office", "security",
and "telecom".  Individual program names are also supported.

All benchmark makefiles support a variety of Makefile targets,
including:

  clean			- clean up all build/test generated outputs
  clean-build		- clean all outputs (except executable)
  build			- build the test program
  run-tests		- run all tests and validate outputs


WorkBench TOPLEVEL RUN SCRIPTS:
-------------------------------

The "runexps.sh" script can be used invoke a specific makefile
target for all WorkBench experiments.  For example, to clean
all the experiment directories:

  ./runexps.sh all clean

The toplevel script supports up to 8 targets, executed in the
sequence specified.  So, for example, to clean, build and
run tests for all experiments, execute the following command:

  ./runexps.sh all clean build run-tests

Errors in the output can be identified with the top-level script
"runcheck.sh".  For example, the following commands will run all
experiments with full validation:

  ./runexps.sh all clean build run-tests |& tee MAKE.log
  ./runcheck.sh MAKE.log

If the runcheck.sh command has any output, at least one of the
experiments failed.  Details are in the log file...


USING ANVIL AND SIMFORGE IN CONJUNCTION WITH THE WorkBench 
----------------------------------------------------------

Several files need to be installed and some environment variables 
need to be modified before WorkBench executables can be instrumented 
in order to perform fast targeted program analysis. 

anvil.dll - This is the Anvil API and is used by your Tool.exe to
instrument/path a binary file.   The anvil.dll should be found
in your environment variable (PATH on windows and LD_LIBRARY_PATH on linux).

simforge.exe - This is the Bitraker ARM simulator, and is used to
run the ARM program on an x86 desktop machine.  The directory
with the simforge.exe should be found in your PATH environment variable.

anvilanalysis.dll or anvilanalysis.so - This is the BitRaker analysis
shared library that is used to enable your code to run efficiently
when using simforge.exe.  Make sure that the directory with this
shared library is found on your search environment variables 
(PATH for windows and LD_LIBRARY_PATH for linux).

Note, that these 3 files are found in the /bin directory for the
Anvil install.  You need to make sure that this directory is found
on the corresponding environment variables listed above.


In addition to the above files, there are 2 files created by the
user of Anvil that are required to perform targeted program analysis.

<yourToolName>.exe - This it the executable you created to perform your
targeted ARM binary analysis containing all of your instrumentation
code.  This is either compiled for windows or linux, depending upon
your Desktop target.  The directory containing this binary should be
specified in your PATH environment variable.

<yourToolName>.dll or <yourToolName>.so - This is the shared library 
(.dll on windows and.so on linux) that contains your analysis code.  
This shared library must be locatable on your search environment variables 
(PATH for windows and LD_LIBRARY_PATH for linux).

Note, that <yourToolName>.exe must have the -i flag specified, and the 
-o flag specified as in the AnvilTemplate tool example.  "-i" specifies 
the input ARM binary name, and "-o" is used to specify the output binary
name.


Anvil SPECIFIC RUN OPTIONS:
---------------------------

Two shell scripts, anvil-build.sh and anvil-run.sh, have been included 
with Workbench to facilitate instrumentation of WorkBench executables 
using Anvil created tools and emulation of instrumented WorkBench 
experiments on Simforge.  

USE OF anvil-build.sh

To instrument all of the executables in a particular benchmark directory
with an Anvil created tool: 

  ./anvil-build.sh <benchMarkName>:build <yourToolName>.exe

To instrument all executables in the Workbench with an Anvil created tool: 

  ./anvil-build.sh all:build <yourToolName>.exe

USE OF anvil-run.sh

To run a particular experiment (that has been instrumented) on Simforge:

  ./anvil-run.sh <experimentName> anvilanalysis.dll

To run all experiments  (all executables in theWorkBench must already be 
instrumented) 

  ./anvil-run.sh all anvilanalysis.dll 

If necessary, special simulator arguments can be passed to anvil-run.sh, e.g.:

  ./anvil-run.sh <experimentName> anvilanalysis.dll -sim_opt1 -sim_opt2 ...