Skip to content

Commit

Permalink
IOTune: Seastar helper tool to estimate proper I/O queue parameters
Browse files Browse the repository at this point in the history
IOTune is a tool that measures the disk's response to varied concurrency, and
determines what is the best set of parameters for Seastar's I/O Queue.

It is supposed to be ran in conditions as close as possible than the expected
deployment. To do that, the tool includes support for two parameters:

--evaluation-directory: the directory in which I/O will be generated.
--cpuset: in which processors to run the test.

The latter is specially useful in scenarions like Amazon EC2, where one or more
processors are usually excluded from the run, as they concentrate interrupts for
the box. Adding those processors to the test can contribute errors and we would
be better off without them.

When the evaluation finishes, the suggested parameters are written to the file
specified by a third parameter:

--options-file: the file where to store the evaluated parameters,
                ~/.config/seastar/io.conf by default.

Estimating the best I/O depth
-----------------------------

To estimate the best I/O depth for this scenario, we will try to find the point
where we first reach 80 % of the maximum throughput. We then multiply this by
the empirical number 3.

In order to reduce testing time, we will run this test in three phases, with
the overarching goal of minimizing the amount of points to query:

Phase 1) Try to find the region where the maximum lives. We don't care if the value
is 100 % accurate, so we run each concurrency not for very long. We also skip many
points between each runs.

Phase 2) Once we know where does the maximum lives, then we run every point in the
region, and we run for longer. The goal here is to estimate what is, in IOPS, the
maximum throughput.

Phase 3) We know, roughly, in which region does the 80th percentile lives, but
because we have ran phase 1 for very little time and skipped many points, we don't
know at which concurrency exactly we should be. We then refine the search running
every point in an interval around the desired point, and for even longer.

Signed-off-by: Glauber Costa <[email protected]>
  • Loading branch information
Glauber Costa committed Feb 15, 2016
1 parent f75306e commit 90b72cc
Show file tree
Hide file tree
Showing 2 changed files with 594 additions and 0 deletions.
Loading

0 comments on commit 90b72cc

Please sign in to comment.