Skip to content

Commit

Permalink
common/options: remove unavailable values of osd_op_queue
Browse files Browse the repository at this point in the history
Signed-off-by: luo rixin <[email protected]>
  • Loading branch information
rosinL committed Oct 10, 2020
1 parent e492447 commit ce2ea1d
Show file tree
Hide file tree
Showing 2 changed files with 46 additions and 109 deletions.
148 changes: 44 additions & 104 deletions doc/rados/configuration/osd-config-ref.rst
Original file line number Diff line number Diff line change
Expand Up @@ -384,22 +384,16 @@ Operations
:Description: This sets the type of queue to be used for prioritizing ops
in the OSDs. Both queues feature a strict sub-queue which is
dequeued before the normal queue. The normal queue is different
between implementations. The original PrioritizedQueue (``prio``) uses a
token bucket system which when there are sufficient tokens will
dequeue high priority queues first. If there are not enough
tokens available, queues are dequeued low priority to high priority.
The WeightedPriorityQueue (``wpq``) dequeues all priorities in
relation to their priorities to prevent starvation of any queue.
WPQ should help in cases where a few OSDs are more overloaded
than others. The new mClock based OpClassQueue
(``mclock_opclass``) prioritizes operations based on which class
between implementations. The WeightedPriorityQueue (``wpq``)
dequeues operations in relation to their priorities to prevent
starvation of any queue. WPQ should help in cases where a few OSDs
are more overloaded than others. The new mClockQueue
(``mclock_scheduler``) prioritizes operations based on which class
they belong to (recovery, scrub, snaptrim, client op, osd subop).
And, the mClock based ClientQueue (``mclock_client``) also
incorporates the client identifier in order to promote fairness
between clients. See `QoS Based on mClock`_. Requires a restart.
See `QoS Based on mClock`_. Requires a restart.

:Type: String
:Valid Choices: prio, wpq, mclock_opclass, mclock_client
:Valid Choices: wpq, mclock_scheduler
:Default: ``wpq``


Expand Down Expand Up @@ -558,7 +552,7 @@ based on `the dmClock algorithm`_. This algorithm allocates the I/O
resources of the Ceph cluster in proportion to weights, and enforces
the constraints of minimum reservation and maximum limitation, so that
the services can compete for the resources fairly. Currently the
*mclock_opclass* operation queue divides Ceph services involving I/O
*mclock_scheduler* operation queue divides Ceph services involving I/O
resources into following buckets:

- client op: the iops issued by client
Expand Down Expand Up @@ -597,12 +591,6 @@ competitor "1". In the case of client ops, it is not clamped by the
limit setting, so it can make use of all the resources if there is no
recovery ongoing.

Along with *mclock_opclass* another mclock operation queue named
*mclock_client* is available. It divides operations based on category
but also divides them based on the client making the request. This
helps not only manage the distribution of resources spent on different
classes of operations but also tries to ensure fairness among clients.

CURRENT IMPLEMENTATION NOTE: the current experimental implementation
does not enforce the limit values. As a first approximation we decided
not to prevent operations that would otherwise enter the operation
Expand Down Expand Up @@ -689,124 +677,76 @@ mClock and dmClock experiments in the ceph-devel mailing list.
:Default: 8 MiB


``osd op queue mclock client op res``

:Description: the reservation of client op.

:Type: Float
:Default: 1000.0


``osd op queue mclock client op wgt``

:Description: the weight of client op.

:Type: Float
:Default: 500.0


``osd op queue mclock client op lim``

:Description: the limit of client op.

:Type: Float
:Default: 1000.0


``osd op queue mclock osd subop res``

:Description: the reservation of osd subop.

:Type: Float
:Default: 1000.0


``osd op queue mclock osd subop wgt``

:Description: the weight of osd subop.

:Type: Float
:Default: 500.0

``osd mclock scheduler client res``

``osd op queue mclock osd subop lim``
:Description: IO proportion reserved for each client (default).

:Description: the limit of osd subop.

:Type: Float
:Default: 0.0


``osd op queue mclock snap res``

:Description: the reservation of snap trimming.

:Type: Float
:Default: 0.0
:Type: Unsigned Integer
:Default: 1


``osd op queue mclock snap wgt``
``osd mclock scheduler client wgt``

:Description: the weight of snap trimming.
:Description: IO share for each client (default) over reservation.

:Type: Float
:Default: 1.0
:Type: Unsigned Integer
:Default: 1


``osd op queue mclock snap lim``
``osd mclock scheduler client lim``

:Description: the limit of snap trimming.
:Description: IO limit for each client (default) over reservation.

:Type: Float
:Default: 0.001
:Type: Unsigned Integer
:Default: 999999


``osd op queue mclock recov res``
``osd mclock scheduler background recovery res``

:Description: the reservation of recovery.
:Description: IO proportion reserved for background recovery (default).

:Type: Float
:Default: 0.0
:Type: Unsigned Integer
:Default: 1


``osd op queue mclock recov wgt``
``osd mclock scheduler background recovery wgt``

:Description: the weight of recovery.
:Description: IO share for each background recovery over reservation.

:Type: Float
:Default: 1.0
:Type: Unsigned Integer
:Default: 1


``osd op queue mclock recov lim``
``osd mclock scheduler background recovery lim``

:Description: the limit of recovery.
:Description: IO limit for background recovery over reservation.

:Type: Float
:Default: 0.001
:Type: Unsigned Integer
:Default: 999999


``osd op queue mclock scrub res``
``osd mclock scheduler background best effort res``

:Description: the reservation of scrub jobs.
:Description: IO proportion reserved for background best_effort (default).

:Type: Float
:Default: 0.0
:Type: Unsigned Integer
:Default: 1


``osd op queue mclock scrub wgt``
``osd mclock scheduler background best effort wgt``

:Description: the weight of scrub jobs.
:Description: IO share for each background best_effort over reservation.

:Type: Float
:Default: 1.0
:Type: Unsigned Integer
:Default: 1


``osd op queue mclock scrub lim``
``osd mclock scheduler background best effort lim``

:Description: the limit of scrub jobs.
:Description: IO limit for background best_effort over reservation.

:Type: Float
:Default: 0.001
:Type: Unsigned Integer
:Default: 999999

.. _the dmClock algorithm: https://www.usenix.org/legacy/event/osdi10/tech/full_papers/Gulati.pdf

Expand Down
7 changes: 2 additions & 5 deletions src/common/options.cc
Original file line number Diff line number Diff line change
Expand Up @@ -2950,13 +2950,10 @@ std::vector<Option> get_global_options() {

Option("osd_op_queue", Option::TYPE_STR, Option::LEVEL_ADVANCED)
.set_default("wpq")
.set_enum_allowed( { "wpq", "prioritized",
"mclock_opclass", "mclock_client", "mclock_scheduler",
"debug_random" } )
.set_enum_allowed( { "wpq", "mclock_scheduler", "debug_random" } )
.set_description("which operation priority queue algorithm to use")
.set_long_description("which operation priority queue algorithm to use; "
"mclock_opclass mclock_client, and "
"mclock_client_profile are currently experimental")
"mclock_scheduler is currently experimental")
.add_see_also("osd_op_queue_cut_off"),

Option("osd_op_queue_cut_off", Option::TYPE_STR, Option::LEVEL_ADVANCED)
Expand Down

0 comments on commit ce2ea1d

Please sign in to comment.