Merge PR ceph#38708 into master

* refs/pull/38708/head:
	doc: update multiple mds scrub docs

Reviewed-by: Jeff Layton <[email protected]>

PendingReleaseNotes

@@ -216,7 +216,7 @@
     ceph daemon <mds of rank 0> scrub_path '~mdsdir' force recursive repair
 
 * CephFS: Scrub is supported in multiple active mds setup. MDS rank 0 handles
-  scrub commands, and forward scrub to other mds if nessesary.
+  scrub commands, and forward scrub to other mds if necessary.
 
 * The following librados API calls have changed:
 

doc/cephfs/scrub.rst

@@ -17,48 +17,51 @@ via a set of scrub commands. Scrub can be classified into two parts:
 This document details commands to initiate and control forward scrub (referred as
 scrub thereafter).
 
+.. warning::
+
+   CephFS forward scrubs are started and manipulated on rank 0. All scrub
+   commands must be directed at rank 0.
+
 Initiate File System Scrub
 ==========================
 
-To start a scrub operation for a directory tree use the following command
+To start a scrub operation for a directory tree use the following command::
 
-::
+   ceph tell mds.<fsname>:0 scrub start <path> [scrubopts] [tag]
 
-   ceph tell mds.a scrub start / recursive
+where ``scrubopts`` is a comma delimited list of ``recursive``, ``force``, or
+``repair`` and ``tag`` is an optional custom string tag (the default is a generated
+UUID). An example command is::
+
+   ceph tell mds.cephfs:0 scrub start / recursive
    {
        "return_code": 0,
        "scrub_tag": "6f0d204c-6cfd-4300-9e02-73f382fd23c1",
        "mode": "asynchronous"
    }
 
-Recursive scrub is asynchronous (as hinted by `mode` in the output above). Scrub tag is
-a random string that can used to monitor the progress of the scrub operation (explained
-further in this document).
-
-Custom tag can also be specified when initiating the scrub operation. Custom tags get
-persisted in the metadata object for every inode in the file system tree that is being
-scrubbed.
+Recursive scrub is asynchronous (as hinted by `mode` in the output above).
+Asynchronous scrubs must be polled using ``scrub status`` to determine the
+status.
 
-::
+The scrub tag is used to differentiate scrubs and also to mark each inode's
+first data object in the default data pool (where the backtrace information is
+stored) with a ``scrub_tag`` extended attribute with the value of the tag. You
+can verify an inode was scrubbed by looking at the extended attribute using the
+RADOS utilities.
 
-   ceph tell mds.a scrub start /a/b/c recursive tag0
-   {
-       "return_code": 0,
-       "scrub_tag": "tag0",
-       "mode": "asynchronous"
-   }
+Scrubs work for multiple active MDS (multiple ranks). The scrub is managed by
+rank 0 and distributed across MDS as appropriate.
 
 
 Monitor (ongoing) File System Scrubs
 ====================================
 
-Status of ongoing scrubs can be monitored using in `scrub status` command. This commands
-lists out ongoing scrubs (identified by the tag) along with the path and options used to
-initiate the scrub.
+Status of ongoing scrubs can be monitored and polled using in `scrub status`
+command. This commands lists out ongoing scrubs (identified by the tag) along
+with the path and options used to initiate the scrub::
 
-::
-
-   ceph tell mds.a scrub status
+   ceph tell mds.cephfs:0 scrub status
    {
        "status": "scrub active (85 inodes in the stack)",
        "scrubs": {
@@ -72,9 +75,7 @@ initiate the scrub.
 `status` shows the number of inodes that are scheduled to be scrubbed at any point in time,
 hence, can change on subsequent `scrub status` invocations. Also, a high level summary of
 scrub operation (which includes the operation state and paths on which scrub is triggered)
-gets displayed in `ceph status`.
-
-::
+gets displayed in `ceph status`::
 
    ceph status
    [...]
@@ -85,26 +86,26 @@ gets displayed in `ceph status`.
 
    [...]
 
+A scrub is complete when it no longer shows up in this list (although that may
+change in future releases). Any damage will be reported via cluster health warnings.
+
 Control (ongoing) File System Scrubs
 ====================================
 
 - Pause: Pausing ongoing scrub operations results in no new or pending inodes being
   scrubbed after in-flight RADOS ops (for the inodes that are currently being scrubbed)
-  finish.
-
-::
+  finish::
 
-   ceph tell mds.a scrub pause
+   ceph tell mds.cephfs:0 scrub pause
    {
        "return_code": 0
    }
 
-`scrub status` after pausing reflects the paused state. At this point, initiating new scrub
-operations (via `scrub start`) would just queue the inode for scrub.
+  The ``scrub status`` after pausing reflects the paused state. At this point,
+  initiating new scrub operations (via ``scrub start``) would just queue the
+  inode for scrub::
 
-::
-
-   ceph tell mds.a scrub status
+   ceph tell mds.cephfs:0 scrub status
    {
        "status": "PAUSED (66 inodes in the stack)",
        "scrubs": {
@@ -115,22 +116,18 @@ operations (via `scrub start`) would just queue the inode for scrub.
        }
    }
 
-- Resume: Resuming kick starts a paused scrub operation.
-
-::
+- Resume: Resuming kick starts a paused scrub operation::
 
-   ceph tell mds.a. scrub resume
+   ceph tell mds.cephfs:0 scrub resume
    {
        "return_code": 0
    }
 
 - Abort: Aborting ongoing scrub operations removes pending inodes from the scrub
   queue (thereby aborting the scrub) after in-flight RADOS ops (for the inodes that
-  are currently being scrubbed) finish.
-
-::
+  are currently being scrubbed) finish::
 
-   ceph tell mds.a. scrub abort
+   ceph tell mds.cephfs:0 scrub abort
    {
        "return_code": 0
    }