Add docformatter which formats Python docstrings to PEP 257 (radian…

…-software#267)

Add [docformatter](https://github.com/PyCQA/docformatter) for Python
docstrings.

By default it outputs diffs but changes in-place with `--in-place`. On
successful change it exits with an error code of `3` (found out by
trial), so I had to add a formatter wrapping-script.

Initially I used `--in-place` with the special `in-place` symbol in
apheleia. But now I tried an approach where I transform the diff into
usable stdout using `patch` instead.

Related to radian-software#266 , where I had used the example of docformatter to ask
how to add scripts with positive exit codes and @raxod502 showed me the
`phpcs` solution.

---------

Co-authored-by: Radon Rosborough <[email protected]>

CHANGELOG.md

@@ -24,6 +24,7 @@ The format is based on [Keep a Changelog].
   ([#263]).
 * [denofmt](https://docs.deno.com/runtime/manual/tools/formatter) for
   js, jsx, ts, tsx, json, jsonc, md files. ([#264])
+* [docformatter](https://github.com/PyCQA/docformatter) for Python docstrings ([#267])
 * [cljfmt](https://github.com/weavejester/cljfmt) for clojure,
   clojurescript, edn files. ([#271])
 
@@ -34,6 +35,7 @@ The format is based on [Keep a Changelog].
 [#261]: https://github.com/radian-software/apheleia/pull/261
 [#263]: https://github.com/radian-software/apheleia/pull/263
 [#264]: https://github.com/radian-software/apheleia/pull/264
+[#267]: https://github.com/radian-software/apheleia/pull/267
 [#271]: https://github.com/radian-software/apheleia/pull/271
 
 ## 4.0 (released 2023-11-23)

apheleia-formatters.el

@@ -56,6 +56,7 @@
     (denofmt-md . ("deno" "fmt" "-" "--ext" "md"))
     (denofmt-ts . ("deno" "fmt" "-" "--ext" "ts"))
     (denofmt-tsx . ("deno" "fmt" "-" "--ext" "tsx"))
+    (docformatter . ("apheleia-docformatter" inplace))
     (dprint . ("dprint" "fmt" "--stdin" filepath))
     (elm-format . ("elm-format" "--yes" "--stdin"))
     (fish-indent . ("fish_indent"))

scripts/formatters/apheleia-docformatter

@@ -0,0 +1,5 @@
+#!/bin/sh
+docformatter --in-place "$@"
+if [ "$?" -eq 3 ]; then
+    exit 0
+fi

test/formatters/apheleia-ft.el

@@ -260,70 +260,63 @@ involve running any formatters."
 Interactively, select a single formatter to test using
 `completing-read'. If FORMATTERS is not provided (or,
 interactively, with prefix argument), fall back to the FORMATTERS
-environment variable, defaulting to all formatters."
+environment variable, defaulting to all formatters.
+
+This takes care of creating temporary file(s), if necessary for
+the provided formatter, for example if `input' or `inplace' is
+used, and substituting them in the command line. You can get the
+name of the file used for input, if any, as a property on the
+returned context."
   (interactive
    (unless (or current-prefix-arg noninteractive)
      (list (completing-read "Formatter: " (apheleia-ft--get-formatters)))))
   (setq-default indent-tabs-mode nil)
   (dolist (formatter (or formatters (apheleia-ft--get-formatters)))
     (dolist (in-file (apheleia-ft--input-files formatter))
-      (let ((extension (file-name-extension in-file))
-            (in-text (apheleia-ft--read-file in-file))
-            ;; The `in-temp-real-file' variable is set to whatever
-            ;; temporary file the formatter will run on (in case it
-            ;; uses the `file' or `filepath' symbol or is a function).
-            (in-temp-real-file nil)
-            (out-temp-file nil)
-            (command (alist-get (intern formatter) apheleia-formatters))
-            (syms nil)
-            (stdout-buffer nil)
-            (stderr-file (make-temp-file "apheleia-ft-stderr-"))
-            (default-directory temporary-file-directory)
-            (exit-status nil)
-            (out-file (replace-regexp-in-string
-                       "/in\\([^/]+\\)" "/out\\1" in-file 'fixedcase))
-            (exec-path
-             (append `(,(expand-file-name
-                         "scripts/formatters"
-                         (file-name-directory
-                          (file-truename
-                           ;; Borrowed with love from Magit
-                           (let ((load-suffixes '(".el")))
-                             (locate-library "apheleia"))))))
-                     exec-path)))
-        ;; Some formatters use the current file-name or buffer-name to interpret the
-        ;; type of file that is being formatted. Some may not be able to determine
-        ;; this from the contents of the file so we set this to force it.
-        (rename-buffer (file-name-nondirectory in-file))
-        (setq stdout-buffer (get-buffer-create
-                             (format "*apheleia-ft-stdout-%S%s" formatter extension)))
-        (with-current-buffer stdout-buffer
-          (erase-buffer))
-        (if (functionp command)
-            (let ((in-temp-file (apheleia-ft--write-temp-file
-                                 in-text extension)))
-              (setq in-temp-real-file in-temp-file)
-              (with-current-buffer (find-file-noselect in-temp-file)
+      (let* ((extension (file-name-extension in-file))
+             (in-text (apheleia-ft--read-file in-file))
+             (in-temp-file (apheleia-ft--write-temp-file
+                            in-text extension))
+             (out-temp-file nil)
+             (command (alist-get (intern formatter) apheleia-formatters))
+             (syms nil)
+             (stdout-buffer nil)
+             (stderr-file (make-temp-file "apheleia-ft-stderr-"))
+             (default-directory temporary-file-directory)
+             (exit-status nil)
+             (out-file (replace-regexp-in-string
+                        "/in\\([^/]+\\)" "/out\\1" in-file 'fixedcase))
+             (exec-path
+              (append `(,(expand-file-name
+                          "scripts/formatters"
+                          (file-name-directory
+                           (file-truename
+                            ;; Borrowed with love from Magit
+                            (let ((load-suffixes '(".el")))
+                              (locate-library "apheleia"))))))
+                      exec-path)))
+        (with-current-buffer (find-file-noselect in-temp-file)
+          ;; Some formatters use the current file-name or buffer-name to interpret the
+          ;; type of file that is being formatted. Some may not be able to determine
+          ;; this from the contents of the file so we set this to force it.
+          (rename-buffer (file-name-nondirectory in-file))
+          (setq stdout-buffer (get-buffer-create
+                               (format "*apheleia-ft-stdout-%S%s" formatter extension)))
+          (with-current-buffer stdout-buffer
+            (erase-buffer))
+          (if (functionp command)
+              (progn
                 (funcall command
                          :buffer (current-buffer)
                          :scratch (current-buffer)
                          :formatter formatter
                          :callback (lambda ()))
-                (copy-to-buffer stdout-buffer (point-min) (point-max))))
-          (let ((in-temp-file (apheleia-ft--write-temp-file
-                               in-text extension)))
-            (with-current-buffer (find-file-noselect in-temp-file)
-              (let ((ctx (apheleia--formatter-context
-                          (intern formatter) command nil nil)))
-                (setq command `(,(apheleia-formatter--arg1 ctx)
-                                ,@(apheleia-formatter--argv ctx))
-                      ;; In this case the real temp file might be
-                      ;; different from the one we generated, because
-                      ;; the context creator might generate another
-                      ;; temporary file to avoid touching our existing
-                      ;; one.
-                      in-temp-real-file (apheleia-formatter--input-fname ctx)
-                      out-temp-file (apheleia-formatter--output-fname ctx))))
+                (copy-to-buffer stdout-buffer (point-min) (point-max)))
+            (let ((ctx (apheleia--formatter-context
+                        (intern formatter) command nil nil)))
+              (setq command `(,(apheleia-formatter--arg1 ctx)
+                              ,@(apheleia-formatter--argv ctx))
+                    out-temp-file (apheleia-formatter--output-fname ctx)))
 
             (with-current-buffer stdout-buffer
               (erase-buffer))
@@ -347,16 +340,15 @@ environment variable, defaulting to all formatters."
               (error
                "Formatter %s exited with status %S" formatter exit-status))))
         ;; Verify that formatter has not touched original file.
-        (when in-temp-real-file
-          (let ((in-text-now (apheleia-ft--read-file in-temp-real-file)))
-            (unless (string= in-text in-text-now)
-              (apheleia-ft--print-diff
-               "original" in-text
-               "updated" in-text-now)
-              (error "Formatter %s modified original file in place" formatter))))
+        (let ((in-text-now (apheleia-ft--read-file in-temp-file)))
+          (unless (string= in-text in-text-now)
+            (apheleia-ft--print-diff
+             "original" in-text
+             "updated" in-text-now)
+            (error "Formatter %s modified original file in place" formatter)))
         ;; Verify that formatter formatted correctly.
         (let ((out-text
-               (if (or (memq 'output syms) (memq 'inplace syms))
+               (if out-temp-file
                    (apheleia-ft--read-file out-temp-file)
                  (with-current-buffer stdout-buffer
                    (buffer-string))))

test/formatters/installers/docformatter.bash

@@ -0,0 +1,2 @@
+apt-get install -y python3-pip
+pip3 install docformatter

test/formatters/samplecode/docformatter/in.py

@@ -0,0 +1,26 @@
+def single_line_doc():
+    """
+    Line break not necessary
+    """
+
+
+def extend_first_line():
+    """First line
+first line continuation
+    """
+
+
+def add_line_break():
+    """First line.
+    Second line.
+    """
+
+
+def long_lines():
+    """
+
+
+Nullam eu ante vel est convallis dignissim.  Fusce suscipit, wisi nec facilisis facilisis, est dui fermentum leo, quis tempor ligula erat quis odio.  Nunc porta vulputate tellus.  Nunc rutrum turpis sed pede.  Sed bibendum.  Aliquam posuere.  Nunc aliquet, augue nec adipiscing interdum, lacus tellus malesuada massa, quis varius mi purus non odio.  Pellentesque condimentum, magna ut suscipit hendrerit, ipsum augue ornare nulla, non luctus diam neque sit amet urna.  Curabitur vulputate vestibulum lorem.  Fusce sagittis, libero non molestie mollis, magna orci ultrices dolor, at vulputate neque nulla lacinia eros.  Sed id ligula quis est convallis tempor.  Curabitur lacinia pulvinar nibh.  Nam a sapien.
+
+
+    """

test/formatters/samplecode/docformatter/out.py

@@ -0,0 +1,30 @@
+def single_line_doc():
+    """Line break not necessary."""
+
+
+def extend_first_line():
+    """First line first line continuation."""
+
+
+def add_line_break():
+    """First line.
+
+    Second line.
+    """
+
+
+def long_lines():
+    """Nullam eu ante vel est convallis dignissim.
+
+    Fusce suscipit, wisi nec facilisis facilisis, est dui fermentum leo,
+    quis tempor ligula erat quis odio.  Nunc porta vulputate tellus.
+    Nunc rutrum turpis sed pede.  Sed bibendum.  Aliquam posuere.  Nunc
+    aliquet, augue nec adipiscing interdum, lacus tellus malesuada
+    massa, quis varius mi purus non odio.  Pellentesque condimentum,
+    magna ut suscipit hendrerit, ipsum augue ornare nulla, non luctus
+    diam neque sit amet urna.  Curabitur vulputate vestibulum lorem.
+    Fusce sagittis, libero non molestie mollis, magna orci ultrices
+    dolor, at vulputate neque nulla lacinia eros.  Sed id ligula quis
+    est convallis tempor.  Curabitur lacinia pulvinar nibh.  Nam a
+    sapien.
+    """