debian: add .requires for specifying python3 deps

we use dh_python3 to define subvar of ${python3:Depends} as a part
of the runtime dependencies of python3 packages, like,
ceph-mgr modules named "ceph-mgr-*", python3 bindings named "python3-*".

but unlike python3 bindings of Ceph APIs, the ceph-mgr modules are
not packaged in a typical python way. in other words, they do not
ship a "dist-info" or an "egg-info" directory. instead, we just
install the python scripts into a directory which can be found by
ceph-mgr, by default it is /usr/share/ceph/mgr/dashboard/plugins.

this does not follow the convention of python packaging or
debian packaging policies related to python package. but it
still makes to put these files in this non-convention place, as
they are not supposed to be python packages consumed by the
outer world -- they are but plugins. and should always work
with the same version of ceph-mgr.

the problem is, despite that we have ${python3:Depends} in
the "Depends" field of packages like ceph-mgr-dashboard, dh_python3
is not able to figure out the dependencies by looking at the
installed files. for instance, we have following "Depends" of
ceph-mgr-dashboard:

Depends: ceph-mgr (= 17.0.0-12481-g805d2320-1focal), python3-cherrypy3, python3-jwt, python3-bcrypt, python3-werkzeug, python3-routes

and in the debian/control file we have:

Depends: ceph-mgr (= ${binary:Version}),
         python3-cherrypy3,
         python3-jwt,
         python3-bcrypt,
         python3-werkzeug,
         python3-routes,
         ${misc:Depends},
         ${python:Depends},
         ${shlibs:Depends},

apparently, none of the subvar is materialized to
a non-empty string.

to improve the packaging, in this change:

* drop all subvars from ceph-mgr-*, as they
  are all implemented in pure python.
* add debian/ceph-mgr-*.requires, it's content
  is replicated with the corresponding requirements.txt
  files.
  * add python3-distutils for distutils, as debian
    and its derivatives package non-essetial part of
    distutils into a separate package, see
    https://packages.debian.org/stable/python3-distutils
* add ${python3:Depends} so dh_python3
  can extract the deps from debian/ceph-mgr-*.pydist
* update the rule for "override_dh_python3" target,
  so dh_python3 can pick up the dependencies specified
  in .requires file.
* remove the python3 dependencies not used by
  ceph-mgr from ceph-mgr's "Depends"

Signed-off-by: Kefu Chai <[email protected]>

debian/ceph-mgr-cephadm.requires

@@ -0,0 +1,4 @@
+CherryPy
+asyncssh
+cryptography
+Jinja2

debian/ceph-mgr-dashboard.requires

@@ -0,0 +1,11 @@
+bcrypt
+CherryPy
+more-itertools
+PyJWT
+pyopenssl
+requests
+Routes
+prettytable
+pytest
+pyyaml
+ceph-common

debian/ceph-mgr-diskprediction-local.requires

@@ -0,0 +1,3 @@
+numpy
+scipy
+scikit-learn

debian/ceph-mgr-k8sevents.requires

@@ -0,0 +1,2 @@
+pyyaml
+kubernetes

debian/ceph-mgr-modules-core.requires

@@ -0,0 +1,6 @@
+natsort
+CherryPy
+pecan
+werkzeug
+requests
+python-dateutil

debian/ceph-mgr-rook.requires

@@ -0,0 +1,2 @@
+kubernetes
+jsonpatch

debian/ceph-mgr.requires

@@ -0,0 +1,5 @@
+pyOpenSSL
+cephfs
+ceph-argparse
+pyyaml
+distutils

debian/control

@@ -226,15 +226,6 @@ Package: ceph-mgr
 Architecture: linux-any
 Depends: ceph-base (= ${binary:Version}),
          ceph-mgr-modules-core (= ${binary:Version}),
-         python3-bcrypt,
-         python3-cherrypy3,
-         python3-natsort,
-         python3-distutils,
-         python3-jwt,
-         python3-openssl,
-         python3-pecan,
-         python3-requests,
-         python3-werkzeug,
          ${misc:Depends},
          ${python3:Depends},
          ${shlibs:Depends},
@@ -256,14 +247,7 @@ Description: manager for the ceph distributed storage system
 Package: ceph-mgr-dashboard
 Architecture: all
 Depends: ceph-mgr (= ${binary:Version}),
-         python3-cherrypy3,
-         python3-jwt,
-         python3-bcrypt,
-         python3-werkzeug,
-         python3-routes,
-         ${misc:Depends},
-         ${python:Depends},
-         ${shlibs:Depends},
+         ${python3:Depends}
 Description: dashboard module for ceph-mgr
  Ceph is a massively scalable, open-source, distributed
  storage system that runs on commodity hardware and delivers object,
@@ -279,12 +263,7 @@ Description: dashboard module for ceph-mgr
 Package: ceph-mgr-diskprediction-local
 Architecture: all
 Depends: ceph-mgr (= ${binary:Version}),
-         python3-numpy,
-         python3-scipy,
-         python3-sklearn,
-         ${misc:Depends},
-         ${python:Depends},
-         ${shlibs:Depends},
+         ${python3:Depends}
 Description: diskprediction-local module for ceph-mgr
  Ceph is a massively scalable, open-source, distributed
  storage system that runs on commodity hardware and delivers object,
@@ -297,9 +276,6 @@ Package: ceph-mgr-modules-core
 Architecture: all
 Depends: ${misc:Depends},
          ${python3:Depends},
-         ${shlibs:Depends},
-         python3-dateutil,
-         python3-openssl,
 Replaces: ceph-mgr (<< 15.1.0)
 Breaks: ceph-mgr (<< 15.1.0)
 Description: ceph manager modules which are always enabled
@@ -313,7 +289,6 @@ Description: ceph manager modules which are always enabled
 Package: ceph-mgr-rook
 Architecture: all
 Depends: ceph-mgr (= ${binary:Version}),
-         python3-jsonpatch
          ${misc:Depends},
          ${python3:Depends},
          ${shlibs:Depends},

debian/py3dist-overrides

@@ -1 +1,4 @@
 cephfs python3-cephfs; PEP386
+ceph_argparse python3-ceph-argparse
+ceph_common python3-ceph-common
+distutils python3-distutils

debian/rules

@@ -142,6 +142,9 @@ override_dh_python3:
 	for binding in rados cephfs rbd rgw; do \
 	  dh_python3 -p python3-$$binding;      \
 	done
+	for pkg in $(basename $(notdir $(wildcard debian/*.requires))); do \
+	  dh_python3 -p $$pkg --requires=debian/$$pkg.requires;            \
+	done
 	dh_python3 -p python3-ceph-argparse
 	dh_python3 -p python3-ceph-common
 	dh_python3 -p cephfs-shell