Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[hailctl] make the output of dry-run copy-pasteable (hail-is#14127)
Here's a diff of `hailctl dataproc start foo --dry-run` on main and on this branch. Notice that the properties and metadata arguments gain a leading and trailing single quote. This ensure that things like `sys_platform!="win32"` are properly transmitted. In `start.py` we just use exec-style invocation, so there's no equivalent issue. ``` 7c7 < --properties=^|||^spark:spark.task.maxFailures=20|||spark:spark.driver.extraJavaOptions=-Xss4M|||spark:spark.executor.extraJavaOptions=-Xss4M|||spark:spark.speculation=true|||hdfs:dfs.replication=1|||dataproc:dataproc.logging.stackdriver.enable=false|||dataproc:dataproc.monitoring.stackdriver.enable=false|||spark:spark.driver.memory=36g|||yarn:yarn.nodemanager.resource.memory-mb=29184|||yarn:yarn.scheduler.maximum-allocation-mb=14592|||spark:spark.executor.cores=4|||spark:spark.executor.memory=5837m|||spark:spark.executor.memoryOverhead=8755m|||spark:spark.memory.storageFraction=0.2|||spark:spark.executorEnv.HAIL_WORKER_OFF_HEAP_MEMORY_PER_CORE_MB=3648 \ --- > '--properties=^|||^spark:spark.task.maxFailures=20|||spark:spark.driver.extraJavaOptions=-Xss4M|||spark:spark.executor.extraJavaOptions=-Xss4M|||spark:spark.speculation=true|||hdfs:dfs.replication=1|||dataproc:dataproc.logging.stackdriver.enable=false|||dataproc:dataproc.monitoring.stackdriver.enable=false|||spark:spark.driver.memory=36g|||yarn:yarn.nodemanager.resource.memory-mb=29184|||yarn:yarn.scheduler.maximum-allocation-mb=14592|||spark:spark.executor.cores=4|||spark:spark.executor.memory=5837m|||spark:spark.executor.memoryOverhead=8755m|||spark:spark.memory.storageFraction=0.2|||spark:spark.executorEnv.HAIL_WORKER_OFF_HEAP_MEMORY_PER_CORE_MB=3648' \ 9c9 < --metadata=^|||^WHEEL=gs://hail-30-day/hailctl/dataproc/dking-dev/0.2.126-a51eabd65859/hail-0.2.126-py3-none-any.whl|||PKGS=aiodns==2.0.0|aiohttp==3.9.1|aiosignal==1.3.1|async-timeout==4.0.3|attrs==23.1.0|avro==1.11.3|azure-common==1.1.28|azure-core==1.29.5|azure-identity==1.15.0|azure-mgmt-core==1.4.0|azure-mgmt-storage==20.1.0|azure-storage-blob==12.19.0|bokeh==3.3.1|boto3==1.33.1|botocore==1.33.1|cachetools==5.3.2|certifi==2023.11.17|cffi==1.16.0|charset-normalizer==3.3.2|click==8.1.7|commonmark==0.9.1|contourpy==1.2.0|cryptography==41.0.7|decorator==4.4.2|deprecated==1.2.14|dill==0.3.7|frozenlist==1.4.0|google-auth==2.23.4|google-auth-oauthlib==0.8.0|humanize==1.1.0|idna==3.6|isodate==0.6.1|janus==1.0.0|jinja2==3.1.2|jmespath==1.0.1|jproperties==2.1.1|markupsafe==2.1.3|msal==1.25.0|msal-extensions==1.0.0|msrest==0.7.1|multidict==6.0.4|nest-asyncio==1.5.8|numpy==1.26.2|oauthlib==3.2.2|orjson==3.9.10|packaging==23.2|pandas==2.1.3|parsimonious==0.10.0|pillow==10.1.0|plotly==5.18.0|portalocker==2.8.2|protobuf==3.20.2|py4j==0.10.9.5|pyasn1==0.5.1|pyasn1-modules==0.3.0|pycares==4.4.0|pycparser==2.21|pygments==2.17.2|pyjwt[crypto]==2.8.0|python-dateutil==2.8.2|python-json-logger==2.0.7|pytz==2023.3.post1|pyyaml==6.0.1|regex==2023.10.3|requests==2.31.0|requests-oauthlib==1.3.1|rich==12.6.0|rsa==4.9|s3transfer==0.8.0|scipy==1.11.4|six==1.16.0|sortedcontainers==2.4.0|tabulate==0.9.0|tenacity==8.2.3|tornado==6.3.3|typer==0.9.0|typing-extensions==4.8.0|tzdata==2023.3|urllib3==1.26.18|uvloop==0.19.0;sys_platform!="win32"|wrapt==1.16.0|xyzservices==2023.10.1|yarl==1.9.3 \ --- > '--metadata=^|||^WHEEL=gs://hail-30-day/hailctl/dataproc/dking-dev/0.2.126-a51eabd65859/hail-0.2.126-py3-none-any.whl|||PKGS=aiodns==2.0.0|aiohttp==3.9.1|aiosignal==1.3.1|async-timeout==4.0.3|attrs==23.1.0|avro==1.11.3|azure-common==1.1.28|azure-core==1.29.5|azure-identity==1.15.0|azure-mgmt-core==1.4.0|azure-mgmt-storage==20.1.0|azure-storage-blob==12.19.0|bokeh==3.3.1|boto3==1.33.1|botocore==1.33.1|cachetools==5.3.2|certifi==2023.11.17|cffi==1.16.0|charset-normalizer==3.3.2|click==8.1.7|commonmark==0.9.1|contourpy==1.2.0|cryptography==41.0.7|decorator==4.4.2|deprecated==1.2.14|dill==0.3.7|frozenlist==1.4.0|google-auth==2.23.4|google-auth-oauthlib==0.8.0|humanize==1.1.0|idna==3.6|isodate==0.6.1|janus==1.0.0|jinja2==3.1.2|jmespath==1.0.1|jproperties==2.1.1|markupsafe==2.1.3|msal==1.25.0|msal-extensions==1.0.0|msrest==0.7.1|multidict==6.0.4|nest-asyncio==1.5.8|numpy==1.26.2|oauthlib==3.2.2|orjson==3.9.10|packaging==23.2|pandas==2.1.3|parsimonious==0.10.0|pillow==10.1.0|plotly==5.18.0|portalocker==2.8.2|protobuf==3.20.2|py4j==0.10.9.5|pyasn1==0.5.1|pyasn1-modules==0.3.0|pycares==4.4.0|pycparser==2.21|pygments==2.17.2|pyjwt[crypto]==2.8.0|python-dateutil==2.8.2|python-json-logger==2.0.7|pytz==2023.3.post1|pyyaml==6.0.1|regex==2023.10.3|requests==2.31.0|requests-oauthlib==1.3.1|rich==12.6.0|rsa==4.9|s3transfer==0.8.0|scipy==1.11.4|six==1.16.0|sortedcontainers==2.4.0|tabulate==0.9.0|tenacity==8.2.3|tornado==6.3.3|typer==0.9.0|typing-extensions==4.8.0|tzdata==2023.3|urllib3==1.26.18|uvloop==0.19.0;sys_platform!="win32"|wrapt==1.16.0|xyzservices==2023.10.1|yarl==1.9.3' \ ```
- Loading branch information