Skip to content

Commit 35eb0a1

Browse files
committed
Write section about JSON schema in changelog.
Also fix markdownlint errors
1 parent df5ab70 commit 35eb0a1

File tree

2 files changed

+50
-12
lines changed

2 files changed

+50
-12
lines changed

CHANGELOG.md

+42-4
Original file line numberDiff line numberDiff line change
@@ -2,9 +2,43 @@
22

33
## v1.10dev
44

5-
### Tools helper code
5+
### Pipeline schema
66

7-
* Allow multiple container tags in `ci.yml` if performing multiple tests in parallel
7+
This release of nf-core/tools introduces a major change / new feature: pipeline schema.
8+
These are [JSON Schema](https://json-schema.org/) files that describe all of the parameters for a given
9+
pipeline with their ID, a description, a longer help text, an optional default value, a variable _type_
10+
(eg. `string` or `boolean`) and more.
11+
12+
The files will be used in a number of places:
13+
14+
* Automatic validation of supplied parameters when running pipelines
15+
* Pipeline execution can be immediately stopped if a required `param` is missing,
16+
or does not conform to the patterns / allowed values in the schema.
17+
* Generation of pipeline command-line help
18+
* Running `nextflow run <pipeline> --help` will use the schema to generate a help text automatically
19+
* Building online documentation on the [nf-core website](https://nf-co.re)
20+
* Integration with 3rd party graphical user interfaces
21+
22+
To support these new schema files, nf-core/tools now comes with a new set of commands: `nf-core schema`.
23+
24+
* Pipeline schema can be generated or updated using `nf-core schema build` - this takes the parameters from
25+
the pipeline config file and prompts the developer for any mismatch between schema and pipeline.
26+
* Once a skeleton Schema file has been built, the command makes use of a new nf-core website tool to provide
27+
a user friendly graphical interface for developers to add content to their schema: [https://nf-co.re/json_schema_build](https://nf-co.re/json_schema_build)
28+
* Pipelines will be automatically tested for valid schema that describe all pipeline parameters using the
29+
`nf-core schema lint` command (also included as part of the main `nf-core lint` command).
30+
* Users can validate their set of pipeline inputs using the `nf-core schema validate` command.
31+
32+
In addition to the new schema commands, the `nf-core launch` command has been completely rewritten from
33+
scratch to make use of the new pipeline schema. This command can use either an interactive command-line
34+
prompt or a rich web interface to help users set parameters for a pipeline run.
35+
36+
The parameter descriptions and help text are fully used and embedded into the launch interfaces to make
37+
this process as user-friendly as possible. We hope that it's particularly well suited to those new to nf-core.
38+
39+
Whilst we appreciate that this new feature will add a little work for pipeline developers, we're excited at
40+
the possibilities that it brings. If you have any feedback or suggestions, please let us know either here on
41+
GitHub or on the nf-core [`#json-schema` Slack channel](https://nfcore.slack.com/channels/json-schema).
842

943
### Template
1044

@@ -15,6 +49,7 @@
1549
* Update `output.md` and add in 'Pipeline information' section describing standard NF and pipeline reporting.
1650
* Build Docker image using GitHub Actions, then push to Docker Hub (instead of building on Docker Hub)
1751
* New Slack channel badge in pipeline readme
52+
* Allow multiple container tags in `ci.yml` if performing multiple tests in parallel
1853
* Add AWS CI tests and full tests GitHub Actions workflows
1954

2055
### Linting
@@ -29,12 +64,15 @@
2964
* Linting code now automatically posts warning / failing results to GitHub PRs as a comment if it can
3065
* Added AWS GitHub Actions workflows linting
3166

32-
### Other
67+
### nf-core/tools Continuous Integration
3368

3469
* Added CI test to check for PRs against `master` in tools repo
3570
* CI PR branch tests fixed & now automatically add a comment on the PR if failing, explaining what is wrong
36-
* Describe alternative installation method via conda with `conda env create`
3771
* Move some of the issue and PR templates into HTML `<!-- comments -->` so that they don't show in issues / PRs
72+
73+
### Other
74+
75+
* Describe alternative installation method via conda with `conda env create`
3876
* Added `macs_gsize` for danRer10, based on [this post](https://biostar.galaxyproject.org/p/18272/)
3977
* nf-core/tools version number now printed underneath header artwork
4078
* Bumped Conda version shipped with nfcore/base to 4.8.2

README.md

+8-8
Original file line numberDiff line numberDiff line change
@@ -226,18 +226,18 @@ Do you want to run this command now? [y/N]: n
226226
### Launch tool options
227227

228228
* `-c`, `--command-only`
229-
* If you prefer not to save your inputs in a JSON file and use `-params-file`, this option will specify all entered params directly in the nextflow command.
229+
* If you prefer not to save your inputs in a JSON file and use `-params-file`, this option will specify all entered params directly in the nextflow command.
230230
* `-p`, `--params-in PATH`
231-
* To use values entered in a previous pipeline run, you can supply the `nf-params.json` file previously generated.
232-
* This will overwrite the pipeline schema defaults before the wizard is launched.
231+
* To use values entered in a previous pipeline run, you can supply the `nf-params.json` file previously generated.
232+
* This will overwrite the pipeline schema defaults before the wizard is launched.
233233
* `-o`, `--params-out PATH`
234-
* Path to save parameters JSON file to. (Default: `nf-params.json`)
234+
* Path to save parameters JSON file to. (Default: `nf-params.json`)
235235
* `-a`, `--save-all`
236-
* Without this option the pipeline will ignore any values that match the pipeline schema defaults.
237-
* This option saves _all_ parameters found to the JSON file.
236+
* Without this option the pipeline will ignore any values that match the pipeline schema defaults.
237+
* This option saves _all_ parameters found to the JSON file.
238238
* `-h`, `--show-hidden`
239-
* A pipeline JSON schema can define some parameters as 'hidden' if they are rarely used or for internal pipeline use only.
240-
* This option forces the wizard to show all parameters, including those labelled as 'hidden'.
239+
* A pipeline JSON schema can define some parameters as 'hidden' if they are rarely used or for internal pipeline use only.
240+
* This option forces the wizard to show all parameters, including those labelled as 'hidden'.
241241

242242
## Downloading pipelines for offline use
243243

0 commit comments

Comments
 (0)