JSON-SCHEMA AS THE EXPERIMENT CONFIG DEFINITION

Linting:
  - There is a linter which enforces various custom checks to ensure that the
    assumptions we make in our system will be enforced by the schemas
  - The linter also enforces basic formatting requirements

Validation:
  - Validation happens in two steps:
    - Sanity check: make sure everything is parsable
    - Validation: make sure everything is present
  - Sanity check is applied to raw user inputs
  - Validation is applied after cluster defaults have been applied
  - The implementation difference between sanity check and validation is the
    inclusion of a single eventuallyRequired schema in the latter.

Null handling:
  - There is no pythonic or golangic way to represent values which were
    provided in the configuration as literal nulls, rather than values which
    were not provided at all.  In theory, you could have singleton
    "NotProvided" pointers, which you would check for every time that you
    checked a value in the schema, but in practice that would be a pain in the
    ass.  Additionally, the golang json-schema library we use treats
    not-present values a nil values anyway.

    Conclusion: null values shall be treated as if they were not present.

Default values:
  - "default" refers to "encoded in json-schema" and not "cluster default"
  - otherwise, there's no way to do client-side defaults, as the master would
    need to be involved in deciding all default values.

Customization strategy:
  - Json-schema alone does not provide the behavior we want out of the box
  - Most json-schema libraries have great support for custom validation logic
  - We have json-schema extensions for the following keywords:
    - checks: Custom messages for arbitrary validation logic
    - compareProperties: Support customizable value comparisons
    - conditional: Enforce a schema whenever some condition is met.
    - disallowProperties: Custom messages when disallowing properties
    - eventuallyRequired: Support two-step validation
    - union: Excellent error messages when validating union types
    - optionalRef: like $ref, but only enforced for non-null values
  - The canonical implementations (with thorough comments) may be found in
    determined_common/schemas/extensions.py

Migrating the experiment config:
  - See pkg/schemas/expconf/parse.go for an example of shimming logic.
  - Bumping the version is only necessary when previously-valid configs become
    no longer valid.  For instance, removing a once-available fields requires a
    version bump, but appending a new field or accepting a new type in an
    existing field does not.
  - To bump versions, take the following steps (assume you are adding v2):
      - create a new `v2` directory, like `schemas/expconf/v2`
      - copy the json schemas files you are updating from the v1 directory into
        the v2 directory.  Also copy any files which referenced any of those
        files; for example, if you update the v1/hyperparameters.json, you'll
        have to copy v1/experiment-config.json as well since its $ref will need
        to now point to the v2/hyperparameters.json.
      - Populate test_cases/v2 with test cases to the new json schemas you just
        made.
      - Make the golang changes:
          - Create the corresponding objects in pkg/schemas/expconf, such as a
            new ExperimentConfigV2 and new HyperparametersV2
          - update expconf/latest.go to point to the correct new object
          - update any uses of that object throughout the codebase to reflect
            the new structure.
          - ensure that `make gen && go test ./pkg/schemas/expconf` passes
      - Currently there are no python or JS changes needed.