[docs] Enum & fixes

Test Plan:
in docs

  make livehtml

see no errors (other than ipynb files)

Reviewers: max, themissinghlink

Reviewed By: themissinghlink

Differential Revision: https://dagster.phacility.com/D1460

docs/sections/api/apidocs/types.rst

@@ -15,8 +15,8 @@ Built-in types
 .. attribute:: Any
 
     Use this type for any input, output, or config field whose type is unconstrained
-    
-    All values are considered to be instances of ``Any``. 
+
+    All values are considered to be instances of ``Any``.
 
     **Examples:**
 
@@ -166,7 +166,7 @@ Built-in types
 
     Use this type to indicate that a string input, output, or config value represents a path. At
     runtime, this will perform an ``isinstance(value, six.string_types)`` -- that is on Python 2,
-    both ``unicode`` and ``str`` will pass this check. 
+    both ``unicode`` and ``str`` will pass this check.
 
     **Examples:**
 
@@ -197,7 +197,7 @@ Built-in types
     it is necessary to use the explicit :py:class:`InputDefinition` API to define them rather than
     the Python 3 type hint syntax.
 
-    All values are considered to be instances of ``Nothing``. 
+    All values are considered to be instances of ``Nothing``.
 
     **Examples:**
 
@@ -207,7 +207,7 @@ Built-in types
         def wait(_) -> Nothing:
             time.sleep(1)
             return
-        
+
         @solid(
             InputDefinition('ready', dagster_type=Nothing)
         )
@@ -430,6 +430,10 @@ builtin types above.
 
 .. autofunction:: PermissiveDict
 
+.. autofunction:: Enum
+
+.. autofunction:: EnumValue
+
 -----
 
 Making New Types

docs/sections/deploying/airflow.rst

@@ -1,4 +1,4 @@
-.. airflow_:
+.. _airflow:
 
 Deploying to Airflow
 --------------------

docs/sections/learn/tutorial/hello_cereal.rst

@@ -92,6 +92,7 @@ You'll see the full stream of events emitted by dagster appear in the console, i
 call to the logging machinery, which will look like:
 
 .. code-block:: console
+
   :emphasize-lines: 1
 
     2019-10-10 11:46:50 - dagster - INFO - system - a91a4cc4-d218-4c2b-800c-aac50fced1a5 - Found 77 cereals
@@ -184,7 +185,7 @@ investigate, in detail, the success or failure of execution, the outputs produce
    :linenos:
    :caption: hello_cereal.py
    :lineno-start: 31
-   :lines: 31-41
+   :lines: 31-40
 
 Now you can use pytest, or your test runner of choice, to run unit tests as you develop your
 data applications.

docs/sections/learn/tutorial/intro_airflow.rst

@@ -100,7 +100,7 @@ More sophisticated setups
 
 This approach requires that Dagster, all of the Python requirements of your Dagster pipelines, and
 your Dagaster pipelines themselves be importable in the environment in which your Airflow tasks
-operate, because under the hood it uses Airflow's :py:class:`airflow:PythonOperator` to define 
+operate, because under the hood it uses Airflow's :py:class:`airflow:PythonOperator` to define
 tasks.
 
 ``dagster-airflow`` also includes facilities for compiling Dagster pipelines to Airflow DAGs whose

docs/sections/learn/tutorial/resources.rst

@@ -173,7 +173,7 @@ The first is to pass an ``environment_dict`` literal to the constructor. Because
 defined in Python, you can do arbitrary computation to construct it -- for instance, picking up
 environment variables, making a call to a secrets store like Hashicorp Vault, etc.
 
-The second is to use the ``from_files`` static constructor, and pass a list of file globs from 
+The second is to use the ``from_files`` static constructor, and pass a list of file globs from
 which to read YAML fragments. Order matters in this case, and keys from later files will overwrite
 keys from earlier files.
 
@@ -188,9 +188,9 @@ From the CLI, use ``-p`` or ``--preset``:
 From Python, you can use :py:func:`execute_pipeline_with_preset <dagster.execute_pipeline_with_preset>`:
 
 .. literalinclude:: ../../../../examples/dagster_examples/intro_tutorial/presets.py
-   :lines: 169
+   :lines: 170
    :dedent: 4
 
-And in Dagit, we can use the "Presets" selector. 
+And in Dagit, we can use the "Presets" selector.
 
 .. thumbnail:: presets.png

docs/sections/learn/tutorial/scheduler.rst

@@ -28,7 +28,7 @@ This is pluggable (and you can write your own), but for now the only implemented
 
 .. literalinclude:: ../../../../examples/dagster_examples/intro_tutorial/scheduler.py
    :linenos:
-   :lines: 38-48
+   :lines: 38-47
    :lineno-start: 38
    :caption: scheduler.py
 

docs/sections/learn/tutorial/types.rst

@@ -55,7 +55,7 @@ Now we can annotate the rest of our pipeline with our new type:
 
 .. literalinclude:: ../../../../examples/dagster_examples/intro_tutorial/custom_types.py
    :lines: 17-43
-   :emphasize-lines: 2, 7, 11 
+   :emphasize-lines: 2, 7, 11
    :linenos:
    :lineno-start: 17
    :caption: custom_types.py
@@ -146,7 +146,7 @@ What we want to be able to do is write:
 
 In order for the Dagster machinery to be able to decode the config value ``{'csv': 'cereal.csv'}``
 into an input of the correct ``LessSimpleDataFrame`` value, we need to write what we call an
-input hydration config. 
+input hydration config.
 
 .. literalinclude:: ../../../../examples/dagster_examples/intro_tutorial/custom_types_3.py
    :lines: 31-37
@@ -217,7 +217,7 @@ Let's see how to use this to emit some summary statistics about our DataFrame ty
    :lines: 17-69
    :linenos:
    :lineno-start: 17
-   :emphasize-lines: 3-9, 33-63
+   :emphasize-lines: 3-9, 33-53
    :caption: custom_types_4.py
 
 A :py:class:`TypeCheck <dagster.TypeCheck>` must include a ``success`` argument describing whether

python_modules/dagster-airflow/dagster_airflow_tests/test_project/build.sh

@@ -53,8 +53,6 @@ rm -rf \
   examples/build \
   examples/dist && \
 \
-# https://kind.sigs.k8s.io/docs/user/quick-start/#loading-an-image-into-your-cluster
-# Can't use a latest tag
 docker build -t dagster-airflow-demo . && \
 docker build -t dagster-airflow-webserver -f Dockerfile-airflow-webserver .
 

python_modules/dagster/dagster/core/types/config.py

@@ -332,6 +332,17 @@ def __init__(self):
 
 
 class EnumValue:
+    '''Define an entry in a :py:func:`Enum`.
+
+    Args:
+        config_value (str):
+            The string representation of the config to accept when passed.
+        python_value (Optional[Any]):
+            The python value to convert the enum entry in to. Defaults to the ``config_value``.
+        description (Optional[str])
+
+    '''
+
     def __init__(self, config_value, python_value=None, description=None):
         self.config_value = check.str_param(config_value, 'config_value')
         self.python_value = config_value if python_value is None else python_value
@@ -368,6 +379,32 @@ def to_python_value(self, config_value):
 
 
 def Enum(name, enum_values):
+    '''
+    Defines a enum configuration type that allows one of a defined set of possible values.
+
+    Args:
+        name (str):
+        enum_values (List[EnumValue]):
+
+    Example:
+        .. code-block:: python
+
+            @solid(
+                config_field=Field(
+                    Enum(
+                        'CowboyType',
+                        [
+                            EnumValue('good'),
+                            EnumValue('bad'),
+                            EnumValue('ugly'),
+                        ]
+                    )
+                )
+            )
+            def resolve_standoff(context):
+                # ...
+    '''
+
     class _EnumType(ConfigEnum):
         def __init__(self):
             super(_EnumType, self).__init__(name=name, enum_values=enum_values)

python_modules/dagster/dagster/core/types/field.py

@@ -62,38 +62,42 @@ def resolve_to_config_type(dagster_type):
 
 class Field(object):
     '''Defines the schema for a configuration field.
-    
+
     Config fields are parsed according to their schemas in order to yield values available at
     pipeline execution time through the config system. Config fields can be set on solids, on custom
     data types (as the :py:func:`@input_hydration_schema <dagster.input_hydration_schema>`), and on
     other pluggable components of the system, such as resources, loggers, and executors.
 
     Args:
-        dagster_type (Any): The type of this field. Users should provide one of the
+        dagster_type (Any):
+            The type of this field. Users should provide one of the
             :ref:`built-in types <builtin>`, a composite constructed using :py:func:`Selector`
             or :py:func:`PermissiveDict`, or a dagster type constructed with
             :py:func:`as_dagster_type`, :py:func:`@dagster_type <dagster_type`, or
             :py:func:`define_python_dagster_type` that has an ``input_hydration_config`` defined.
             Note that these constructs can be nested -- i.e., a :py:class:`Dict` can itself contain
             :py:class:`Fields <Field>` of other types, etc.
-        default_value (Any): A default value for this field, conformant to the schema set by the
+        default_value (Any):
+            A default value for this field, conformant to the schema set by the
             ``dagster_type`` argument. If a default value is provided, ``is_optional`` should be
             ``True``.
-        is_optional (bool): Whether the presence of this field is optional. If ``is_optional`` is
-            ``False``, no default value should be provided.
-        description (str): A human-readable description of this config field.
-Examples:
-    
-
-    .. code-block:: python
-
-        @solid(
-            config_field=Field(
-                Dict({'word': Field(String, default_value='foo'), 'repeats': Field(Int)})
+        is_optional (bool):
+            Whether the presence of this field is optional. If ``is_optional`` is ``False``, no
+            default value should be provided.
+        description (str):
+            A human-readable description of this config field.
+
+    Examples:
+        .. code-block:: python
+
+            @solid(
+                config_field=Field(
+                    Dict({'word': Field(String, default_value='foo'), 'repeats': Field(Int)})
+                )
             )
-        )
-        def repeat_word(context):
-            return context.solid_config['word'] * context.solid_config['repeats']
+            def repeat_word(context):
+                return context.solid_config['word'] * context.solid_config['repeats']
+
     '''
 
     def __init__(