Merge branch 'master' into fix-url-reminders-link

Author: m-vdb

CHANGELOG.rst

@@ -17,6 +17,26 @@ This project adheres to `Semantic Versioning`_ starting with version 1.0.
 
 .. towncrier release notes start
 
+[1.9.7] - 2020-04-23
+^^^^^^^^^^^^^^^^^^^^
+
+Improvements
+------------
+- `#4606 <https://github.com/rasahq/rasa/issues/4606>`_: The stream reading timeout for ``rasa shell` is now configurable by using the
+  environment variable ``RASA_SHELL_STREAM_READING_TIMEOUT_IN_SECONDS``.
+  This can help to fix problems when using ``rasa shell`` with custom actions which run
+  10 seconds or longer.
+
+Bugfixes
+--------
+- `#5709 <https://github.com/rasahq/rasa/issues/5709>`_: Reverted changes in 1.9.6 that led to model incompatibility. Upgrade to 1.9.7 to fix 
+  ``self.sequence_lengths_for(tf_batch_data[TEXT_SEQ_LENGTH][0]) IndexError: list index out of range`` 
+  error without needing to retrain earlier 1.9 models.
+
+  Therefore, all 1.9 models `except for 1.9.6` will be compatible; a model trained on 1.9.6 will need
+  to be retrained on 1.9.7.
+
+
 [1.9.6] - 2020-04-15
 ^^^^^^^^^^^^^^^^^^^^
 

changelog/3419.improvement.rst

@@ -0,0 +1,3 @@
+Include the source filename of a story in the failed stories
+
+Include the source filename of a story in the failed stories to make it easier to identify the file which contains the failed story.

changelog/3765.feature.rst

@@ -0,0 +1,18 @@
+Add support for entities with roles and grouping of entities in Rasa NLU.
+
+You can now define a role and/or group label in addition to the entity type for entities.
+Use the role label if an entity can play different roles in your assistant.
+For example, a city can be a destination or a departure city.
+The group label can be used to group multiple entities together.
+For example, you could group different pizza orders, so that you know what toppings goes with which pizza and
+what size which pizza has.
+For more details see :ref:`entities-roles-groups`.
+
+To fill slots from entities with a specific role/group, you need to either use forms or use a custom action.
+We updated the tracker method ``get_latest_entity_values`` to take an optional role/group label.
+If you want to use a form, you can add the specific role/group label of interest to the slot mapping function
+``from_entity`` (see :ref:`forms``).
+
+.. note::
+
+    Composite entities are currently just supported by the :ref:``diet-classifier`` and :ref:``CRFEntityExtractor``.

changelog/5230.bugfix.rst

@@ -1 +1 @@
-Fixed issue where posting to certain callback channel URLs would return a 500 error on successful posts due to invalid response format
+Fixed issue where posting to certain callback channel URLs would return a 500 error on successful posts due to invalid response format.

changelog/5465.feature.rst

@@ -0,0 +1,29 @@
+Update training data format for NLU to support entities with a role or group label.
+
+You can now specify synonyms, roles, and groups of entities using the following data format:
+Markdown:
+
+.. code-block:: none
+
+  [LA]{"entity": "location", "role": "city", "group": "CA", "value": "Los Angeles"}
+
+JSON:
+
+.. code-block:: none
+
+  "entities": [
+      {
+          "start": 10,
+          "end": 12,
+          "value": "Los Angeles",
+          "entity": "location",
+          "role": "city",
+          "group": "CA",
+      }
+  ]
+
+The markdown format ``[LA](location:Los Angeles)`` is deprecated. To update your training data file just
+execute the following command on the terminal of your choice:
+``sed -i .deprecated -E 's/\[(.*)\]\((.*):(.*)\)/\[\1\]\{"entity": "\2", "value": "\3"\}/g' nlu.md``
+
+For more information about the new data format see :ref:`training-data-format`.

changelog/5556.misc.rst renamed to changelog/5556.misc.rst

@@ -0,0 +1 @@
+Remove regularization gradient for variables that don't have prediction gradient.

changelog/5669.improvement.rst

@@ -0,0 +1,2 @@
+Raise a warning in ``CRFEntityExtractor`` and ``DIETClassifier`` if entities are not correctly annotated in the
+training data, e.g. their start and end values do not match any start and end values of tokens.

changelog/5672.improvement.rst

@@ -0,0 +1,3 @@
+Change default values for hyper-parameters in ``EmbeddingIntentClassifier`` and ``DIETClassifier``
+
+Use ``scale_loss=False`` in ``DIETClassifier``. Reduce the number of dense dimensions for sparse features of text from 512 to 256 in ``EmbeddingIntentClassifier``.

changelog/5717.improvement.rst

@@ -0,0 +1,56 @@
+## intent:affirm
+- yes
+- yep
+- yeah
+- indeed
+- that's right
+- ok
+- great
+- right, thank you
+- correct
+- great choice
+- sounds really good
+
+## intent:goodbye
+- bye
+- goodbye
+- good bye
+- stop
+- end
+- farewell
+- Bye bye
+- have a good one
+
+## intent:greet
+- hey
+- howdy
+- hey there
+- hello
+- hi
+- good morning
+- good evening
+- dear sir
+
+## intent:chitchat
+- What's your name?
+- What can I call you?
+- How's the weather?
+- Is it too hot outside?
+
+## intent:book_flight
+- i'm looking for a flight
+- I want to book a flight
+- i'm looking for a flight to [Berlin]{"entity": "location", "role": "to"}
+- show me flights from [Amsterdam]{"entity": "location", "role": "from"}
+- show me flights to [London]{"entity": "location", "role": "to"}
+- i am looking for a flight from [SF]{"entity": "location", "value": "San Fransisco", "role": "from"} to [New York]{"entity": "location", "role": "to"}
+- search for flights
+- from [Madrid]{"entity": "location", "role": "from"} to [Munich]{"entity": "location", "role": "to"}
+- any flight to [Liverpool]{"entity": "location", "role": "to"}
+
+## intent:order_pizza
+- i want a [large]{"entity": "size", "group": "1"} pizza with [tomato]{"entity": "topping", "group": "1"} and a [small]{"entity": "size", "group": "2"} pizza with [bacon]{"entity": "topping", "group": "2"}
+- one [large]{"entity": "size", "group": "1"} with [pepperoni]{"entity": "topping", "group": "1"} and a [medium]{"entity": "size", "group": "2"} with [mushrooms]{"entity": "topping", "group": "2"}
+- I would like a [medium]{"entity": "size", "group": "1"} standard pizza and a [medium]{"entity": "size", "group": "2"} pizza with [extra cheese]{"entity": "topping", "group": "2"}
+- [large]{"entity": "size", "group": "1"} with [onions]{"entity": "topping", "group": "1"} and [small]{"entity": "size", "group": "2"} with [olives]{"entity": "topping", "group": "2"}
+- a pizza with [onions]{"entity": "topping", "group": "1"} in [medium]{"entity": "size", "group": "1"} and one with [mushrooms]{"entity": "topping", "group": "2"} in [small]{"entity": "size", "group": "2"} please