Merge branch 'master' into 1.4.x-merge

Author: wochinge

CHANGELOG.rst

@@ -7,6 +7,33 @@ Rasa Change Log
 All notable changes to this project will be documented in this file.
 This project adheres to `Semantic Versioning`_ starting with version 1.0.
 
+
+[Unreleased 1.5.0a1] - `master`_
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Added
+-----
+- Added data validator that checks if domain object returned is empty. If so, exit early from the command ``rasa data validate``
+- Added the KeywordIntentClassifier
+- Added documentation for ``AugmentedMemoizationPolicy``
+- Fall back to ``InMemoryTrackerStore`` in case there is any problem with the current
+  tracker store
+
+Changed
+-------
+- Do not retrain the entire Core model if only the ``templates`` section of the domain is changed.
+- Upgraded ``jsonschema`` version
+
+Removed
+-------
+
+Fixed
+-----
+- ``MultiProjectImporter`` now imports files in the order of the import statements
+- Fixed server hanging forever on leaving ``rasa shell`` before first message
+- Fixed rasa init showing traceback error when user does Keyboard Interrupt before choosing a project path
+- ``CountVectorsFeaturizer`` featurizes intents only if its analyzer is set to ``word``
+
 [Unreleased 1.4.4]
 ^^^^^^^^^^^^^^^^^^
 
@@ -241,7 +268,6 @@ Changed
 -------
 - Pin gast to == 0.2.2
 
-
 [1.3.0] - 2019-09-05
 ^^^^^^^^^^^^^^^^^^^^
 

Makefile

@@ -46,7 +46,7 @@ types:
 	pytype --keep-going rasa
 
 prepare-tests-macos: prepare-tests-files
-	brew install graphviz
+	brew install graphviz wget
 
 prepare-tests-ubuntu: prepare-tests-files
 	sudo apt-get -y install graphviz graphviz-dev python3-tk

data/test_config/keyword_classifier_config.yml

@@ -0,0 +1,2 @@
+pipeline:
+  - name: "KeywordIntentClassifier"

docs/core/knowledge-bases.rst

@@ -358,7 +358,7 @@ The default mapping looks like:
           "8": lambda l: l[7],
           "9": lambda l: l[8],
           "10": lambda l: l[9],
-          "ANY": lambda l: random.choice(list),
+          "ANY": lambda l: random.choice(l),
           "LAST": lambda l: l[-1],
       }
 
@@ -526,7 +526,7 @@ You can customize your ``InMemoryKnowledgeBase`` by overwriting the following fu
           "8": lambda l: l[7],
           "9": lambda l: l[8],
           "10": lambda l: l[9],
-          "ANY": lambda l: random.choice(list),
+          "ANY": lambda l: random.choice(l),
           "LAST": lambda l: l[-1],
       }
 

docs/core/policies.rst

@@ -417,6 +417,24 @@ training data. It predicts the next action with confidence ``1.0``
 if this exact conversation exists in the training data, otherwise it
 predicts ``None`` with confidence ``0.0``.
 
+Augmented Memoization Policy
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The ``AugmentedMemoizationPolicy`` remembers examples from training
+stories for up to ``max_history`` turns, just like the ``MemoizationPolicy``.
+Additionally, it has a forgetting mechanism that will forget a certain amount
+of steps in the conversation history and try to find a match in your stories
+with the reduced history. It predicts the next action with confidence ``1.0``
+if a match is found, otherwise it predicts ``None`` with confidence ``0.0``.
+
+.. note::
+
+  If you have dialogues where some slots that are set during
+  prediction time might not be set in training stories (e.g. in training
+  stories starting with a reminder not all previous slots are set),
+  make sure to add the relevant stories without slots to your training
+  data as well.
+
 .. _fallback-policy:
 
 Fallback Policy

docs/core/stories.rst

@@ -176,12 +176,6 @@ at a time):
    in the main stories, the first event can be an action or an utterance
    as well.
 
-.. note::
-   Unlike regular stories, checkpoints are not restricted to starting with an
-   input from the user. As long as the checkpoint is inserted at the right points
-   in the main stories, the first event can be an action or an utterance
-   as well.
-
 
 OR Statements
 ~~~~~~~~~~~~~

docs/nlu/components.rst

@@ -275,23 +275,8 @@ CountVectorsFeaturizer
 Intent Classifiers
 ------------------
 
-KeywordIntentClassifier
-~~~~~~~~~~~~~~~~~~~~~~~
 
-:Short: Simple keyword matching intent classifier. Not intended to be used.
-:Outputs: ``intent``
-:Requires: nothing
-:Output-Example:
-
-    .. code-block:: json
-
-        {
-            "intent": {"name": "greet", "confidence": 0.98343}
-        }
 
-:Description:
-    This classifier is mostly used as a placeholder. It is able to recognize `hello` and
-    `goodbye` intents by searching for these keywords in the passed messages.
 
 MitieIntentClassifier
 ~~~~~~~~~~~~~~~~~~~~~
@@ -485,6 +470,41 @@ EmbeddingIntentClassifier
               See `starspace paper <https://arxiv.org/abs/1709.03856>`_ for details.
 
 
+.. _keyword_intent_classifier:
+
+KeywordIntentClassifier
+~~~~~~~~~~~~~~~~~~~~~~~
+
+:Short: Simple keyword matching intent classifier, intended for small, short-term projects.
+:Outputs: ``intent``
+:Requires: nothing
+
+:Output-Example:
+
+    .. code-block:: json
+
+        {
+            "intent": {"name": "greet", "confidence": 1.0}
+        }
+
+:Description:
+    This classifier works by searching a message for keywords.
+    The matching is case sensitive by default and searches only for exact matches of the keyword-string in the user message.
+    The keywords for an intent are the examples of that intent in the NLU training data.
+    This means the entire example is the keyword, not the individual words in the example.
+
+    .. note:: This classifier is intended only for small projects or to get started. If
+              you have few NLU training data you can use one of our pipelines
+              :ref:`choosing-a-pipeline`.
+
+:Configuration:
+
+    .. code-block:: yaml
+
+        pipeline:
+        - name: "KeywordIntentClassifier"
+          case_sensitive: True
+
 Selectors
 ----------
 
@@ -496,6 +516,7 @@ Response Selector
 :Short: Response Selector
 :Outputs: A dictionary with key as ``direct_response_intent`` and value containing ``response`` and ``ranking``
 :Requires: A featurizer
+
 :Output-Example:
 
     .. code-block:: json

docs/user-guide/validate-files.rst

@@ -41,6 +41,8 @@ actions.
 
 **verify_all():** Runs all verifications above.
 
+**verify_domain_validity():** Check if domain is valid.
+
 To use these functions it is necessary to create a `Validator` object and initialize the logger. See the following code:
 
 .. code-block:: python

examples/moodbot/data/nlu.md

@@ -14,11 +14,11 @@
 - good afternoon
 
 ## intent:goodbye
+- good afternoon
 - cu
 - good by
 - cee you later
 - good night
-- good afternoon
 - bye
 - goodbye
 - have a nice day
@@ -42,7 +42,6 @@
 
 ## intent:mood_great
 - perfect
-- very good
 - great
 - amazing
 - feeling like a king
@@ -51,7 +50,7 @@
 - I am great
 - I am amazing
 - I am going to save the world
-- super
+- super stoked
 - extremely good
 - so so perfect
 - so good
@@ -67,7 +66,7 @@
 - sad
 - very sad
 - unhappy
-- not so good
+- not good
 - not very good
 - extremly sad
 - so saad

rasa/cli/data.py

@@ -103,5 +103,9 @@ def validate_files(args):
     )
 
     validator = loop.run_until_complete(Validator.from_importer(file_importer))
+    domain_is_valid = validator.verify_domain_validity()
+    if not domain_is_valid:
+        sys.exit(1)
+
     everything_is_alright = validator.verify_all(not args.fail_on_warnings)
     sys.exit(0) if everything_is_alright else sys.exit(1)