Skip to content

Commit 82e5298

Browse files
authored
Merge pull request RasaHQ#5416 from RasaHQ/rasa-init-tests
`rasa init` should include tests by default
2 parents f3a3fb7 + e3cfad6 commit 82e5298

File tree

9 files changed

+339
-276
lines changed

9 files changed

+339
-276
lines changed

changelog/5416.improvement.rst

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
Change ``rasa init`` to include ``tests/conversation_tests.md`` file by default.

docs/user-guide/evaluating-models.rst

Lines changed: 34 additions & 37 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,40 @@ Evaluating Models
1616
If you are looking to tune the hyperparameters of your NLU model,
1717
check out this `tutorial <https://blog.rasa.com/rasa-nlu-in-depth-part-3-hyperparameters/>`_.
1818

19+
.. _end_to_end_evaluation:
20+
21+
End-to-End Evaluation
22+
---------------------
23+
24+
Rasa Open Source lets you evaluate dialogues end-to-end, running through
25+
test conversations and making sure that both NLU and Core make correct predictions.
26+
27+
To do this, you need some stories in the end-to-end format,
28+
which includes both the NLU output and the original text.
29+
Here is an example:
30+
31+
.. code-block:: story
32+
33+
## end-to-end story 1
34+
* greet: hello
35+
- utter_ask_howcanhelp
36+
* inform: show me [chinese](cuisine) restaurants
37+
- utter_ask_location
38+
* inform: in [Paris](location)
39+
- utter_ask_price
40+
41+
By default Rasa saves tests to ``tests/conversation_tests.md``. You can evaluate your model
42+
against them by running:
43+
44+
.. code-block:: bash
45+
46+
$ rasa test
47+
48+
.. note::
49+
50+
Make sure your model file in ``models`` is a combined ``core``
51+
and ``nlu`` model. If it does not contain an NLU model, Core will use
52+
the default ``RegexInterpreter``.
1953

2054
.. _nlu-evaluation:
2155

@@ -227,40 +261,3 @@ you.
227261
.. note::
228262
This training process can take a long time, so we'd suggest letting it run
229263
somewhere in the background where it can't be interrupted.
230-
231-
232-
.. _end_to_end_evaluation:
233-
234-
End-to-End Evaluation
235-
---------------------
236-
237-
Rasa lets you evaluate dialogues end-to-end, running through
238-
test conversations and making sure that both NLU and Core make correct predictions.
239-
240-
To do this, you need some stories in the end-to-end format,
241-
which includes both the NLU output and the original text.
242-
Here is an example:
243-
244-
.. code-block:: story
245-
246-
## end-to-end story 1
247-
* greet: hello
248-
- utter_ask_howcanhelp
249-
* inform: show me [chinese](cuisine) restaurants
250-
- utter_ask_location
251-
* inform: in [Paris](location)
252-
- utter_ask_price
253-
254-
255-
If you've saved end-to-end stories as a file called ``e2e_stories.md``,
256-
you can evaluate your model against them by running:
257-
258-
.. code-block:: bash
259-
260-
$ rasa test --stories e2e_stories.md --e2e
261-
262-
.. note::
263-
264-
Make sure your model file in ``models`` is a combined ``core``
265-
and ``nlu`` model. If it does not contain an NLU model, Core will use
266-
the default ``RegexInterpreter``.

0 commit comments

Comments
 (0)