@@ -16,6 +16,40 @@ Evaluating Models
16
16
If you are looking to tune the hyperparameters of your NLU model,
17
17
check out this `tutorial <https://blog.rasa.com/rasa-nlu-in-depth-part-3-hyperparameters/ >`_.
18
18
19
+ .. _end_to_end_evaluation :
20
+
21
+ End-to-End Evaluation
22
+ ---------------------
23
+
24
+ Rasa Open Source lets you evaluate dialogues end-to-end, running through
25
+ test conversations and making sure that both NLU and Core make correct predictions.
26
+
27
+ To do this, you need some stories in the end-to-end format,
28
+ which includes both the NLU output and the original text.
29
+ Here is an example:
30
+
31
+ .. code-block :: story
32
+
33
+ ## end-to-end story 1
34
+ * greet: hello
35
+ - utter_ask_howcanhelp
36
+ * inform: show me [chinese](cuisine) restaurants
37
+ - utter_ask_location
38
+ * inform: in [Paris](location)
39
+ - utter_ask_price
40
+
41
+ By default Rasa saves tests to ``tests/conversation_tests.md ``. You can evaluate your model
42
+ against them by running:
43
+
44
+ .. code-block :: bash
45
+
46
+ $ rasa test
47
+
48
+ .. note ::
49
+
50
+ Make sure your model file in ``models `` is a combined ``core ``
51
+ and ``nlu `` model. If it does not contain an NLU model, Core will use
52
+ the default ``RegexInterpreter ``.
19
53
20
54
.. _nlu-evaluation :
21
55
@@ -227,40 +261,3 @@ you.
227
261
.. note ::
228
262
This training process can take a long time, so we'd suggest letting it run
229
263
somewhere in the background where it can't be interrupted.
230
-
231
-
232
- .. _end_to_end_evaluation :
233
-
234
- End-to-End Evaluation
235
- ---------------------
236
-
237
- Rasa lets you evaluate dialogues end-to-end, running through
238
- test conversations and making sure that both NLU and Core make correct predictions.
239
-
240
- To do this, you need some stories in the end-to-end format,
241
- which includes both the NLU output and the original text.
242
- Here is an example:
243
-
244
- .. code-block :: story
245
-
246
- ## end-to-end story 1
247
- * greet: hello
248
- - utter_ask_howcanhelp
249
- * inform: show me [chinese](cuisine) restaurants
250
- - utter_ask_location
251
- * inform: in [Paris](location)
252
- - utter_ask_price
253
-
254
-
255
- If you've saved end-to-end stories as a file called ``e2e_stories.md ``,
256
- you can evaluate your model against them by running:
257
-
258
- .. code-block :: bash
259
-
260
- $ rasa test --stories e2e_stories.md --e2e
261
-
262
- .. note ::
263
-
264
- Make sure your model file in ``models `` is a combined ``core ``
265
- and ``nlu `` model. If it does not contain an NLU model, Core will use
266
- the default ``RegexInterpreter ``.
0 commit comments