[v0.1.5] Add LLM caching, audio support, tree-of-thoughts and virtual…

… tool example, ...

- Add persistent Database for caching LLM calls
- Support Audio as part of the prompt
- Support using `gen` outside `ppl` function, similar usage as litellm's `completion`
- Add example reimplementing tree-of-thoughts with parallelization (6x speedup)
- Add example for (simplified) emulate tools using LLMs and function docstring
- Allow using schema dict to specify available tools for LLMs
- Allow specifying docstring as `SystemMessage` in the `ppl` decorator function
- Simplified the example for defining concepts in prompts
- Add tests for caching and `gen` outside `ppl` function
- some re-organize for imports

README.md

@@ -153,6 +153,7 @@ For a more comprehensive tutorial, please refer to the [tutorial](https://appl-t
 For more detailed usage and examples, please refer to the [cookbook](https://appl-team.github.io/appl/cookbook).
 
 APPL can be used to reproduce some popular LM-based applications easily, such as:
+* [Tree of Thoughts](https://github.com/princeton-nlp/tree-of-thought-llm)[[APPL implementation](examples/advanced/tree_of_thoughts/)]: deliberate problem solving with Large Language Models.
 * [Wordware's TwitterPersonality](https://twitter.wordware.ai/)[[APPL implementation](https://github.com/appl-team/TwitterPersonality)]: analyzes your tweets to determine your Twitter personality.
 
 ## Citation and Acknowledgment

docs/tutorials/3_appl_function.md

@@ -18,11 +18,11 @@ As you have seen in the [QA examples](./2_qa_example.md), you can define prompts
 ??? question "How about docstrings in *APPL functions*?"
     Docstring is a special expression statement in Python. There are two cases for docstrings in *APPL functions*:
 
-    1. If the docstring is triple-quoted, it will **NOT** be captured as a prompt by default. To also include the docstring as a part of the prompt, you may specify `include_docstring=True` in the `@ppl` decorator, like
+    1. If the docstring is triple-quoted, it will **NOT** be captured as a prompt by default. To also include the docstring as a part of the prompt (as a system message or user message), you may specify `docstring_as` to be "system" or "user" in the `@ppl` decorator, like
         ```python
-        @ppl(include_docstring=True)
+        @ppl(docstring_as="system")
         def my_function():
-            """Docstring as a part of the prompt.
+            """Docstring as system message.
 
             Details."""
             "Another prompt."
@@ -32,12 +32,12 @@ As you have seen in the [QA examples](./2_qa_example.md), you can define prompts
         ```
         Outputs:
         ```
-        Docstring as a part of the prompt.
+        Docstring as system message.
 
         Details.
         Another prompt.
         ```
-    1. Otherwise, it will be captured as a prompt. But if the content is not meant to be the docstring of the function, it is recommended to use f-string instead.
+    2. Otherwise, it will be captured as a prompt. But if the content is not meant to be the docstring of the function, it is recommended to use f-string instead.
         ```python
         @ppl
         def my_function():

docs/tutorials/6_prompt_coding.md

@@ -21,7 +21,7 @@ You can also nest the compositors to create more complex structures, like:
 ```python linenums="1"
 @ppl
 def compose(items: list[str]):
-    with Tagged("div"):  # (1)
+    with Tagged("div", indent_inside=4):  # (1)
         "Display the items:"
         with NumberedList(indent=4): # (2)
             items # (3)
@@ -30,7 +30,7 @@ def compose(items: list[str]):
 print(compose(["item1", "item2"]))
 ```
 
-1. The default indentation inside the tag is 4 spaces, can be changed by setting the `indent_inside` parameter.
+1. The default is no indentation inside the tag, can be changed by setting the `indent_inside` parameter.
 2. Explicitly set the indentation for the content inside the list.
 3. Fits different number of items.
 
@@ -81,10 +81,10 @@ With the help of modern IDEs like VSCode, you can easily navigate to the definit
 We use the example in [PromptCoder](https://github.com/dhh1995/PromptCoder?tab=readme-ov-file#usage) to illustrate the usage of these prompt coding helpers:
 
 ```python linenums="1"
-import appl
 from appl import BracketedDefinition as Def
-from appl import define, empty_line, ppl, records
+from appl import define, ppl, records
 from appl.compositor import *
+from appl.const import EMPTY
 
 # (1)
 class InputReq(Def):
@@ -94,40 +94,26 @@ class OutputReq(Def):
     name = "Output Requirement"
 
 @ppl
-def requirements(opr: str):
-    "Requirements"
+def get_prompt(opr: str, language: str):
+    "## Requirements"
     with NumberedList():
         InputReq(desc="The input should be two numbers.") # (2)
         OutputReq(desc=f"The output should be the {opr} of the two numbers.") # (3)
-    return records()
-
-@ppl
-def instruction(language: str):
-    "Instruction"
-    with LineSeparated():
-        f"Write a function in {language} that satisfies the {InputReq} and {OutputReq}." # (4)
-    return records()
-
-@ppl
-def get_prompt(opr: str, language: str):
-    with LineSeparated(indexing="##"):
-        requirements(opr)  # (5)
-        empty_line()  # (6)
-        instruction(language)
+    EMPTY
+    "## Instruction"
+    f"Write a function in {language} that satisfies the {InputReq} and {OutputReq}." # (4)
     return records()
 ```
 
 1. Declare the input and output requirements classes for reference.
     Alternatively, but not recommended, you can define classes as follows:
     `InputReq = define("Input Requirement")` and `OutputReq = define("Output Requirement")`.
     But then VSCode cannot recognize it as a class.
-2. Complete the input requirement with a description.
-3. Complete the output requirement with a description.
+2. Instantiate the input requirement with a description.
+3. Instantiate the output requirement with a description.
 4. The naming can be used to distinguish:
     - variable naming (e.g. language): the dynamic input.
     - class naming (e.g. InputReq): the reference to the concept.
-5. The returned prompt will be formatted using the compositor.
-6. Create an empty line regardless of other compositors.
 
 The result of `get_prompt("sum", "Python")` will be:
 ```md

docs/tutorials/7_tracing.md

@@ -1,6 +1,21 @@
-# Using Tracing
+# Using Caching and Tracing
 
-APPL supports tracing APPL functions and LM calls to facilitate users to understand and debug the program executions. The trace is useful for reproducing (potentially partial) execution results by loading cached responses of the LM calls, which enables failure recovery and avoids the extra costs of resending these calls. This also unlocks the possibility of conveniently debugging one specific LM call out of the whole program.
+APPL supports tracing APPL functions and LM calls to facilitate users to understand and debug the program executions. Both the trace and the persistent LLM caching are useful for reproducing (potentially partial) execution results by loading cached responses of the LM calls, which enables failure recovery and avoids the extra costs of resending these calls. This also unlocks the possibility of conveniently debugging one specific LM call out of the whole program.
+
+## Enabling APPL Caching
+The persistent LLM caching (default path: `~/.appl/caches/cache.db`) is automatically enabled since v0.1.5.
+LLM calls with temperature 0 will look up the cache first, use the cached responses if found, generate and cache the responses otherwise.
+
+```yaml title="appl.yaml"
+settings:
+  caching:
+    enabled: true # default to enable the caching
+    folder: "~/.appl/caches" # The folder to store the cache files
+    max_size: 100000  # Maximum number of entries in cache
+    time_to_live: 43200 # Time-to-live in minutes (30 days)
+    cleanup_interval: 1440 # Cleanup interval in minutes (1 day)
+    allow_temp_greater_than_0: false # Whether to cache the generation results with temperature to be greater than 0
+```
 
 ## Enabling APPL Tracing
 
@@ -38,7 +53,7 @@ You can reproduce the execution results from a previous trace by specifying the
 $ APPL_RESUME_TRACE=<path to the trace file> python answer_questions.py
 ```
 
-Then each LM call will be loaded from the trace file if it exists. Such loading can be useful for:
+Then each LM call will be loaded from the trace file if it exists (loading from the trace is of higher priority than the persistent cache). Such loading can be useful for:
 
 - Debugging a specific LM call: the LM calls before that can be loaded from the trace file, therefore no need to resend them with extra costs.
 - Reproducible results: the trace file can be shared with others to reproduce the same results.