init

.gitignore

@@ -0,0 +1,27 @@
+*.egg-info
+
+**/__pycache__
+
+workspace_cache/
+
+XAgent/logs/
+running_records/
+
+XAgentServer/localstorage
+XAgentServer/logs/
+XAgentServer/prod_server_envs.py
+
+.DS_Store
+.vscode
+.idea
+*_running_records
+
+data/**
+data/server.db
+ToolServerNode/logs/**
+**/workspace/**
+download_tmp_file
+local_workspace/**
+!local_workspace/readme.md
+sync.py
+assets/private.yml

README.md

@@ -0,0 +1,158 @@
+<div align= "center">
+    <h1> 🪢 XAgent</h1>
+</div>
+
+<div align="center">
+
+
+[![Discord](https://dcbadge.vercel.app/api/server/xagent?style=flat)](https://discord.gg/xagent) [![Twitter](https://img.shields.io/twitter/follow/xagent?style=social)](https://twitter.com/XAgent) [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
+
+</div>
+
+
+<p align="center">
+  <a href="#Quickstart">Tutorial</a> •
+  <a href="#Demo">Demo</a> •
+  <a href="#Blog">Blog</a> •
+  <a href="#Citation">Citation</a>
+
+
+</p>
+
+</div>
+
+
+# 📖 Introduction
+XAgent is a open-source experimental Large Language Model (LLM) driven autonomous agent that can automatically solve various tasks. 
+It is designed to be a general-purpose agent that can be applied to a wide range of tasks. Currently, XAgent is still in its early stage, and we are working hard to improve it.
+
+🏆 Our goal is to create a super-intelligent agent that can solve any given task!
+
+## 🪢 XAgent
+XAgent is designed with the following features:
+- **Autonomous**: XAgent can automatically solve various tasks without human participation.
+- **Safety**: XAgent is designed to run safely. All actions are constrained inside a docker container. Run it anyway!
+- **Extensible**: XAgent is designed to be extensible. You can easily add new tools to enhance agent's abilities, and even new agents！
+- **GUI**: XAgent provides a friendly GUI for users to interact with the agent. You can also use the command line interface to interact with the agent.
+- **Cooperation with Human**: XAgent can collaborate with you to tackle tasks. It not only has the capability to follow your guidance in solving complex tasks on the go, but also can seek your assistance when it encounters challenges.
+
+XAgent is composed of three parts:
+- **🤖 Dispatcher** is responsible for dynmically instantiate and dispatching tasks to different agents. It allows us to add new agents and improve the agents' abilities.
+- **🧐 Planner** is responsible for generate and rectify plans for tasks. It divides a task into subtasks and generates milestones for them, allowing agents to solve tasks step by step.
+- **🦾 Actor** is responsible for conducting actions to achieve goals and finish subtasks. The actor utilizes various tools to solve subtasks, and it can also collaborate with human to solve tasks.
+
+## 🧰 ToolServer
+ToolServer is the server provide XAgent with powerful and safe tools to solve tasks. It is a docker container that provides a safe environment for XAgent to run.
+Currently, ToolServer provides the following tools:
+- **📝 File Editor** provide a text editing tool that can write, read, and modify files.
+- **📘 Python Notebook** provide a interactive python notebook that can run python code to validate ideas, draw figures, etc.
+- **🌏 Web Browser** provide a web browser that can search and visit webpages.
+- **🖥️ Shell** provide a bash shell tool that can execute any shell commands, even install programs and host services.
+- **🧩 Rapid API** provide a tool to retrieve apis from Rapid API and calling them, which provides a wide range of apis for XAgent to use. See [ToolBench](https://github.com/OpenBMB/ToolBench) to get more information about the Rapid API collections.
+You can also easily add new tools to ToolServer to enhance agent's abilities.
+
+<div><a id="Quickstart"></a></div>
+
+# ✨ Quickstart
+## 🛠️ Build and Setup ToolServer
+ToolServer is where XAgent's action takes place. It is a docker container that provides a safe environment for XAgent to run.
+So you should install `docker` and `docker-compose` first. 
+After that, you should build the docker image for ToolServer and start the docker container.
+```bash
+cd ToolServer
+bash build_all.sh && docker-compose up
+```
+Refer [here](ToolServer/README.md) for the detailed information about our ToolServer.
+
+## 🎮 Setup and Run XAgent
+After setting up ToolServer, you can start to run XAgent.
+- Install requirements (Require Python >= 3.10)
+```bash
+pip install -r requirements.txt
+```
+
+- Configure XAgent
+You should configure XAgent in `config.yml` before running it. 
+At least one of openai key is provided in `config.yml`, which is used to access OpenAI API.
+We highly recommand to use `gpt-4-32k` to run XAgent, `gpt-4` is also ok for most simple tasks.
+In any case, at least one `gpt-3.5-turbo-16k` api key should be provided as a backup model.
+We do not test or recommand to use `gpt-3.5-turbo` to run XAgent due to very limited context length, you should not try to run XAgent on that.
+
+- Run XAgent
+```bash
+python run.py --task "put your task here" --model "gpt-4"
+```
+The local workspace for your XAgent is in `local_workspace`, where you can find all the files generated by XAgent throughout the running process. Besides, in `running_records` you can find all the intermediate steps information, e.g. task statuses, LLM's input-output pairs, used tools, etc.
+
+- Run XAgent with GUI
+```bash
+bash XAgentServer/dockerfiles/build.sh
+cd XAgentServer
+docker compose up
+docker exec XAgent-Server systemctl start nginx
+```
+Build the docker image for XAgent-Server and start the docker container.
+You will see the XAgent Server listening on port `8000`.
+Refer [here](XAgentServer/README.md) for the detailed information about our GUI Demo.
+
+<div><a id="Demo"></a></div>
+
+# 🎬 Demo
+You can check our live demo on [XAgent Official Website](https://www.x-agent.net/).
+
+We also provide a video demo of using XAgent here.
+
+<div align="center">
+
+<video width="320" height="240" controls>
+  <source src="assets/demo.mp4" type="video/mp4">
+</video>
+
+</div>
+
+Here we also show some cases of XAgent solving tasks:
+## Case 1. Data Analysis: Demonstrating the Effectiveness of Dual-Loop Mechanism
+We start with a case of aiding users in intricate data analysis. Here, our user submitted an `iris.zip` file to XAgent, seeking assistance in data analysis. XAgent swiftly broke down the task into four sub-tasks: (1) data inspection and comprehension, (2) verification of the system's Python environment for relevant data analysis libraries, (3) crafting data analysis code for data processing and analysis, and (4) compiling an analytical report based on the Python code's execution results.
+Here is a figure draw by XAgent.
+![Data Statics by XAgent](assets/readme/statistics.png)
+
+
+## Case 2. Recommendation: A New Paradigm of Human-Agent Interaction
+Empowered with the unique capability to actively seek human assistance and collaborate in problem-solving, XAgent continues to redefine the boundaries of human-agent cooperation. As depicted in srceenshot below, a user sought XAgent's aid in recommending some great restaurants for a friendly gathering, yet failed to provide specific details. Recognizing the insufficiency of the provided information, XAgent employed the AskForHumanHelp tool, prompting human intervention to elicit the user's preferred location, budget constraints, culinary preferences, and any dietary restrictions. Armed with this valuable feedback, XAgent seamlessly generated tailored restaurant recommendations, ensuring a personalized and satisfying experience for the user and their friends.
+![Illustration of Ask for Human Help of XAgent](assets/readme/ask_for_human_help.png)
+
+## Case 3. Training Model: A Sophisticated Tool User
+XAgent not only tackles mundane tasks but also serves as an invaluable aid in complex tasks such as model training. Here we show a scenario where a user desires to analyze movie reviews and evaluate the public sentiment surrounding particular films. In response, XAgent promptly initiates the process by downloading the IMDB dataset to train a cutting-edge BERT model (see screenshot below), harnessing the power of deep learning. Armed with this trained BERT model, XAgent seamlessly navigates the intricate nuances of movie reviews, offering insightful predictions regarding the public's perception of various films.
+![bert_1](assets/readme/bert_1.png)
+![bert_2](assets/readme/bert_2.png)
+![bert_3](assets/readme/bert_3.png)
+
+## 📊 Evaluation
+We conduct human preference evaluation to evaluate XAgent's performance. We prepare over 50 realworld complex tasks for evaluation, which can be categorized into 5 class: Search and Report, Coding and Developing, Data Analysis, Math and Life Assistant.
+We compare the result of XAgent with [AutoGPT](https://github.com/Significant-Gravitas/AutoGPT), the result shows a total win of XAgent over AutoGPT. You can download records for XAgent [here](https://drive.google.com/drive/folders/1_slFNXUBQ1CGiNLMCCYoQXZYNGAAdHok?usp=sharing).
+
+![HumanPrefer](assets/readme/agent_comparison.png)
+
+We report a significant improvement of XAgent over AutoGPT in term of human preference.
+
+We also evaluate XAgent on the following benchmarks:
+![Benchmarks](assets/readme/eval_on_dataset.png)
+
+
+<div><a id="Blog"></a></div>
+
+# 🖌️ Blog
+
+Coming Soon.
+
+<div><a id="Citation"></a></div>
+
+# Citation
+If you find our repo useful, please kindly consider cite:
+```angular2
+@misc{xagent2023,
+      title={XAgent: An Autonomous Agent for Complex Task Solving}, 
+      author={XAgent Team},
+      year={2023},
+}
+```

ToolServer/.env

@@ -0,0 +1,3 @@
+DB_USERNAME=admin
+DB_PASSWORD=xagentmongodb
+DB_COLLECTION=TSM

ToolServer/README.md

@@ -0,0 +1,189 @@
+# 🧰 ToolServer
+
+ToolServer is the server provide XAgent with powerful and safe tools to solve tasks. It is a docker container that provides a safe environment for XAgent to run.
+
+ToolServer is composed of three parts:
+- **ToolServerManager** is responsible for creating and managing ToolServerNode instances.
+- **ToolServerMonitor** is responsible for monitoring the status of ToolServerNode instances. Automatic detect instances status and removing the instances that are not working properly.
+- **ToolServerNode** is responsible for providing tools to solve tasks. It is a docker container that provides a safe environment for XAgent to run.
+
+Currently, ToolServer provides the following tools:
+- **📝 File Editor** provide a text editing tool that can write, read, and modify files.
+- **📘 Python Notebook** provide a interactive python notebook that can run python code to validate ideas, draw figures, etc.
+- **🌏 Web Browser** provide a web browser that can search and visit webpages.
+- **🖥️ Shell** provide a bash shell tool that can execute any shell commands, even install programs and host services.
+- **🧩 Rapid API** provide a tool to retrieve apis from Rapid API and calling them, which provides a wide range of apis for XAgent to use. See [ToolBench](https://github.com/OpenBMB/ToolBench) to get more information about the Rapid API collections.
+You can also easily add new tools to ToolServer to enhance agent's abilities.
+
+## ⚡️ Configurations
+Configurations for ToolServer are stored in `ToolServer/config/`. You can change them and rebuild images to apply the changes.
+Notes:
+- Change `node.privileged` to `false` in `manager.yml` if you don't want to used docker in ToolServerNode. This will disable the ability to run docker commands in ToolServerNode.
+- Change `idling_close_minutes` in `monitor.yml` to change the time that ToolServerMonitor will wait before closing idle ToolServerNode instances.
+- Add your api keys in `node.yml` to enable bing search and rapid api.
+- Change api timeout for Toolserver in `docker-compose.yml` by altering values after `-t` in `services.ToolServerManager.command` if you encounter timeout error of ToolServer.
+
+## 🛠️ Build and Setup ToolServer
+All docker image build files are stored in `ToolServer/dockerfiles`, together with the `build.sh` script. 
+You can build them one by one, or build all of them with following command:
+```bash
+bash ToolServer/build_all.sh
+```
+This will build all the docker images for ToolServerManager, ToolServerMonitor and ToolServerNode.
+
+After building the docker images, you can start the docker container with following command:
+```bash
+cd ToolServer
+docker-compose up
+```
+Note that you should install `docker` and `docker-compose` first.
+
+## 🧩 API Documentation
+### /get_cookies
+This path will return a cookie that contains the node_id of the ToolServerNode instance.
+All the following requests should use this cookie to identify the ToolServerNode instance.
+
+### /get_available_tools
+This path will return all registered tools in ToolServerNode, together with their parameters.
+```JSON
+{
+    "available_envs":[
+        {
+            "name":"env1",
+            "description":"description1",
+            "tools":["tool1","tool2"] //at most 50 tools, the rest will not be returned
+        },
+    ],
+    "available_tools":[
+        "tool1",
+        "tool2", //hidden tools will not be returned
+    ],
+    "tools_json":[
+        {
+            "name":"tool1",
+            "description":"description1",
+            "parameters":{
+                "type":"object",
+                "properties":{
+                    "param1":{
+                        "type":"string",
+                        "description":"description1"
+                    },
+                    "param2":{
+                        "type":"integer",
+                        "description":"description2"
+                    }
+                },
+                "required":["param1","param2"]
+            }
+        },
+    ]
+}
+```
+
+### /retrieving_tools
+Giving a question, return related tools. Rapid API will also be returned.
+Arguments:
+```JSON
+{
+    "question":"question",
+    "top_k":10
+}
+```
+Return:
+```JSON
+{
+    "retrieved_tools":[
+        "tool1",
+        "tool2"
+    ],
+    "tools_json":[
+        {
+            //tool1 json
+        },
+        {
+            //tool2 json
+        }
+    ]
+}
+```
+
+### /get_json_schema_for_tools
+Return the json schema for the given tools.
+Arguments:
+```JSON
+{
+    "tools":["tool1","tool2"]
+}
+```
+Return:
+```JSON
+{
+    "tools_json":[
+        {
+            //tool1 json
+        },
+        {
+            //tool2 json
+        }
+    ],
+    "missing_tools":[
+
+    ]
+}
+```
+
+### /get_json_schema_for_envs
+Return the json schema for the given envs.
+Arguments:
+```JSON
+{
+    "envs":["env1","env2"]
+}
+```
+Return:
+```JSON
+{
+    "envs_json":[
+        {
+            "name":"env1",
+            "description":"description1",
+            "tools":["tool1","tool2"]
+        }
+    ],
+    "missing_envs":[
+
+    ]
+}
+```
+
+### /execute_tool
+Execute the given tool with the given parameters.
+Arguments:
+```JSON
+{
+    "tool":"tool1",
+    "parameters":{
+        "param1":"value1",
+        "param2":"value2"
+    }
+}
+```
+Return is dependent on the tool.
+The return http code 450 standfor need further calling to finish tool execution.
+When return http code 450, the return value will be like:
+```JSON
+{
+    "detail":{
+        "type":"retry",
+        "next_calling":"ShellEnv_read_stdout",
+        "arguments":{}
+    }
+}
+```
+
+### /close_session
+Close the ToolServerNode instance.
+
+### /release_session
+Close and delete ToolServerNode instance.