Empowering AI agents to operate their own computers through APIs
______ ______ __ __ ______ ______ __ ______ __ __
/\ ___\ /\ __ \ /\ "-.\ \ /\ ___\ /\ __ \ /\ \ /\ ___\ /\ \_\ \
\ \ \____\ \ \/\ \\ \ \-. \\ \___ \\ \ \/\ \\ \ \____\ \ __\ \ \____ \
\ \_____\\ \_____\\ \_\\"\_\\/\_____\\ \_____\\ \_____\\ \_____\\/\_____\
\/_____/ \/_____/ \/_/ \/_/ \/_____/ \/_____/ \/_____/ \/_____/ \/_____/
Anthropic's Computer use feature allows users to operate their computer through tool calls, becoming a super agent for computer operations. However, in the official demo, the chat interface and desktop are integrated and cannot be used separately or accessed remotely through API calls.
ConsoleY is a Docker-based remote desktop environment that containerizes the Ubuntu desktop environment. It can not only be accessed through a browser but also provides remote control capabilities through an API service implemented in Rust, making it compatible with any local or cloud-based AI assistant that supports tool calls.
- 🖥️ Docker-based Ubuntu desktop environment, accessible via browser
- 🚀 Complete computer operation API service built with Rust, including desktop operations, file operations, Shell command execution, etc., with hot-reload support
- Docker
- Docker Desktop
- Docker Compose
# Clone the repository
git clone [email protected]:consoley/consoley.git
cd consoley
# Build development environment image
docker-compose build
# Start development environment container
docker-compose up -dAfter the Docker container starts, the API service will automatically start and can be accessed at http://localhost:8090. You can check the health status of the API service at http://localhost:8090/health.
The raw desktop GUI can be accessed at http://localhost:6070.
# Stop development environment container
docker-compose down# Install frontend dependencies
npm install
# Start desktop
npm run devAfter it, you can access the more controlled desktop through your browser at http://localhost:1420, which allows to toggle the "Allow human operation" switch.
The API interface design follows Anthropic's Computer use feature, supporting desktop operations, file operations, and Shell commands through a single endpoint. For more details, please refer to the API Documentation.