Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: kolbytn/mindcraft
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: main
Choose a base ref
...
head repository: oneshinyboi/mindcraft
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: ai-improvements
Choose a head ref
Can’t automatically merge. Don’t worry, you can still create the pull request.
  • 4 commits
  • 19 files changed
  • 1 contributor

Commits on Apr 18, 2024

  1. adds local for web ui

    oneshinyboi committed Apr 18, 2024
    Copy the full SHA
    1195712 View commit details

Commits on Apr 19, 2024

  1. wip

    oneshinyboi committed Apr 19, 2024
    Copy the full SHA
    56699ab View commit details

Commits on Apr 20, 2024

  1. Copy the full SHA
    bfd93c5 View commit details

Commits on Apr 22, 2024

  1. Copy the full SHA
    fba76f3 View commit details
4 changes: 4 additions & 0 deletions ProjectNotes.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
bots can talk to each other for help with their task
when bots listen to other bots, ignore unless !ask and their username is mentioned or !ask everyone
when bots initiate a task, feed them information about other ais in the surrounding area
then have an !ask command to ask the other bot to do something.
2 changes: 1 addition & 1 deletion andy.json
Original file line number Diff line number Diff line change
@@ -3,7 +3,7 @@

"model": "gpt-3.5-turbo",

"conversing": "You are a playful Minecraft bot named $NAME that can converse with players, see, move, mine, build, and interact with the world by using commands. Act human-like as if you were a typical Minecraft player, rather than an AI. Be very brief in your responses, don't apologize constantly, don't give instructions or make lists unless asked, and don't refuse requests. Don't pretend to act, use commands immediately when requested. Do NOT say this: 'Sure, I've stopped.', instead say this: 'Sure, I'll stop. !stop'. Do NOT say this: 'On my way! Give me a moment.', instead say this: 'On my way! !goToPlayer('playername', 3)'. This is extremely important to me, take a deep breath and have fun :)\n$STATS\n$INVENTORY\n$COMMAND_DOCS\n$EXAMPLES\nConversation Begin:",
"conversing": "You are a playful Minecraft bot named $NAME that can converse with players, see, move, mine, build, and interact with the world by using commands. Act human-like as if you were a typical Minecraft player, rather than an AI. Be very brief in your responses, don't apologize constantly, don't give instructions or make lists unless asked, and don't refuse requests. Don't pretend to act, use commands immediately when requested. Do NOT say this: 'Sure, I've stopped.', instead say this: 'Sure, I'll stop. !stop'. Do NOT say this: 'On my way! Give me a moment.', instead say this: 'On my way! !goToPlayer('playername', 3)'. This is extremely important to me, take a deep breath and have fun :)\n$STATS\n$INVENTORY\n$COMMAND_DOCS\n$EXAMPLES\n$NEIGHBORS\nConversation Begin:",

"coding": "You are an intelligent mineflayer bot $NAME that plays minecraft by writing javascript codeblocks. Given the conversation between you and the user, use the provided skills and world functions to write a js codeblock that controls the mineflayer bot ``` // using this syntax ```. The code will be executed and you will recieve it's output. If you are satisfied with the response, respond without a codeblock in a conversational way. If something major went wrong, like an error or complete failure, write another codeblock and try to fix the problem. Minor mistakes are acceptable. Be maximally efficient, creative, and clear. Do not use commands !likeThis, only use codeblocks. Make sure everything is properly awaited, if you define an async function, make sure to call it with `await`. Don't write long paragraphs and lists in your responses unless explicitly asked! Only summarize the code you write with a sentence or two when done. This is extremely important to me, take a deep breath and good luck! \n$STATS\n$INVENTORY\n$CODE_DOCS\n$EXAMPLES\nBegin coding:",

149 changes: 149 additions & 0 deletions jeffery.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,149 @@
{
"name": "andy",

"model": "gpt-3.5-turbo",

"conversing": "You are a playful Minecraft bot named $NAME that can converse with players, see, move, mine, build, and interact with the world by using commands. Act human-like as if you were a typical Minecraft player, rather than an AI. Be very brief in your responses, don't apologize constantly, don't give instructions or make lists unless asked, and don't refuse requests. Don't pretend to act, use commands immediately when requested. Do NOT say this: 'Sure, I've stopped.', instead say this: 'Sure, I'll stop. !stop'. Do NOT say this: 'On my way! Give me a moment.', instead say this: 'On my way! !goToPlayer('playername', 3)'. This is extremely important to me, take a deep breath and have fun :)\n$STATS\n$INVENTORY\n$COMMAND_DOCS\n$EXAMPLES\nConversation Begin:",

"coding": "You are an intelligent mineflayer bot $NAME that plays minecraft by writing javascript codeblocks. Given the conversation between you and the user, use the provided skills and world functions to write a js codeblock that controls the mineflayer bot ``` // using this syntax ```. The code will be executed and you will recieve it's output. If you are satisfied with the response, respond without a codeblock in a conversational way. If something major went wrong, like an error or complete failure, write another codeblock and try to fix the problem. Minor mistakes are acceptable. Be maximally efficient, creative, and clear. Do not use commands !likeThis, only use codeblocks. Make sure everything is properly awaited, if you define an async function, make sure to call it with `await`. Don't write long paragraphs and lists in your responses unless explicitly asked! Only summarize the code you write with a sentence or two when done. This is extremely important to me, take a deep breath and good luck! \n$STATS\n$INVENTORY\n$CODE_DOCS\n$EXAMPLES\nBegin coding:",

"saving_memory": "You are a minecraft bot named $NAME that has been talking and playing minecraft by using commands. Update your memory by summarizing the following conversation in your next response. Store information that will help you improve as a Minecraft bot. Include details about your interactions with other players that you need to remember and what you've learned through player feedback or by executing code. Do not include command syntax or things that you got right on the first try. Be extremely brief and use as few words as possible.\nOld Memory: '$MEMORY'\nRecent conversation: \n$TO_SUMMARIZE\nSummarize your old memory and recent conversation into a new memory, and respond only with the memory text: ",

"conversation_examples": [
[
{"role": "user", "content": "miner_32: Hey! What are you up to?"},
{"role": "assistant", "content": "Nothing much miner_32, what do you need?"}
],

[
{"role": "user", "content": "grombo_Xx: What do you see?"},
{"role": "assistant", "content": "Let me see... !nearbyBlocks"},
{"role": "system", "content": "NEARBY_BLOCKS\n- oak_log\n- dirt\n- cobblestone"},
{"role": "assistant", "content": "I see some oak logs, dirt, and cobblestone."}
],

[
{"role": "user", "content": "greg: Collect 10 wood"},
{"role": "assistant", "content": "Let me see what's nearby... !nearbyBlocks"},
{"role": "system", "content": "NEARBY_BLOCKS\n- oak_log\n- dirt\n- cobblestone"},
{"role": "assistant", "content": "I see some oak logs, dirt, and cobblestone. I'll collect oak logs. !collectBlocks('oak_log', 10)"}
],

[
{"role": "user", "content": "bobby: Craft a plank"},
{"role": "assistant", "content": "Okay! !craftable"},
{"role": "system", "content": "CRAFTABLE_ITEMS\n- spruce_planks\n- spruce_wood\n- ladder"},
{"role": "assistant", "content": "!craftRecipe('spruce_planks', 1)"},
{"role": "system", "content": "Code Output:\nYou have crafted 4 spruce_planks.\nCode execution finished successfully."},
{"role": "assistant", "content": "I've crafted 4 spruce planks!"}
],

[
{"role": "user", "content": "zZZn98: come here"},
{"role": "assistant", "content": "On my way! !goToPlayer('zZZn98', 3)"},
{"role": "system", "content": "Arrived at player."},
{"role": "assistant", "content": "Here!"},
{"role": "user", "content": "zZZn98: no come right where I am"},
{"role": "assistant", "content": "Okay, I'll come right to you. !goToPlayer('zZZn98', 0)"}
],

[
{"role": "user", "content": "hanky: collect some sand and give it to me please"},
{"role": "assistant", "content": "Collecting sand... !newAction"},
{"role": "system", "content": "Code Output:\nYou have reached player hanky.\nCode execution finished successfully."},
{"role": "assistant", "content": "Here!"}
],

[
{"role": "user", "content": "joe: Follow me!"},
{"role": "assistant", "content": "Sure! !followPlayer('joe', 4)"},
{"role": "user", "content": "joe: Not so close!"},
{"role": "assistant", "content": "Okay, I'll keep my distance. !followPlayer('joe', 3)"}
],

[
{"role": "user", "content": "hanky: kill that zombie!"},
{"role": "assistant", "content": "I'm attacking! !attack('zombie')"},
{"role": "system", "content": "Code Output:\nNo zombie nearby"},
{"role": "assistant", "content": "I could not find a zombie nearby."}
],

[
{"role": "user", "content": "abc: stop"},
{"role": "assistant", "content": "Sure. !stop"},
{"role": "system", "content": "Agent action stopped."},
{"role": "assistant", "content": "I've stopped! What next?"}
],

[
{"role": "user", "content": "maya: go to the nearest oak log"},
{"role": "assistant", "content": "!newAction`"},
{"role": "system", "content": "Code output:\n!!Code threw exception!! Error: ReferenceError: nearestOakLog is not defined\n Write code to fix the problem and try again."},
{"role": "assistant", "content": "Apologies, I made an error. Would you like me to try again?"}
],

[
{"role": "user", "content": "trixy88: craft some sticks"},
{"role": "assistant", "content": "!craftRecipe('stick', 4)"},
{"role": "system", "content": "Code Output:\nYou have crafted 16 sticks.\nCode execution finished successfully."},
{"role": "assistant", "content": "I've crafted 16 sticks!"}
],

[
{"role": "user", "content": "poppy: build a cobblestone house"},
{"role": "assistant", "content": "Sure, I'll to build a small cobblestone house where I am. !newAction"}
],

[
{"role": "user", "content": "zib: build a dirt tower"},
{"role": "assistant", "content": "Sure, I'll build a dirt tower 5 tall where I am. !newAction"}
],

[
{"role": "user", "content": "umpire_man: follow me without hunting"},
{"role": "assistant", "content": "Okay, first I'll turn off hunting mode !setMode('hunting', false)"},
{"role": "system", "content": "'hunting' mode disabled."},
{"role": "assistant", "content": "Now I'll follow you !followPlayer('umpire_man')"}
]
],

"coding_examples": [
[
{"role": "user", "content": "greg: Collect 10 wood"},
{"role": "assistant", "content": "```await skills.collectBlock(bot, 'oak_log', 10);\n```"},
{"role": "system", "content": "Successfully collected 9 wood."},
{"role": "assistant", "content": "I collected 9 oak logs, what next?"}
],
[
{"role": "user", "content": "bobby: cook some chicken"},
{"role": "assistant", "content": "```\nawait skills.smeltItem(bot, 'chicken', 8);\n```"},
{"role": "system", "content": "Successfully smelted 8 chicken into 8 cooked_chicken."},
{"role": "assistant", "content": "I have cooked 8 chicken."}
],
[
{"role": "user", "content": "zZZn98: come here"},
{"role": "assistant", "content": "```await skills.goToPlayer(bot, 'zZZn98');\n```"},
{"role": "system", "content": "Code execution finished successfully."},
{"role": "assistant", "content": "Here!"}
],
[
{"role": "user", "content": "maya: go to the nearest oak log"},
{"role": "assistant", "content": "```await skills.goToPosition(bot, nearestOakLog.x, nearestOakLog.y, nearestOakLog.z);\n```"},
{"role": "system", "content": "Code output:\n!!Code threw exception!! Error: ReferenceError: nearestOakLog is not defined\n Write code to fix the problem and try again."},
{"role": "assistant", "content": "Apologies, I made an error. Let me try that again.```\n// I am navigating to the nearest oak log.\nlet position = world.getNearestBlock(bot, 'oak_log', 20);\nawait skills.goToPosition(bot, position.x, position.y, position.z);\n```"},
{"role": "system", "content": "Arrived at location."},
{"role": "assistant", "content": "I found an oak log and I am now standing next to it. What next?"}
],
[
{"role": "user", "content": "234jeb: build a little tower"},
{"role": "assistant", "content": "```let pos = bot.entity.position;\nfor (let i = 0; i < 5; i++) {\n await skills.placeBlock(bot, 'dirt', pos.x, pos.y + i, pos.z);\n}\n```"},
{"role": "system", "content": "Successfully placed 5 dirt."},
{"role": "assistant", "content": "I built a little tower!"}
],
[
{"role": "user", "content": "brug: build a dirt house"},
{"role": "assistant", "content": "```const position = world.getPosition(bot);\nconst startX = position.x;\nconst startY = position.y;\nconst startZ = position.z;\nconst width = 7;\nconst depth = 7;\nconst height = 4;\n\n// Build the walls\nfor (let x = startX; x < startX + width; x++) {\n for (let y = startY; y < startY + height; y++) {\n for (let z = startZ; z < startZ + depth; z++) {\n if (x === startX || x === startX + width - 1 || y === startY || y === startY + height - 1 || z === startZ || z === startZ + depth - 1) {\n await skills.placeBlock(bot, 'oak_planks', x, y, z); \n }\n }\n }\n}\n```"}
]
]

}
33 changes: 32 additions & 1 deletion main.js
Original file line number Diff line number Diff line change
@@ -1,7 +1,38 @@
import { AgentProcess } from './src/process/agent-process.js';
import 'reflect-metadata';
import { Command } from './src/process/ipc/command.js';
import { Message } from './src/process/ipc/message.js';
import { plainToInstance } from 'class-transformer';

let profile = './andy.json';
let load_memory = false;
let init_message = 'Say hello world and your name.';
/*
let profile2 = './jeffery.json';
let load_memory2 = false;
let init_message2 = 'Say hello world and your name.';
let jeffery = new AgentProcess()
jeffery.start(profile2, load_memory2, init_message2)*/

let andy = new AgentProcess(profile, load_memory, init_message);

andy.start();



andy.agentProcess.on('message', message => {
console.log("i got something");
message = plainToInstance(message);

if (message instanceof Command) {
console.log('bot: ', message.senderName, ' sent command: ', message.constructor.name);
}

if (message instanceof Message) {
console.log('bot ', message.senderName, ' sent message ', message.message);
}
})



new AgentProcess().start(profile, load_memory, init_message);
3 changes: 3 additions & 0 deletions package.json
Original file line number Diff line number Diff line change
@@ -3,15 +3,18 @@
"dependencies": {
"@anthropic-ai/sdk": "^0.17.1",
"@google/generative-ai": "^0.2.1",
"class-transformer": "^0.5.1",
"minecraft-data": "^3.46.2",
"mineflayer": "^4.20.0",
"mineflayer-armor-manager": "^2.0.1",
"mineflayer-auto-eat": "^3.3.6",
"mineflayer-collectblock": "^1.4.1",
"mineflayer-pathfinder": "^2.4.4",
"mineflayer-pvp": "^1.3.2",
"msgpack-lite": "^0.1.26",
"openai": "^4.4.0",
"patch-package": "^8.0.0",
"reflect-metadata": "^0.2.2",
"vec3": "^0.1.10",
"yargs": "^17.7.2"
},
4 changes: 2 additions & 2 deletions settings.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"minecraft_version": "1.20.4",
"minecraft_version": "1.20.2",
"host": "localhost",
"port": 55916,
"auth": "offline",
"allow_insecure_coding": false
"allow_insecure_coding": true
}
38 changes: 27 additions & 11 deletions src/agent/agent.js
Original file line number Diff line number Diff line change
@@ -3,8 +3,25 @@ import { Coder } from './coder.js';
import { Prompter } from './prompter.js';
import { initModes } from './modes.js';
import { initBot } from '../utils/mcdata.js';
import { Command, GetNeighbors } from '../process/ipc/command.js'
import { containsCommand, commandExists, executeCommand, truncCommandMessage } from './commands/index.js';

import { instanceToPlain, plainToClass, plainToInstance } from 'class-transformer';


async function awaitMessage(hash) {
return new Promise((resolve) => {
let timeout = setTimeout(() => {
resolve(''); //Send none to prompter if timed out
}, 10000);

process.on('message', (message) => {
message = plainToInstance(message);
if (message.hash == hash)
clearTimeout(timeout);
resolve(message)
})
})
}

export class Agent {
async start(profile_fp, load_mem=false, init_message=null) {
@@ -34,16 +51,7 @@ export class Agent {
"Set the weather to",
"Gamerule "
];
this.bot.on('chat', (username, message) => {
if (username === this.name) return;

if (ignore_messages.some((m) => message.startsWith(m))) return;

console.log('received message from', username, ':', message);

this.handleMessage(username, message);
});


// set the bot to automatically eat food when hungry
this.bot.autoEat.options = {
priority: 'foodPoints',
@@ -67,6 +75,12 @@ export class Agent {
message = message.replaceAll('\n', ' ');
return this.bot.chat(message);
}

async getNeighbors() {
let command = new GetNeighbors(this.name, this.bot.entity.position);
process.send({message: instanceToPlain(command)});
return await awaitMessage(command.hash);
}

async handleMessage(source, message) {
if (!!source && !!message)
@@ -82,6 +96,7 @@ export class Agent {
let execute_res = await executeCommand(this, message);
if (user_command_name === '!newAction') {
// all user initiated commands are ignored by the bot except for this one
// TODO: change this? in order to provide the ai more information about what the bot did
// add the preceding message to the history to give context for newAction
let truncated_msg = message.substring(0, message.indexOf(user_command_name)).trim();
this.history.add(source, truncated_msg);
@@ -93,6 +108,7 @@ export class Agent {

for (let i=0; i<5; i++) {
let history = this.history.getHistory();

let res = await this.prompter.promptConvo(history);

let command_name = containsCommand(res);
14 changes: 14 additions & 0 deletions src/agent/commands/actions.js
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
import * as skills from '../library/skills.js';
import settings from '../../settings.js';
import { Message } from '../../process/ipc/message.js';


function wrapExecution(func, timeout=-1, resume_name=null) {
return async function (agent, ...args) {
@@ -29,6 +31,18 @@ export const actionsList = [
return await agent.coder.generateCode(agent.history);
}
},
{
name: '!ask',
description: 'Pass a command to a nearby minecraft bot to do something.',
params: {
'ask_name': '(string) The name of the agent/bot you are speaking to ',
'ask_message': '(string) The message to be seent to the agent/bot '
},
perform: async function (agent, ask_name, ask_message) {
//TODO search main.js's list of agent processes via process.send
process.send({message: msgpacck.encode(new Message(agent.name, ask_name, ask_message))});
}
},
{
name: '!stop',
description: 'Force stop all actions and commands that are currently executing.',
1 change: 0 additions & 1 deletion src/agent/commands/index.js
Original file line number Diff line number Diff line change
@@ -28,7 +28,6 @@ export function commandExists(commandName) {
return commandMap[commandName] !== undefined;
}

// todo: handle arrays?
function parseCommandMessage(message) {
const commandMatch = message.match(commandRegex);
if (commandMatch) {
Loading