Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Task errors and agent message logs #279

Merged
merged 1 commit into from
Jun 27, 2024
Merged

Conversation

nerfZael
Copy link
Contributor

@nerfZael nerfZael commented Jun 26, 2024

Changes:

  • Added table task_errors since a task can have multiple errors (e.g. when running, when building transactions, etc). Tasks still has an error column, but that is now only used for errors that stop AutoTx from running
  • Increased max consecutive auto reply to 50 in main chat and 20 in delegate researcher (was 10 and 10) since this is effectively the same as max_rounds for our case
  • Implemented logging for all agent messages while auto tx is running. This should help greatly when debugging.
  • Endpoints for fetching logs

@nerfZael nerfZael requested review from dOrgJelli and cbrzn June 26, 2024 21:21
@nerfZael
Copy link
Contributor Author

/workflows/benchmarks agents/token/send 3

@nerfZael nerfZael changed the base branch from main to dev June 26, 2024 22:21
Copy link

github-actions bot commented Jun 26, 2024

Finished benchmarks
Download artifacts

Test Run Summary

  • Run from: ./autotx/tests/agents/token/send
  • Base path: autotx/tests/agents/token/send/test_send.py::
  • Iterations: 3
  • Total Cost: $0.10
  • Total Success Rate (%): ${\color{none} \LARGE \texttt {100.00} \large \texttt {} }$

Detailed Results

Test Name Success Rate (%) Passes Fails Avg Time Avg Cost
test_send_erc20 ${\color{none} \large \texttt {100} \normalsize \texttt {} }$ ${\color{none} \large \texttt {3}}$ ${\color{none} \large \texttt {0}}$ 21s $0.00
test_send_erc20_parallel ${\color{none} \large \texttt {100} \normalsize \texttt {} }$ ${\color{none} \large \texttt {3}}$ ${\color{none} \large \texttt {0}}$ 23s $0.01
test_send_eth_multiple ${\color{none} \large \texttt {100} \normalsize \texttt {} }$ ${\color{none} \large \texttt {3}}$ ${\color{none} \large \texttt {0}}$ 28s $0.01
test_send_native ${\color{none} \large \texttt {100} \normalsize \texttt {} }$ ${\color{none} \large \texttt {3}}$ ${\color{none} \large \texttt {0}}$ 15s $0.00
test_send_native_sequential ${\color{none} \large \texttt {100} \normalsize \texttt {} }$ ${\color{none} \large \texttt {3}}$ ${\color{none} \large \texttt {0}}$ 20s $0.01

Total run time: 5.37 minutes

@nerfZael nerfZael merged commit fe67c6e into dev Jun 27, 2024
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant