promptsearch

Inspired by Promptbreeder: Self-Referential Self-Improvement Via Prompt Evolution

Searches for better prompts given a model, dataset and evaluation.

Need to set up weave notion of model, dataset and evaluation - see https://github.com/wandb/weave

ps = PromptSearch(model=model, dataset=dataset, evaluation=evaluation)
ps.steps(10)

Try example on hellaswag dataset

PYTHONPATH=. python examples/hellaswag/hellaswag.py