Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Better documentation #7

Closed
I-I-IT opened this issue Mar 4, 2025 · 7 comments
Closed

Better documentation #7

I-I-IT opened this issue Mar 4, 2025 · 7 comments
Assignees

Comments

@I-I-IT
Copy link

I-I-IT commented Mar 4, 2025

Hi, I have run a test batch today, and they are a couple of small things which could be improved in the docs.

Config
Specify that the user shouldn't directly input the API key in the config file, but rather input the name of the apiin the config file and then write export NAME_API_KEY=[redacted] in the terminal.

I also had trouble with uv that didn't want to use my venv and they recommended --active which didn't work. Listing all the dependencies might be useful, maybe as an annex.

Also, I noticed that in utils.py
from sympy.parsing.latex import parse_latex import sympy

It seems to me that import should be first, unless I am missing something.

Another thing I had trouble is to view results.I don't understand what path we need to input: the results path or the competition path? Either way, no results display.

@JasperDekoninck JasperDekoninck self-assigned this Mar 4, 2025
@JasperDekoninck
Copy link
Collaborator

Hi!

Thanks for the feedback. I updated the README with an alternative to UV (using conda and pip install -e .). I also updated the docs to clarify that the API keys should be added as environment variables.

@JasperDekoninck
Copy link
Collaborator

JasperDekoninck commented Mar 4, 2025

In order to view results, you should put
uv run python scripts/app.py --comp path/to/competition
where path/to/competition is the path relative to the data folder. So if you are running on AIME 2025 I, this would be aime/aime_2025_I. I am assuming that this relative path is a bit unintuitive?

@I-I-IT
Copy link
Author

I-I-IT commented Mar 5, 2025

I just have this. No data displayed.
Image

@JasperDekoninck
Copy link
Collaborator

It seems from the picture you are still passing data to the path? Could you try with
uv run python scripts/app.py --comp hmmt/hmmt_feb_2025

@I-I-IT
Copy link
Author

I-I-IT commented Mar 6, 2025

When I am doing this I just have a blank page.

@JasperDekoninck
Copy link
Collaborator

Hmm, this is strange. I cannot reproduce the issue locally. Are you sure you already ran some models on the HMMT competition and that their results are stored in data/hmmt/hmmt_feb_2025? If so, can you give me more details what models you ran and the configs you used for them?

@I-I-IT
Copy link
Author

I-I-IT commented Mar 12, 2025

Late reply, but I now got it to work.

Image

@I-I-IT I-I-IT closed this as completed Mar 12, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants