the page endpoint is not intended for scraping

[extreme4all]

osrs-hiscores/src/scrape/requests.py

Line 34 in e9bbcf5

url = "https://secure.runescape.com/m=hiscore_oldschool/overall"

in the past this endpoint was strictly protected by imperva, it seems to be less protected now, but extensive scraping of unintended pages might cause issues.

[lukearend]

Gotcha. I'll get in touch with you and the other bot detection people about data collection in the future. Is there currently an intended API for this kind of scraping?

[extreme4all]

i believe:
https://secure.runescape.com/m=hiscore_oldschool/index_lite.ws?player={player['name']}

The runescape documentation on it
https://runescape.wiki/w/Application_programming_interface#Hiscores_Lite_2