Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Option to only get metadata #81

Open
KeinNiemand opened this issue Jun 13, 2022 · 4 comments
Open

Option to only get metadata #81

KeinNiemand opened this issue Jun 13, 2022 · 4 comments

Comments

@KeinNiemand
Copy link
Contributor

An option to only get metadata like the number of chapters or the last updated date would be really usefull to make scripts that only redownload things that have updated

@kemayo
Copy link
Owner

kemayo commented Jun 13, 2022

It's possible, but only for some sites. Or, rather, only possible without a full download for some sites -- the ones that e.g. walk through a full story via next-chapter links would be problematic.

@KeinNiemand
Copy link
Contributor Author

Even if it takes a full download/nearly a full download (no need to do generate an epup, do a lot of formatting etc.) and therfore still take almost as long as a full download getting just metadata would still be useful since it woild allow me to see what story have updated by their file modified date without having to look inside the files or look at the alerts I'm getting from the sites the storys are downloaded from.

@KeinNiemand
Copy link
Contributor Author

Also even if for siteres where it has to go trough links it still has to go trough every links to get say the chapter count it would still be faster then a full download as it dosn't have to do any formatting/proccessing/converting to epub, ...

@ClaasJG
Copy link
Contributor

ClaasJG commented Jul 6, 2022

I once considered to write a PR which would extend the Side class to allow the creation of a Side specific 'data bundle' which could be used by the same Site implementation again to check if there is a potential update.

Such a 'data bundle' could be the last update date if available or for an custom adapter which uses a next selector it could contain the link to the last page as well as the hash of the last page. This would allow to quickly check if there could be an update. (i.e. extract the last chapter again and check if the hash changed or check if a 'next' link became available).

It is an open question, if this 'data bundle' should be included in the epub or written to an extra file or to stdout.
While I think it would be nicer to have the 'data bundle' embedded in the epub it would be easier to generate an extra file. Leech may not know the name of an epub until it got completely downloaded and therefor it could be hard to relate links / custom jsons and the generated epubs.

While you asked for metadata like the number of chapters I think the possibility to generate such a 'data bundle' paired with the possibility to ask leech if there may be an update would also solve your problem? If there is interest in such a solution I would try to implement it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants