You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Thanks for this super useful package. I want to restrict the crawl to certain URL specifications, but capture all links on the crawled pages regardless of whether they match the filter. I can't get this to work in practice. An example:
Page https://beta.companieshouse.gov.uk/company/02906991/officers (which is crawled) includes links such as https://beta.companieshouse.gov.uk/officers/... but these pages are not included in the results. E.g:
Thanks for this super useful package. I want to restrict the crawl to certain URL specifications, but capture all links on the crawled pages regardless of whether they match the filter. I can't get this to work in practice. An example:
Page
https://beta.companieshouse.gov.uk/company/02906991/officers
(which is crawled) includes links such ashttps://beta.companieshouse.gov.uk/officers/...
but these pages are not included in the results. E.g:Shouldn't this links be captured, since I have provided no
dataUrlfilter
argument? Or am I missing something here?The text was updated successfully, but these errors were encountered: