Regional Daily Newspapers Example: Ouest-France
Ouest-France (http://www.ouest-france.fr/) is the biggest regional daily newspaper title with 47 editions. In the past, we tested the deposit of PDF files without success.
There is no free online access to the PDF versions of the newspapers (http://www.ouestfrance-enligne.com/). A password is required to access the publications after subscription. We added the password into the Heritrix profile but:
- The login/password is available for 3 months only
- Often, the crawler gets disconnected
- Any false URL causes a disconnect and leads to the login page
- But Heritrix enters the password only once a job (the page is then marked as “already seen” and is not collected again)
So to date, we crawl the articles but not the integral PDF versions.