NetarchiveSuite
Space shortcuts
Page tree

Versions Compared

Old Version 3

changes.mady.by.user Anders Klindt Myrvoll

Saved on

compared with

New Version 4

changes.mady.by.user Anders Klindt Myrvoll

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Status of the production sites

Netarkivet

Panel
  • NAS and Bitmagasinet. We are having quite some difficulties getting everything perfect, but we are working hard to iron out any issues.  Seems like it´s mainly Haddop-issues
  • In the future how do we test NAS-releases as a community?
  • We are planning to test RHEL 8 - but we need a stable test-system first, so we can se implications form upgrading. Any feedback from others who have updated would be appreciated 
  • We have a contact at YouTube/Google and they have asked us for IP- ranges, useragent and motivations - so there is hope to harvest YouTube in the future like BnF 
  • We are awaiting that the IIPC steering comitee decides to proceed (or not) with  User-Friendly High Fidelity Browser-Based Crawling for All - IIPC Tools Portfolio Funding Proposal 2022-2023
  • Broad harvest is postphoned until january/we have a working system
  • We still do selective and special harvests.



Panel

BnF


Panel

First of all, we are pleased to welcome, in our team, Kevin Locoh-Donou, data engineer for the LIFRANUM research project, for a period of 9 months.

On the occasion of "Fantastic Futures" the 3rd international conference on Artificial Intelligence (AI) for librairies, archives and museums which took place at the BnF on December 10th 2021, the digital legal deposit service highlights its AI websites collection:

Finally, our 2021 broad crawl ended on November 14th and lasted a little less than 5 weeks. 2.5 billion URLs were crawled for a total of 114TB.

...