Page tree

Versions Compared


  • This line was added.
  • This line was removed.
  • Formatting was changed.


Update on NAS latest tests and developments

In Denmark we have two new developers working, for the moment, full-time on webarchiving - Rasmus and Peter. The major effort is on replacing our current backend which will require

  • Reimplementing access (getRecord) to in a secure way that avoids the need to go through JMS and ftp
  • Reimplementing all essential batch jobs in more modern mass-processing framework ie hadoop

As far as Heritrix 3 work is concerned, the main issue is what to do with our various homebrewed heritrix extensions. For each extension we should either

  • Retain our extension (the default)
  • Move to the newest equivalent Heritrix extension
  • Merge our extension with the latest heritrix version

In each case we will need to analyse the code to decide what makes sense.

Status of the production sites