NetarchiveSuite

More work on NAS-2683

We should also check deployment

We should also check deployment

Yes, but i) I wanted to be able to log the duplicates because we still don't understand what's really happening, and ii) What if the same id is returned by both queries ie for some reason the same ...

Yes, but
i) I wanted to be able to log the duplicates because we still don't understand what's really happening, and
ii) What if the same id is returned by both queries ie for some reason the same harvestId shows up as both fullharvest and partialharvest? It should never happen, but the schema actually allows it.

Nothing. I just wanted to put the configuration key in a variable so I could log it.

Nothing. I just wanted to put the configuration key in a variable so I could log it.

no followup required

no followup required

Ǹo followup required

Ǹo followup required

What do you want to prevent here?

What do you want to prevent here?

Instead of this, just use SELECT distinct in lines 672 and 676

Instead of this, just use SELECT distinct in lines 672 and 676

Work on NAS-2683 - now works with seeds coming from a form

Max number of running h3 instances. So 80 or so now?

Max number of running h3 instances. So 80 or so now?

Is it possible to define the configuration and executions tags in pluginManagement so they don't need to be repeated in each module?

Is it possible to define the configuration and executions tags in pluginManagement so they don't need to be repeated in each module?

Not an easy one to review, but should be easy to test. If it builds and harvests then you haven't removed anything vital.

Not an easy one to review, but should be easy to test. If it builds and harvests then you haven't removed anything vital.

Since these are useful tools they should be documented with example usage in https://sbforge.org/display/NASDOC/Tools+in+the+Common+Module .

Since these are useful tools they should be documented with example usage in https://sbforge.org/display/NASDOC/Tools+in+the+Common+Module .

NAS-2685 Simultaneous processing of HD
NAS-2685 Simultaneous processing of HD
This may not fix the bug, but should make diagnosis easier.

This may not fix the bug, but should make diagnosis easier.

More logging.

Added a lot of logging and renamed one method to make its functionality clearer.

Added a sanity test that harvest ids returned are unique.

Fixed bug in crawllog caching. Work on optimizing frontier queue viewery. Removed useless auto refresh from most pages.

The class ./harvester/harvester-core/src/main/java/dk/netarkivet/harvester/harvesting/report/Heritrix1Constants.java should also be removed

The class ./harvester/harvester-core/src/main/java/dk/netarkivet/harvester/harvesting/report/Heritrix1Constants.java
should also be removed

NAS-2673 - more work