Generation of snapshot jobs takes too long

It seems that NetarchiveSuite performs poorly, when generating jobs for a snapshot harvest, at least when using the postgreSQL database.

It could, however, also be an algorithmic issue.

The performance, we have seen in Netarkivet prod and test-environment, is 1 job generated each 65 minutes handling 10000 domains. The number of domains in netarkivet domains database is now close to 2 mio domains.

That means over 200 hours (= 9 days) to generate the full complement of jobs in the snapshot harvest.