Details
-
New Feature
-
Resolution: Fixed
-
Major
-
3.3
-
None
-
SB/KB
-
Rough
Description
This feature was requested at the NetarchiveSuite workshop in September 2007:
It is possible for a crawl to greatly exceed its limits since objects from other domains aren't counted towards domain size limits even when they're inline images. When another domains items are downloaded as images they should be counted towards the limit both in Heritrix and in the historical info. Essentially no downloaded material should remain unaccounted for.