Details
-
Bug
-
Resolution: Fixed
-
Blocker
-
None
-
0.5
-
None
Description
The DomainSensitiveFrontier has turned out to be slow on crawls with lots of
seeds (e.g. 10.000)
Further more is the max-docs setting applied on a pr. host basis instead of a
pr. domain basis.
Suggest we shift to the default BdbFrontier
NOTE: This bug is originally from Bugzilla bug_id=395.