Uploaded image for project: 'NetarchiveSuite'
  1. NetarchiveSuite
  2. NAS-385

HarvestControllerServer does not die cleanly on full disk

    XMLWordPrintable

Details

    • Bug
    • Resolution: Duplicate
    • Major
    • None
    • 0.5
    • Common
    • None

    Description

      I made a little virtual disk and filled it up with zeroes before trying
      to harvest siemens.dk into it
      A section of the output log:
      Jun 27 2005 12:52:48 PM dk.netarkivet.harvestcontroller.HeritrixLauncher doOneCrawl
      FINE: timestamp discovered queued downloaded doc/s(avg)
      KB/s(avg) dl-failures busy-thread mem-use-KB heap-size-KB
      2005-06-27T12:52:48Z 569 357 212 0.0(0.22)
      0(1) 2
      5 24480 25784
      Jun 27 2005 12:52:48 PM dk.netarkivet.harvestcontroller.HeritrixLauncher doOneCrawl
      WARNING: Aborting crawl due to slow or non-responding webserver(s)
      Jun 27 2005 12:52:48 PM
      dk.netarkivet.harvestcontroller.HeritrixLauncher$SimpleCrawlStatusListener
      crawlEnding
      FINE: Crawl ending: Aborting because of slow or non-responding webserver(s)
      Jun 27 2005 12:52:49 PM org.archive.util.DevUtils warnHandle
      WARNING: java.io.IOException: No space left on device
      at java.io.FileOutputStream.writeBytes(Native Method)
      at java.io.FileOutputStream.write(FileOutputStream.java:260)
      at
      org.archive.io.RecyclingFastBufferedOutputStream.close(RecyclingFastBufferedOutputStream.java:137)
      at
      org.archive.io.RecordingOutputStream.closeDiskStream(RecordingOutputStream.java:271)
      at
      org.archive.io.RecordingOutputStream.closeRecorder(RecordingOutputStream.java:277)
      at
      org.archive.io.RecordingInputStream.closeRecorder(RecordingInputStream.java:302)
      at org.archive.util.HttpRecorder.closeRecorders(HttpRecorder.java:231)
      at org.archive.crawler.framework.ToeThread.run(ToeThread.java:181)

      Jun 27 2005 12:53:08 PM dk.netarkivet.harvestcontroller.HeritrixLauncher doOneCrawl
      FINE: timestamp discovered queued downloaded doc/s(avg)
      KB/s(avg) dl-failures busy-thread mem-use-KB heap-size-KB
      2005-06-27T12:53:08Z 569 357 212 0.0(0.22)
      0(1) 2
      5 24993 25784
      Jun 27 2005 12:53:28 PM dk.netarkivet.harvestcontroller.HeritrixLauncher doOneCrawl

      Apparently the attempt to end the crawl throws an exception with the result
      that the crawl never really ends. This is potentially quite severe as it would
      not be obvious from the monitor application that the harvest controller is no
      longer functioning.
      NOTE: This bug is originally from Bugzilla bug_id=383.
      This bug was previously assigned to Unassigned.
      This bug was originally submitted by CSR.

      Attachments

        Activity

          People

            Unassigned Unassigned
            Anonymous Anonymous
            Watchers:
            0 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: