Uploaded image for project: 'NetarchiveSuite'
  1. NetarchiveSuite
  2. NAS-1000

Timeout kills non finished job because harvester gets stuck?

    XMLWordPrintable

Details

    • Bug
    • Resolution: Won't Fix
    • Critical
    • None
    • 3.2
    • None

    Description

      We have a few times seen timeouts like this:
      2007-11-20T02:52:38Z 2517973 1027719 1448063 0(10.26) 0(358) 44769 32 1798988 1974400 1
      243140 12234
      2007-11-20T02:53:18Z 2517973 1026653 1448063 0(10.26) 0(358) 45835 2 1774945 1974400 1
      243140 12222
      2007-11-20T02:54:06Z 2517973 1025484 1448063 0(10.25) 0(358) 47004 2 1787212 1974400 1
      243140 12064
      2007-11-20T02:54:42Z 2517973 1024283 1448063 0(10.25) 0(358) 48205 1 1783009 1974400 1
      243140 12193
      2007-11-20T02:55:14Z 2517973 1023206 1448063 0(10.25) 0(358) 49282 1 1778815 1974400 1
      243140 12181
      2007-11-20T03:16:13Z 2517973 1022062 1448063 0(10.16) 0(355) 50426 3 1768812 1974400 1.5
      243140 12024
      2007-11-20T03:16:33Z 2517973 1022060 1448065 0.1(10.16) 0(355) 50428 50 1782017 1974400 1.72
      243140 11884
      2007-11-20T03:25:32Z 2517974 1021061 1448068 0.01(10.12) 0(354) 51424 36 1780381 1974400 1
      243140 12155
      20071120032533 CRAWL ENDING - Aborting because of inactivity
      2007-11-20T03:25:52Z 2517974 1021058 1448068 0(10.12) 0(353) 51429 28 1785490 1974400 1.46
      243140 11872
      2007-11-20T03:25:58Z 2517974 1021030 1448068 0(10.11) 0(353) 51457 0 1793030 1974400 2.77
      243140 11872
      20071120032600 CRAWL ENDED - Aborting because of inactivity
      2 interesting things:

      • the harvester stopped logging for many minutes
      • there are a lot of objects still in the queue.
        In this situation we do not want to abort the job.
        I'm not sure if our usage of the harvester status allows for checking this situation (no status for some time). If it's possible it should check the timestamp (1st field in each line) and if this is not updated we should reset the timeout

      Attachments

        Activity

          People

            kfc Kåre Fiedler Christiansen (Inactive)
            bja Bjarne Andersen
            Watchers:
            0 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: