Uploaded image for project: 'NetarchiveSuite'
  1. NetarchiveSuite
  2. NAS-1818

WaybackIndexer takes too long to start

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • Major
    • I50, 3.19.0
    • None
    • Wayback
    • None
    • SB/KB
    • Timeboxed
    • Hide

      TBD by CSR in connection with release 3.20

      Sparsely tested by starting the program dk.netarkivet.wayback.indexer.WaybackIndexerApplication

      Used the settings file belonging to the IndexServerApplication with the following addition:

      <wayback><indexer><replicaId>KB</replicaId></indexer></wayback>
      
      Show
      TBD by CSR in connection with release 3.20 Sparsely tested by starting the program dk.netarkivet.wayback.indexer.WaybackIndexerApplication Used the settings file belonging to the IndexServerApplication with the following addition: <wayback><indexer><replicaId>KB</replicaId></indexer></wayback>

    Description

      The run method in WaybackIndexer looks like
      public void run()

      { log.info("Starting producer thread"); FileNameHarvester.harvest(); IndexerQueue.getInstance().populate(); }

      so it has to run through every file in the archive before it sets anything to harvest.
      Better to have

      IndexerQueue.getInstance().populate();
      FileNameHarvester.harvest();
      IndexerQueue.getInstance().populate();
      So it can at least start right away.

      Attachments

        Activity

          People

            svc Søren Vejrup Carlsen (Inactive)
            csr Colin Rosenthal
            Colin Rosenthal Colin Rosenthal
            Watchers:
            0 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

                Estimated:
                Original Estimate - 0.5h
                0.5h
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 0.5h
                0.5h