Uploaded image for project: 'NetarchiveSuite'
  1. NetarchiveSuite
  2. NAS-2037

Thread accumulation in IndexServerApplication

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • Critical
    • I50, 3.18.3, 3.19.0
    • 3.18.0
    • Archive
    • None
    • SB/KB
    • Hide

      Verify that threads like

      "pool-155-thread-1" prio=10 tid=0x7e46e400 nid=0x7ec1 waiting on condition [0x7d1a4000]
         java.lang.Thread.State: WAITING (parking)
              at sun.misc.Unsafe.park(Native Method)
              - parking to wait for  <0x93987228> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
              at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
              at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
              at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
              at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
              at java.lang.Thread.run(Thread.java:662)
      

      are not accumulating.

      This can be tested by scheduling netarkivet.dk to be harvested hourly for 7 hours in a row.
      The time spent can be reduced by creating several hourly netarkivet.dk selective harvests.

      After these harvests are finished

      jstack <Indexserver-application pid> > jstack.out
      

      "grep ThreadPoolExecutor.getTask jstack.out" should only then result in less than 5 hits.

      Show
      Verify that threads like "pool-155-thread-1" prio=10 tid=0x7e46e400 nid=0x7ec1 waiting on condition [0x7d1a4000] java.lang. Thread .State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x93987228> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907) at java.lang. Thread .run( Thread .java:662) are not accumulating. This can be tested by scheduling netarkivet.dk to be harvested hourly for 7 hours in a row. The time spent can be reduced by creating several hourly netarkivet.dk selective harvests. After these harvests are finished jstack <Indexserver-application pid> > jstack.out "grep ThreadPoolExecutor.getTask jstack.out" should only then result in less than 5 hits.

    Description

      The new parallelisation of the index generation in CrawlLogIndexCache using ThreadPoolExexecutors are not shutting the executors down after end of use.

      Thus threads accumulate until we can't create any more native thread.

      The workaround is to restart the IndexServerApplication once in a while

      But the fix is quite simple to implement

      Attachments

        Activity

          People

            svc Søren Vejrup Carlsen (Inactive)
            svc Søren Vejrup Carlsen (Inactive)
            Mikis Seth Sørensen Mikis Seth Sørensen (Inactive)
            Watchers:
            0 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

                Estimated:
                Original Estimate - 3h
                3h
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 3h
                3h