Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /home/cibuild02/workspace/NetarchiveSuite-Colin-1-Click/monitor/monitor-test/src/main/resources
[INFO]
[INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ monitor-test ---
[INFO] No sources to compile
[INFO]
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ monitor-test ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 1 resource
[INFO] Copying 8 resources
[INFO]
[INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @ monitor-test ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 4 source files to /home/cibuild02/workspace/NetarchiveSuite-Colin-1-Click/monitor/monitor-test/target/test-classes
[INFO]
[INFO] --- maven-surefire-plugin:2.17:test (default-test) @ monitor-test ---
[INFO] Tests are skipped.
[INFO]
[INFO] --- maven-jgit-buildnumber-plugin:1.2.9:extract-buildnumber (git-buildnumber) @ monitor-test ---
[INFO]
[INFO] --- maven-jar-plugin:2.4:jar (default-jar) @ monitor-test ---
[WARNING] JAR will be empty - no content was marked for inclusion!
[INFO] Building jar: /home/cibuild02/workspace/NetarchiveSuite-Colin-1-Click/monitor/monitor-test/target/monitor-test-7.4-SNAPSHOT.jar
[INFO]
[INFO] --- maven-source-plugin:2.2.1:jar-no-fork (attach-sources-no-fork) @ monitor-test ---
[INFO] No sources in project. Archive not created.
[JENKINS] Archiving disabled
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building NetarchiveSuite - wayback resourceStore 7.4-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ wayback-resourcestore ---
[INFO]
[INFO] --- jacoco-maven-plugin:0.7.9:prepare-agent (agent) @ wayback-resourcestore ---
[INFO] argLine set to -javaagent:/home/cibuild02/.m2/repository/org/jacoco/org.jacoco.agent/0.7.9/org.jacoco.agent-0.7.9-runtime.jar=destfile=/home/cibuild02/workspace/NetarchiveSuite-Colin-1-Click/wayback/wayback-resourcestore/../target/jacoco.exec,append=true
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ wayback-resourcestore ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /home/cibuild02/workspace/NetarchiveSuite-Colin-1-Click/wayback/wayback-resourcestore/src/main/resources
[INFO]
[INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ wayback-resourcestore ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 7 source files to /home/cibuild02/workspace/NetarchiveSuite-Colin-1-Click/wayback/wayback-resourcestore/target/classes
[INFO]
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ wayback-resourcestore ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /home/cibuild02/workspace/NetarchiveSuite-Colin-1-Click/wayback/wayback-resourcestore/src/test/resources
[INFO]
[INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @ wayback-resourcestore ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 4 source files to /home/cibuild02/workspace/NetarchiveSuite-Colin-1-Click/wayback/wayback-resourcestore/target/test-classes
[INFO]
[INFO] --- maven-surefire-plugin:2.17:test (default-test) @ wayback-resourcestore ---
[INFO] Tests are skipped.
[INFO]
[INFO] --- maven-jgit-buildnumber-plugin:1.2.9:extract-buildnumber (git-buildnumber) @ wayback-resourcestore ---
[INFO]
[INFO] --- maven-jar-plugin:2.4:jar (default-jar) @ wayback-resourcestore ---
[INFO] Building jar: /home/cibuild02/workspace/NetarchiveSuite-Colin-1-Click/wayback/wayback-resourcestore/target/wayback-resourcestore-7.4-SNAPSHOT.jar
[INFO]
[INFO] --- maven-source-plugin:2.2.1:jar-no-fork (attach-sources-no-fork) @ wayback-resourcestore ---
[INFO] Building jar: /home/cibuild02/workspace/NetarchiveSuite-Colin-1-Click/wayback/wayback-resourcestore/target/wayback-resourcestore-7.4-SNAPSHOT-sources.jar
[JENKINS] Archiving disabled
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building NetarchiveSuite - wayback - test 7.4-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ wayback-test ---
[INFO]
[INFO] --- jacoco-maven-plugin:0.7.9:prepare-agent (agent) @ wayback-test ---
[INFO] argLine set to -javaagent:/home/cibuild02/.m2/repository/org/jacoco/org.jacoco.agent/0.7.9/org.jacoco.agent-0.7.9-runtime.jar=destfile=/home/cibuild02/workspace/NetarchiveSuite-Colin-1-Click/wayback/wayback-test/../target/jacoco.exec,append=true
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ wayback-test ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /home/cibuild02/workspace/NetarchiveSuite-Colin-1-Click/wayback/wayback-test/src/main/resources
[INFO]
[INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ wayback-test ---
[INFO] No sources to compile
[INFO]
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ wayback-test ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 1 resource
[INFO] Copying 28 resources
[INFO]
[INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @ wayback-test ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 17 source files to /home/cibuild02/workspace/NetarchiveSuite-Colin-1-Click/wayback/wayback-test/target/test-classes
[WARNING] /home/cibuild02/workspace/NetarchiveSuite-Colin-1-Click/wayback/wayback-test/src/test/java/dk/netarkivet/wayback/hadoop/CDXMapperTester.java: Some input files use or override a deprecated API.
[WARNING] /home/cibuild02/workspace/NetarchiveSuite-Colin-1-Click/wayback/wayback-test/src/test/java/dk/netarkivet/wayback/hadoop/CDXMapperTester.java: Recompile with -Xlint:deprecation for details.
[INFO]
[INFO] --- maven-surefire-plugin:2.17:test (default-test) @ wayback-test ---
[INFO] Tests are skipped.
[INFO]
[INFO] --- maven-jgit-buildnumber-plugin:1.2.9:extract-buildnumber (git-buildnumber) @ wayback-test ---
[INFO]
[INFO] --- maven-jar-plugin:2.4:jar (default-jar) @ wayback-test ---
[WARNING] JAR will be empty - no content was marked for inclusion!
[INFO] Building jar: /home/cibuild02/workspace/NetarchiveSuite-Colin-1-Click/wayback/wayback-test/target/wayback-test-7.4-SNAPSHOT.jar
[INFO]
[INFO] --- maven-source-plugin:2.2.1:jar-no-fork (attach-sources-no-fork) @ wayback-test ---
[INFO] No sources in project. Archive not created.
[JENKINS] Archiving disabled
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building NetarchiveSuite - integration tests 7.4-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ integration-test ---
[INFO]
[INFO] --- maven-jgit-buildnumber-plugin:1.2.9:extract-buildnumber (git-buildnumber) @ integration-test ---
[INFO]
[INFO] --- maven-source-plugin:2.2.1:jar-no-fork (attach-sources-no-fork) @ integration-test ---
[JENKINS] Archiving disabled
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building NetarchiveSuite System test 7.4-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ system-test ---
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ system-test ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /home/cibuild02/workspace/NetarchiveSuite-Colin-1-Click/integration-test/system-test/src/main/resources
[INFO]
[INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ system-test ---
[INFO] No sources to compile
[INFO]
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ system-test ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 22 resources
[INFO]
[INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @ system-test ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 34 source files to /home/cibuild02/workspace/NetarchiveSuite-Colin-1-Click/integration-test/system-test/target/test-classes
[WARNING] /home/cibuild02/workspace/NetarchiveSuite-Colin-1-Click/integration-test/system-test/src/test/java/dk/netarkivet/systemtest/SeleniumSession.java: /home/cibuild02/workspace/NetarchiveSuite-Colin-1-Click/integration-test/system-test/src/test/java/dk/netarkivet/systemtest/SeleniumSession.java uses unchecked or unsafe operations.
[WARNING] /home/cibuild02/workspace/NetarchiveSuite-Colin-1-Click/integration-test/system-test/src/test/java/dk/netarkivet/systemtest/SeleniumSession.java: Recompile with -Xlint:unchecked for details.
[INFO]
[INFO] --- maven-surefire-plugin:2.17:test (default-test) @ system-test ---
[INFO] Tests are skipped.
[INFO]
[INFO] --- maven-jgit-buildnumber-plugin:1.2.9:extract-buildnumber (git-buildnumber) @ system-test ---
[INFO]
[INFO] --- maven-jar-plugin:2.4:jar (default-jar) @ system-test ---
[WARNING] JAR will be empty - no content was marked for inclusion!
[INFO] Building jar: /home/cibuild02/workspace/NetarchiveSuite-Colin-1-Click/integration-test/system-test/target/system-test-7.4-SNAPSHOT.jar
[INFO]
[INFO] --- maven-source-plugin:2.2.1:jar-no-fork (attach-sources-no-fork) @ system-test ---
[INFO] No sources in project. Archive not created.
[JENKINS] Archiving disabled
[JENKINS] Archiving disabled
[JENKINS] Archiving disabled
[JENKINS] Archiving disabled
[JENKINS] Archiving disabled
[JENKINS] Archiving disabled
[JENKINS] Archiving disabled
[JENKINS] Archiving disabled
[JENKINS] Archiving disabled
[JENKINS] Archiving disabled
[JENKINS] Archiving disabled
[JENKINS] Archiving disabled
[JENKINS] Archiving disabled
[JENKINS] Archiving disabled
[JENKINS] Archiving disabled
[JENKINS] Archiving disabled
[JENKINS] Archiving disabled
[JENKINS] Archiving disabled
[JENKINS] Archiving disabled
[JENKINS] Archiving disabled
[JENKINS] Archiving disabled
[JENKINS] Archiving disabled
[JENKINS] Archiving disabled
[JENKINS] Archiving disabled
[JENKINS] Archiving disabled
[JENKINS] Archiving disabled
[JENKINS] Archiving disabled
[JENKINS] Archiving disabled
[JENKINS] Archiving disabled
[JENKINS] Archiving disabled
[JENKINS] Archiving disabled
[JENKINS] Archiving disabled
[JENKINS] Archiving disabled
[JENKINS] Archiving disabled
[JENKINS] Archiving disabled
[JENKINS] Archiving disabled
[JENKINS] Archiving disabled
[JENKINS] Archiving disabled
[JENKINS] Archiving disabled
[JENKINS] Archiving disabled
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] NetarchiveSuite .................................... SUCCESS [ 1.840 s]
[INFO] NetarchiveSuite - build-tools ...................... SUCCESS [ 1.691 s]
[INFO] NetarchiveSuite - common ........................... SUCCESS [ 0.149 s]
[INFO] NetarchiveSuite - test utils ....................... SUCCESS [ 2.186 s]
[INFO] NetarchiveSuite - common - core .................... SUCCESS [ 8.010 s]
[INFO] NetarchiveSuite - common - test .................... SUCCESS [ 4.834 s]
[INFO] NetarchiveSuite - archive .......................... SUCCESS [ 0.129 s]
[INFO] NetarchiveSuite - archive - core ................... SUCCESS [ 3.689 s]
[INFO] NetarchiveSuite - harvester ........................ SUCCESS [ 0.141 s]
[INFO] NetarchiveSuite - harvester - core ................. SUCCESS [ 6.907 s]
[INFO] NetarchiveSuite - archive - test ................... SUCCESS [ 3.057 s]
[INFO] NetarchiveSuite - bitpreservation - webapp ......... SUCCESS [ 1.129 s]
[INFO] NetarchiveSuite - Harvest Scheduler ................ SUCCESS [ 2.703 s]
[INFO] NetarchiveSuite - monitor .......................... SUCCESS [ 0.141 s]
[INFO] NetarchiveSuite - monitor - core ................... SUCCESS [ 2.017 s]
[INFO] NetarchiveSuite - wayback .......................... SUCCESS [ 0.119 s]
[INFO] NetarchiveSuite - wayback - indexer ................ SUCCESS [ 2.540 s]
[INFO] NetarchiveSuite - deploy ........................... SUCCESS [ 0.153 s]
[INFO] NetarchiveSuite - deploy - core .................... SUCCESS [ 2.992 s]
[INFO] NetarchiveSuite - harvester - heritrix3 ............ SUCCESS [ 0.152 s]
[INFO] NetarchiveSuite - Heritrix3 Extensions ............. SUCCESS [ 3.774 s]
[INFO] NetarchiveSuite - heritrix3 controller ............. SUCCESS [ 2.511 s]
[INFO] hadoop-uber-jar .................................... SUCCESS [ 8.221 s]
[INFO] hadoop-uber-jar-invoker ............................ SUCCESS [ 9.144 s]
[INFO] NetarchiveSuite - Status - Webapp .................. SUCCESS [ 1.074 s]
[INFO] NetarchiveSuite - qa - webapp ...................... SUCCESS [ 0.489 s]
[INFO] NetarchiveSuite - history - webapp ................. SUCCESS [ 1.842 s]
[INFO] NetarchiveSuite - harvestDefinition - webapp ....... SUCCESS [ 1.189 s]
[INFO] NetarchiveSuite - harvestChannel - webapp .......... SUCCESS [ 0.525 s]
[INFO] NetarchiveSuite - heritrix3 monitor ................ SUCCESS [ 20.061 s]
[INFO] NetarchiveSuite - deploy - distribution ............ SUCCESS [ 18.589 s]
[INFO] NetarchiveSuite - deploy - test .................... SUCCESS [ 2.702 s]
[INFO] NetarchiveSuite - Heritrix3 Bundler ................ SUCCESS [ 17.850 s]
[INFO] NetarchiveSuite - harvester - test ................. SUCCESS [ 3.952 s]
[INFO] NetarchiveSuite - monitor - test ................... SUCCESS [ 1.696 s]
[INFO] NetarchiveSuite - wayback resourceStore ............ SUCCESS [ 1.685 s]
[INFO] NetarchiveSuite - wayback - test ................... SUCCESS [ 1.888 s]
[INFO] NetarchiveSuite - integration tests ................ SUCCESS [ 0.113 s]
[INFO] NetarchiveSuite System test ........................ SUCCESS [ 1.841 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 02:37 min
[INFO] Finished at: 2022-07-11T09:47:48+02:00
[INFO] Final Memory: 144M/344M
[INFO] ------------------------------------------------------------------------
channel stopped
[NetarchiveSuite-Colin-1-Click] $ /bin/bash /tmp/hudson3464755983644839005.sh
Do a total cleanup of system Test6
start tunning cleanup script
end part of script
--------------------------------------------
KILLING MACHINE: netarkdv@sb-test-har-001.statsbiblioteket.dk
Killing all applications on: sb-test-har-001.statsbiblioteket.dk
Killing linux application: HarvestControllerApplication_sbhigh_h3
--------------------------------------------
--------------------------------------------
KILLING MACHINE: devel@kb-test-adm-001.kb.dk
Killing all applications on: kb-test-adm-001.kb.dk
Killing linux application: GUIApplication
Killing linux application: ArcRepositoryApplication
Killing linux application: HarvestJobManagerApplication
--------------------------------------------
KILLING MACHINE: devel@kb-test-har-003.kb.dk
Killing all applications on: kb-test-har-003.kb.dk
Killing linux application: HarvestControllerApplication_kblow001_h3
--------------------------------------------
KILLING MACHINE: devel@kb-test-har-005.kb.dk
Killing all applications on: kb-test-har-005.kb.dk
Killing linux application: HarvestControllerApplication_browser_1
Killing linux application: HarvestControllerApplication_browser_2
--------------------------------------------
KILLING MACHINE: devel@kb-test-har-004.kb.dk
Killing all applications on: kb-test-har-004.kb.dk
Killing linux application: HarvestControllerApplication_kblow002_h3
Killing linux application: HarvestControllerApplication_kbhigh_h3
--------------------------------------------
KILLING MACHINE: devel@kb-test-acs-001.kb.dk
Killing all applications on: kb-test-acs-001.kb.dk
Killing linux application: IndexServerApplication
Killing linux application: ViewerProxyApplication
--------------------------------------------
KILLING MACHINE: devel@kb-test-way-001.kb.dk
Killing all applications on: kb-test-way-001.kb.dk
Killing linux application: WaybackIndexerApplication
Killing linux application: AggregatorApplication
--------------------------------------------
running cleanup_bitmag
Resetting Collection Test6
Delete content in collection=Test6
09:48:55.583 DEBUG o.b.c.settings.XMLFileSettingsLoader - Loading the settings file '/home/devel/ResetCollection/bitrepository-client/bin/../conf/RepositorySettings.xml'.
09:48:55.875 DEBUG org.bitrepository.common.JaxbHelper - Creating JAXBHelper based on schema from: jar:file:/home/devel/ResetCollection/bitrepository-client/lib/bitrepository-reference-settings-1.9.0.1.jar!/xsd/ReferenceSettings.xsd
09:48:55.907 DEBUG o.b.c.settings.XMLFileSettingsLoader - Loading the settings file '/home/devel/ResetCollection/bitrepository-client/bin/../conf/ReferenceSettings.xml'.
09:48:56.606 DEBUG o.b.p.security.BasicSecurityManager - X509CertificateHolder found
09:48:56.622 DEBUG o.b.p.security.BasicSecurityManager - Got something, that we don't (yet) recognize. Class: PEMKeyPair
09:48:56.623 DEBUG o.b.p.security.BasicSecurityManager - PrivateKeyInfo found
09:48:56.779 INFO o.b.p.activemq.ActiveMQMessageBus - Initializing ActiveMQMessageBus:'org.bitrepository.settings.repositorysettings.MessageBusConfiguration@13c9d689[name=netarkiv-aar-bit-master-devel-01 broker, url=failover:(ssl://netarkiv-aar-bit-master-devel-01.statsbiblioteket.dk:61612?keepAlive=true)]'.
09:48:56.780 DEBUG org.bitrepository.common.JaxbHelper - Creating JAXBHelper based on schema from: jar:file:/home/devel/ResetCollection/bitrepository-client/lib/bitrepository-message-xml-xsd-30.jar!/xsd/BitRepositoryMessages.xsd
09:48:57.624 INFO o.a.a.t.failover.FailoverTransport - Successfully connected to ssl://netarkiv-aar-bit-master-devel-01.statsbiblioteket.dk:61612?keepAlive=true
09:48:57.740 DEBUG o.b.p.activemq.ActiveMQMessageBus - ActiveMQConnection initialized for 'org.bitrepository.settings.repositorysettings.MessageBusConfiguration@13c9d689[name=netarkiv-aar-bit-master-devel-01 broker, url=failover:(ssl://netarkiv-aar-bit-master-devel-01.statsbiblioteket.dk:61612?keepAlive=true)]'.
09:48:57.743 INFO o.b.p.activemq.ActiveMQMessageBus - Settings component filter to: [kb-test-har-003.kb.dk-devel-5269]
09:48:57.746 DEBUG o.b.c.c.m.CollectionBasedConversationMediator - Initializing the CollectionBasedConversationMediator
09:48:57.747 DEBUG o.b.p.activemq.ActiveMQMessageBus - Adding listener 'org.bitrepository.client.conversation.mediator.CollectionBasedConversationMediator@4a7f959b' to destination: 'topic://netarkiv-devel-kb-test-har-003.kb.dk-devel-5269' on message-bus 'netarkiv-aar-bit-master-devel-01 broker'.
09:48:57.748 DEBUG o.b.p.activemq.ActiveMQMessageBus - Retrieving message consumer on destination 'topic://netarkiv-devel-kb-test-har-003.kb.dk-devel-5269' for listener 'org.bitrepository.client.conversation.mediator.CollectionBasedConversationMediator@4a7f959b'. Key: 'topic://netarkiv-devel-kb-test-har-003.kb.dk-devel-5269#1249875355'.
09:48:57.748 DEBUG o.b.p.activemq.ActiveMQMessageBus - No consumer known. Creating new for key 'topic://netarkiv-devel-kb-test-har-003.kb.dk-devel-5269#1249875355'.
Checksum: Count: FileID:
09:48:57.788 INFO o.b.a.g.ConversationBasedGetChecksumsClient - Requesting the checksums for file '' with the specifications 'org.bitrepository.bitrepositoryelements.ChecksumSpecTYPE@73e22a3d[checksumType=MD5, checksumSalt=<null>(default), otherChecksumType=<null>(default)]null' with query [ContributorQuery{componentID='netarkiv-tape-devel-01, minTimestamp=Thu Jan 01 01:00:00 CET 1970, maxTimestamp=null, maxNumberOfResults=10000}, ContributorQuery{componentID='netarkiv-disk-devel-01, minTimestamp=Thu Jan 01 01:00:00 CET 1970, maxTimestamp=null, maxNumberOfResults=10000}, ContributorQuery{componentID='netarkiv-disk-devel-02, minTimestamp=Thu Jan 01 01:00:00 CET 1970, maxTimestamp=null, maxNumberOfResults=10000}, ContributorQuery{componentID='netarkiv-checksum-devel-01, minTimestamp=Thu Jan 01 01:00:00 CET 1970, maxTimestamp=null, maxNumberOfResults=10000}, ContributorQuery{componentID='netarkiv-kbh-disk-devel-01, minTimestamp=Thu Jan 01 01:00:00 CET 1970, maxTimestamp=null, maxNumberOfResults=10000}].
09:48:57.807 DEBUG org.bitrepository.common.JaxbHelper - Creating JAXBHelper based on schema from: jar:file:/home/devel/ResetCollection/bitrepository-client/lib/bitrepository-message-xml-xsd-30.jar!/versioning/ProtocolVersionDefinition.xsd
09:48:58.362 DEBUG o.b.p.m.l.GetChecksumsMessageLogger - Sent IdentifyPillarsForGetChecksumsRequest(b3bce940) collectionID Test6, destination topic://netarkiv-devel: FileIDs=AllFiles
09:48:58.363 DEBUG o.b.commandline.GetChecksumsCmd - Received event: b3bce940: GET_CHECKSUMS: IDENTIFY_REQUEST_SENT: , Identifying pillars for GetChecksums
09:48:58.740 DEBUG o.b.p.m.l.GetChecksumsMessageLogger - Received IdentifyPillarsForGetChecksumsResponse(b3bce940) collectionID Test6, from netarkiv-kbh-disk-devel-01: IDENTIFICATION_POSITIVE(Checksum operation available.)
09:48:58.761 DEBUG o.b.commandline.GetChecksumsCmd - Received event: b3bce940: GET_CHECKSUMS: COMPONENT_IDENTIFIED: ContributorID netarkiv-kbh-disk-devel-01
09:48:58.761 DEBUG o.b.c.c.s.ContributorResponseStatus - Received response from: netarkiv-kbh-disk-devel-01
09:48:58.762 DEBUG o.b.c.c.s.ContributorResponseStatus - Expected contributors: [netarkiv-disk-devel-02, netarkiv-checksum-devel-01, netarkiv-tape-devel-01, netarkiv-disk-devel-01, netarkiv-kbh-disk-devel-01], components that have not answered: [netarkiv-disk-devel-02, netarkiv-checksum-devel-01, netarkiv-tape-devel-01, netarkiv-disk-devel-01]
09:48:59.017 DEBUG o.b.p.m.l.GetChecksumsMessageLogger - Received IdentifyPillarsForGetChecksumsResponse(b3bce940) collectionID Test6, from netarkiv-disk-devel-01: IDENTIFICATION_POSITIVE(IDENTIFICATION_POSITIVE)
09:48:59.026 DEBUG o.b.commandline.GetChecksumsCmd - Received event: b3bce940: GET_CHECKSUMS: COMPONENT_IDENTIFIED: ContributorID netarkiv-disk-devel-01
09:48:59.026 DEBUG o.b.c.c.s.ContributorResponseStatus - Received response from: netarkiv-disk-devel-01
09:48:59.026 DEBUG o.b.c.c.s.ContributorResponseStatus - Expected contributors: [netarkiv-disk-devel-02, netarkiv-checksum-devel-01, netarkiv-tape-devel-01, netarkiv-disk-devel-01, netarkiv-kbh-disk-devel-01], components that have not answered: [netarkiv-disk-devel-02, netarkiv-checksum-devel-01, netarkiv-tape-devel-01]
09:48:59.285 DEBUG o.b.p.m.l.GetChecksumsMessageLogger - Received IdentifyPillarsForGetChecksumsResponse(b3bce940) collectionID Test6, from netarkiv-tape-devel-01: IDENTIFICATION_POSITIVE(IDENTIFICATION_POSITIVE)
09:48:59.299 DEBUG o.b.commandline.GetChecksumsCmd - Received event: b3bce940: GET_CHECKSUMS: COMPONENT_IDENTIFIED: ContributorID netarkiv-tape-devel-01
09:48:59.299 DEBUG o.b.c.c.s.ContributorResponseStatus - Received response from: netarkiv-tape-devel-01
09:48:59.299 DEBUG o.b.c.c.s.ContributorResponseStatus - Expected contributors: [netarkiv-disk-devel-02, netarkiv-checksum-devel-01, netarkiv-tape-devel-01, netarkiv-disk-devel-01, netarkiv-kbh-disk-devel-01], components that have not answered: [netarkiv-disk-devel-02, netarkiv-checksum-devel-01]
09:48:59.527 DEBUG o.b.p.m.l.GetChecksumsMessageLogger - Received IdentifyPillarsForGetChecksumsResponse(b3bce940) collectionID Test6, from netarkiv-checksum-devel-01: IDENTIFICATION_POSITIVE(Operation acknowledged and accepted.)
09:48:59.530 DEBUG o.b.commandline.GetChecksumsCmd - Received event: b3bce940: GET_CHECKSUMS: COMPONENT_IDENTIFIED: ContributorID netarkiv-checksum-devel-01
09:48:59.530 DEBUG o.b.c.c.s.ContributorResponseStatus - Received response from: netarkiv-checksum-devel-01
09:48:59.530 DEBUG o.b.c.c.s.ContributorResponseStatus - Expected contributors: [netarkiv-disk-devel-02, netarkiv-checksum-devel-01, netarkiv-tape-devel-01, netarkiv-disk-devel-01, netarkiv-kbh-disk-devel-01], components that have not answered: [netarkiv-disk-devel-02]
09:49:07.804 DEBUG o.b.commandline.GetChecksumsCmd - Received event: b3bce940: GET_CHECKSUMS: IDENTIFY_TIMEOUT: , Time has run out for looking up contributors
The following contributors didn't respond: [netarkiv-disk-devel-02]
09:49:07.805 DEBUG o.b.commandline.GetChecksumsCmd - Received event: b3bce940: GET_CHECKSUMS: COMPONENT_FAILED: ContributorID netarkiv-disk-devel-02 ResponseCode: null, Timeout for identifying contributor
09:49:07.806 DEBUG o.b.commandline.GetChecksumsCmd - Received event: b3bce940: GET_CHECKSUMS: IDENTIFICATION_COMPLETE: Contributors: [netarkiv-checksum-devel-01, netarkiv-kbh-disk-devel-01, netarkiv-disk-devel-01, netarkiv-tape-devel-01]
09:49:07.808 DEBUG o.b.commandline.GetChecksumsCmd - Received event: b3bce940: GET_CHECKSUMS: REQUEST_SENT: ContributorID [netarkiv-checksum-devel-01, netarkiv-tape-devel-01, netarkiv-disk-devel-01, netarkiv-kbh-disk-devel-01], Sending GetFileIDsRequest's
09:49:08.167 DEBUG o.b.p.m.l.GetChecksumsMessageLogger - Sent GetChecksumsRequest(b3bce940) collectionID Test6, destination queue://netarkiv-tape-devel-01: FileIDs=AllFiles, ChecksumRequestForExistingFile=org.bitrepository.bitrepositoryelements.ChecksumSpecTYPE@73e22a3d[checksumType=MD5, checksumSalt=<null>(default), otherChecksumType=<null>(default)], MaxNumberOfResults=10000, MinTimestamp=1970-01-01T01:00:00.000+01:00
09:49:08.459 DEBUG o.b.p.m.l.GetChecksumsMessageLogger - Sent GetChecksumsRequest(b3bce940) collectionID Test6, destination queue://netarkiv-disk-devel-01: FileIDs=AllFiles, ChecksumRequestForExistingFile=org.bitrepository.bitrepositoryelements.ChecksumSpecTYPE@73e22a3d[checksumType=MD5, checksumSalt=<null>(default), otherChecksumType=<null>(default)], MaxNumberOfResults=10000, MinTimestamp=1970-01-01T01:00:00.000+01:00
09:49:08.865 DEBUG o.b.p.m.l.GetChecksumsMessageLogger - Sent GetChecksumsRequest(b3bce940) collectionID Test6, destination topic://netarkiv-devel-netarkiv-checksum-devel-01-contributor: FileIDs=AllFiles, ChecksumRequestForExistingFile=org.bitrepository.bitrepositoryelements.ChecksumSpecTYPE@73e22a3d[checksumType=MD5, checksumSalt=<null>(default), otherChecksumType=<null>(default)], MaxNumberOfResults=10000, MinTimestamp=1970-01-01T01:00:00.000+01:00
09:49:08.884 DEBUG o.b.p.m.l.GetChecksumsMessageLogger - Received GetChecksumsProgressResponse(b3bce940) collectionID Test6, from netarkiv-tape-devel-01: OPERATION_ACCEPTED_PROGRESS(OPERATION_ACCEPTED_PROGRESS)
09:49:08.890 DEBUG o.b.commandline.GetChecksumsCmd - Received event: b3bce940: GET_CHECKSUMS: PROGRESS: ContributorID netarkiv-tape-devel-01, OPERATION_ACCEPTED_PROGRESS
09:49:09.360 DEBUG o.b.p.m.l.GetChecksumsMessageLogger - Sent GetChecksumsRequest(b3bce940) collectionID Test6, destination topic://netarkiv-devel.netarkiv-kbh-disk-devel-01: FileIDs=AllFiles, ChecksumRequestForExistingFile=org.bitrepository.bitrepositoryelements.ChecksumSpecTYPE@73e22a3d[checksumType=MD5, checksumSalt=<null>(default), otherChecksumType=<null>(default)], MaxNumberOfResults=10000, MinTimestamp=1970-01-01T01:00:00.000+01:00
09:49:09.395 DEBUG o.b.p.m.l.GetChecksumsMessageLogger - Received GetChecksumsFinalResponse(b3bce940) collectionID Test6, from netarkiv-tape-devel-01: OPERATION_COMPLETED(OPERATION_COMPLETED), NumberOfChecksums=0, ChecksumRequestForExistingFile=org.bitrepository.bitrepositoryelements.ChecksumSpecTYPE@7e6d319f[checksumType=MD5, checksumSalt=<null>(default), otherChecksumType=<null>(default)], PartialResult=false
09:49:09.404 DEBUG o.b.c.c.s.ContributorResponseStatus - Received response from: netarkiv-tape-devel-01
09:49:09.404 DEBUG o.b.c.c.s.ContributorResponseStatus - Expected contributors: [netarkiv-checksum-devel-01, netarkiv-tape-devel-01, netarkiv-disk-devel-01, netarkiv-kbh-disk-devel-01], components that have not answered: [netarkiv-checksum-devel-01, netarkiv-disk-devel-01, netarkiv-kbh-disk-devel-01]
09:49:09.595 DEBUG o.b.p.m.l.GetChecksumsMessageLogger - Received GetChecksumsProgressResponse(b3bce940) collectionID Test6, from netarkiv-disk-devel-01: OPERATION_ACCEPTED_PROGRESS(OPERATION_ACCEPTED_PROGRESS)
09:49:09.596 DEBUG o.b.commandline.GetChecksumsCmd - Received event: b3bce940: GET_CHECKSUMS: PROGRESS: ContributorID netarkiv-disk-devel-01, OPERATION_ACCEPTED_PROGRESS
09:49:09.831 DEBUG o.b.p.m.l.GetChecksumsMessageLogger - Received GetChecksumsFinalResponse(b3bce940) collectionID Test6, from netarkiv-disk-devel-01: OPERATION_COMPLETED(OPERATION_COMPLETED), NumberOfChecksums=0, ChecksumRequestForExistingFile=org.bitrepository.bitrepositoryelements.ChecksumSpecTYPE@7b06a3b[checksumType=MD5, checksumSalt=<null>(default), otherChecksumType=<null>(default)], PartialResult=false
09:49:09.837 DEBUG o.b.c.c.s.ContributorResponseStatus - Received response from: netarkiv-disk-devel-01
09:49:09.837 DEBUG o.b.c.c.s.ContributorResponseStatus - Expected contributors: [netarkiv-checksum-devel-01, netarkiv-tape-devel-01, netarkiv-disk-devel-01, netarkiv-kbh-disk-devel-01], components that have not answered: [netarkiv-checksum-devel-01, netarkiv-kbh-disk-devel-01]
09:49:10.131 DEBUG o.b.p.m.l.GetChecksumsMessageLogger - Received GetChecksumsProgressResponse(b3bce940) collectionID Test6, from netarkiv-checksum-devel-01: OPERATION_ACCEPTED_PROGRESS(Starting to calculate checksums.)
09:49:10.149 DEBUG o.b.commandline.GetChecksumsCmd - Received event: b3bce940: GET_CHECKSUMS: PROGRESS: ContributorID netarkiv-checksum-devel-01, Starting to calculate checksums.
09:49:10.331 DEBUG o.b.p.m.l.GetChecksumsMessageLogger - Received GetChecksumsFinalResponse(b3bce940) collectionID Test6, from netarkiv-checksum-devel-01: OPERATION_COMPLETED, NumberOfChecksums=0, ChecksumRequestForExistingFile=org.bitrepository.bitrepositoryelements.ChecksumSpecTYPE@f065590[checksumType=MD5, checksumSalt=<null>(default), otherChecksumType=<null>(default)], PartialResult=false
09:49:10.332 DEBUG o.b.c.c.s.ContributorResponseStatus - Received response from: netarkiv-checksum-devel-01
09:49:10.333 DEBUG o.b.c.c.s.ContributorResponseStatus - Expected contributors: [netarkiv-checksum-devel-01, netarkiv-tape-devel-01, netarkiv-disk-devel-01, netarkiv-kbh-disk-devel-01], components that have not answered: [netarkiv-kbh-disk-devel-01]
09:49:10.545 DEBUG o.b.p.m.l.GetChecksumsMessageLogger - Received GetChecksumsFinalResponse(b3bce940) collectionID Test6, from netarkiv-kbh-disk-devel-01: OPERATION_COMPLETED(Completed checksum retrieval.), NumberOfChecksums=0, ChecksumRequestForExistingFile=org.bitrepository.bitrepositoryelements.ChecksumSpecTYPE@23aff414[checksumType=MD5, checksumSalt=<null>(default), otherChecksumType=<null>(default)], PartialResult=false
09:49:10.546 DEBUG o.b.c.c.s.ContributorResponseStatus - Received response from: netarkiv-kbh-disk-devel-01
09:49:10.546 DEBUG o.b.c.c.s.ContributorResponseStatus - Expected contributors: [netarkiv-checksum-devel-01, netarkiv-tape-devel-01, netarkiv-disk-devel-01, netarkiv-kbh-disk-devel-01], components that have not answered: []
Deleting file FileID: with checksum Checksum: in collection Test6 from pillars netarkiv-tape-devel-01 netarkiv-disk-devel-01 netarkiv-checksum-devel-01 netarkiv-kbh-disk-devel-01
09:49:11.820 DEBUG org.bitrepository.common.JaxbHelper - Creating JAXBHelper based on schema from: jar:file:/home/devel/ResetCollection/bitrepository-client/lib/bitrepository-repository-settings-xsd-12.jar!/xsd/RepositorySettings.xsd
09:49:11.892 DEBUG o.b.c.settings.XMLFileSettingsLoader - Loading the settings file '/home/devel/ResetCollection/bitrepository-client/bin/../conf/RepositorySettings.xml'.
09:49:12.165 DEBUG org.bitrepository.common.JaxbHelper - Creating JAXBHelper based on schema from: jar:file:/home/devel/ResetCollection/bitrepository-client/lib/bitrepository-reference-settings-1.9.0.1.jar!/xsd/ReferenceSettings.xsd
09:49:12.202 DEBUG o.b.c.settings.XMLFileSettingsLoader - Loading the settings file '/home/devel/ResetCollection/bitrepository-client/bin/../conf/ReferenceSettings.xml'.
09:49:12.874 DEBUG o.b.p.security.BasicSecurityManager - X509CertificateHolder found
09:49:12.895 DEBUG o.b.p.security.BasicSecurityManager - Got something, that we don't (yet) recognize. Class: PEMKeyPair
09:49:12.895 DEBUG o.b.p.security.BasicSecurityManager - PrivateKeyInfo found
09:49:13.056 INFO o.b.p.activemq.ActiveMQMessageBus - Initializing ActiveMQMessageBus:'org.bitrepository.settings.repositorysettings.MessageBusConfiguration@13c9d689[name=netarkiv-aar-bit-master-devel-01 broker, url=failover:(ssl://netarkiv-aar-bit-master-devel-01.statsbiblioteket.dk:61612?keepAlive=true)]'.
09:49:13.057 DEBUG org.bitrepository.common.JaxbHelper - Creating JAXBHelper based on schema from: jar:file:/home/devel/ResetCollection/bitrepository-client/lib/bitrepository-message-xml-xsd-30.jar!/xsd/BitRepositoryMessages.xsd
09:49:14.064 INFO o.a.a.t.failover.FailoverTransport - Successfully connected to ssl://netarkiv-aar-bit-master-devel-01.statsbiblioteket.dk:61612?keepAlive=true
09:49:14.195 DEBUG o.b.p.activemq.ActiveMQMessageBus - ActiveMQConnection initialized for 'org.bitrepository.settings.repositorysettings.MessageBusConfiguration@13c9d689[name=netarkiv-aar-bit-master-devel-01 broker, url=failover:(ssl://netarkiv-aar-bit-master-devel-01.statsbiblioteket.dk:61612?keepAlive=true)]'.
09:49:14.198 INFO o.b.p.activemq.ActiveMQMessageBus - Settings component filter to: [kb-test-har-003.kb.dk-devel-5302]
09:49:14.201 DEBUG o.b.c.c.m.CollectionBasedConversationMediator - Initializing the CollectionBasedConversationMediator
09:49:14.201 DEBUG o.b.p.activemq.ActiveMQMessageBus - Adding listener 'org.bitrepository.client.conversation.mediator.CollectionBasedConversationMediator@4d5b6aac' to destination: 'topic://netarkiv-devel-kb-test-har-003.kb.dk-devel-5302' on message-bus 'netarkiv-aar-bit-master-devel-01 broker'.
09:49:14.202 DEBUG o.b.p.activemq.ActiveMQMessageBus - Retrieving message consumer on destination 'topic://netarkiv-devel-kb-test-har-003.kb.dk-devel-5302' for listener 'org.bitrepository.client.conversation.mediator.CollectionBasedConversationMediator@4d5b6aac'. Key: 'topic://netarkiv-devel-kb-test-har-003.kb.dk-devel-5302#1297836716'.
09:49:14.202 DEBUG o.b.p.activemq.ActiveMQMessageBus - No consumer known. Creating new for key 'topic://netarkiv-devel-kb-test-har-003.kb.dk-devel-5302#1297836716'.
09:49:14.222 DEBUG o.b.commandline.DeleteFileCmd - Performing the DeleteFile operation.
java.lang.IllegalArgumentException: String hexString, 'Checksum:', must be an even number of characters.
at org.bitrepository.common.ArgumentValidator.checkTrue(ArgumentValidator.java:160)
at org.bitrepository.common.utils.Base16Utils.encodeBase16(Base16Utils.java:64)
at org.bitrepository.commandline.CommandLineClient.getChecksumDataForDeleteValidation(CommandLineClient.java:302)
at org.bitrepository.commandline.DeleteFileCmd.deleteTheFile(DeleteFileCmd.java:125)
at org.bitrepository.commandline.DeleteFileCmd.performOperation(DeleteFileCmd.java:107)
at org.bitrepository.commandline.CommandLineClient.runCommand(CommandLineClient.java:68)
at org.bitrepository.commandline.DeleteFileCmd.main(DeleteFileCmd.java:47)
xargs: ssh: exited with status 255; aborting
calling cleanup_hosts
running cleanup_hosts
Removing installation on sb-test-har-001.statsbiblioteket.dk
Removing installation on kb-test-har-003.kb.dk
Removing installation on kb-test-har-004.kb.dk
Removing installation on kb-test-har-005.kb.dk
Removing installation on kb-test-adm-001.kb.dk
Removing installation on kb-test-acs-001.kb.dk
Removing installation on kb-test-way-001.kb.dk
calling cleanup_linuxbarhosts
running cleanup_linuxbarhosts
Removing installation on sb-test-bar-001.statsbiblioteket.dk
Removing bitarchive on sb-test-bar-001.statsbiblioteket.dk
Calling cleanup_windowsbarhosts
running cleanup_windowsbarhosts
Removing C:\bitarkiv\Test6 on kb-test-bar-01.bitarkiv.kb.dk
ssh: connect to host kb-test-bar-01.bitarkiv.kb.dk port 22: Connection timed out
Removing E:\bitarchive_1\Test6 on kb-test-bar-01.bitarkiv.kb.dk
ssh: connect to host kb-test-bar-01.bitarkiv.kb.dk port 22: Connection timed out
Removing F:\bitarchive_2\Test6 on kb-test-bar-01.bitarkiv.kb.dk
ssh: connect to host kb-test-bar-01.bitarkiv.kb.dk port 22: Connection timed out
Removing g:\bitarchive_3\Test6 on kb-test-bar-01.bitarkiv.kb.dk
ssh: connect to host kb-test-bar-01.bitarkiv.kb.dk port 22: Connection timed out
Removing C:\Users\ba-devel\Test6 on kb-test-bar-01.bitarkiv.kb.dk
ssh: connect to host kb-test-bar-01.bitarkiv.kb.dk port 22: Connection timed out
TESTX=Test6
PORT=8078
MAILRECEIVERS=csr@kb.dk
VERSION=CSR
DEPLOYCONF=deploy_config_bitmag.xml
H3ZIP=CSR
BITMAGCONF=/home/devel/bitmag_test_config/client_config
Using bundler file /home/devel/nas_versions/bundler/NetarchiveSuite-heritrix3-bundler-CSR.zip
Creating distribution files in /home/devel/prepared_software/Test6.
Will read source files using encoding 'UTF-8' (defaulted)
Branch in settings not found: common.warcRecordService
Branch in settings not found: common.arcrepositoryClient.bitrepository.getFileIDsMaxResults
Branch in settings not found: common.hadoop.kerberos
Branch in settings not found: common.hadoop.username
Branch in settings not found: common.hadoop.mapred.hadoopUberJar
Branch in settings not found: common.hadoop.mapred.inputFilesParentDir
Branch in settings not found: common.hadoop.mapred.cdxJob
Branch in settings not found: common.hadoop.mapred.metadataExtractionJob
Branch in settings not found: common.hadoop.mapred.metadataCDXExtractionJob
Branch in settings not found: common.hadoop.mapred.crawlLogExtractionJob
Branch in settings not found: common.fileResolver.class
Branch in settings not found: common.fileResolver.baseUrl
Branch in settings not found: common.fileResolver.keyfile
Branch in settings not found: common.webinterface.metadata_cache_dir
Branch in settings not found: harvester.harvesting.heritrix3.bundle
Branch in settings not found: harvester.harvesting.heritrix3.certificate
Branch in settings not found: harvester.harvesting.heritrix3.bundle
Branch in settings not found: harvester.harvesting.heritrix3.certificate
Branch in settings not found: harvester.harvesting.umbra
Branch in settings not found: harvester.harvesting.heritrix3.bundle
Branch in settings not found: harvester.harvesting.heritrix3.certificate
Branch in settings not found: harvester.harvesting.umbra
Branch in settings not found: harvester.harvesting.heritrix3.bundle
Branch in settings not found: harvester.harvesting.heritrix3.bundle
Machine 'kb-test-adm-001.kb.dk' configured with encoding 'UTF-8' (defaulted)
Machine 'kb-test-har-003.kb.dk' configured with encoding 'UTF-8' (defaulted)
NetarchiveSuite-heritrix3-bundler.zip -> /home/devel/Test6/NetarchiveSuite-heritrix3-bundler.zip
/home/devel/quickstart-nas-5.0-h3/h3server.jks -> /home/devel/Test6/h3server.jks
Machine 'kb-test-har-005.kb.dk' configured with encoding 'UTF-8' (defaulted)
NetarchiveSuite-heritrix3-bundler.zip -> /home/devel/Test6/NetarchiveSuite-heritrix3-bundler.zip
/home/devel/quickstart-nas-5.0-h3/h3server.jks -> /home/devel/Test6/h3server.jks
Machine 'kb-test-har-004.kb.dk' configured with encoding 'UTF-8' (defaulted)
NetarchiveSuite-heritrix3-bundler.zip -> /home/devel/Test6/NetarchiveSuite-heritrix3-bundler.zip
Machine 'kb-test-acs-001.kb.dk' configured with encoding 'UTF-8' (defaulted)
Machine 'kb-test-way-001.kb.dk' configured with encoding 'UTF-8' (defaulted)
Machine 'sb-test-har-001.statsbiblioteket.dk' configured with encoding 'UTF-8' (defaulted)
/home/devel/prepared_software/Test6/NetarchiveSuite-heritrix3-bundler.zip -> /home/netarkdv/Test6/NetarchiveSuite-heritrix3-bundler.zip
09:54:42.990 [main] DEBUG dk.netarkivet.deploy.XmlStructure - Element baseDir is not a branch in the tree [archive, bitpreservation, baseDir]. Null returned
09:54:42.999 [main] DEBUG dk.netarkivet.deploy.XmlStructure - Element is not text. Null returned. Path given to getLeafValue archive
09:54:43.001 [main] DEBUG dk.netarkivet.deploy.XmlStructure - Element baseDir is not a branch in the tree [archive, bitpreservation, baseDir]. Null returned
09:54:43.001 [main] DEBUG dk.netarkivet.deploy.XmlStructure - Element is not text. Null returned. Path given to getLeafValue archive
09:54:43.001 [main] DEBUG dk.netarkivet.deploy.XmlStructure - Element baseDir is not a branch in the tree [archive, bitpreservation, baseDir]. Null returned
09:54:43.002 [main] DEBUG dk.netarkivet.deploy.XmlStructure - Element is not text. Null returned. Path given to getLeafValue archive
09:54:43.002 [main] DEBUG dk.netarkivet.deploy.XmlStructure - Element baseDir is not a branch in the tree [archive, bitpreservation, baseDir]. Null returned
09:54:43.002 [main] DEBUG dk.netarkivet.deploy.XmlStructure - Element is not text. Null returned. Path given to getLeafValue archive
09:54:43.003 [main] DEBUG dk.netarkivet.deploy.XmlStructure - Element baseDir is not a branch in the tree [archive, bitpreservation, baseDir]. Null returned
09:54:43.003 [main] DEBUG dk.netarkivet.deploy.XmlStructure - Element is not text. Null returned. Path given to getLeafValue archive
09:54:43.004 [main] DEBUG dk.netarkivet.deploy.XmlStructure - Element baseDir is not a branch in the tree [archive, bitpreservation, baseDir]. Null returned
09:54:43.004 [main] DEBUG dk.netarkivet.deploy.XmlStructure - Element is not text. Null returned. Path given to getLeafValue archive
09:54:43.111 [main] DEBUG dk.netarkivet.deploy.XmlStructure - Element baseDir is not a branch in the tree [archive, bitpreservation, baseDir]. Null returned
09:54:43.111 [main] DEBUG dk.netarkivet.deploy.XmlStructure - Element is not text. Null returned. Path given to getLeafValue archive
--------------------------------------------
INSTALLING TO MACHINE: devel@kb-test-adm-001.kb.dk
copying NetarchiveSuite-CSR.zip to:kb-test-adm-001.kb.dk
deleting devel@kb-test-adm-001.kb.dk:/home/devel/Test6/lib
unzipping NetarchiveSuite-CSR.zip at:kb-test-adm-001.kb.dk
Creating directories.
preparing for copying of settings and scripts
copying settings and scripts
make scripts executable
make password and access files readonly
--------------------------------------------
INSTALLING TO MACHINE: devel@kb-test-har-003.kb.dk
copying NetarchiveSuite-CSR.zip to:kb-test-har-003.kb.dk
deleting devel@kb-test-har-003.kb.dk:/home/devel/Test6/lib
unzipping NetarchiveSuite-CSR.zip at:kb-test-har-003.kb.dk
Creating directories.
preparing for copying of settings and scripts
copying settings and scripts
make scripts executable
make password and access files readonly
--------------------------------------------
INSTALLING TO MACHINE: devel@kb-test-har-005.kb.dk
copying NetarchiveSuite-CSR.zip to:kb-test-har-005.kb.dk
deleting devel@kb-test-har-005.kb.dk:/home/devel/Test6/lib
unzipping NetarchiveSuite-CSR.zip at:kb-test-har-005.kb.dk
Creating directories.
preparing for copying of settings and scripts
copying settings and scripts
make scripts executable
make password and access files readonly
--------------------------------------------
INSTALLING TO MACHINE: devel@kb-test-har-004.kb.dk
copying NetarchiveSuite-CSR.zip to:kb-test-har-004.kb.dk
deleting devel@kb-test-har-004.kb.dk:/home/devel/Test6/lib
unzipping NetarchiveSuite-CSR.zip at:kb-test-har-004.kb.dk
Creating directories.
preparing for copying of settings and scripts
copying settings and scripts
make scripts executable
make password and access files readonly
--------------------------------------------
INSTALLING TO MACHINE: devel@kb-test-acs-001.kb.dk
copying NetarchiveSuite-CSR.zip to:kb-test-acs-001.kb.dk
deleting devel@kb-test-acs-001.kb.dk:/home/devel/Test6/lib
unzipping NetarchiveSuite-CSR.zip at:kb-test-acs-001.kb.dk
Creating directories.
preparing for copying of settings and scripts
copying settings and scripts
make scripts executable
make password and access files readonly
--------------------------------------------
INSTALLING TO MACHINE: devel@kb-test-way-001.kb.dk
copying NetarchiveSuite-CSR.zip to:kb-test-way-001.kb.dk
deleting devel@kb-test-way-001.kb.dk:/home/devel/Test6/lib
unzipping NetarchiveSuite-CSR.zip at:kb-test-way-001.kb.dk
Creating directories.
preparing for copying of settings and scripts
copying settings and scripts
make scripts executable
make password and access files readonly
--------------------------------------------
--------------------------------------------
INSTALLING TO MACHINE: netarkdv@sb-test-har-001.statsbiblioteket.dk
copying NetarchiveSuite-CSR.zip to:sb-test-har-001.statsbiblioteket.dk
deleting netarkdv@sb-test-har-001.statsbiblioteket.dk:/home/netarkdv/Test6/lib
unzipping NetarchiveSuite-CSR.zip at:sb-test-har-001.statsbiblioteket.dk
Creating directories.
preparing for copying of settings and scripts
copying settings and scripts
make scripts executable
make password and access files readonly
--------------------------------------------
Initializing database for system Test6
+ host=devel@kb-test-adm-001
+ scp -p /home/devel/devel-config/scripts/db/setup_admindb.sh /home/devel/devel-config/scripts/db/setup_harvestdb.sh /home/devel/devel-config/scripts/db/teardown_admindb.sh /home/devel/devel-config/scripts/db/teardown_harvestdb.sh /home/devel/devel-config/scripts/../resources/harvestdb.testdata.sql /home/devel/devel-config/scripts/../resources/default_orderxml.cxml devel@kb-test-adm-001:bin/
+ ssh devel@kb-test-adm-001
Pseudo-terminal will not be allocated because stdin is not a terminal.
Dette system må kun tilgås af brugere autoriserede af Det Kgl. Bibliotek.
This system is only to be used by users authorized by
the Royal Danish Library.
Destroying test6_admindb
DROP DATABASE
Creating empty test6_admindb
CREATE DATABASE
Initialising test6_admindb
CREATE TABLE
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
CREATE TABLE
CREATE SEQUENCE
ALTER TABLE
CREATE TABLE
CREATE SEQUENCE
ALTER TABLE
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE TABLE
CREATE SEQUENCE
ALTER TABLE
CREATE INDEX
CREATE TABLE
CREATE SEQUENCE
ALTER TABLE
Destroying test6_harvestdb
DROP DATABASE
CREATE DATABASE
CREATE TABLE
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
GRANT
CREATE TABLE
CREATE SEQUENCE
ALTER TABLE
CREATE INDEX
CREATE INDEX
GRANT
GRANT
CREATE TABLE
CREATE SEQUENCE
ALTER TABLE
CREATE INDEX
CREATE INDEX
CREATE INDEX
GRANT
GRANT
CREATE TABLE
GRANT
CREATE TABLE
GRANT
CREATE TABLE
CREATE SEQUENCE
ALTER TABLE
CREATE INDEX
CREATE INDEX
GRANT
GRANT
CREATE TABLE
CREATE SEQUENCE
ALTER TABLE
CREATE INDEX
CREATE INDEX
GRANT
GRANT
CREATE TABLE
CREATE SEQUENCE
ALTER TABLE
CREATE INDEX
GRANT
GRANT
CREATE TABLE
CREATE SEQUENCE
ALTER TABLE
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
GRANT
GRANT
CREATE TABLE
CREATE INDEX
GRANT
CREATE TABLE
GRANT
CREATE TABLE
CREATE INDEX
GRANT
CREATE TABLE
GRANT
CREATE TABLE
CREATE SEQUENCE
ALTER TABLE
GRANT
GRANT
CREATE TABLE
CREATE SEQUENCE
ALTER TABLE
GRANT
GRANT
CREATE TABLE
CREATE INDEX
CREATE INDEX
GRANT
CREATE TABLE
CREATE INDEX
GRANT
CREATE TABLE
CREATE SEQUENCE
ALTER TABLE
CREATE INDEX
GRANT
GRANT
CREATE TABLE
CREATE SEQUENCE
ALTER TABLE
CREATE INDEX
CREATE INDEX
GRANT
GRANT
CREATE TABLE
CREATE INDEX
CREATE INDEX
CREATE INDEX
GRANT
CREATE TABLE
CREATE INDEX
CREATE INDEX
GRANT
CREATE TABLE
GRANT
CREATE TABLE
CREATE SEQUENCE
ALTER TABLE
GRANT
CREATE TABLE
CREATE SEQUENCE
ALTER TABLE
GRANT
CREATE TABLE
CREATE SEQUENCE
ALTER TABLE
GRANT
INSERT 0 1
INSERT 0 1
CREATE TABLE
CREATE SEQUENCE
ALTER TABLE
CREATE INDEX
GRANT
GRANT
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE INDEX
CREATE SEQUENCE
CREATE TABLE
ALTER TABLE
CREATE INDEX
GRANT
GRANT
GRANT
INSERT 0 1
INSERT 0 1
INSERT 0 1
SET
SET
SET
SET
SET
SET
SET
SET
SET
COPY 16
COPY 16
setval
--------
17
(1 row)
COPY 16
setval
--------
17
(1 row)
COPY 3
setval
--------
17
(1 row)
COPY 2
setval
--------
5
(1 row)
COPY 4
setval
--------
4
(1 row)
COPY 25
COPY 16
setval
--------
17
(1 row)
09:56:38.601 [MLog-Init-Reporter] INFO com.mchange.v2.log.MLog - MLog clients using slf4j logging.
09:56:38.637 [MLog-Init-Reporter] DEBUG com.mchange.v2.log.MLog - Reading VM config for path list /com/mchange/v2/log/default-mchange-log.properties, /mchange-commons.properties, /c3p0.properties, hocon:/reference,/application,/c3p0,/, /mchange-log.properties, /
09:56:38.637 [MLog-Init-Reporter] DEBUG com.mchange.v2.log.MLog - The configuration file for resource identifier '/mchange-commons.properties' could not be found. Skipping.
09:56:38.638 [MLog-Init-Reporter] DEBUG com.mchange.v2.log.MLog - The configuration file for resource identifier '/c3p0.properties' could not be found. Skipping.
09:56:38.638 [MLog-Init-Reporter] DEBUG com.mchange.v2.log.MLog - The configuration file for resource identifier 'hocon:/reference,/application,/c3p0,/' could not be found. Skipping.
09:56:38.638 [MLog-Init-Reporter] DEBUG com.mchange.v2.log.MLog - The configuration file for resource identifier '/mchange-log.properties' could not be found. Skipping.
09:56:38.649 [main] DEBUG com.mchange.v2.cfg.MConfig - The configuration file for resource identifier '/mchange-commons.properties' could not be found. Skipping.
09:56:38.649 [main] DEBUG com.mchange.v2.cfg.MConfig - The configuration file for resource identifier '/mchange-log.properties' could not be found. Skipping.
09:56:38.649 [main] DEBUG com.mchange.v2.cfg.MConfig - The configuration file for resource identifier 'hocon:/reference,/application,/c3p0,/' could not be found. Skipping.
09:56:38.649 [main] DEBUG com.mchange.v2.cfg.MConfig - The configuration file for resource identifier '/c3p0.properties' could not be found. Skipping.
09:56:39.185 [main] INFO com.mchange.v2.c3p0.C3P0Registry - Initializing c3p0-0.9.5.4 [built 23-March-2019 23:00:48 -0700; debug? true; trace: 10]
09:56:39.288 [main] DEBUG com.mchange.v2.c3p0.management.DynamicPooledDataSourceManagerMBean - MBean: com.mchange.v2.c3p0:type=PooledDataSource,identityToken=10bdx52ap1tjxrrf1ldowu8|f4168b8,name=10bdx52ap1tjxrrf1ldowu8|f4168b8 registered.
09:56:39.370 [main] DEBUG com.mchange.v2.c3p0.management.DynamicPooledDataSourceManagerMBean - MBean: com.mchange.v2.c3p0:type=PooledDataSource,identityToken=10bdx52ap1tjxrrf1ldowu8|f4168b8,name=10bdx52ap1tjxrrf1ldowu8|f4168b8 unregistered, in order to be reregistered after update.
09:56:39.371 [main] DEBUG com.mchange.v2.c3p0.management.DynamicPooledDataSourceManagerMBean - MBean: com.mchange.v2.c3p0:type=PooledDataSource,identityToken=10bdx52ap1tjxrrf1ldowu8|f4168b8,name=10bdx52ap1tjxrrf1ldowu8|f4168b8 registered.
09:56:39.402 [main] INFO dk.netarkivet.harvester.datamodel.HarvestDBConnection - Connection pool initialized with the following values:
- minPoolSize=5
- maxPoolSize=20
- acquireIncrement=5
- maxStatements=0
- maxStatementsPerConnection=0
- idleConnTestPeriod=0
- idleConnTestQuery='null'
- idleConnTestOnCheckin=false
09:56:39.528 [main] INFO com.mchange.v2.c3p0.impl.AbstractPoolBackedDataSource - Initializing c3p0 pool... com.mchange.v2.c3p0.ComboPooledDataSource [ acquireIncrement -> 5, acquireRetryAttempts -> 30, acquireRetryDelay -> 1000, autoCommitOnClose -> false, automaticTestTable -> null, breakAfterAcquireFailure -> false, checkoutTimeout -> 0, connectionCustomizerClassName -> null, connectionTesterClassName -> com.mchange.v2.c3p0.impl.DefaultConnectionTester, contextClassLoaderSource -> caller, dataSourceName -> 10bdx52ap1tjxrrf1ldowu8|f4168b8, debugUnreturnedConnectionStackTraces -> false, description -> null, driverClass -> org.postgresql.Driver, extensions -> {}, factoryClassLocation -> null, forceIgnoreUnresolvedTransactions -> false, forceSynchronousCheckins -> false, forceUseNamedDriverClass -> false, identityToken -> 10bdx52ap1tjxrrf1ldowu8|f4168b8, idleConnectionTestPeriod -> 0, initialPoolSize -> 3, jdbcUrl -> jdbc:postgresql://localhost:5432/test6_harvestdb, maxAdministrativeTaskTime -> 0, maxConnectionAge -> 3600, maxIdleTime -> 0, maxIdleTimeExcessConnections -> 0, maxPoolSize -> 20, maxStatements -> 0, maxStatementsPerConnection -> 0, minPoolSize -> 5, numHelperThreads -> 3, preferredTestQuery -> null, privilegeSpawnedThreads -> false, properties -> {user=******, password=******}, propertyCycle -> 0, statementCacheNumDeferredCloseThreads -> 0, testConnectionOnCheckin -> false, testConnectionOnCheckout -> false, unreturnedConnectionTimeout -> 0, userOverrides -> {}, usesTraditionalReflectiveProxies -> false ]
09:56:39.592 [main] DEBUG com.mchange.v2.cfg.MConfig - The configuration file for resource identifier '/mchange-commons.properties' could not be found. Skipping.
09:56:39.592 [main] DEBUG com.mchange.v2.cfg.MConfig - The configuration file for resource identifier '/mchange-log.properties' could not be found. Skipping.
09:56:39.592 [main] DEBUG com.mchange.v2.cfg.MConfig - The configuration file for resource identifier '/c3p0.properties' could not be found. Skipping.
09:56:39.592 [main] DEBUG com.mchange.v2.cfg.MConfig - The configuration file for resource identifier 'hocon:/reference,/application,/c3p0,/' could not be found. Skipping.
09:56:39.593 [main] WARN com.mchange.v2.resourcepool.BasicResourcePool - Bad pool size config, start 3 < min 5. Using 5 as start.
09:56:39.600 [main] DEBUG com.mchange.v2.resourcepool.BasicResourcePool - com.mchange.v2.resourcepool.BasicResourcePool@5be6e01c config: [start -> 5; min -> 5; max -> 20; inc -> 5; num_acq_attempts -> 30; acq_attempt_delay -> 1000; check_idle_resources_delay -> 0; max_resource_age -> 3600000; max_idle_time -> 0; excess_max_idle_time -> 0; destroy_unreturned_resc_time -> 0; expiration_enforcement_delay -> 900000; break_on_acquisition_failure -> false; debug_store_checkout_exceptions -> false; force_synchronous_checkins -> false]
09:56:39.600 [main] DEBUG com.mchange.v2.c3p0.impl.C3P0PooledConnectionPoolManager - Created new pool for auth, username (masked): 'de******'.
09:56:39.600 [main] DEBUG com.mchange.v2.resourcepool.BasicResourcePool - acquire test -- pool size: 0; target_pool_size: 5; desired target? 1
09:56:39.600 [main] DEBUG com.mchange.v2.resourcepool.BasicResourcePool - awaitAvailable(): [unknown]
09:56:40.033 [main] DEBUG com.mchange.v2.c3p0.impl.NewProxyPreparedStatement - com.mchange.v2.c3p0.impl.NewProxyPreparedStatement@3911c2a7 [wrapping: SELECT version FROM schemaversions WHERE tablename = 'ordertemplates'] closed orphaned ResultSet: com.mchange.v2.c3p0.impl.NewProxyResultSet@1725dc0f [wrapping: null]
09:56:40.055 [main] DEBUG com.mchange.v2.c3p0.impl.NewProxyPreparedStatement - com.mchange.v2.c3p0.impl.NewProxyPreparedStatement@2145b572 [wrapping: SELECT COUNT(*) FROM ordertemplates WHERE name = 'default_orderxml'] closed orphaned ResultSet: com.mchange.v2.c3p0.impl.NewProxyResultSet@15eb5ee5 [wrapping: null]
09:56:40.107 [main] DEBUG com.mchange.v2.c3p0.impl.NewProxyPreparedStatement - com.mchange.v2.c3p0.impl.NewProxyPreparedStatement@35ef1869 [wrapping: SELECT COUNT(*) FROM ordertemplates WHERE name = 'default_orderxml'] closed orphaned ResultSet: com.mchange.v2.c3p0.impl.NewProxyResultSet@3c407114 [wrapping: null]
The template 'default_orderxml' has now been updated.
+ set +x
--------------------------------------------
STARTING MACHINE: devel@kb-test-adm-001.kb.dk
Starting all applications on: kb-test-adm-001.kb.dk
Starting linux application: GUIApplication
Starting linux application: ArcRepositoryApplication
Starting linux application: HarvestJobManagerApplication
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/devel/Test6/lib/logback-classic-1.2.3.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/devel/Test6/lib/slf4j-log4j12-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
09:56:42,625 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [/home/devel/Test6/conf/logback_HarvestJobManagerApplication.xml] at [file:/home/devel/Test6/conf/logback_HarvestJobManagerApplication.xml]
09:56:43,049 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - debug attribute not set
09:56:43,055 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
09:56:43,121 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [STDOUT]
09:56:43,200 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
09:56:43,516 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
09:56:43,548 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [FILE]
09:56:43,708 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@1742920067 - No compression will be used
09:56:43,709 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@1742920067 - Will use the pattern ./log/HarvestJobManagerApplication.%d{yyyy-MM-dd}.%i.log for the active file
09:56:43,743 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@5d47c63f - The date pattern is 'yyyy-MM-dd' from file name pattern './log/HarvestJobManagerApplication.%d{yyyy-MM-dd}.%i.log'.
09:56:43,743 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@5d47c63f - Roll-over at midnight.
09:56:43,777 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@5d47c63f - Setting initial period to Mon Jul 11 09:56:43 CEST 2022
09:56:43,780 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@5d47c63f - SizeAndTimeBasedFNATP is deprecated. Use SizeAndTimeBasedRollingPolicy instead
09:56:43,780 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@5d47c63f - For more information see http://logback.qos.ch/manual/appenders.html#SizeAndTimeBasedRollingPolicy
09:56:43,794 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
09:56:43,818 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - Active log file name: ./log/HarvestJobManagerApplication.log
09:56:43,819 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - File property is set to [./log/HarvestJobManagerApplication.log]
09:56:43,825 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [dk.netarkivet.monitor.logging.CachingSLF4JAppender]
09:56:45,983 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [MONITOR]
09:56:45,999 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to WARN
09:56:45,999 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[ROOT]
09:56:46,002 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [FILE] to Logger[ROOT]
09:56:46,002 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [MONITOR] to Logger[ROOT]
09:56:46,003 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [dk.netarkivet] to DEBUG
09:56:46,003 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
09:56:46,012 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@5ea434c8 - Registering current configuration as safe fallback point
2022-07-11 09:56:46.040 DEBUG dk.netarkivet.common.utils.Settings.invoke0 - Value found in loaded data: SB,KB
SLF4J: A number (1) of logging calls during the initialization phase have been intercepted and are
SLF4J: now being replayed. These are subject to the filtering rules of the underlying logging system.
SLF4J: See also http://www.slf4j.org/codes.html#replay
2022-07-11 09:56:46.178 INFO d.n.c.distribute.JMSConnectionSunMQ.<init> - Creating instance of dk.netarkivet.common.distribute.JMSConnectionSunMQ
2022-07-11 09:56:46.186 DEBUG d.n.common.distribute.JMSConnection.initConnection - Initializing a JMS connection dk.netarkivet.common.distribute.JMSConnectionSunMQ
2022-07-11 09:56:46.191 INFO d.n.c.distribute.JMSConnectionSunMQ.getConnectionFactory - Establishing SunMQ JMS Connection to 'kb-test-adm-001.kb.dk:7676'
--------------------------------------------
STARTING MACHINE: devel@kb-test-har-003.kb.dk
Starting all applications on: kb-test-har-003.kb.dk
Starting linux application: HarvestControllerApplication_kblow001_h3
09:56:47,190 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [/home/devel/Test6/conf/logback_HarvestControllerApplication_kblow001_h3.xml] at [file:/home/devel/Test6/conf/logback_HarvestControllerApplication_kblow001_h3.xml]
09:56:47,397 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - debug attribute not set
09:56:47,400 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
09:56:47,409 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [STDOUT]
09:56:47,423 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
09:56:47,485 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
09:56:47,490 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [FILE]
09:56:47,516 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@858952163 - No compression will be used
09:56:47,517 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@858952163 - Will use the pattern ./log/HarvestControllerApplication_kblow001_h3.%d{yyyy-MM-dd}.%i.log for the active file
09:56:47,521 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@479d31f3 - The date pattern is 'yyyy-MM-dd' from file name pattern './log/HarvestControllerApplication_kblow001_h3.%d{yyyy-MM-dd}.%i.log'.
09:56:47,521 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@479d31f3 - Roll-over at midnight.
09:56:47,525 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@479d31f3 - Setting initial period to Mon Jul 11 09:56:47 CEST 2022
09:56:47,526 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@479d31f3 - SizeAndTimeBasedFNATP is deprecated. Use SizeAndTimeBasedRollingPolicy instead
09:56:47,526 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@479d31f3 - For more information see http://logback.qos.ch/manual/appenders.html#SizeAndTimeBasedRollingPolicy
09:56:47,527 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
09:56:47,530 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - Active log file name: ./log/HarvestControllerApplication_kblow001_h3.log
09:56:47,530 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - File property is set to [./log/HarvestControllerApplication_kblow001_h3.log]
09:56:47,531 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [dk.netarkivet.monitor.logging.CachingSLF4JAppender]
09:56:48,062 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [MONITOR]
09:56:48,068 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to WARN
09:56:48,068 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[ROOT]
09:56:48,068 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [FILE] to Logger[ROOT]
09:56:48,068 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [MONITOR] to Logger[ROOT]
09:56:48,069 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [dk.netarkivet] to DEBUG
09:56:48,069 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
09:56:48,070 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@40ef3420 - Registering current configuration as safe fallback point
2022-07-11 09:56:48.075 DEBUG dk.netarkivet.common.utils.Settings.invoke0 - Value found in loaded data: SB,KB
SLF4J: A number (1) of logging calls during the initialization phase have been intercepted and are
SLF4J: now being replayed. These are subject to the filtering rules of the underlying logging system.
SLF4J: See also http://www.slf4j.org/codes.html#replay
Starting dk.netarkivet.harvester.heritrix3.HarvestControllerServer
Version: 7.4-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/530d443d1c093ced66f5fc473e516c6b5c0373f6)
2022-07-11 09:56:48.082 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Starting dk.netarkivet.harvester.heritrix3.HarvestControllerServer
Version: 7.4-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/530d443d1c093ced66f5fc473e516c6b5c0373f6)
Java VM: 1.8.0_332
2022-07-11 09:56:48.083 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Java VM: 1.8.0_332
java.home: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.332.b09-1.el7_9.x86_64/jre
2022-07-11 09:56:48.084 INFO d.n.common.utils.ApplicationUtils.logAndPrint - java.home: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.332.b09-1.el7_9.x86_64/jre
Working dir: /home/devel/Test6
2022-07-11 09:56:48.084 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Working dir: /home/devel/Test6
2022-07-11 09:56:48.084 INFO d.n.common.utils.ApplicationUtils.startApp - Using settings files '/home/devel/Test6/conf/settings_HarvestControllerApplication_kblow001_h3.xml'
2022-07-11 09:56:48.212 INFO d.n.m.d.JMSMonitorRegistryClient.register - Registering this client for monitoring every 1 minutes, using hostname 'kb-test-har-003.kb.dk' and JMX/RMI ports 8170/8270
2022-07-11 09:56:48.212 INFO d.n.c.m.MBeanConnectorCreator.exposeJMXMBeanServer - Registered mbean server in registry on port 8170 communicating on port 8270 using password file 'conf/jmxremote.password'.
Service URL is service:jmx:rmi://kb-test-har-003.kb.dk:8270/jndi/rmi://kb-test-har-003.kb.dk:8170/jmxrmi
dk.netarkivet.harvester.heritrix3.HarvestControllerServer Running
2022-07-11 09:56:48.218 INFO d.n.common.utils.ApplicationUtils.logAndPrint - dk.netarkivet.harvester.heritrix3.HarvestControllerServer Running
2022-07-11 09:56:48.220 INFO d.n.c.distribute.JMSConnectionSunMQ.<init> - Creating instance of dk.netarkivet.common.distribute.JMSConnectionSunMQ
2022-07-11 09:56:48.220 DEBUG d.n.common.distribute.JMSConnection.initConnection - Initializing a JMS connection dk.netarkivet.common.distribute.JMSConnectionSunMQ
2022-07-11 09:56:48.221 INFO d.n.c.distribute.JMSConnectionSunMQ.getConnectionFactory - Establishing SunMQ JMS Connection to 'kb-test-adm-001.kb.dk:7676'
2022-07-11 09:56:48.227 INFO d.n.h.h.HarvestControllerServer.<init> - Starting HarvestControllerServer.
2022-07-11 09:56:48.227 INFO d.n.h.h.HarvestControllerServer.<init> - Bound to harvest channel 'LOWPRIORITY'
2022-07-11 09:56:48.227 INFO d.n.h.h.HarvestControllerServer.<init> - Serverdir: 'harvester_low'
2022-07-11 09:56:48.228 INFO d.n.h.h.HarvestControllerServer.<init> - Harvesting requires at least 400000000 bytes free.
2022-07-11 09:56:48.367 DEBUG d.n.c.d.bitrepository.BitmagUtils.initialize - Bitrepository settings will be loaded from /home/devel/extra_configs/bitmag_config.
2022-07-11 09:56:48.367 DEBUG d.n.c.d.bitrepository.BitmagUtils.initialize - Path to Bitrepository settings is /home/devel/extra_configs/bitmag_config.
2022-07-11 09:56:48.374 INFO d.n.c.d.bitrepository.HostNameUtils.getHostName - Local hostname (provided by getCanonicalHostName): kb-test-har-003.kb.dk
2022-07-11 09:56:49.829 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - BitmagArcRepositoryClient will timeout on each get request after 300000 milliseconds.
2022-07-11 09:56:49.830 DEBUG d.n.common.distribute.JMSConnection.setListener - Adding dk.netarkivet.archive.arcrepository.distribute.BitmagArcRepositoryClient@5c3b6c6e as listener to Test6_COMMON_THIS_REPOS_CLIENT_130_226_228_9_HCS_KBLOW001_H3
2022-07-11 09:56:49.837 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - BitmagArcRepositoryClient listens for replies on channel '[Queue 'Test6_COMMON_THIS_REPOS_CLIENT_130_226_228_9_HCS_KBLOW001_H3']'
2022-07-11 09:56:49.838 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - Getting bitmag config from settings.common.arcrepositoryClient.bitrepository.settingsDir=/home/devel/extra_configs/bitmag_config
2022-07-11 09:56:49.838 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - Using 'Test6' as default collectionID
2022-07-11 09:56:49.842 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - Storing tempfiles in folder 'arcrepositoryTemp'
2022-07-11 09:56:49.859 INFO d.n.common.utils.HttpsClientBuilder.<init> - Creating Aggressive retry handler for https queries.
2022-07-11 09:56:49.925 INFO d.n.c.utils.BasicTwoWaySSLProvider.loadPrivateKey - Key file '/home/devel/extra_configs/https_config/https_key.pem' with private key and certificate does not exist!
2022-07-11 09:56:49.953 DEBUG d.n.h.h.HarvestControllerServer.<init> - Obtained JMS connection.
2022-07-11 09:56:49.954 INFO d.n.h.h.HarvestControllerServer.<init> - SEND_READY_DELAY used by HarvestControllerServer is 300
2022-07-11 09:56:49.954 INFO d.n.h.heritrix3.PostProcessing.processOldJobs - Looking for unprocessed crawldata in 'harvester_low'
2022-07-11 09:56:49.955 DEBUG d.n.common.distribute.JMSConnection.setListener - Adding dk.netarkivet.harvester.heritrix3.HarvestControllerServer@7c56e013 as listener to Test6_COMMON_HCHAN_VAL_RESP
2022-07-11 09:56:49.966 INFO d.n.h.h.HarvestControllerServer.<init> - Requested to check the validity of harvest channel 'LOWPRIORITY'
2022-07-11 09:56:49.978 INFO d.n.h.h.HarvestControllerServer.visit - Received message stating that channel 'LOWPRIORITY' is valid.
2022-07-11 09:56:49.979 INFO d.n.h.h.HarvestControllerServer.beginListeningIfSpaceAvailable - Starts to listen to new jobs on queue '[Queue 'Test6_COMMON_JOB_SNAPSHOT_LOWPRIORITY']'
2022-07-11 09:56:49.980 DEBUG d.n.common.distribute.JMSConnection.setListener - Adding dk.netarkivet.harvester.heritrix3.HarvestControllerServer@7c56e013 as listener to Test6_COMMON_JOB_SNAPSHOT_LOWPRIORITY
--------------------------------------------
STARTING MACHINE: devel@kb-test-har-005.kb.dk
Starting all applications on: kb-test-har-005.kb.dk
Starting linux application: HarvestControllerApplication_browser_1
Starting linux application: HarvestControllerApplication_browser_2
09:56:52,728 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [/home/devel/Test6/conf/logback_HarvestControllerApplication_browser_1.xml] at [file:/home/devel/Test6/conf/logback_HarvestControllerApplication_browser_1.xml]
09:56:53,113 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - debug attribute not set
09:56:53,120 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
09:56:53,141 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [STDOUT]
09:56:53,180 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
09:56:53,286 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
09:56:53,291 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [FILE]
09:56:53,322 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@447981768 - No compression will be used
09:56:53,323 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@447981768 - Will use the pattern ./log/HarvestControllerApplication_browser_1.%d{yyyy-MM-dd}.%i.log for the active file
09:56:53,325 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@43195e57 - The date pattern is 'yyyy-MM-dd' from file name pattern './log/HarvestControllerApplication_browser_1.%d{yyyy-MM-dd}.%i.log'.
09:56:53,325 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@43195e57 - Roll-over at midnight.
09:56:53,334 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@43195e57 - Setting initial period to Mon Jul 11 09:56:53 CEST 2022
09:56:53,334 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@43195e57 - SizeAndTimeBasedFNATP is deprecated. Use SizeAndTimeBasedRollingPolicy instead
09:56:53,334 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@43195e57 - For more information see http://logback.qos.ch/manual/appenders.html#SizeAndTimeBasedRollingPolicy
09:56:53,336 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
09:56:53,338 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - Active log file name: ./log/HarvestControllerApplication_browser_1.log
09:56:53,338 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - File property is set to [./log/HarvestControllerApplication_browser_1.log]
09:56:53,339 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [dk.netarkivet.monitor.logging.CachingSLF4JAppender]
09:56:54,322 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [MONITOR]
09:56:54,324 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to WARN
09:56:54,324 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[ROOT]
09:56:54,340 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [FILE] to Logger[ROOT]
09:56:54,340 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [MONITOR] to Logger[ROOT]
09:56:54,341 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [dk.netarkivet] to DEBUG
09:56:54,341 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
09:56:54,341 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@333291e3 - Registering current configuration as safe fallback point
2022-07-11 09:56:54.346 DEBUG dk.netarkivet.common.utils.Settings.invoke0 - Value found in loaded data: SB,KB
SLF4J: A number (1) of logging calls during the initialization phase have been intercepted and are
SLF4J: now being replayed. These are subject to the filtering rules of the underlying logging system.
SLF4J: See also http://www.slf4j.org/codes.html#replay
Starting dk.netarkivet.harvester.heritrix3.HarvestControllerServer
Version: 7.4-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/530d443d1c093ced66f5fc473e516c6b5c0373f6)
2022-07-11 09:56:54.352 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Starting dk.netarkivet.harvester.heritrix3.HarvestControllerServer
Version: 7.4-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/530d443d1c093ced66f5fc473e516c6b5c0373f6)
Java VM: 1.8.0_312
2022-07-11 09:56:54.353 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Java VM: 1.8.0_312
java.home: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.312.b07-1.el7_9.x86_64/jre
2022-07-11 09:56:54.353 INFO d.n.common.utils.ApplicationUtils.logAndPrint - java.home: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.312.b07-1.el7_9.x86_64/jre
Working dir: /home/devel/Test6
2022-07-11 09:56:54.353 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Working dir: /home/devel/Test6
2022-07-11 09:56:54.357 INFO d.n.common.utils.ApplicationUtils.startApp - Using settings files '/home/devel/Test6/conf/settings_HarvestControllerApplication_browser_1.xml'
2022-07-11 09:56:54.551 INFO d.n.m.d.JMSMonitorRegistryClient.register - Registering this client for monitoring every 1 minutes, using hostname 'kb-test-har-005.kb.dk' and JMX/RMI ports 8170/8270
2022-07-11 09:56:54.552 INFO d.n.c.m.MBeanConnectorCreator.exposeJMXMBeanServer - Registered mbean server in registry on port 8170 communicating on port 8270 using password file 'conf/jmxremote.password'.
Service URL is service:jmx:rmi://kb-test-har-005.kb.dk:8270/jndi/rmi://kb-test-har-005.kb.dk:8170/jmxrmi
2022-07-11 09:56:54.563 INFO d.n.c.distribute.JMSConnectionSunMQ.<init> - Creating instance of dk.netarkivet.common.distribute.JMSConnectionSunMQ
2022-07-11 09:56:54.563 DEBUG d.n.common.distribute.JMSConnection.initConnection - Initializing a JMS connection dk.netarkivet.common.distribute.JMSConnectionSunMQ
2022-07-11 09:56:54.564 INFO d.n.c.distribute.JMSConnectionSunMQ.getConnectionFactory - Establishing SunMQ JMS Connection to 'kb-test-adm-001.kb.dk:7676'
dk.netarkivet.harvester.heritrix3.HarvestControllerServer Running
2022-07-11 09:56:54.564 INFO d.n.common.utils.ApplicationUtils.logAndPrint - dk.netarkivet.harvester.heritrix3.HarvestControllerServer Running
2022-07-11 09:56:54.574 INFO d.n.h.h.HarvestControllerServer.<init> - Starting HarvestControllerServer.
2022-07-11 09:56:54.574 INFO d.n.h.h.HarvestControllerServer.<init> - Bound to harvest channel 'BROWSER'
2022-07-11 09:56:54.586 INFO d.n.h.h.HarvestControllerServer.<init> - Serverdir: 'harvester_browser_1'
2022-07-11 09:56:54.587 INFO d.n.h.h.HarvestControllerServer.<init> - Harvesting requires at least 400000000 bytes free.
2022-07-11 09:56:54.771 DEBUG d.n.c.d.bitrepository.BitmagUtils.initialize - Bitrepository settings will be loaded from /home/devel/extra_configs/bitmag_config.
2022-07-11 09:56:54.771 DEBUG d.n.c.d.bitrepository.BitmagUtils.initialize - Path to Bitrepository settings is /home/devel/extra_configs/bitmag_config.
2022-07-11 09:56:54.777 INFO d.n.c.d.bitrepository.HostNameUtils.getHostName - Local hostname (provided by getCanonicalHostName): kb-test-har-005.kb.dk
2022-07-11 09:56:56.827 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - BitmagArcRepositoryClient will timeout on each get request after 300000 milliseconds.
2022-07-11 09:56:56.828 DEBUG d.n.common.distribute.JMSConnection.setListener - Adding dk.netarkivet.archive.arcrepository.distribute.BitmagArcRepositoryClient@26425897 as listener to Test6_COMMON_THIS_REPOS_CLIENT_130_226_228_16_HCS_BROWSER_1
2022-07-11 09:56:56.846 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - BitmagArcRepositoryClient listens for replies on channel '[Queue 'Test6_COMMON_THIS_REPOS_CLIENT_130_226_228_16_HCS_BROWSER_1']'
2022-07-11 09:56:56.847 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - Getting bitmag config from settings.common.arcrepositoryClient.bitrepository.settingsDir=/home/devel/extra_configs/bitmag_config
2022-07-11 09:56:56.847 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - Using 'Test6' as default collectionID
2022-07-11 09:56:56.851 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - Storing tempfiles in folder 'arcrepositoryTemp'
2022-07-11 09:56:56.870 INFO d.n.common.utils.HttpsClientBuilder.<init> - Creating Aggressive retry handler for https queries.
2022-07-11 09:56:56.940 INFO d.n.c.utils.BasicTwoWaySSLProvider.loadPrivateKey - Key file '/home/devel/extra_configs/https_config/https_key.pem' with private key and certificate does not exist!
2022-07-11 09:56:56.966 DEBUG d.n.h.h.HarvestControllerServer.<init> - Obtained JMS connection.
2022-07-11 09:56:56.967 INFO d.n.h.h.HarvestControllerServer.<init> - SEND_READY_DELAY used by HarvestControllerServer is 300
2022-07-11 09:56:56.968 INFO d.n.h.heritrix3.PostProcessing.processOldJobs - Looking for unprocessed crawldata in 'harvester_browser_1'
2022-07-11 09:56:56.968 DEBUG d.n.common.distribute.JMSConnection.setListener - Adding dk.netarkivet.harvester.heritrix3.HarvestControllerServer@7bc10d84 as listener to Test6_COMMON_HCHAN_VAL_RESP
2022-07-11 09:56:57.004 INFO d.n.h.h.HarvestControllerServer.<init> - Requested to check the validity of harvest channel 'BROWSER'
2022-07-11 09:56:57.010 INFO d.n.h.h.HarvestControllerServer.visit - Received message stating that channel 'BROWSER' is valid.
2022-07-11 09:56:57.012 INFO d.n.h.h.HarvestControllerServer.beginListeningIfSpaceAvailable - Starts to listen to new jobs on queue '[Queue 'Test6_COMMON_JOB_PARTIAL_BROWSER']'
2022-07-11 09:56:57.012 DEBUG d.n.common.distribute.JMSConnection.setListener - Adding dk.netarkivet.harvester.heritrix3.HarvestControllerServer@7bc10d84 as listener to Test6_COMMON_JOB_PARTIAL_BROWSER
09:56:52,653 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [/home/devel/Test6/conf/logback_HarvestControllerApplication_browser_2.xml] at [file:/home/devel/Test6/conf/logback_HarvestControllerApplication_browser_2.xml]
09:56:53,004 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - debug attribute not set
09:56:53,021 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
09:56:53,027 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [STDOUT]
09:56:53,061 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
09:56:53,144 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
09:56:53,148 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [FILE]
09:56:53,202 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@447981768 - No compression will be used
09:56:53,203 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@447981768 - Will use the pattern ./log/HarvestControllerApplication_browser_2.%d{yyyy-MM-dd}.%i.log for the active file
09:56:53,213 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@43195e57 - The date pattern is 'yyyy-MM-dd' from file name pattern './log/HarvestControllerApplication_browser_2.%d{yyyy-MM-dd}.%i.log'.
09:56:53,213 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@43195e57 - Roll-over at midnight.
09:56:53,217 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@43195e57 - Setting initial period to Mon Jul 11 09:56:53 CEST 2022
09:56:53,217 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@43195e57 - SizeAndTimeBasedFNATP is deprecated. Use SizeAndTimeBasedRollingPolicy instead
09:56:53,217 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@43195e57 - For more information see http://logback.qos.ch/manual/appenders.html#SizeAndTimeBasedRollingPolicy
09:56:53,218 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
09:56:53,222 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - Active log file name: ./log/HarvestControllerApplication_browser_2.log
09:56:53,222 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - File property is set to [./log/HarvestControllerApplication_browser_2.log]
09:56:53,223 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [dk.netarkivet.monitor.logging.CachingSLF4JAppender]
09:56:53,930 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [MONITOR]
09:56:53,932 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to WARN
09:56:53,932 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[ROOT]
09:56:53,934 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [FILE] to Logger[ROOT]
09:56:53,934 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [MONITOR] to Logger[ROOT]
09:56:53,952 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [dk.netarkivet] to DEBUG
09:56:53,952 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
09:56:53,953 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@333291e3 - Registering current configuration as safe fallback point
2022-07-11 09:56:53.958 DEBUG dk.netarkivet.common.utils.Settings.invoke0 - Value found in loaded data: SB,KB
SLF4J: A number (1) of logging calls during the initialization phase have been intercepted and are
SLF4J: now being replayed. These are subject to the filtering rules of the underlying logging system.
SLF4J: See also http://www.slf4j.org/codes.html#replay
Starting dk.netarkivet.harvester.heritrix3.HarvestControllerServer
Version: 7.4-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/530d443d1c093ced66f5fc473e516c6b5c0373f6)
2022-07-11 09:56:53.969 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Starting dk.netarkivet.harvester.heritrix3.HarvestControllerServer
Version: 7.4-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/530d443d1c093ced66f5fc473e516c6b5c0373f6)
Java VM: 1.8.0_312
2022-07-11 09:56:53.970 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Java VM: 1.8.0_312
java.home: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.312.b07-1.el7_9.x86_64/jre
2022-07-11 09:56:53.970 INFO d.n.common.utils.ApplicationUtils.logAndPrint - java.home: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.312.b07-1.el7_9.x86_64/jre
Working dir: /home/devel/Test6
2022-07-11 09:56:53.970 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Working dir: /home/devel/Test6
2022-07-11 09:56:53.974 INFO d.n.common.utils.ApplicationUtils.startApp - Using settings files '/home/devel/Test6/conf/settings_HarvestControllerApplication_browser_2.xml'
2022-07-11 09:56:54.198 INFO d.n.m.d.JMSMonitorRegistryClient.register - Registering this client for monitoring every 1 minutes, using hostname 'kb-test-har-005.kb.dk' and JMX/RMI ports 8175/8275
2022-07-11 09:56:54.198 INFO d.n.c.m.MBeanConnectorCreator.exposeJMXMBeanServer - Registered mbean server in registry on port 8175 communicating on port 8275 using password file 'conf/jmxremote.password'.
Service URL is service:jmx:rmi://kb-test-har-005.kb.dk:8275/jndi/rmi://kb-test-har-005.kb.dk:8175/jmxrmi
dk.netarkivet.harvester.heritrix3.HarvestControllerServer Running
2022-07-11 09:56:54.203 INFO d.n.common.utils.ApplicationUtils.logAndPrint - dk.netarkivet.harvester.heritrix3.HarvestControllerServer Running
2022-07-11 09:56:54.229 INFO d.n.c.distribute.JMSConnectionSunMQ.<init> - Creating instance of dk.netarkivet.common.distribute.JMSConnectionSunMQ
2022-07-11 09:56:54.230 DEBUG d.n.common.distribute.JMSConnection.initConnection - Initializing a JMS connection dk.netarkivet.common.distribute.JMSConnectionSunMQ
2022-07-11 09:56:54.230 INFO d.n.c.distribute.JMSConnectionSunMQ.getConnectionFactory - Establishing SunMQ JMS Connection to 'kb-test-adm-001.kb.dk:7676'
2022-07-11 09:56:54.276 INFO d.n.h.h.HarvestControllerServer.<init> - Starting HarvestControllerServer.
2022-07-11 09:56:54.276 INFO d.n.h.h.HarvestControllerServer.<init> - Bound to harvest channel 'BROWSER'
2022-07-11 09:56:54.277 INFO d.n.h.h.HarvestControllerServer.<init> - Serverdir: 'harvester_browser_2'
2022-07-11 09:56:54.277 INFO d.n.h.h.HarvestControllerServer.<init> - Harvesting requires at least 400000000 bytes free.
2022-07-11 09:56:54.459 DEBUG d.n.c.d.bitrepository.BitmagUtils.initialize - Bitrepository settings will be loaded from /home/devel/extra_configs/bitmag_config.
2022-07-11 09:56:54.460 DEBUG d.n.c.d.bitrepository.BitmagUtils.initialize - Path to Bitrepository settings is /home/devel/extra_configs/bitmag_config.
2022-07-11 09:56:54.471 INFO d.n.c.d.bitrepository.HostNameUtils.getHostName - Local hostname (provided by getCanonicalHostName): kb-test-har-005.kb.dk
2022-07-11 09:56:56.742 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - BitmagArcRepositoryClient will timeout on each get request after 300000 milliseconds.
2022-07-11 09:56:56.743 DEBUG d.n.common.distribute.JMSConnection.setListener - Adding dk.netarkivet.archive.arcrepository.distribute.BitmagArcRepositoryClient@6f8f9349 as listener to Test6_COMMON_THIS_REPOS_CLIENT_130_226_228_16_HCS_BROWSER_2
2022-07-11 09:56:56.786 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - BitmagArcRepositoryClient listens for replies on channel '[Queue 'Test6_COMMON_THIS_REPOS_CLIENT_130_226_228_16_HCS_BROWSER_2']'
2022-07-11 09:56:56.787 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - Getting bitmag config from settings.common.arcrepositoryClient.bitrepository.settingsDir=/home/devel/extra_configs/bitmag_config
2022-07-11 09:56:56.787 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - Using 'Test6' as default collectionID
2022-07-11 09:56:56.791 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - Storing tempfiles in folder 'arcrepositoryTemp'
2022-07-11 09:56:56.809 INFO d.n.common.utils.HttpsClientBuilder.<init> - Creating Aggressive retry handler for https queries.
2022-07-11 09:56:56.916 INFO d.n.c.utils.BasicTwoWaySSLProvider.loadPrivateKey - Key file '/home/devel/extra_configs/https_config/https_key.pem' with private key and certificate does not exist!
2022-07-11 09:56:56.979 DEBUG d.n.h.h.HarvestControllerServer.<init> - Obtained JMS connection.
2022-07-11 09:56:56.981 INFO d.n.h.h.HarvestControllerServer.<init> - SEND_READY_DELAY used by HarvestControllerServer is 300
2022-07-11 09:56:56.981 INFO d.n.h.heritrix3.PostProcessing.processOldJobs - Looking for unprocessed crawldata in 'harvester_browser_2'
2022-07-11 09:56:56.982 DEBUG d.n.common.distribute.JMSConnection.setListener - Adding dk.netarkivet.harvester.heritrix3.HarvestControllerServer@2c3dec30 as listener to Test6_COMMON_HCHAN_VAL_RESP
2022-07-11 09:56:57.006 INFO d.n.h.h.HarvestControllerServer.visit - Received message stating that channel 'BROWSER' is valid.
2022-07-11 09:56:57.008 INFO d.n.h.h.HarvestControllerServer.beginListeningIfSpaceAvailable - Starts to listen to new jobs on queue '[Queue 'Test6_COMMON_JOB_PARTIAL_BROWSER']'
2022-07-11 09:56:57.008 DEBUG d.n.common.distribute.JMSConnection.setListener - Adding dk.netarkivet.harvester.heritrix3.HarvestControllerServer@2c3dec30 as listener to Test6_COMMON_JOB_PARTIAL_BROWSER
2022-07-11 09:56:57.019 INFO d.n.h.h.HarvestControllerServer.<init> - Requested to check the validity of harvest channel 'BROWSER'
--------------------------------------------
STARTING MACHINE: devel@kb-test-har-004.kb.dk
Starting all applications on: kb-test-har-004.kb.dk
Starting linux application: HarvestControllerApplication_kblow002_h3
Starting linux application: HarvestControllerApplication_kbhigh_h3
09:56:59,022 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [/home/devel/Test6/conf/logback_HarvestControllerApplication_kbhigh_h3.xml] at [file:/home/devel/Test6/conf/logback_HarvestControllerApplication_kbhigh_h3.xml]
09:57:00,002 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - debug attribute not set
09:57:00,018 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
09:57:00,040 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [STDOUT]
09:57:00,106 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
09:57:00,248 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
09:57:00,260 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [FILE]
09:57:00,348 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@1704629915 - No compression will be used
09:57:00,353 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@1704629915 - Will use the pattern ./log/HarvestControllerApplication_kbhigh_h3.%d{yyyy-MM-dd}.%i.log for the active file
09:57:00,357 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@76908cc0 - The date pattern is 'yyyy-MM-dd' from file name pattern './log/HarvestControllerApplication_kbhigh_h3.%d{yyyy-MM-dd}.%i.log'.
09:57:00,357 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@76908cc0 - Roll-over at midnight.
09:57:00,372 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@76908cc0 - Setting initial period to Mon Jul 11 09:57:00 CEST 2022
09:57:00,373 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@76908cc0 - SizeAndTimeBasedFNATP is deprecated. Use SizeAndTimeBasedRollingPolicy instead
09:57:00,373 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@76908cc0 - For more information see http://logback.qos.ch/manual/appenders.html#SizeAndTimeBasedRollingPolicy
09:57:00,382 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
09:57:00,396 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - Active log file name: ./log/HarvestControllerApplication_kbhigh_h3.log
09:57:00,396 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - File property is set to [./log/HarvestControllerApplication_kbhigh_h3.log]
09:57:00,400 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [dk.netarkivet.monitor.logging.CachingSLF4JAppender]
09:57:01,627 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [MONITOR]
09:57:01,639 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to WARN
09:57:01,639 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[ROOT]
09:57:01,640 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [FILE] to Logger[ROOT]
09:57:01,640 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [MONITOR] to Logger[ROOT]
09:57:01,641 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [dk.netarkivet] to DEBUG
09:57:01,641 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
09:57:01,642 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@2473d930 - Registering current configuration as safe fallback point
2022-07-11 09:57:01.653 DEBUG dk.netarkivet.common.utils.Settings.invoke0 - Value found in loaded data: SB,KB
SLF4J: A number (1) of logging calls during the initialization phase have been intercepted and are
SLF4J: now being replayed. These are subject to the filtering rules of the underlying logging system.
SLF4J: See also http://www.slf4j.org/codes.html#replay
Starting dk.netarkivet.harvester.heritrix3.HarvestControllerServer
Version: 7.4-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/530d443d1c093ced66f5fc473e516c6b5c0373f6)
2022-07-11 09:57:01.676 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Starting dk.netarkivet.harvester.heritrix3.HarvestControllerServer
Version: 7.4-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/530d443d1c093ced66f5fc473e516c6b5c0373f6)
Java VM: 1.8.0_25
2022-07-11 09:57:01.679 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Java VM: 1.8.0_25
java.home: /usr/local/java/jdk1.8.0_25_x64/jre
2022-07-11 09:57:01.680 INFO d.n.common.utils.ApplicationUtils.logAndPrint - java.home: /usr/local/java/jdk1.8.0_25_x64/jre
Working dir: /home/devel/Test6
2022-07-11 09:57:01.683 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Working dir: /home/devel/Test6
2022-07-11 09:57:01.685 INFO d.n.common.utils.ApplicationUtils.startApp - Using settings files '/home/devel/Test6/conf/settings_HarvestControllerApplication_kbhigh_h3.xml'
2022-07-11 09:57:02.100 INFO d.n.m.d.JMSMonitorRegistryClient.register - Registering this client for monitoring every 1 minutes, using hostname 'kb-test-har-004.kb.dk' and JMX/RMI ports 8172/8272
2022-07-11 09:57:02.101 INFO d.n.c.m.MBeanConnectorCreator.exposeJMXMBeanServer - Registered mbean server in registry on port 8172 communicating on port 8272 using password file 'conf/jmxremote.password'.
Service URL is service:jmx:rmi://kb-test-har-004.kb.dk:8272/jndi/rmi://kb-test-har-004.kb.dk:8172/jmxrmi
dk.netarkivet.harvester.heritrix3.HarvestControllerServer Running
2022-07-11 09:57:02.139 INFO d.n.common.utils.ApplicationUtils.logAndPrint - dk.netarkivet.harvester.heritrix3.HarvestControllerServer Running
2022-07-11 09:57:02.162 INFO d.n.c.distribute.JMSConnectionSunMQ.<init> - Creating instance of dk.netarkivet.common.distribute.JMSConnectionSunMQ
2022-07-11 09:57:02.162 DEBUG d.n.common.distribute.JMSConnection.initConnection - Initializing a JMS connection dk.netarkivet.common.distribute.JMSConnectionSunMQ
2022-07-11 09:57:02.163 INFO d.n.c.distribute.JMSConnectionSunMQ.getConnectionFactory - Establishing SunMQ JMS Connection to 'kb-test-adm-001.kb.dk:7676'
2022-07-11 09:57:02.186 INFO d.n.h.h.HarvestControllerServer.<init> - Starting HarvestControllerServer.
2022-07-11 09:57:02.187 INFO d.n.h.h.HarvestControllerServer.<init> - Bound to harvest channel 'HIGHPRIORITY'
2022-07-11 09:57:02.187 INFO d.n.h.h.HarvestControllerServer.<init> - Serverdir: 'harvester_high'
2022-07-11 09:57:02.188 INFO d.n.h.h.HarvestControllerServer.<init> - Harvesting requires at least 400000000 bytes free.
2022-07-11 09:57:02.570 DEBUG d.n.c.d.bitrepository.BitmagUtils.initialize - Bitrepository settings will be loaded from /home/devel/extra_configs/bitmag_config.
2022-07-11 09:57:02.581 DEBUG d.n.c.d.bitrepository.BitmagUtils.initialize - Path to Bitrepository settings is /home/devel/extra_configs/bitmag_config.
2022-07-11 09:57:02.610 INFO d.n.c.d.bitrepository.HostNameUtils.getHostName - Local hostname (provided by getCanonicalHostName): kb-test-har-004.kb.dk
09:56:58,899 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [/home/devel/Test6/conf/logback_HarvestControllerApplication_kblow002_h3.xml] at [file:/home/devel/Test6/conf/logback_HarvestControllerApplication_kblow002_h3.xml]
09:56:59,778 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - debug attribute not set
09:56:59,799 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
09:56:59,842 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [STDOUT]
09:56:59,903 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
09:57:00,088 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
09:57:00,107 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [FILE]
09:57:00,216 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@1704629915 - No compression will be used
09:57:00,220 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@1704629915 - Will use the pattern ./log/HarvestControllerApplication_kblow002_h3.%d{yyyy-MM-dd}.%i.log for the active file
09:57:00,228 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@76908cc0 - The date pattern is 'yyyy-MM-dd' from file name pattern './log/HarvestControllerApplication_kblow002_h3.%d{yyyy-MM-dd}.%i.log'.
09:57:00,228 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@76908cc0 - Roll-over at midnight.
09:57:00,244 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@76908cc0 - Setting initial period to Mon Jul 11 09:57:00 CEST 2022
09:57:00,246 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@76908cc0 - SizeAndTimeBasedFNATP is deprecated. Use SizeAndTimeBasedRollingPolicy instead
09:57:00,246 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@76908cc0 - For more information see http://logback.qos.ch/manual/appenders.html#SizeAndTimeBasedRollingPolicy
09:57:00,254 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
09:57:00,273 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - Active log file name: ./log/HarvestControllerApplication_kblow002_h3.log
09:57:00,273 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - File property is set to [./log/HarvestControllerApplication_kblow002_h3.log]
09:57:00,274 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [dk.netarkivet.monitor.logging.CachingSLF4JAppender]
09:57:01,262 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [MONITOR]
09:57:01,276 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to WARN
09:57:01,276 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[ROOT]
09:57:01,278 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [FILE] to Logger[ROOT]
09:57:01,279 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [MONITOR] to Logger[ROOT]
09:57:01,281 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [dk.netarkivet] to DEBUG
09:57:01,281 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
09:57:01,285 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@2473d930 - Registering current configuration as safe fallback point
2022-07-11 09:57:01.303 DEBUG dk.netarkivet.common.utils.Settings.invoke0 - Value found in loaded data: SB,KB
SLF4J: A number (1) of logging calls during the initialization phase have been intercepted and are
SLF4J: now being replayed. These are subject to the filtering rules of the underlying logging system.
SLF4J: See also http://www.slf4j.org/codes.html#replay
Starting dk.netarkivet.harvester.heritrix3.HarvestControllerServer
Version: 7.4-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/530d443d1c093ced66f5fc473e516c6b5c0373f6)
2022-07-11 09:57:01.338 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Starting dk.netarkivet.harvester.heritrix3.HarvestControllerServer
Version: 7.4-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/530d443d1c093ced66f5fc473e516c6b5c0373f6)
Java VM: 1.8.0_25
2022-07-11 09:57:01.342 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Java VM: 1.8.0_25
java.home: /usr/local/java/jdk1.8.0_25_x64/jre
2022-07-11 09:57:01.346 INFO d.n.common.utils.ApplicationUtils.logAndPrint - java.home: /usr/local/java/jdk1.8.0_25_x64/jre
Working dir: /home/devel/Test6
2022-07-11 09:57:01.350 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Working dir: /home/devel/Test6
2022-07-11 09:57:01.355 INFO d.n.common.utils.ApplicationUtils.startApp - Using settings files '/home/devel/Test6/conf/settings_HarvestControllerApplication_kblow002_h3.xml'
2022-07-11 09:57:01.775 INFO d.n.m.d.JMSMonitorRegistryClient.register - Registering this client for monitoring every 1 minutes, using hostname 'kb-test-har-004.kb.dk' and JMX/RMI ports 8170/8270
2022-07-11 09:57:01.784 INFO d.n.c.m.MBeanConnectorCreator.exposeJMXMBeanServer - Registered mbean server in registry on port 8170 communicating on port 8270 using password file 'conf/jmxremote.password'.
Service URL is service:jmx:rmi://kb-test-har-004.kb.dk:8270/jndi/rmi://kb-test-har-004.kb.dk:8170/jmxrmi
dk.netarkivet.harvester.heritrix3.HarvestControllerServer Running
2022-07-11 09:57:01.826 INFO d.n.common.utils.ApplicationUtils.logAndPrint - dk.netarkivet.harvester.heritrix3.HarvestControllerServer Running
2022-07-11 09:57:01.831 INFO d.n.c.distribute.JMSConnectionSunMQ.<init> - Creating instance of dk.netarkivet.common.distribute.JMSConnectionSunMQ
2022-07-11 09:57:01.831 DEBUG d.n.common.distribute.JMSConnection.initConnection - Initializing a JMS connection dk.netarkivet.common.distribute.JMSConnectionSunMQ
2022-07-11 09:57:01.834 INFO d.n.c.distribute.JMSConnectionSunMQ.getConnectionFactory - Establishing SunMQ JMS Connection to 'kb-test-adm-001.kb.dk:7676'
2022-07-11 09:57:01.873 INFO d.n.h.h.HarvestControllerServer.<init> - Starting HarvestControllerServer.
2022-07-11 09:57:01.874 INFO d.n.h.h.HarvestControllerServer.<init> - Bound to harvest channel 'LOWPRIORITY'
2022-07-11 09:57:01.874 INFO d.n.h.h.HarvestControllerServer.<init> - Serverdir: 'harvester_low'
2022-07-11 09:57:01.875 INFO d.n.h.h.HarvestControllerServer.<init> - Harvesting requires at least 400000000 bytes free.
2022-07-11 09:57:02.263 DEBUG d.n.c.d.bitrepository.BitmagUtils.initialize - Bitrepository settings will be loaded from /home/devel/extra_configs/bitmag_config.
2022-07-11 09:57:02.283 DEBUG d.n.c.d.bitrepository.BitmagUtils.initialize - Path to Bitrepository settings is /home/devel/extra_configs/bitmag_config.
2022-07-11 09:57:02.297 INFO d.n.c.d.bitrepository.HostNameUtils.getHostName - Local hostname (provided by getCanonicalHostName): kb-test-har-004.kb.dk
--------------------------------------------
STARTING MACHINE: devel@kb-test-acs-001.kb.dk
Starting all applications on: kb-test-acs-001.kb.dk
Starting linux application: IndexServerApplication
Starting linux application: ViewerProxyApplication
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/devel/Test6/lib/logback-classic-1.2.3.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/devel/Test6/lib/slf4j-log4j12-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
09:57:04,084 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [/home/devel/Test6/conf/logback_IndexServerApplication.xml] at [file:/home/devel/Test6/conf/logback_IndexServerApplication.xml]
09:57:04,354 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - debug attribute not set
09:57:04,357 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
09:57:04,365 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [STDOUT]
09:57:04,377 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
09:57:04,430 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
09:57:04,435 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [FILE]
09:57:04,462 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@29183965 - No compression will be used
09:57:04,464 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@29183965 - Will use the pattern ./log/IndexServerApplication.%d{yyyy-MM-dd}.%i.log for the active file
09:57:04,467 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@55183b20 - The date pattern is 'yyyy-MM-dd' from file name pattern './log/IndexServerApplication.%d{yyyy-MM-dd}.%i.log'.
09:57:04,467 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@55183b20 - Roll-over at midnight.
09:57:04,472 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@55183b20 - Setting initial period to Mon Jul 11 09:57:04 CEST 2022
09:57:04,473 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@55183b20 - SizeAndTimeBasedFNATP is deprecated. Use SizeAndTimeBasedRollingPolicy instead
09:57:04,473 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@55183b20 - For more information see http://logback.qos.ch/manual/appenders.html#SizeAndTimeBasedRollingPolicy
09:57:04,475 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
09:57:04,484 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - Active log file name: ./log/IndexServerApplication.log
09:57:04,484 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - File property is set to [./log/IndexServerApplication.log]
09:57:04,485 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [dk.netarkivet.monitor.logging.CachingSLF4JAppender]
09:57:05,100 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [MONITOR]
09:57:05,102 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to WARN
09:57:05,102 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[ROOT]
09:57:05,103 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [FILE] to Logger[ROOT]
09:57:05,103 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [MONITOR] to Logger[ROOT]
09:57:05,104 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [dk.netarkivet] to DEBUG
09:57:05,104 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
09:57:05,104 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@4f83df68 - Registering current configuration as safe fallback point
SLF4J: Actual binding is of type [ch.qos.logback.classic.util.ContextSelectorStaticBinder]
2022-07-11 09:57:05.110 DEBUG dk.netarkivet.common.utils.Settings.invoke0 - Value found in loaded data: SB,KB
SLF4J: A number (1) of logging calls during the initialization phase have been intercepted and are
SLF4J: now being replayed. These are subject to the filtering rules of the underlying logging system.
SLF4J: See also http://www.slf4j.org/codes.html#replay
Starting dk.netarkivet.harvester.indexserver.IndexServer
Version: 7.4-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/530d443d1c093ced66f5fc473e516c6b5c0373f6)
2022-07-11 09:57:05.122 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Starting dk.netarkivet.harvester.indexserver.IndexServer
Version: 7.4-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/530d443d1c093ced66f5fc473e516c6b5c0373f6)
Java VM: 1.8.0_332
2022-07-11 09:57:05.123 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Java VM: 1.8.0_332
java.home: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.332.b09-1.el7_9.x86_64/jre
2022-07-11 09:57:05.123 INFO d.n.common.utils.ApplicationUtils.logAndPrint - java.home: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.332.b09-1.el7_9.x86_64/jre
Working dir: /home/devel/Test6
2022-07-11 09:57:05.134 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Working dir: /home/devel/Test6
2022-07-11 09:57:05.135 INFO d.n.common.utils.ApplicationUtils.startApp - Using settings files '/home/devel/Test6/conf/settings_IndexServerApplication.xml'
2022-07-11 09:57:05.329 INFO d.n.m.d.JMSMonitorRegistryClient.register - Registering this client for monitoring every 1 minutes, using hostname 'kb-test-acs-001.kb.dk' and JMX/RMI ports 8371/8471
2022-07-11 09:57:05.330 INFO d.n.c.m.MBeanConnectorCreator.exposeJMXMBeanServer - Registered mbean server in registry on port 8371 communicating on port 8471 using password file 'conf/jmxremote.password'.
Service URL is service:jmx:rmi://kb-test-acs-001.kb.dk:8471/jndi/rmi://kb-test-acs-001.kb.dk:8371/jmxrmi
2022-07-11 09:57:05.339 INFO d.n.c.distribute.JMSConnectionSunMQ.<init> - Creating instance of dk.netarkivet.common.distribute.JMSConnectionSunMQ
2022-07-11 09:57:05.340 DEBUG d.n.common.distribute.JMSConnection.initConnection - Initializing a JMS connection dk.netarkivet.common.distribute.JMSConnectionSunMQ
2022-07-11 09:57:05.340 INFO d.n.c.distribute.JMSConnectionSunMQ.getConnectionFactory - Establishing SunMQ JMS Connection to 'kb-test-adm-001.kb.dk:7676'
dk.netarkivet.harvester.indexserver.IndexServer Running
2022-07-11 09:57:05.352 INFO d.n.common.utils.ApplicationUtils.logAndPrint - dk.netarkivet.harvester.indexserver.IndexServer Running
2022-07-11 09:57:05.367 INFO d.n.h.indexserver.FileBasedCache.<init> - Metadata cache for 'cdxdata' uses directory '/home/devel/Test6/cache/cdxdata'
2022-07-11 09:57:05.391 DEBUG d.n.c.d.bitrepository.BitmagUtils.initialize - Bitrepository settings will be loaded from /home/devel/extra_configs/bitmag_config.
2022-07-11 09:57:05.392 DEBUG d.n.c.d.bitrepository.BitmagUtils.initialize - Path to Bitrepository settings is /home/devel/extra_configs/bitmag_config.
2022-07-11 09:57:05.398 INFO d.n.c.d.bitrepository.HostNameUtils.getHostName - Local hostname (provided by getCanonicalHostName): kb-test-acs-001.kb.dk
2022-07-11 09:57:07.195 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - BitmagArcRepositoryClient will timeout on each get request after 300000 milliseconds.
2022-07-11 09:57:07.196 DEBUG d.n.common.distribute.JMSConnection.setListener - Adding dk.netarkivet.archive.arcrepository.distribute.BitmagArcRepositoryClient@4fc5e095 as listener to Test6_COMMON_THIS_REPOS_CLIENT_130_226_228_5_IS
2022-07-11 09:57:07.211 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - BitmagArcRepositoryClient listens for replies on channel '[Queue 'Test6_COMMON_THIS_REPOS_CLIENT_130_226_228_5_IS']'
2022-07-11 09:57:07.212 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - Getting bitmag config from settings.common.arcrepositoryClient.bitrepository.settingsDir=/home/devel/extra_configs/bitmag_config
2022-07-11 09:57:07.212 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - Using 'Test6' as default collectionID
2022-07-11 09:57:07.230 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - Storing tempfiles in folder 'arcrepositoryTemp'
2022-07-11 09:57:07.259 INFO d.n.common.utils.HttpsClientBuilder.<init> - Creating Aggressive retry handler for https queries.
2022-07-11 09:57:07.360 DEBUG d.n.c.utils.BasicTwoWaySSLProvider.loadPrivateKey - X509CertificateHolder found
2022-07-11 09:57:07.361 DEBUG d.n.c.utils.BasicTwoWaySSLProvider.loadPrivateKey - PrivateKeyInfo found
2022-07-11 09:57:07.418 INFO d.n.h.indexserver.RawMetadataCache.<init> - Metadata cache for 'cdxdata' is fetching metadata with urls matching 'metadata://[^/]*/crawl/index/cdx.*' and mimetype matching 'application/x-cdx'. Migration of duplicate records is disabled
2022-07-11 09:57:07.418 INFO d.n.h.indexserver.FileBasedCache.<init> - Metadata cache for 'cdxindex' uses directory '/home/devel/Test6/cache/cdxindex'
2022-07-11 09:57:07.422 INFO d.n.h.indexserver.FileBasedCache.<init> - Metadata cache for 'crawllog' uses directory '/home/devel/Test6/cache/crawllog'
2022-07-11 09:57:07.423 INFO d.n.h.indexserver.RawMetadataCache.<init> - Metadata cache for 'crawllog' is fetching metadata with urls matching 'metadata://[^/]*/crawl/logs/crawl\.log.*' and mimetype matching 'text/plain'. Migration of duplicate records is disabled
2022-07-11 09:57:07.423 INFO d.n.h.indexserver.FileBasedCache.<init> - Metadata cache for 'dedupcrawllogindex' uses directory '/home/devel/Test6/cache/dedupcrawllogindex'
2022-07-11 09:57:07.424 INFO d.n.h.indexserver.FileBasedCache.<init> - Metadata cache for 'cdxdata' uses directory '/home/devel/Test6/cache/cdxdata'
2022-07-11 09:57:07.424 INFO d.n.h.indexserver.RawMetadataCache.<init> - Metadata cache for 'cdxdata' is fetching metadata with urls matching 'metadata://[^/]*/crawl/index/cdx.*' and mimetype matching 'application/x-cdx'. Migration of duplicate records is disabled
2022-07-11 09:57:07.425 INFO d.n.h.indexserver.FileBasedCache.<init> - Metadata cache for 'crawllog' uses directory '/home/devel/Test6/cache/crawllog'
2022-07-11 09:57:07.425 INFO d.n.h.indexserver.RawMetadataCache.<init> - Metadata cache for 'crawllog' is fetching metadata with urls matching 'metadata://[^/]*/crawl/logs/crawl\.log.*' and mimetype matching 'text/plain'. Migration of duplicate records is disabled
2022-07-11 09:57:07.425 INFO d.n.h.indexserver.FileBasedCache.<init> - Metadata cache for 'fullcrawllogindex' uses directory '/home/devel/Test6/cache/fullcrawllogindex'
2022-07-11 09:57:07.426 INFO d.n.h.indexserver.FileBasedCache.<init> - Metadata cache for 'cdxdata' uses directory '/home/devel/Test6/cache/cdxdata'
2022-07-11 09:57:07.426 INFO d.n.h.indexserver.RawMetadataCache.<init> - Metadata cache for 'cdxdata' is fetching metadata with urls matching 'metadata://[^/]*/crawl/index/cdx.*' and mimetype matching 'application/x-cdx'. Migration of duplicate records is disabled
2022-07-11 09:57:07.427 DEBUG d.n.h.indexserver.FileBasedCache.cache - Waiting to enter synchronization on /home/devel/Test6/cache/cdxindex/empty-cache.working
2022-07-11 09:57:07.427 DEBUG d.n.h.indexserver.FileBasedCache.cache - locking filechannel for file '/home/devel/Test6/cache/cdxindex/empty-cache.working' (thread = main)
2022-07-11 09:57:07.432 DEBUG d.n.common.utils.ProcessUtils.runProcess - Running external program: sort /home/devel/Test6/cache/cdxindex/empty-cache -o /home/devel/Test6/cache/cdxindex/empty-cache.unsorted with environment LANG=C
2022-07-11 09:57:07.462 DEBUG d.n.h.indexserver.FileBasedCache.cache - release lock on filechannel sun.nio.ch.FileChannelImpl@18f20260
2022-07-11 09:57:07.462 INFO d.n.h.indexserver.FileBasedCache.getIndex - Generated index '/home/devel/Test6/cache/cdxindex/empty-cache' of id '[]', request was for '[]'
2022-07-11 09:57:07.463 DEBUG d.n.h.indexserver.FileBasedCache.cache - Waiting to enter synchronization on /home/devel/Test6/cache/dedupcrawllogindex/empty-cache.working
2022-07-11 09:57:07.463 DEBUG d.n.h.indexserver.FileBasedCache.cache - locking filechannel for file '/home/devel/Test6/cache/dedupcrawllogindex/empty-cache.working' (thread = main)
2022-07-11 09:57:07.463 INFO d.n.h.indexserver.CrawlLogIndexCache.prepareCombine - Starting to generate dedupcrawllogindex for the 0 jobs: []
2022-07-11 09:57:07.464 INFO d.n.h.indexserver.CrawlLogIndexCache.combine - Starting combine task #1. This combines a dataset with 0 crawl logs (thread = main)
2022-07-11 09:57:07.633 INFO d.n.h.indexserver.CrawlLogIndexCache.combine - Adding the final 0 subindices to main index. Forcing index to contain max 42 files (related to combine task #1)
2022-07-11 09:57:07.640 INFO d.n.h.indexserver.CrawlLogIndexCache.combine - Adding operation completed (combine task #1)!
2022-07-11 09:57:07.641 INFO d.n.h.indexserver.CrawlLogIndexCache.combine - Closed index (related to combine task #1
2022-07-11 09:57:07.642 INFO d.n.h.indexserver.CrawlLogIndexCache.combine - Gzip-compressing the individual 2 index files of combine task # 1
2022-07-11 09:57:07.649 INFO d.n.h.indexserver.CrawlLogIndexCache.combine - Completed combine task #1 that combined a dataset with 0 crawl logs (entries in combined index: 0) - compressed index has size 72 bytes
2022-07-11 09:57:07.649 DEBUG d.n.h.indexserver.FileBasedCache.cache - release lock on filechannel sun.nio.ch.FileChannelImpl@55e7a35c
2022-07-11 09:57:07.650 INFO d.n.h.indexserver.FileBasedCache.getIndex - Generated index '/home/devel/Test6/cache/dedupcrawllogindex/empty-cache' of id '[]', request was for '[]'
2022-07-11 09:57:07.650 DEBUG d.n.h.indexserver.FileBasedCache.cache - Waiting to enter synchronization on /home/devel/Test6/cache/fullcrawllogindex/empty-cache.working
2022-07-11 09:57:07.650 DEBUG d.n.h.indexserver.FileBasedCache.cache - locking filechannel for file '/home/devel/Test6/cache/fullcrawllogindex/empty-cache.working' (thread = main)
2022-07-11 09:57:07.650 INFO d.n.h.indexserver.CrawlLogIndexCache.prepareCombine - Starting to generate fullcrawllogindex for the 0 jobs: []
2022-07-11 09:57:07.651 INFO d.n.h.indexserver.CrawlLogIndexCache.combine - Starting combine task #1. This combines a dataset with 0 crawl logs (thread = main)
2022-07-11 09:57:07.653 INFO d.n.h.indexserver.CrawlLogIndexCache.combine - Adding the final 0 subindices to main index. Forcing index to contain max 42 files (related to combine task #1)
2022-07-11 09:57:07.655 INFO d.n.h.indexserver.CrawlLogIndexCache.combine - Adding operation completed (combine task #1)!
2022-07-11 09:57:07.656 INFO d.n.h.indexserver.CrawlLogIndexCache.combine - Closed index (related to combine task #1
2022-07-11 09:57:07.656 INFO d.n.h.indexserver.CrawlLogIndexCache.combine - Gzip-compressing the individual 2 index files of combine task # 1
2022-07-11 09:57:07.657 INFO d.n.h.indexserver.CrawlLogIndexCache.combine - Completed combine task #1 that combined a dataset with 0 crawl logs (entries in combined index: 0) - compressed index has size 72 bytes
2022-07-11 09:57:07.658 DEBUG d.n.h.indexserver.FileBasedCache.cache - release lock on filechannel sun.nio.ch.FileChannelImpl@5922ae77
2022-07-11 09:57:07.658 INFO d.n.h.indexserver.FileBasedCache.getIndex - Generated index '/home/devel/Test6/cache/fullcrawllogindex/empty-cache' of id '[]', request was for '[]'
2022-07-11 09:57:07.698 INFO d.n.c.utils.hadoop.HadoopJobUtils.doKerberosLogin - Kerberos: /home/devel/extra_configs/kerberos_config/krb5.conf, /home/devel/extra_configs/kerberos_config/nat-nas-devel.keytab, nat-nas-devel@KBHPC.KB.DK
2022-07-11 09:57:08.013 WARN o.a.hadoop.util.NativeCodeLoader.<clinit> - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
09:57:03,893 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [/home/devel/Test6/conf/logback_ViewerProxyApplication.xml] at [file:/home/devel/Test6/conf/logback_ViewerProxyApplication.xml]
09:57:04,187 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - debug attribute not set
09:57:04,188 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
09:57:04,196 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [STDOUT]
09:57:04,209 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
09:57:04,307 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
09:57:04,313 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [FILE]
09:57:04,353 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@117009527 - No compression will be used
09:57:04,354 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@117009527 - Will use the pattern ./log/ViewerProxyApplication.%d{yyyy-MM-dd}.%i.log for the active file
09:57:04,357 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@be64738 - The date pattern is 'yyyy-MM-dd' from file name pattern './log/ViewerProxyApplication.%d{yyyy-MM-dd}.%i.log'.
09:57:04,357 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@be64738 - Roll-over at midnight.
09:57:04,362 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@be64738 - Setting initial period to Mon Jul 11 09:57:04 CEST 2022
09:57:04,362 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@be64738 - SizeAndTimeBasedFNATP is deprecated. Use SizeAndTimeBasedRollingPolicy instead
09:57:04,362 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@be64738 - For more information see http://logback.qos.ch/manual/appenders.html#SizeAndTimeBasedRollingPolicy
09:57:04,364 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
09:57:04,371 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - Active log file name: ./log/ViewerProxyApplication.log
09:57:04,371 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - File property is set to [./log/ViewerProxyApplication.log]
09:57:04,373 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [dk.netarkivet.monitor.logging.CachingSLF4JAppender]
09:57:05,106 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [MONITOR]
09:57:05,108 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to WARN
09:57:05,108 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[ROOT]
09:57:05,109 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [FILE] to Logger[ROOT]
09:57:05,109 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [MONITOR] to Logger[ROOT]
09:57:05,118 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [dk.netarkivet] to DEBUG
09:57:05,118 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
09:57:05,119 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@3ba9ad43 - Registering current configuration as safe fallback point
2022-07-11 09:57:05.126 DEBUG dk.netarkivet.common.utils.Settings.invoke0 - Value found in loaded data: SB,KB
SLF4J: A number (1) of logging calls during the initialization phase have been intercepted and are
SLF4J: now being replayed. These are subject to the filtering rules of the underlying logging system.
SLF4J: See also http://www.slf4j.org/codes.html#replay
Starting dk.netarkivet.viewerproxy.ViewerProxy
Version: 7.4-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/530d443d1c093ced66f5fc473e516c6b5c0373f6)
2022-07-11 09:57:05.139 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Starting dk.netarkivet.viewerproxy.ViewerProxy
Version: 7.4-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/530d443d1c093ced66f5fc473e516c6b5c0373f6)
Java VM: 1.8.0_332
2022-07-11 09:57:05.140 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Java VM: 1.8.0_332
java.home: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.332.b09-1.el7_9.x86_64/jre
2022-07-11 09:57:05.140 INFO d.n.common.utils.ApplicationUtils.logAndPrint - java.home: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.332.b09-1.el7_9.x86_64/jre
Working dir: /home/devel/Test6
2022-07-11 09:57:05.141 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Working dir: /home/devel/Test6
2022-07-11 09:57:05.141 INFO d.n.common.utils.ApplicationUtils.startApp - Using settings files '/home/devel/Test6/conf/settings_ViewerProxyApplication.xml'
2022-07-11 09:57:05.332 INFO d.n.m.d.JMSMonitorRegistryClient.register - Registering this client for monitoring every 1 minutes, using hostname 'kb-test-acs-001.kb.dk' and JMX/RMI ports 8370/8470
2022-07-11 09:57:05.333 INFO d.n.c.m.MBeanConnectorCreator.exposeJMXMBeanServer - Registered mbean server in registry on port 8370 communicating on port 8470 using password file 'conf/jmxremote.password'.
Service URL is service:jmx:rmi://kb-test-acs-001.kb.dk:8470/jndi/rmi://kb-test-acs-001.kb.dk:8370/jmxrmi
dk.netarkivet.viewerproxy.ViewerProxy Running
2022-07-11 09:57:05.335 INFO d.n.common.utils.ApplicationUtils.logAndPrint - dk.netarkivet.viewerproxy.ViewerProxy Running
2022-07-11 09:57:05.345 INFO d.n.c.distribute.JMSConnectionSunMQ.<init> - Creating instance of dk.netarkivet.common.distribute.JMSConnectionSunMQ
2022-07-11 09:57:05.346 DEBUG d.n.common.distribute.JMSConnection.initConnection - Initializing a JMS connection dk.netarkivet.common.distribute.JMSConnectionSunMQ
2022-07-11 09:57:05.346 INFO d.n.c.distribute.JMSConnectionSunMQ.getConnectionFactory - Establishing SunMQ JMS Connection to 'kb-test-adm-001.kb.dk:7676'
2022-07-11 09:57:05.384 DEBUG d.n.c.d.bitrepository.BitmagUtils.initialize - Bitrepository settings will be loaded from /home/devel/extra_configs/bitmag_config.
2022-07-11 09:57:05.385 DEBUG d.n.c.d.bitrepository.BitmagUtils.initialize - Path to Bitrepository settings is /home/devel/extra_configs/bitmag_config.
2022-07-11 09:57:05.390 INFO d.n.c.d.bitrepository.HostNameUtils.getHostName - Local hostname (provided by getCanonicalHostName): kb-test-acs-001.kb.dk
2022-07-11 09:57:07.215 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - BitmagArcRepositoryClient will timeout on each get request after 300000 milliseconds.
2022-07-11 09:57:07.216 DEBUG d.n.common.distribute.JMSConnection.setListener - Adding dk.netarkivet.archive.arcrepository.distribute.BitmagArcRepositoryClient@b6b1987 as listener to Test6_COMMON_THIS_REPOS_CLIENT_130_226_228_5_VP
2022-07-11 09:57:07.227 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - BitmagArcRepositoryClient listens for replies on channel '[Queue 'Test6_COMMON_THIS_REPOS_CLIENT_130_226_228_5_VP']'
2022-07-11 09:57:07.227 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - Getting bitmag config from settings.common.arcrepositoryClient.bitrepository.settingsDir=/home/devel/extra_configs/bitmag_config
2022-07-11 09:57:07.227 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - Using 'Test6' as default collectionID
2022-07-11 09:57:07.232 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - Storing tempfiles in folder 'arcrepositoryTemp'
2022-07-11 09:57:07.259 INFO d.n.common.utils.HttpsClientBuilder.<init> - Creating Aggressive retry handler for https queries.
2022-07-11 09:57:07.322 DEBUG d.n.c.utils.BasicTwoWaySSLProvider.loadPrivateKey - X509CertificateHolder found
2022-07-11 09:57:07.324 DEBUG d.n.c.utils.BasicTwoWaySSLProvider.loadPrivateKey - PrivateKeyInfo found
2022-07-11 09:57:07.435 INFO d.n.h.indexserver.FileBasedCache.<init> - Metadata cache for 'FULL_CRAWL_LOG' uses directory '/home/devel/Test6/cache/FULL_CRAWL_LOG'
2022-07-11 09:57:07.484 INFO d.n.viewerproxy.ARCArchiveAccess.<init> - Constructed instance of ARCArchiveAccess with TryToLookupUriAsFtp: false
2022-07-11 09:57:07.535 INFO dk.netarkivet.viewerproxy.WebProxy.<init> - Starting viewerproxy jetty on port 8078
--------------------------------------------
STARTING MACHINE: devel@kb-test-way-001.kb.dk
Starting all applications on: kb-test-way-001.kb.dk
Starting linux application: WaybackIndexerApplication
Starting linux application: AggregatorApplication
09:57:11,202 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [/home/devel/Test6/conf/logback_AggregatorApplication.xml] at [file:/home/devel/Test6/conf/logback_AggregatorApplication.xml]
09:57:11,622 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - debug attribute not set
09:57:11,625 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
09:57:11,669 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [STDOUT]
09:57:11,733 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
09:57:11,956 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
09:57:12,005 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [FILE]
09:57:12,141 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@477289012 - No compression will be used
09:57:12,144 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@477289012 - Will use the pattern ./log/AggregatorApplication.%d{yyyy-MM-dd}.%i.log for the active file
09:57:12,158 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@6b0c2d26 - The date pattern is 'yyyy-MM-dd' from file name pattern './log/AggregatorApplication.%d{yyyy-MM-dd}.%i.log'.
09:57:12,158 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@6b0c2d26 - Roll-over at midnight.
09:57:12,193 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@6b0c2d26 - Setting initial period to Mon Jul 11 09:57:12 CEST 2022
09:57:12,193 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@6b0c2d26 - SizeAndTimeBasedFNATP is deprecated. Use SizeAndTimeBasedRollingPolicy instead
09:57:12,193 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@6b0c2d26 - For more information see http://logback.qos.ch/manual/appenders.html#SizeAndTimeBasedRollingPolicy
09:57:12,199 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
09:57:12,208 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - Active log file name: ./log/AggregatorApplication.log
09:57:12,208 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - File property is set to [./log/AggregatorApplication.log]
09:57:12,215 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [dk.netarkivet.monitor.logging.CachingSLF4JAppender]
09:57:13,566 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [MONITOR]
09:57:13,572 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to WARN
09:57:13,572 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[ROOT]
09:57:13,573 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [FILE] to Logger[ROOT]
09:57:13,573 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [MONITOR] to Logger[ROOT]
09:57:13,582 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [dk.netarkivet] to DEBUG
09:57:13,582 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
09:57:13,582 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@3d3fcdb0 - Registering current configuration as safe fallback point
2022-07-11 09:57:13.606 DEBUG dk.netarkivet.common.utils.Settings.invoke0 - Value found in loaded data: SB,KB
SLF4J: A number (1) of logging calls during the initialization phase have been intercepted and are
SLF4J: now being replayed. These are subject to the filtering rules of the underlying logging system.
SLF4J: See also http://www.slf4j.org/codes.html#replay
Starting dk.netarkivet.wayback.aggregator.AggregationWorker
Version: 7.4-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/530d443d1c093ced66f5fc473e516c6b5c0373f6)
2022-07-11 09:57:13.645 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Starting dk.netarkivet.wayback.aggregator.AggregationWorker
Version: 7.4-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/530d443d1c093ced66f5fc473e516c6b5c0373f6)
Java VM: 1.8.0_25
2022-07-11 09:57:13.646 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Java VM: 1.8.0_25
java.home: /usr/local/java.old/jdk1.8.0_25_x64/jre
2022-07-11 09:57:13.647 INFO d.n.common.utils.ApplicationUtils.logAndPrint - java.home: /usr/local/java.old/jdk1.8.0_25_x64/jre
Working dir: /home/devel/Test6
2022-07-11 09:57:13.648 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Working dir: /home/devel/Test6
2022-07-11 09:57:13.650 INFO d.n.common.utils.ApplicationUtils.startApp - Using settings files '/home/devel/Test6/conf/settings_AggregatorApplication.xml'
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/devel/Test6/lib/logback-classic-1.2.3.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/devel/Test6/lib/slf4j-log4j12-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
--------------------------------------------
--------------------------------------------
STARTING MACHINE: netarkdv@sb-test-har-001.statsbiblioteket.dk
Starting all applications on: sb-test-har-001.statsbiblioteket.dk
Starting linux application: HarvestControllerApplication_sbhigh_h3
09:57:14,278 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [/home/netarkdv/Test6/conf/logback_HarvestControllerApplication_sbhigh_h3.xml] at [file:/home/netarkdv/Test6/conf/logback_HarvestControllerApplication_sbhigh_h3.xml]
09:57:14,384 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - debug attribute not set
09:57:14,386 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
09:57:14,390 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [STDOUT]
09:57:14,397 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
09:57:14,426 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
09:57:14,428 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [FILE]
09:57:14,441 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@858952163 - No compression will be used
09:57:14,441 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@858952163 - Will use the pattern ./log/HarvestControllerApplication_sbhigh_h3.%d{yyyy-MM-dd}.%i.log for the active file
09:57:14,443 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@479d31f3 - The date pattern is 'yyyy-MM-dd' from file name pattern './log/HarvestControllerApplication_sbhigh_h3.%d{yyyy-MM-dd}.%i.log'.
09:57:14,443 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@479d31f3 - Roll-over at midnight.
09:57:14,445 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@479d31f3 - Setting initial period to Mon Jul 11 09:57:14 CEST 2022
09:57:14,445 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@479d31f3 - SizeAndTimeBasedFNATP is deprecated. Use SizeAndTimeBasedRollingPolicy instead
09:57:14,445 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@479d31f3 - For more information see http://logback.qos.ch/manual/appenders.html#SizeAndTimeBasedRollingPolicy
09:57:14,446 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
09:57:14,448 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - Active log file name: ./log/HarvestControllerApplication_sbhigh_h3.log
09:57:14,448 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - File property is set to [./log/HarvestControllerApplication_sbhigh_h3.log]
09:57:14,448 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [dk.netarkivet.monitor.logging.CachingSLF4JAppender]
09:57:14,703 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [MONITOR]
09:57:14,705 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to WARN
09:57:14,705 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[ROOT]
09:57:14,705 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [FILE] to Logger[ROOT]
09:57:14,705 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [MONITOR] to Logger[ROOT]
09:57:14,705 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [dk.netarkivet] to DEBUG
09:57:14,705 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
09:57:14,706 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@40ef3420 - Registering current configuration as safe fallback point
2022-07-11 09:57:14.709 DEBUG dk.netarkivet.common.utils.Settings.invoke0 - Value found in loaded data: SB,KB
SLF4J: A number (1) of logging calls during the initialization phase have been intercepted and are
SLF4J: now being replayed. These are subject to the filtering rules of the underlying logging system.
SLF4J: See also http://www.slf4j.org/codes.html#replay
Starting dk.netarkivet.harvester.heritrix3.HarvestControllerServer
Version: 7.4-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/530d443d1c093ced66f5fc473e516c6b5c0373f6)
2022-07-11 09:57:14.722 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Starting dk.netarkivet.harvester.heritrix3.HarvestControllerServer
Version: 7.4-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/530d443d1c093ced66f5fc473e516c6b5c0373f6)
Java VM: 1.8.0_332
2022-07-11 09:57:14.722 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Java VM: 1.8.0_332
java.home: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.332.b09-2.el8_6.x86_64/jre
2022-07-11 09:57:14.722 INFO d.n.common.utils.ApplicationUtils.logAndPrint - java.home: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.332.b09-2.el8_6.x86_64/jre
Working dir: /home/netarkdv/Test6
2022-07-11 09:57:14.734 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Working dir: /home/netarkdv/Test6
2022-07-11 09:57:14.734 INFO d.n.common.utils.ApplicationUtils.startApp - Using settings files '/home/netarkdv/Test6/conf/settings_HarvestControllerApplication_sbhigh_h3.xml'
2022-07-11 09:57:14.824 INFO d.n.m.d.JMSMonitorRegistryClient.register - Registering this client for monitoring every 1 minutes, using hostname 'sb-test-har-001.statsbiblioteket.dk' and JMX/RMI ports 8170/8270
2022-07-11 09:57:14.824 INFO d.n.c.m.MBeanConnectorCreator.exposeJMXMBeanServer - Registered mbean server in registry on port 8170 communicating on port 8270 using password file 'conf/jmxremote.password'.
Service URL is service:jmx:rmi://sb-test-har-001.statsbiblioteket.dk:8270/jndi/rmi://sb-test-har-001.statsbiblioteket.dk:8170/jmxrmi
dk.netarkivet.harvester.heritrix3.HarvestControllerServer Running
2022-07-11 09:57:14.828 INFO d.n.common.utils.ApplicationUtils.logAndPrint - dk.netarkivet.harvester.heritrix3.HarvestControllerServer Running
2022-07-11 09:57:14.836 INFO d.n.h.h.HarvestControllerServer.<init> - Starting HarvestControllerServer.
2022-07-11 09:57:14.836 INFO d.n.h.h.HarvestControllerServer.<init> - Bound to harvest channel 'HIGHPRIORITY'
2022-07-11 09:57:14.837 INFO d.n.c.distribute.JMSConnectionSunMQ.<init> - Creating instance of dk.netarkivet.common.distribute.JMSConnectionSunMQ
2022-07-11 09:57:14.839 DEBUG d.n.common.distribute.JMSConnection.initConnection - Initializing a JMS connection dk.netarkivet.common.distribute.JMSConnectionSunMQ
2022-07-11 09:57:14.840 INFO d.n.c.distribute.JMSConnectionSunMQ.getConnectionFactory - Establishing SunMQ JMS Connection to 'kb-test-adm-001.kb.dk:7676'
2022-07-11 09:57:14.836 INFO d.n.h.h.HarvestControllerServer.<init> - Serverdir: 'harvester_high'
2022-07-11 09:57:14.842 INFO d.n.h.h.HarvestControllerServer.<init> - Harvesting requires at least 400000000 bytes free.
2022-07-11 09:57:15.001 DEBUG d.n.c.d.bitrepository.BitmagUtils.initialize - Bitrepository settings will be loaded from /home/netarkdv/extra_configs/bitmag_config.
2022-07-11 09:57:15.001 DEBUG d.n.c.d.bitrepository.BitmagUtils.initialize - Path to Bitrepository settings is /home/netarkdv/extra_configs/bitmag_config.
2022-07-11 09:57:15.004 INFO d.n.c.d.bitrepository.HostNameUtils.getHostName - Local hostname (provided by getCanonicalHostName): sb-test-har-001.statsbiblioteket.dk
2022-07-11 09:57:15.799 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - BitmagArcRepositoryClient will timeout on each get request after 300000 milliseconds.
2022-07-11 09:57:15.799 DEBUG d.n.common.distribute.JMSConnection.setListener - Adding dk.netarkivet.archive.arcrepository.distribute.BitmagArcRepositoryClient@1c7fd41f as listener to Test6_COMMON_THIS_REPOS_CLIENT_130_225_27_140_HCS_SBHIGH_H3
2022-07-11 09:57:15.825 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - BitmagArcRepositoryClient listens for replies on channel '[Queue 'Test6_COMMON_THIS_REPOS_CLIENT_130_225_27_140_HCS_SBHIGH_H3']'
2022-07-11 09:57:15.826 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - Getting bitmag config from settings.common.arcrepositoryClient.bitrepository.settingsDir=/home/netarkdv/extra_configs/bitmag_config
2022-07-11 09:57:15.826 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - Using 'Test6' as default collectionID
2022-07-11 09:57:15.829 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - Storing tempfiles in folder 'arcrepositoryTemp'
2022-07-11 09:57:15.835 INFO d.n.common.utils.HttpsClientBuilder.<init> - Creating Aggressive retry handler for https queries.
2022-07-11 09:57:15.860 INFO d.n.c.utils.BasicTwoWaySSLProvider.loadPrivateKey - Key file '/home/devel/extra_configs/https_config/https_key.pem' with private key and certificate does not exist!
2022-07-11 09:57:15.876 DEBUG d.n.h.h.HarvestControllerServer.<init> - Obtained JMS connection.
2022-07-11 09:57:15.877 INFO d.n.h.h.HarvestControllerServer.<init> - SEND_READY_DELAY used by HarvestControllerServer is 300
2022-07-11 09:57:15.877 INFO d.n.h.heritrix3.PostProcessing.processOldJobs - Looking for unprocessed crawldata in 'harvester_high'
2022-07-11 09:57:15.877 DEBUG d.n.common.distribute.JMSConnection.setListener - Adding dk.netarkivet.harvester.heritrix3.HarvestControllerServer@1de5f0ef as listener to Test6_COMMON_HCHAN_VAL_RESP
2022-07-11 09:57:15.919 INFO d.n.h.h.HarvestControllerServer.<init> - Requested to check the validity of harvest channel 'HIGHPRIORITY'
2022-07-11 09:57:15.929 INFO d.n.h.h.HarvestControllerServer.visit - Received message stating that channel 'HIGHPRIORITY' is valid.
2022-07-11 09:57:15.930 INFO d.n.h.h.HarvestControllerServer.beginListeningIfSpaceAvailable - Starts to listen to new jobs on queue '[Queue 'Test6_COMMON_JOB_PARTIAL_HIGHPRIORITY']'
2022-07-11 09:57:15.930 DEBUG d.n.common.distribute.JMSConnection.setListener - Adding dk.netarkivet.harvester.heritrix3.HarvestControllerServer@1de5f0ef as listener to Test6_COMMON_JOB_PARTIAL_HIGHPRIORITY
--------------------------------------------
Completed the script all_test.sh
Finished: SUCCESS
Console Output
Help us localize this page