etarkiv-checksum-devel-01: OPERATION_COMPLETED
15:01:07.156 DEBUG o.b.c.c.s.ContributorResponseStatus - Received response from: netarkiv-checksum-devel-01
15:01:07.156 DEBUG o.b.c.c.s.ContributorResponseStatus - Expected contributors: [netarkiv-checksum-devel-01], components that have not answered: []
15:01:07.156 DEBUG o.b.commandline.DeleteFileCmd - Complete: 0a2e5d44: DELETE_FILE for file 1-1-20210304152022685-00000-ciblee_2015_kb-test-har-004.kb.dk.warc: COMPLETE:
15:01:07.156 INFO o.b.commandline.DeleteFileCmd - Results of the DeleteFile operation for the file '1-1-20210304152022685-00000-ciblee_2015_kb-test-har-004.kb.dk.warc':
15:01:07.156 DEBUG o.b.commandline.DeleteFileCmd - 0a2e5d44: DELETE_FILE for file 1-1-20210304152022685-00000-ciblee_2015_kb-test-har-004.kb.dk.warc: COMPLETE:
15:01:08.368 DEBUG org.bitrepository.common.JaxbHelper - Creating JAXBHelper based on schema from: jar:file:/home/devel/ResetCollection/bitrepository-client/lib/bitrepository-repository-settings-xsd-12.jar!/xsd/RepositorySettings.xsd
15:01:08.488 DEBUG o.b.c.settings.XMLFileSettingsLoader - Loading the settings file '/home/devel/ResetCollection/bitrepository-client/bin/../conf/RepositorySettings.xml'.
15:01:08.753 DEBUG org.bitrepository.common.JaxbHelper - Creating JAXBHelper based on schema from: jar:file:/home/devel/ResetCollection/bitrepository-client/lib/bitrepository-reference-settings-1.9.0.1.jar!/xsd/ReferenceSettings.xsd
15:01:08.785 DEBUG o.b.c.settings.XMLFileSettingsLoader - Loading the settings file '/home/devel/ResetCollection/bitrepository-client/bin/../conf/ReferenceSettings.xml'.
15:01:09.367 DEBUG o.b.p.security.BasicSecurityManager - X509CertificateHolder found
15:01:09.385 DEBUG o.b.p.security.BasicSecurityManager - Got something, that we don't (yet) recognize. Class: PEMKeyPair
15:01:09.385 DEBUG o.b.p.security.BasicSecurityManager - PrivateKeyInfo found
15:01:09.538 INFO o.b.p.activemq.ActiveMQMessageBus - Initializing ActiveMQMessageBus:'org.bitrepository.settings.repositorysettings.MessageBusConfiguration@14cd1699[name=netarkiv-aar-bit-master-devel-01 broker, url=failover:(ssl://netarkiv-aar-bit-master-devel-01.statsbiblioteket.dk:61612?keepAlive=true)]'.
15:01:09.538 DEBUG org.bitrepository.common.JaxbHelper - Creating JAXBHelper based on schema from: jar:file:/home/devel/ResetCollection/bitrepository-client/lib/bitrepository-message-xml-xsd-30.jar!/xsd/BitRepositoryMessages.xsd
15:01:10.358 INFO o.a.a.t.failover.FailoverTransport - Successfully connected to ssl://netarkiv-aar-bit-master-devel-01.statsbiblioteket.dk:61612?keepAlive=true
15:01:10.458 DEBUG o.b.p.activemq.ActiveMQMessageBus - ActiveMQConnection initialized for 'org.bitrepository.settings.repositorysettings.MessageBusConfiguration@14cd1699[name=netarkiv-aar-bit-master-devel-01 broker, url=failover:(ssl://netarkiv-aar-bit-master-devel-01.statsbiblioteket.dk:61612?keepAlive=true)]'.
15:01:10.461 INFO o.b.p.activemq.ActiveMQMessageBus - Settings component filter to: [kb-test-har-003.kb.dk-devel-24047]
15:01:10.465 DEBUG o.b.c.c.m.CollectionBasedConversationMediator - Initializing the CollectionBasedConversationMediator
15:01:10.465 DEBUG o.b.p.activemq.ActiveMQMessageBus - Adding listener 'org.bitrepository.client.conversation.mediator.CollectionBasedConversationMediator@3514a4c0' to destination: 'topic://netarkiv-devel-kb-test-har-003.kb.dk-devel-24047' on message-bus 'netarkiv-aar-bit-master-devel-01 broker'.
15:01:10.465 DEBUG o.b.p.activemq.ActiveMQMessageBus - Retrieving message consumer on destination 'topic://netarkiv-devel-kb-test-har-003.kb.dk-devel-24047' for listener 'org.bitrepository.client.conversation.mediator.CollectionBasedConversationMediator@3514a4c0'. Key: 'topic://netarkiv-devel-kb-test-har-003.kb.dk-devel-24047#890545344'.
15:01:10.466 DEBUG o.b.p.activemq.ActiveMQMessageBus - No consumer known. Creating new for key 'topic://netarkiv-devel-kb-test-har-003.kb.dk-devel-24047#890545344'.
15:01:10.488 DEBUG o.b.commandline.DeleteFileCmd - Performing the DeleteFile operation.
15:01:10.499 DEBUG o.b.commandline.DeleteFileCmd - Initiating the DeleteFile conversation.
15:01:10.502 INFO o.b.m.d.ConversationBasedDeleteFileClient - Requesting the deletion of the file '1-1-20210304152022685-00000-ciblee_2015_kb-test-har-004.kb.dk.warc' from the pillar 'netarkiv-kbh-disk-devel-01' with checksum 'org.bitrepository.bitrepositoryelements.ChecksumDataForFileTYPE@6f46426d[checksumSpec=org.bitrepository.bitrepositoryelements.ChecksumSpecTYPE@73700b80[checksumType=MD5, checksumSalt=<null>(default), otherChecksumType=<null>(default)], checksumValue={-30,43,59,119,100,87,-127,-68,-34,-29,37,-26,-9,-23,64,20}, calculationTimestamp=2021-03-09T15:01:10.490+01:00]', while requested checksum 'null'. And the audit trail information 'null'.
15:01:10.520 DEBUG org.bitrepository.common.JaxbHelper - Creating JAXBHelper based on schema from: jar:file:/home/devel/ResetCollection/bitrepository-client/lib/bitrepository-message-xml-xsd-30.jar!/versioning/ProtocolVersionDefinition.xsd
15:01:11.053 DEBUG o.b.p.m.l.DeleteFileMessageLogger - Sent IdentifyPillarsForDeleteFileRequest(559da1c3) collectionID Test3, to netarkiv-kbh-disk-devel-01, destination topic://netarkiv-devel: FileID=1-1-20210304152022685-00000-ciblee_2015_kb-test-har-004.kb.dk.warc
15:01:11.054 DEBUG o.b.commandline.DeleteFileCmd - Received event: 559da1c3: DELETE_FILE for file 1-1-20210304152022685-00000-ciblee_2015_kb-test-har-004.kb.dk.warc: IDENTIFY_REQUEST_SENT: , Identifying pillars for delete file
15:01:11.373 DEBUG o.b.p.m.l.DeleteFileMessageLogger - Received IdentifyPillarsForDeleteFileResponse(559da1c3) collectionID Test3, from netarkiv-kbh-disk-devel-01: IDENTIFICATION_POSITIVE(File 1-1-20210304152022685-00000-ciblee_2015_kb-test-har-004.kb.dk.warc is in collection and can be deleted.)
15:01:11.388 DEBUG o.b.commandline.DeleteFileCmd - Received event: 559da1c3: DELETE_FILE for file 1-1-20210304152022685-00000-ciblee_2015_kb-test-har-004.kb.dk.warc: COMPONENT_IDENTIFIED: ContributorID netarkiv-kbh-disk-devel-01
15:01:11.389 DEBUG o.b.c.c.s.ContributorResponseStatus - Received response from: netarkiv-kbh-disk-devel-01
15:01:11.389 DEBUG o.b.c.c.s.ContributorResponseStatus - Expected contributors: [netarkiv-kbh-disk-devel-01], components that have not answered: []
15:01:11.389 DEBUG o.b.commandline.DeleteFileCmd - Received event: 559da1c3: DELETE_FILE for file 1-1-20210304152022685-00000-ciblee_2015_kb-test-har-004.kb.dk.warc: IDENTIFICATION_COMPLETE: Contributors: [netarkiv-kbh-disk-devel-01]
15:01:11.390 DEBUG o.b.commandline.DeleteFileCmd - Received event: 559da1c3: DELETE_FILE for file 1-1-20210304152022685-00000-ciblee_2015_kb-test-har-004.kb.dk.warc: REQUEST_SENT: ContributorID [netarkiv-kbh-disk-devel-01], Sending request for deleting file
15:01:11.968 DEBUG o.b.p.m.l.DeleteFileMessageLogger - Received IdentifyPillarsForDeleteFileResponse(559da1c3) collectionID Test3, from netarkiv-tape-devel-01: FILE_NOT_FOUND_FAILURE(Cannot serve request, file not on store)
15:01:12.127 DEBUG o.b.p.m.l.DeleteFileMessageLogger - Sent DeleteFileRequest(559da1c3) collectionID Test3, to netarkiv-kbh-disk-devel-01, destination topic://netarkiv-devel.netarkiv-kbh-disk-devel-01: FileID=1-1-20210304152022685-00000-ciblee_2015_kb-test-har-004.kb.dk.warc, ChecksumDataForExistingFile=org.bitrepository.bitrepositoryelements.ChecksumDataForFileTYPE@6f46426d[checksumSpec=org.bitrepository.bitrepositoryelements.ChecksumSpecTYPE@73700b80[checksumType=MD5, checksumSalt=<null>(default), otherChecksumType=<null>(default)], checksumValue={-30,43,59,119,100,87,-127,-68,-34,-29,37,-26,-9,-23,64,20}, calculationTimestamp=2021-03-09T15:01:10.490+01:00]
15:01:12.127 DEBUG o.b.c.c.ConversationEventMonitor - 559da1c3: DELETE_FILE for file 1-1-20210304152022685-00000-ciblee_2015_kb-test-har-004.kb.dk.warc:Ignoring message from irrelevant component netarkiv-tape-devel-01
15:01:12.281 DEBUG o.b.p.m.l.DeleteFileMessageLogger - Received IdentifyPillarsForDeleteFileResponse(559da1c3) collectionID Test3, from netarkiv-disk-devel-01: FILE_NOT_FOUND_FAILURE(Cannot serve request, file not on store)
15:01:12.282 DEBUG o.b.c.c.ConversationEventMonitor - 559da1c3: DELETE_FILE for file 1-1-20210304152022685-00000-ciblee_2015_kb-test-har-004.kb.dk.warc:Ignoring message from irrelevant component netarkiv-disk-devel-01
15:01:12.490 DEBUG o.b.p.m.l.DeleteFileMessageLogger - Received DeleteFileFinalResponse(559da1c3) collectionID Test3, from netarkiv-kbh-disk-devel-01: OPERATION_COMPLETED(1-1-20210304152022685-00000-ciblee_2015_kb-test-har-004.kb.dk.warc deleted.)
15:01:12.491 DEBUG o.b.c.c.s.ContributorResponseStatus - Received response from: netarkiv-kbh-disk-devel-01
15:01:12.492 DEBUG o.b.c.c.s.ContributorResponseStatus - Expected contributors: [netarkiv-kbh-disk-devel-01], components that have not answered: []
15:01:12.492 DEBUG o.b.commandline.DeleteFileCmd - Complete: 559da1c3: DELETE_FILE for file 1-1-20210304152022685-00000-ciblee_2015_kb-test-har-004.kb.dk.warc: COMPLETE:
15:01:12.492 INFO o.b.commandline.DeleteFileCmd - Results of the DeleteFile operation for the file '1-1-20210304152022685-00000-ciblee_2015_kb-test-har-004.kb.dk.warc':
15:01:12.492 DEBUG o.b.commandline.DeleteFileCmd - 559da1c3: DELETE_FILE for file 1-1-20210304152022685-00000-ciblee_2015_kb-test-har-004.kb.dk.warc: COMPLETE:
calling cleanup_hosts
running cleanup_hosts
Removing installation on sb-test-har-001.statsbiblioteket.dk
Removing installation on kb-test-har-003.kb.dk
Removing installation on kb-test-har-004.kb.dk
Removing installation on kb-test-har-005.kb.dk
Removing installation on kb-test-adm-001.kb.dk
Removing installation on kb-test-acs-001.kb.dk
Removing installation on kb-test-way-001.kb.dk
calling cleanup_linuxbarhosts
running cleanup_linuxbarhosts
Removing installation on sb-test-bar-001.statsbiblioteket.dk
Removing bitarchive on sb-test-bar-001.statsbiblioteket.dk
Calling cleanup_windowsbarhosts
running cleanup_windowsbarhosts
Removing C:\bitarkiv\Test3 on kb-test-bar-01.bitarkiv.kb.dk
The system cannot find the file specified.
Removing E:\bitarchive_1\Test3 on kb-test-bar-01.bitarkiv.kb.dk
The system cannot find the file specified.
Removing F:\bitarchive_2\Test3 on kb-test-bar-01.bitarkiv.kb.dk
The system cannot find the file specified.
Removing g:\bitarchive_3\Test3 on kb-test-bar-01.bitarkiv.kb.dk
The system cannot find the file specified.
TESTX=Test3
PORT=8076
MAILRECEIVERS=rbkr@kb.dk
VERSION=RBKR
DEPLOYCONF=deploy_config_bitmag.xml
H3ZIP=RBKR
BITMAGCONF=/home/devel/bitmag_test_config/client_config
Using bundler file /home/devel/nas_versions/bundler/NetarchiveSuite-heritrix3-bundler-RBKR.zip
Creating distribution files in /home/devel/prepared_software/Test3.
Will read source files using encoding 'UTF-8' (defaulted)
Branch in settings not found: common.warcRecordService
Branch in settings not found: common.arcrepositoryClient.bitrepository.getFileIDsMaxResults
Branch in settings not found: common.hadoop.kerberos
Branch in settings not found: common.hadoop.username
Branch in settings not found: common.hadoop.mapred.hadoopUberJar
Branch in settings not found: common.hadoop.mapred.inputFilesParentDir
Branch in settings not found: common.hadoop.mapred.cdxJob
Branch in settings not found: common.hadoop.mapred.metadataExtractionJob
Branch in settings not found: common.hadoop.mapred.metadataCDXExtractionJob
Branch in settings not found: common.hadoop.mapred.crawlLogExtractionJob
Branch in settings not found: common.fileResolver.baseUrl
Branch in settings not found: common.fileResolver.keyfile
Branch in settings not found: harvester.harvesting.heritrix3.bundle
Branch in settings not found: harvester.harvesting.heritrix3.certificate
Branch in settings not found: harvester.harvesting.heritrix3.bundle
Branch in settings not found: harvester.harvesting.heritrix3.certificate
Branch in settings not found: harvester.harvesting.umbra
Branch in settings not found: harvester.harvesting.heritrix3.bundle
Branch in settings not found: harvester.harvesting.heritrix3.certificate
Branch in settings not found: harvester.harvesting.umbra
Branch in settings not found: harvester.harvesting.heritrix3.bundle
Branch in settings not found: harvester.harvesting.heritrix3.bundle
Machine 'kb-test-adm-001.kb.dk' configured with encoding 'UTF-8' (defaulted)
Machine 'kb-test-har-003.kb.dk' configured with encoding 'UTF-8' (defaulted)
NetarchiveSuite-heritrix3-bundler.zip -> /home/devel/Test3/NetarchiveSuite-heritrix3-bundler.zip
/home/devel/quickstart-nas-5.0-h3/h3server.jks -> /home/devel/Test3/h3server.jks
Machine 'kb-test-har-005.kb.dk' configured with encoding 'UTF-8' (defaulted)
NetarchiveSuite-heritrix3-bundler.zip -> /home/devel/Test3/NetarchiveSuite-heritrix3-bundler.zip
/home/devel/quickstart-nas-5.0-h3/h3server.jks -> /home/devel/Test3/h3server.jks
Machine 'kb-test-har-004.kb.dk' configured with encoding 'UTF-8' (defaulted)
NetarchiveSuite-heritrix3-bundler.zip -> /home/devel/Test3/NetarchiveSuite-heritrix3-bundler.zip
Machine 'kb-test-acs-001.kb.dk' configured with encoding 'UTF-8' (defaulted)
Machine 'kb-test-way-001.kb.dk' configured with encoding 'UTF-8' (defaulted)
Machine 'sb-test-har-001.statsbiblioteket.dk' configured with encoding 'UTF-8' (defaulted)
/home/devel/prepared_software/Test3/NetarchiveSuite-heritrix3-bundler.zip -> /home/netarkdv/Test3/NetarchiveSuite-heritrix3-bundler.zip
15:01:54.253 [main] DEBUG dk.netarkivet.deploy.XmlStructure - Element baseDir is not a branch in the tree [archive, bitpreservation, baseDir]. Null returned
15:01:54.258 [main] DEBUG dk.netarkivet.deploy.XmlStructure - Element is not text. Null returned. Path given to getLeafValue archive
15:01:54.259 [main] DEBUG dk.netarkivet.deploy.XmlStructure - Element baseDir is not a branch in the tree [archive, bitpreservation, baseDir]. Null returned
15:01:54.259 [main] DEBUG dk.netarkivet.deploy.XmlStructure - Element is not text. Null returned. Path given to getLeafValue archive
15:01:54.259 [main] DEBUG dk.netarkivet.deploy.XmlStructure - Element baseDir is not a branch in the tree [archive, bitpreservation, baseDir]. Null returned
15:01:54.259 [main] DEBUG dk.netarkivet.deploy.XmlStructure - Element is not text. Null returned. Path given to getLeafValue archive
15:01:54.260 [main] DEBUG dk.netarkivet.deploy.XmlStructure - Element baseDir is not a branch in the tree [archive, bitpreservation, baseDir]. Null returned
15:01:54.260 [main] DEBUG dk.netarkivet.deploy.XmlStructure - Element is not text. Null returned. Path given to getLeafValue archive
15:01:54.260 [main] DEBUG dk.netarkivet.deploy.XmlStructure - Element baseDir is not a branch in the tree [archive, bitpreservation, baseDir]. Null returned
15:01:54.261 [main] DEBUG dk.netarkivet.deploy.XmlStructure - Element is not text. Null returned. Path given to getLeafValue archive
15:01:54.261 [main] DEBUG dk.netarkivet.deploy.XmlStructure - Element baseDir is not a branch in the tree [archive, bitpreservation, baseDir]. Null returned
15:01:54.261 [main] DEBUG dk.netarkivet.deploy.XmlStructure - Element is not text. Null returned. Path given to getLeafValue archive
15:01:54.310 [main] DEBUG dk.netarkivet.deploy.XmlStructure - Element baseDir is not a branch in the tree [archive, bitpreservation, baseDir]. Null returned
15:01:54.310 [main] DEBUG dk.netarkivet.deploy.XmlStructure - Element is not text. Null returned. Path given to getLeafValue archive
--------------------------------------------
INSTALLING TO MACHINE: devel@kb-test-adm-001.kb.dk
copying NetarchiveSuite-RBKR.zip to:kb-test-adm-001.kb.dk
deleting devel@kb-test-adm-001.kb.dk:/home/devel/Test3/lib
unzipping NetarchiveSuite-RBKR.zip at:kb-test-adm-001.kb.dk
Creating directories.
preparing for copying of settings and scripts
copying settings and scripts
make scripts executable
make password and access files readonly
--------------------------------------------
INSTALLING TO MACHINE: devel@kb-test-har-003.kb.dk
copying NetarchiveSuite-RBKR.zip to:kb-test-har-003.kb.dk
deleting devel@kb-test-har-003.kb.dk:/home/devel/Test3/lib
unzipping NetarchiveSuite-RBKR.zip at:kb-test-har-003.kb.dk
Creating directories.
preparing for copying of settings and scripts
copying settings and scripts
make scripts executable
make password and access files readonly
--------------------------------------------
INSTALLING TO MACHINE: devel@kb-test-har-005.kb.dk
copying NetarchiveSuite-RBKR.zip to:kb-test-har-005.kb.dk
deleting devel@kb-test-har-005.kb.dk:/home/devel/Test3/lib
unzipping NetarchiveSuite-RBKR.zip at:kb-test-har-005.kb.dk
Creating directories.
preparing for copying of settings and scripts
copying settings and scripts
make scripts executable
make password and access files readonly
--------------------------------------------
INSTALLING TO MACHINE: devel@kb-test-har-004.kb.dk
copying NetarchiveSuite-RBKR.zip to:kb-test-har-004.kb.dk
deleting devel@kb-test-har-004.kb.dk:/home/devel/Test3/lib
unzipping NetarchiveSuite-RBKR.zip at:kb-test-har-004.kb.dk
Creating directories.
preparing for copying of settings and scripts
copying settings and scripts
make scripts executable
make password and access files readonly
--------------------------------------------
INSTALLING TO MACHINE: devel@kb-test-acs-001.kb.dk
copying NetarchiveSuite-RBKR.zip to:kb-test-acs-001.kb.dk
deleting devel@kb-test-acs-001.kb.dk:/home/devel/Test3/lib
unzipping NetarchiveSuite-RBKR.zip at:kb-test-acs-001.kb.dk
Creating directories.
preparing for copying of settings and scripts
copying settings and scripts
make scripts executable
make password and access files readonly
--------------------------------------------
INSTALLING TO MACHINE: devel@kb-test-way-001.kb.dk
copying NetarchiveSuite-RBKR.zip to:kb-test-way-001.kb.dk
deleting devel@kb-test-way-001.kb.dk:/home/devel/Test3/lib
unzipping NetarchiveSuite-RBKR.zip at:kb-test-way-001.kb.dk
Creating directories.
preparing for copying of settings and scripts
copying settings and scripts
make scripts executable
make password and access files readonly
--------------------------------------------
--------------------------------------------
INSTALLING TO MACHINE: netarkdv@sb-test-har-001.statsbiblioteket.dk
copying NetarchiveSuite-RBKR.zip to:sb-test-har-001.statsbiblioteket.dk
deleting netarkdv@sb-test-har-001.statsbiblioteket.dk:/home/netarkdv/Test3/lib
unzipping NetarchiveSuite-RBKR.zip at:sb-test-har-001.statsbiblioteket.dk
Creating directories.
preparing for copying of settings and scripts
copying settings and scripts
make scripts executable
make password and access files readonly
--------------------------------------------
Initializing database for system Test3
+ host=devel@kb-test-adm-001
+ scp -p /home/devel/devel-config/scripts/db/setup_admindb.sh /home/devel/devel-config/scripts/db/setup_harvestdb.sh /home/devel/devel-config/scripts/db/teardown_admindb.sh /home/devel/devel-config/scripts/db/teardown_harvestdb.sh /home/devel/devel-config/scripts/../resources/harvestdb.testdata.sql /home/devel/devel-config/scripts/../resources/default_orderxml.cxml devel@kb-test-adm-001:bin/
+ ssh devel@kb-test-adm-001
Pseudo-terminal will not be allocated because stdin is not a terminal.
Destroying test3_admindb
DEBUG: transaction ID wrap limit is 2257721481, limited by database "test2056_harvestdb"
DROP DATABASE
Creating empty test3_admindb
DEBUG: transaction ID wrap limit is 2257721481, limited by database "test2056_harvestdb"
CREATE DATABASE
Initialising test3_admindb
CREATE TABLE
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "replica_pkey" for table "replica"
NOTICE: CREATE TABLE / UNIQUE will create implicit index "replica_replica_id_key" for table "replica"
NOTICE: CREATE TABLE / UNIQUE will create implicit index "replica_replica_name_key" for table "replica"
CREATE TABLE
CREATE SEQUENCE
ALTER TABLE
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "replicafileinfo_pkey" for table "replicafileinfo"
CREATE TABLE
CREATE SEQUENCE
ALTER TABLE
CREATE INDEX
CREATE INDEX
CREATE INDEX
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "file_pkey" for table "file"
CREATE TABLE
CREATE SEQUENCE
ALTER TABLE
CREATE INDEX
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "segment_pkey" for table "segment"
CREATE TABLE
CREATE SEQUENCE
ALTER TABLE
Destroying test3_harvestdb
DEBUG: transaction ID wrap limit is 2257721481, limited by database "test2056_harvestdb"
DROP DATABASE
DEBUG: transaction ID wrap limit is 2257721481, limited by database "test2056_harvestdb"
CREATE DATABASE
CREATE TABLE
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
GRANT
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "domains_pkey" for table "domains"
NOTICE: CREATE TABLE / UNIQUE will create implicit index "domains_name_key" for table "domains"
CREATE TABLE
CREATE SEQUENCE
ALTER TABLE
CREATE INDEX
CREATE INDEX
GRANT
GRANT
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "configurations_pkey" for table "configurations"
CREATE TABLE
CREATE SEQUENCE
ALTER TABLE
CREATE INDEX
CREATE INDEX
CREATE INDEX
GRANT
GRANT
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "config_passwords_pkey" for table "config_passwords"
CREATE TABLE
GRANT
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "config_seedlists_pkey" for table "config_seedlists"
CREATE TABLE
GRANT
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "seedlists_pkey" for table "seedlists"
CREATE TABLE
CREATE SEQUENCE
ALTER TABLE
CREATE INDEX
CREATE INDEX
GRANT
GRANT
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "passwords_pkey" for table "passwords"
CREATE TABLE
CREATE SEQUENCE
ALTER TABLE
CREATE INDEX
CREATE INDEX
GRANT
GRANT
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "ownerinfo_pkey" for table "ownerinfo"
CREATE TABLE
CREATE SEQUENCE
ALTER TABLE
CREATE INDEX
GRANT
GRANT
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "historyinfo_pkey" for table "historyinfo"
CREATE TABLE
CREATE SEQUENCE
ALTER TABLE
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
GRANT
GRANT
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "harvestdefinitions_pkey" for table "harvestdefinitions"
NOTICE: CREATE TABLE / UNIQUE will create implicit index "harvestdefinitions_name_key" for table "harvestdefinitions"
CREATE TABLE
CREATE INDEX
GRANT
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "fullharvests_pkey" for table "fullharvests"
CREATE TABLE
GRANT
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "partialharvests_pkey" for table "partialharvests"
CREATE TABLE
CREATE INDEX
GRANT
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "harvest_configs_pkey" for table "harvest_configs"
CREATE TABLE
GRANT
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "schedules_pkey" for table "schedules"
NOTICE: CREATE TABLE / UNIQUE will create implicit index "schedules_name_key" for table "schedules"
CREATE TABLE
CREATE SEQUENCE
ALTER TABLE
GRANT
GRANT
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "ordertemplates_pkey" for table "ordertemplates"
NOTICE: CREATE TABLE / UNIQUE will create implicit index "ordertemplates_name_key" for table "ordertemplates"
CREATE TABLE
CREATE SEQUENCE
ALTER TABLE
GRANT
GRANT
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "jobs_pkey" for table "jobs"
CREATE TABLE
CREATE INDEX
CREATE INDEX
GRANT
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "job_configs_pkey" for table "job_configs"
CREATE TABLE
CREATE INDEX
GRANT
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "global_crawler_trap_lists_pkey" for table "global_crawler_trap_lists"
NOTICE: CREATE TABLE / UNIQUE will create implicit index "global_crawler_trap_lists_name_key" for table "global_crawler_trap_lists"
CREATE TABLE
CREATE SEQUENCE
ALTER TABLE
CREATE INDEX
GRANT
GRANT
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "global_crawler_trap_expressions_pkey" for table "global_crawler_trap_expressions"
CREATE TABLE
CREATE SEQUENCE
ALTER TABLE
CREATE INDEX
CREATE INDEX
GRANT
GRANT
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "pkrunningjobshistory" for table "runningjobshistory"
CREATE TABLE
CREATE INDEX
CREATE INDEX
CREATE INDEX
GRANT
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "pkrunningjobsmonitor" for table "runningjobsmonitor"
CREATE TABLE
CREATE INDEX
CREATE INDEX
GRANT
NOTICE: CREATE TABLE / UNIQUE will create implicit index "pkfrontierreportlines" for table "frontierreportmonitor"
CREATE TABLE
GRANT
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "extendedfieldtype_pkey" for table "extendedfieldtype"
CREATE TABLE
CREATE SEQUENCE
ALTER TABLE
GRANT
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "extendedfield_pkey" for table "extendedfield"
CREATE TABLE
CREATE SEQUENCE
ALTER TABLE
GRANT
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "extendedfieldvalue_pkey" for table "extendedfieldvalue"
CREATE TABLE
CREATE SEQUENCE
ALTER TABLE
GRANT
INSERT 0 1
INSERT 0 1
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "harvestchannel_pkey" for table "harvestchannel"
NOTICE: CREATE TABLE / UNIQUE will create implicit index "harvestchannel_name_key" for table "harvestchannel"
CREATE TABLE
CREATE SEQUENCE
ALTER TABLE
CREATE INDEX
GRANT
GRANT
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
CREATE TABLE
NOTICE: ALTER TABLE / ADD PRIMARY KEY will create implicit index "eav_type_attribute_pkey" for table "eav_type_attribute"
ALTER TABLE
CREATE INDEX
CREATE SEQUENCE
CREATE TABLE
NOTICE: ALTER TABLE / ADD PRIMARY KEY will create implicit index "eav_attribute_pkey" for table "eav_attribute"
ALTER TABLE
CREATE INDEX
GRANT
GRANT
GRANT
INSERT 0 1
INSERT 0 1
INSERT 0 1
SET
SET
SET
SET
SET
SET
SET
SET
SET
setval
--------
17
(1 row)
setval
--------
17
(1 row)
setval
--------
17
(1 row)
setval
--------
5
(1 row)
setval
--------
4
(1 row)
setval
--------
17
(1 row)
15:03:55.657 [main] INFO com.mchange.v2.log.MLog - MLog clients using log4j logging.
15:03:56.000 [main] INFO com.mchange.v2.c3p0.C3P0Registry - Initializing c3p0-0.9.2.1 [built 20-March-2013 10:47:27 +0000; debug? true; trace: 10]
15:03:56.090 [main] DEBUG com.mchange.v2.c3p0.management.DynamicPooledDataSourceManagerMBean - MBean: com.mchange.v2.c3p0:type=PooledDataSource,identityToken=10bdx52ag6gr1ggy4bfvu|150c158,name=10bdx52ag6gr1ggy4bfvu|150c158 registered.
15:03:56.174 [main] DEBUG com.mchange.v2.c3p0.management.DynamicPooledDataSourceManagerMBean - MBean: com.mchange.v2.c3p0:type=PooledDataSource,identityToken=10bdx52ag6gr1ggy4bfvu|150c158,name=10bdx52ag6gr1ggy4bfvu|150c158 unregistered, in order to be reregistered after update.
15:03:56.174 [main] DEBUG com.mchange.v2.c3p0.management.DynamicPooledDataSourceManagerMBean - MBean: com.mchange.v2.c3p0:type=PooledDataSource,identityToken=10bdx52ag6gr1ggy4bfvu|150c158,name=10bdx52ag6gr1ggy4bfvu|150c158 registered.
15:03:56.202 [main] INFO dk.netarkivet.harvester.datamodel.HarvestDBConnection - Connection pool initialized with the following values:
- minPoolSize=5
- maxPoolSize=20
- acquireIncrement=5
- maxStatements=0
- maxStatementsPerConnection=0
- idleConnTestPeriod=0
- idleConnTestQuery='null'
- idleConnTestOnCheckin=false
15:03:56.282 [main] INFO com.mchange.v2.c3p0.impl.AbstractPoolBackedDataSource - Initializing c3p0 pool... com.mchange.v2.c3p0.ComboPooledDataSource [ acquireIncrement -> 5, acquireRetryAttempts -> 30, acquireRetryDelay -> 1000, autoCommitOnClose -> false, automaticTestTable -> null, breakAfterAcquireFailure -> false, checkoutTimeout -> 0, connectionCustomizerClassName -> null, connectionTesterClassName -> com.mchange.v2.c3p0.impl.DefaultConnectionTester, dataSourceName -> 10bdx52ag6gr1ggy4bfvu|150c158, debugUnreturnedConnectionStackTraces -> false, description -> null, driverClass -> org.postgresql.Driver, factoryClassLocation -> null, forceIgnoreUnresolvedTransactions -> false, identityToken -> 10bdx52ag6gr1ggy4bfvu|150c158, idleConnectionTestPeriod -> 0, initialPoolSize -> 3, jdbcUrl -> jdbc:postgresql://localhost:5432/test3_harvestdb, maxAdministrativeTaskTime -> 0, maxConnectionAge -> 3600, maxIdleTime -> 0, maxIdleTimeExcessConnections -> 0, maxPoolSize -> 20, maxStatements -> 0, maxStatementsPerConnection -> 0, minPoolSize -> 5, numHelperThreads -> 3, preferredTestQuery -> null, properties -> {user=******, password=******}, propertyCycle -> 0, statementCacheNumDeferredCloseThreads -> 0, testConnectionOnCheckin -> false, testConnectionOnCheckout -> false, unreturnedConnectionTimeout -> 0, userOverrides -> {}, usesTraditionalReflectiveProxies -> false ]
15:03:56.306 [main] DEBUG com.mchange.v2.resourcepool.BasicResourcePool - incremented pending_acquires: 1
15:03:56.307 [main] DEBUG com.mchange.v2.resourcepool.BasicResourcePool - Starting acquisition series. Incremented pending_acquires [1], attempts_remaining: 30
15:03:56.307 [main] DEBUG com.mchange.v2.async.ThreadPoolAsynchronousRunner - com.mchange.v2.async.ThreadPoolAsynchronousRunner@1040f2f: Adding task to queue -- com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask@c87930
15:03:56.307 [main] DEBUG com.mchange.v2.resourcepool.BasicResourcePool - incremented pending_acquires: 2
15:03:56.307 [main] DEBUG com.mchange.v2.resourcepool.BasicResourcePool - Starting acquisition series. Incremented pending_acquires [2], attempts_remaining: 30
15:03:56.307 [main] DEBUG com.mchange.v2.async.ThreadPoolAsynchronousRunner - com.mchange.v2.async.ThreadPoolAsynchronousRunner@1040f2f: Adding task to queue -- com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask@cff1a
15:03:56.308 [main] DEBUG com.mchange.v2.resourcepool.BasicResourcePool - incremented pending_acquires: 3
15:03:56.308 [main] DEBUG com.mchange.v2.resourcepool.BasicResourcePool - Starting acquisition series. Incremented pending_acquires [3], attempts_remaining: 30
15:03:56.308 [main] DEBUG com.mchange.v2.async.ThreadPoolAsynchronousRunner - com.mchange.v2.async.ThreadPoolAsynchronousRunner@1040f2f: Adding task to queue -- com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask@15710fc
15:03:56.308 [main] DEBUG com.mchange.v2.resourcepool.BasicResourcePool - incremented pending_acquires: 4
15:03:56.308 [main] DEBUG com.mchange.v2.resourcepool.BasicResourcePool - Starting acquisition series. Incremented pending_acquires [4], attempts_remaining: 30
15:03:56.308 [main] DEBUG com.mchange.v2.async.ThreadPoolAsynchronousRunner - com.mchange.v2.async.ThreadPoolAsynchronousRunner@1040f2f: Adding task to queue -- com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask@5f1e4e
15:03:56.309 [main] DEBUG com.mchange.v2.resourcepool.BasicResourcePool - incremented pending_acquires: 5
15:03:56.309 [main] DEBUG com.mchange.v2.resourcepool.BasicResourcePool - Starting acquisition series. Incremented pending_acquires [5], attempts_remaining: 30
15:03:56.309 [main] DEBUG com.mchange.v2.async.ThreadPoolAsynchronousRunner - com.mchange.v2.async.ThreadPoolAsynchronousRunner@1040f2f: Adding task to queue -- com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask@1f9eccc
15:03:56.318 [main] DEBUG com.mchange.v2.resourcepool.BasicResourcePool - com.mchange.v2.resourcepool.BasicResourcePool@aa2946 config: [start -> 3; min -> 5; max -> 20; inc -> 5; num_acq_attempts -> 30; acq_attempt_delay -> 1000; check_idle_resources_delay -> 0; mox_resource_age -> 3600000; max_idle_time -> 0; excess_max_idle_time -> 0; destroy_unreturned_resc_time -> 0; expiration_enforcement_delay -> 900000; break_on_acquisition_failure -> false; debug_store_checkout_exceptions -> false]
15:03:56.318 [main] DEBUG com.mchange.v2.c3p0.impl.C3P0PooledConnectionPoolManager - Created new pool for auth, username (masked): 'de******'.
15:03:56.318 [main] DEBUG com.mchange.v2.resourcepool.BasicResourcePool - acquire test -- pool size: 0; target_pool_size: 5; desired target? 1
15:03:56.318 [main] DEBUG com.mchange.v2.resourcepool.BasicResourcePool - awaitAvailable(): [unknown]
15:03:56.319 [main] DEBUG com.mchange.v2.resourcepool.BasicResourcePool - trace com.mchange.v2.resourcepool.BasicResourcePool@aa2946 [managed: 0, unused: 0, excluded: 0]
15:03:56.469 [C3P0PooledConnectionPoolManager[identityToken->10bdx52ag6gr1ggy4bfvu|150c158]-HelperThread-#0] DEBUG com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool - com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager@1e91528.acquireResource() returning.
15:03:56.470 [C3P0PooledConnectionPoolManager[identityToken->10bdx52ag6gr1ggy4bfvu|150c158]-HelperThread-#2] DEBUG com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool - com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager@1e91528.acquireResource() returning.
15:03:56.471 [C3P0PooledConnectionPoolManager[identityToken->10bdx52ag6gr1ggy4bfvu|150c158]-HelperThread-#0] DEBUG com.mchange.v2.resourcepool.BasicResourcePool - trace com.mchange.v2.resourcepool.BasicResourcePool@aa2946 [managed: 1, unused: 1, excluded: 0]
15:03:56.471 [C3P0PooledConnectionPoolManager[identityToken->10bdx52ag6gr1ggy4bfvu|150c158]-HelperThread-#0] DEBUG com.mchange.v2.resourcepool.BasicResourcePool - decremented pending_acquires: 4
15:03:56.472 [C3P0PooledConnectionPoolManager[identityToken->10bdx52ag6gr1ggy4bfvu|150c158]-HelperThread-#0] DEBUG com.mchange.v2.resourcepool.BasicResourcePool - Acquisition series terminated successfully. Decremented pending_acquires [4], attempts_remaining: 30
15:03:56.472 [C3P0PooledConnectionPoolManager[identityToken->10bdx52ag6gr1ggy4bfvu|150c158]-HelperThread-#2] DEBUG com.mchange.v2.resourcepool.BasicResourcePool - trace com.mchange.v2.resourcepool.BasicResourcePool@aa2946 [managed: 2, unused: 1, excluded: 0] (e.g. com.mchange.v2.c3p0.impl.NewPooledConnection@1e0acdb)
15:03:56.472 [C3P0PooledConnectionPoolManager[identityToken->10bdx52ag6gr1ggy4bfvu|150c158]-HelperThread-#2] DEBUG com.mchange.v2.resourcepool.BasicResourcePool - decremented pending_acquires: 3
15:03:56.472 [C3P0PooledConnectionPoolManager[identityToken->10bdx52ag6gr1ggy4bfvu|150c158]-HelperThread-#2] DEBUG com.mchange.v2.resourcepool.BasicResourcePool - Acquisition series terminated successfully. Decremented pending_acquires [3], attempts_remaining: 30
15:03:56.473 [main] DEBUG com.mchange.v2.resourcepool.BasicResourcePool - trace com.mchange.v2.resourcepool.BasicResourcePool@aa2946 [managed: 2, unused: 1, excluded: 0] (e.g. com.mchange.v2.c3p0.impl.NewPooledConnection@1e0acdb)
15:03:56.475 [C3P0PooledConnectionPoolManager[identityToken->10bdx52ag6gr1ggy4bfvu|150c158]-HelperThread-#1] DEBUG com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool - com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager@1e91528.acquireResource() returning.
15:03:56.475 [C3P0PooledConnectionPoolManager[identityToken->10bdx52ag6gr1ggy4bfvu|150c158]-HelperThread-#1] DEBUG com.mchange.v2.resourcepool.BasicResourcePool - trace com.mchange.v2.resourcepool.BasicResourcePool@aa2946 [managed: 3, unused: 2, excluded: 0] (e.g. com.mchange.v2.c3p0.impl.NewPooledConnection@1e0acdb)
15:03:56.475 [C3P0PooledConnectionPoolManager[identityToken->10bdx52ag6gr1ggy4bfvu|150c158]-HelperThread-#1] DEBUG com.mchange.v2.resourcepool.BasicResourcePool - decremented pending_acquires: 2
15:03:56.475 [C3P0PooledConnectionPoolManager[identityToken->10bdx52ag6gr1ggy4bfvu|150c158]-HelperThread-#1] DEBUG com.mchange.v2.resourcepool.BasicResourcePool - Acquisition series terminated successfully. Decremented pending_acquires [2], attempts_remaining: 30
15:03:56.484 [C3P0PooledConnectionPoolManager[identityToken->10bdx52ag6gr1ggy4bfvu|150c158]-HelperThread-#0] DEBUG com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool - com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager@1e91528.acquireResource() returning.
15:03:56.484 [C3P0PooledConnectionPoolManager[identityToken->10bdx52ag6gr1ggy4bfvu|150c158]-HelperThread-#0] DEBUG com.mchange.v2.resourcepool.BasicResourcePool - trace com.mchange.v2.resourcepool.BasicResourcePool@aa2946 [managed: 4, unused: 3, excluded: 0] (e.g. com.mchange.v2.c3p0.impl.NewPooledConnection@1e0acdb)
15:03:56.484 [C3P0PooledConnectionPoolManager[identityToken->10bdx52ag6gr1ggy4bfvu|150c158]-HelperThread-#0] DEBUG com.mchange.v2.resourcepool.BasicResourcePool - decremented pending_acquires: 1
15:03:56.484 [C3P0PooledConnectionPoolManager[identityToken->10bdx52ag6gr1ggy4bfvu|150c158]-HelperThread-#0] DEBUG com.mchange.v2.resourcepool.BasicResourcePool - Acquisition series terminated successfully. Decremented pending_acquires [1], attempts_remaining: 30
15:03:56.485 [C3P0PooledConnectionPoolManager[identityToken->10bdx52ag6gr1ggy4bfvu|150c158]-HelperThread-#2] DEBUG com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool - com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager@1e91528.acquireResource() returning.
15:03:56.485 [C3P0PooledConnectionPoolManager[identityToken->10bdx52ag6gr1ggy4bfvu|150c158]-HelperThread-#2] DEBUG com.mchange.v2.resourcepool.BasicResourcePool - trace com.mchange.v2.resourcepool.BasicResourcePool@aa2946 [managed: 5, unused: 4, excluded: 0] (e.g. com.mchange.v2.c3p0.impl.NewPooledConnection@1e0acdb)
15:03:56.485 [C3P0PooledConnectionPoolManager[identityToken->10bdx52ag6gr1ggy4bfvu|150c158]-HelperThread-#2] DEBUG com.mchange.v2.resourcepool.BasicResourcePool - decremented pending_acquires: 0
15:03:56.485 [C3P0PooledConnectionPoolManager[identityToken->10bdx52ag6gr1ggy4bfvu|150c158]-HelperThread-#2] DEBUG com.mchange.v2.resourcepool.BasicResourcePool - Acquisition series terminated successfully. Decremented pending_acquires [0], attempts_remaining: 30
15:03:56.574 [main] DEBUG com.mchange.v2.c3p0.impl.NewProxyPreparedStatement - com.mchange.v2.c3p0.impl.NewProxyPreparedStatement@1938003 closed orphaned ResultSet: com.mchange.v2.c3p0.impl.NewProxyResultSet@1a561cf
15:03:56.583 [main] DEBUG com.mchange.v2.async.ThreadPoolAsynchronousRunner - com.mchange.v2.async.ThreadPoolAsynchronousRunner@1040f2f: Adding task to queue -- com.mchange.v2.resourcepool.BasicResourcePool$1RefurbishCheckinResourceTask@646153
15:03:56.584 [main] DEBUG com.mchange.v2.resourcepool.BasicResourcePool - trace com.mchange.v2.resourcepool.BasicResourcePool@aa2946 [managed: 5, unused: 4, excluded: 0] (e.g. com.mchange.v2.c3p0.impl.NewPooledConnection@1e0acdb)
15:03:56.594 [main] DEBUG com.mchange.v2.resourcepool.BasicResourcePool - trace com.mchange.v2.resourcepool.BasicResourcePool@aa2946 [managed: 5, unused: 4, excluded: 0] (e.g. com.mchange.v2.c3p0.impl.NewPooledConnection@1e0acdb)
15:03:56.603 [main] DEBUG com.mchange.v2.c3p0.impl.NewProxyPreparedStatement - com.mchange.v2.c3p0.impl.NewProxyPreparedStatement@12f2cc1 closed orphaned ResultSet: com.mchange.v2.c3p0.impl.NewProxyResultSet@16a4e4b
15:03:56.605 [main] DEBUG com.mchange.v2.async.ThreadPoolAsynchronousRunner - com.mchange.v2.async.ThreadPoolAsynchronousRunner@1040f2f: Adding task to queue -- com.mchange.v2.resourcepool.BasicResourcePool$1RefurbishCheckinResourceTask@79870
15:03:56.606 [main] DEBUG com.mchange.v2.resourcepool.BasicResourcePool - trace com.mchange.v2.resourcepool.BasicResourcePool@aa2946 [managed: 5, unused: 4, excluded: 0] (e.g. com.mchange.v2.c3p0.impl.NewPooledConnection@1e0acdb)
15:03:56.673 [main] DEBUG com.mchange.v2.resourcepool.BasicResourcePool - trace com.mchange.v2.resourcepool.BasicResourcePool@aa2946 [managed: 5, unused: 4, excluded: 0] (e.g. com.mchange.v2.c3p0.impl.NewPooledConnection@1e0acdb)
15:03:56.674 [main] DEBUG com.mchange.v2.c3p0.impl.NewProxyPreparedStatement - com.mchange.v2.c3p0.impl.NewProxyPreparedStatement@1761df8 closed orphaned ResultSet: com.mchange.v2.c3p0.impl.NewProxyResultSet@417213
15:03:56.706 [main] DEBUG com.mchange.v2.async.ThreadPoolAsynchronousRunner - com.mchange.v2.async.ThreadPoolAsynchronousRunner@1040f2f: Adding task to queue -- com.mchange.v2.resourcepool.BasicResourcePool$1RefurbishCheckinResourceTask@1dd4eda
15:03:56.707 [main] DEBUG com.mchange.v2.resourcepool.BasicResourcePool - trace com.mchange.v2.resourcepool.BasicResourcePool@aa2946 [managed: 5, unused: 4, excluded: 0] (e.g. com.mchange.v2.c3p0.impl.NewPooledConnection@1e0acdb)
15:03:56.708 [main] DEBUG com.mchange.v2.c3p0.impl.NewProxyConnection - com.mchange.v2.c3p0.impl.NewProxyConnection@611244: close() called more than once.
The template 'default_orderxml' has now been updated.
+ set +x
--------------------------------------------
STARTING MACHINE: devel@kb-test-adm-001.kb.dk
Starting all applications on: kb-test-adm-001.kb.dk
Starting linux application: GUIApplication
Starting linux application: ArcRepositoryApplication
Starting linux application: HarvestJobManagerApplication
15:03:58,313 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [/home/devel/Test3/conf/logback_ArcRepositoryApplication.xml] at [file:/home/devel/Test3/conf/logback_ArcRepositoryApplication.xml]
15:03:59,060 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - debug attribute not set
15:03:59,063 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
15:03:59,082 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [STDOUT]
15:03:59,121 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
15:03:59,287 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
15:03:59,299 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [FILE]
15:03:59,350 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@379243 - No compression will be used
15:03:59,352 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@379243 - Will use the pattern ./log/ArcRepositoryApplication.%d{yyyy-MM-dd}.%i.log for the active file
15:03:59,355 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@4cf7cf - The date pattern is 'yyyy-MM-dd' from file name pattern './log/ArcRepositoryApplication.%d{yyyy-MM-dd}.%i.log'.
15:03:59,355 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@4cf7cf - Roll-over at midnight.
15:03:59,365 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@4cf7cf - Setting initial period to Tue Mar 09 15:03:59 CET 2021
15:03:59,365 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@4cf7cf - SizeAndTimeBasedFNATP is deprecated. Use SizeAndTimeBasedRollingPolicy instead
15:03:59,365 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@4cf7cf - For more information see http://logback.qos.ch/manual/appenders.html#SizeAndTimeBasedRollingPolicy
15:03:59,367 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
15:03:59,372 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - Active log file name: ./log/ArcRepositoryApplication.log
15:03:59,372 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - File property is set to [./log/ArcRepositoryApplication.log]
15:03:59,375 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [dk.netarkivet.monitor.logging.CachingSLF4JAppender]
15:04:00,336 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [MONITOR]
15:04:00,346 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to WARN
15:04:00,346 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[ROOT]
15:04:00,351 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [FILE] to Logger[ROOT]
15:04:00,352 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [MONITOR] to Logger[ROOT]
15:04:00,352 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [dk.netarkivet] to DEBUG
15:04:00,352 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
15:04:00,353 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@1d6b6c7 - Registering current configuration as safe fallback point
2021-03-09 15:04:00.383 DEBUG dk.netarkivet.common.utils.Settings.invoke0 - Value found in loaded data: SB,KB
SLF4J: A number (1) of logging calls during the initialization phase have been intercepted and are
SLF4J: now being replayed. These are subject to the filtering rules of the underlying logging system.
SLF4J: See also http://www.slf4j.org/codes.html#replay
Starting dk.netarkivet.archive.arcrepository.ArcRepository
Version: 5.7-IIPCH3-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/cb5f59cb8465f2eea112989e4ed878851c79ff87)
2021-03-09 15:04:00.407 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Starting dk.netarkivet.archive.arcrepository.ArcRepository
Version: 5.7-IIPCH3-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/cb5f59cb8465f2eea112989e4ed878851c79ff87)
Java VM: 1.8.0_20
2021-03-09 15:04:00.408 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Java VM: 1.8.0_20
java.home: /usr/java/jdk1.8.0_20/jre
2021-03-09 15:04:00.409 INFO d.n.common.utils.ApplicationUtils.logAndPrint - java.home: /usr/java/jdk1.8.0_20/jre
Working dir: /home/devel/Test3
2021-03-09 15:04:00.411 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Working dir: /home/devel/Test3
2021-03-09 15:04:00.411 INFO d.n.common.utils.ApplicationUtils.startApp - Using settings files '/home/devel/Test3/conf/settings_ArcRepositoryApplication.xml'
2021-03-09 15:04:00.685 INFO d.n.m.d.JMSMonitorRegistryClient.register - Registering this client for monitoring every 1 minutes, using hostname 'kb-test-adm-001.kb.dk' and JMX/RMI ports 8151/8251
2021-03-09 15:04:00.685 INFO d.n.c.m.MBeanConnectorCreator.exposeJMXMBeanServer - Registered mbean server in registry on port 8151 communicating on port 8251 using password file 'conf/jmxremote.password'.
Service URL is service:jmx:rmi://kb-test-adm-001.kb.dk:8251/jndi/rmi://kb-test-adm-001.kb.dk:8151/jmxrmi
dk.netarkivet.archive.arcrepository.ArcRepository Running
2021-03-09 15:04:00.696 INFO d.n.common.utils.ApplicationUtils.logAndPrint - dk.netarkivet.archive.arcrepository.ArcRepository Running
2021-03-09 15:04:00.711 INFO d.n.c.distribute.JMSConnectionSunMQ.<init> - Creating instance of dk.netarkivet.common.distribute.JMSConnectionSunMQ
2021-03-09 15:04:00.711 DEBUG d.n.common.distribute.JMSConnection.initConnection - Initializing a JMS connection dk.netarkivet.common.distribute.JMSConnectionSunMQ
2021-03-09 15:04:00.711 INFO d.n.c.distribute.JMSConnectionSunMQ.getConnectionFactory - Establishing SunMQ JMS Connection to 'kb-test-adm-001.kb.dk:7676'
2021-03-09 15:04:01.273 INFO d.n.a.a.ArchiveDBConnection.initDataSource - Using jdbc url: jdbc:postgresql://localhost:5432/test3_admindb
2021-03-09 15:04:01.275 INFO d.n.a.a.ArchiveDBConnection.initDataSource - Connection pool initialized with the following values:
- minPoolSize=5
- maxPoolSize=10
- acquireIncrement=5
- maxStatements=0
- maxStatementsPerConnection=0
- idleConnTestPeriod=30
- idleConnTestQuery='SELECT COUNT(*) FROM replica'
- idleConnTestOnCheckin=false
2021-03-09 15:04:01.772 DEBUG d.n.a.a.ReplicaCacheDatabase.initialiseDB - IDs for replicas already in the database:
2021-03-09 15:04:01.780 INFO d.n.a.a.ReplicaCacheDatabase.initialiseDB - Inserting replica 'BITARCHIVEReplica (SB) SBN' in database.
2021-03-09 15:04:01.781 DEBUG d.n.a.a.ReplicaCacheHelpers.insertReplicaIntoDB - Executing insert, conditional on SB not already existing in the database.
2021-03-09 15:04:01.793 DEBUG d.n.a.a.ReplicaCacheHelpers.insertReplicaIntoDB - Insert statement for SB returned 1
2021-03-09 15:04:01.796 INFO d.n.a.a.ReplicaCacheDatabase.initialiseDB - Inserting replica 'BITARCHIVEReplica (KB) KBN' in database.
2021-03-09 15:04:01.796 DEBUG d.n.a.a.ReplicaCacheHelpers.insertReplicaIntoDB - Executing insert, conditional on KB not already existing in the database.
2021-03-09 15:04:01.799 DEBUG d.n.a.a.ReplicaCacheHelpers.insertReplicaIntoDB - Insert statement for KB returned 1
2021-03-09 15:04:01.802 INFO d.n.a.a.ReplicaCacheDatabase.<init> - Initialization of database successful
2021-03-09 15:04:01.831 INFO d.n.a.a.d.ArcRepositoryServer.<init> - Listening for arc repository messages on channel '[Queue 'Test3_COMMON_THE_REPOS']'
2021-03-09 15:04:01.834 DEBUG d.n.common.distribute.JMSConnection.setListener - Adding dk.netarkivet.archive.arcrepository.distribute.ArcRepositoryServer@10090a4 as listener to Test3_COMMON_THE_REPOS
2021-03-09 15:04:01.864 INFO d.n.a.arcrepository.ArcRepository.<init> - Starting the ArcRepository
15:03:57,811 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [/home/devel/Test3/conf/logback_GUIApplication.xml] at [file:/home/devel/Test3/conf/logback_GUIApplication.xml]
15:03:58,345 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - debug attribute not set
15:03:58,348 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
15:03:58,366 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [STDOUT]
15:03:58,402 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
15:03:58,479 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
15:03:58,485 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [FILE]
15:03:58,528 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@28594521 - No compression will be used
15:03:58,536 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@28594521 - Will use the pattern ./log/GUIApplication.%d{yyyy-MM-dd}.%i.log for the active file
15:03:58,546 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@107227e - The date pattern is 'yyyy-MM-dd' from file name pattern './log/GUIApplication.%d{yyyy-MM-dd}.%i.log'.
15:03:58,546 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@107227e - Roll-over at midnight.
15:03:58,555 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@107227e - Setting initial period to Tue Mar 09 15:03:58 CET 2021
15:03:58,556 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@107227e - SizeAndTimeBasedFNATP is deprecated. Use SizeAndTimeBasedRollingPolicy instead
15:03:58,556 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@107227e - For more information see http://logback.qos.ch/manual/appenders.html#SizeAndTimeBasedRollingPolicy
15:03:58,558 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
15:03:58,563 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - Active log file name: ./log/GUIApplication.log
15:03:58,563 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - File property is set to [./log/GUIApplication.log]
15:03:58,564 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [dk.netarkivet.monitor.logging.CachingSLF4JAppender]
15:03:59,530 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [MONITOR]
15:03:59,543 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to WARN
15:03:59,543 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[ROOT]
15:03:59,545 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [FILE] to Logger[ROOT]
15:03:59,545 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [MONITOR] to Logger[ROOT]
15:03:59,545 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [dk.netarkivet] to DEBUG
15:03:59,545 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
15:03:59,546 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@417213 - Registering current configuration as safe fallback point
2021-03-09 15:03:59.573 DEBUG dk.netarkivet.common.utils.Settings.invoke0 - Value found in loaded data: SB,KB
SLF4J: A number (1) of logging calls during the initialization phase have been intercepted and are
SLF4J: now being replayed. These are subject to the filtering rules of the underlying logging system.
SLF4J: See also http://www.slf4j.org/codes.html#replay
Starting dk.netarkivet.common.webinterface.GUIWebServer
Version: 5.7-IIPCH3-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/cb5f59cb8465f2eea112989e4ed878851c79ff87)
2021-03-09 15:03:59.599 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Starting dk.netarkivet.common.webinterface.GUIWebServer
Version: 5.7-IIPCH3-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/cb5f59cb8465f2eea112989e4ed878851c79ff87)
Java VM: 1.8.0_20
2021-03-09 15:03:59.601 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Java VM: 1.8.0_20
java.home: /usr/java/jdk1.8.0_20/jre
2021-03-09 15:03:59.602 INFO d.n.common.utils.ApplicationUtils.logAndPrint - java.home: /usr/java/jdk1.8.0_20/jre
Working dir: /home/devel/Test3
2021-03-09 15:03:59.604 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Working dir: /home/devel/Test3
2021-03-09 15:03:59.604 INFO d.n.common.utils.ApplicationUtils.startApp - Using settings files '/home/devel/Test3/conf/settings_GUIApplication.xml'
2021-03-09 15:03:59.943 INFO d.n.m.d.JMSMonitorRegistryClient.register - Registering this client for monitoring every 1 minutes, using hostname 'kb-test-adm-001.kb.dk' and JMX/RMI ports 8150/8250
2021-03-09 15:03:59.945 INFO d.n.c.m.MBeanConnectorCreator.exposeJMXMBeanServer - Registered mbean server in registry on port 8150 communicating on port 8250 using password file 'conf/jmxremote.password'.
Service URL is service:jmx:rmi://kb-test-adm-001.kb.dk:8250/jndi/rmi://kb-test-adm-001.kb.dk:8150/jmxrmi
dk.netarkivet.common.webinterface.GUIWebServer Running
2021-03-09 15:03:59.947 INFO d.n.common.utils.ApplicationUtils.logAndPrint - dk.netarkivet.common.webinterface.GUIWebServer Running
2021-03-09 15:03:59.959 DEBUG dk.netarkivet.common.utils.Settings.debug - Value found in loaded data: webpages/HarvestDefinition.war,webpages/History.war,webpages/HarvestChannel.war,webpages/QA.war,webpages/Status.war
2021-03-09 15:03:59.959 DEBUG dk.netarkivet.common.utils.Settings.debug - Value found in loaded data: dk.netarkivet.harvester.webinterface.DefinitionsSiteSection,dk.netarkivet.harvester.webinterface.HistorySiteSection,dk.netarkivet.harvester.webinterface.HarvestChannelSiteSection,dk.netarkivet.viewerproxy.webinterface.QASiteSection,dk.netarkivet.monitor.webinterface.StatusSiteSection
2021-03-09 15:03:59.959 INFO d.n.common.webinterface.GUIWebServer.<init> - Starting webserver. Port: 8076 deployment directories: 'webpages/HarvestDefinition.war,webpages/History.war,webpages/HarvestChannel.war,webpages/QA.war,webpages/Status.war' classes: 'dk.netarkivet.harvester.webinterface.DefinitionsSiteSection,dk.netarkivet.harvester.webinterface.HistorySiteSection,dk.netarkivet.harvester.webinterface.HarvestChannelSiteSection,dk.netarkivet.viewerproxy.webinterface.QASiteSection,dk.netarkivet.monitor.webinterface.StatusSiteSection'
2021-03-09 15:03:59.983 DEBUG d.n.common.webinterface.GUIWebServer.<init> - GUI using tempdir /home/devel/Test3/tmpdircommon
2021-03-09 15:03:59.984 DEBUG d.n.common.webinterface.GUIWebServer.<init> - GUI using basedir /home/devel/Test3
2021-03-09 15:03:59.998 INFO d.n.common.webinterface.GUIWebServer.<init> - Deleted existing tempdir '/home/devel/Test3/tmpdircommon'
2021-03-09 15:03:59.999 INFO d.n.c.distribute.JMSConnectionSunMQ.<init> - Creating instance of dk.netarkivet.common.distribute.JMSConnectionSunMQ
2021-03-09 15:04:00.003 DEBUG d.n.common.distribute.JMSConnection.initConnection - Initializing a JMS connection dk.netarkivet.common.distribute.JMSConnectionSunMQ
2021-03-09 15:04:00.003 INFO d.n.c.distribute.JMSConnectionSunMQ.getConnectionFactory - Establishing SunMQ JMS Connection to 'kb-test-adm-001.kb.dk:7676'
2021-03-09 15:04:00.275 INFO d.n.common.webinterface.GUIWebServer.<init> - Scanning for taglibs is disabled as tomcat.util.scan.StandardJarScanFilter.jarsToSkip is unset.
2021-03-09 15:04:01.461 WARN o.a.hadoop.util.NativeCodeLoader.<clinit> - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2021-03-09 15:04:02.239 DEBUG dk.netarkivet.common.utils.Settings.debug - Value found in loaded data: dk.netarkivet.harvester.webinterface.DefinitionsSiteSection,dk.netarkivet.harvester.webinterface.HistorySiteSection,dk.netarkivet.harvester.webinterface.HarvestChannelSiteSection,dk.netarkivet.viewerproxy.webinterface.QASiteSection,dk.netarkivet.monitor.webinterface.StatusSiteSection
2021-03-09 15:04:02.239 DEBUG d.n.common.webinterface.SiteSection.getSections - Loading 5 site section(s).
2021-03-09 15:04:02.240 DEBUG d.n.common.webinterface.SiteSection.getSections - Loading site section dk.netarkivet.harvester.webinterface.DefinitionsSiteSection.
2021-03-09 15:04:02.245 DEBUG d.n.common.webinterface.SiteSection.getSections - Loading site section dk.netarkivet.harvester.webinterface.HistorySiteSection.
2021-03-09 15:04:02.249 DEBUG d.n.common.webinterface.SiteSection.getSections - Loading site section dk.netarkivet.harvester.webinterface.HarvestChannelSiteSection.
2021-03-09 15:04:02.251 DEBUG d.n.common.webinterface.SiteSection.getSections - Loading site section dk.netarkivet.viewerproxy.webinterface.QASiteSection.
2021-03-09 15:04:02.252 DEBUG d.n.common.webinterface.SiteSection.getSections - Loading site section dk.netarkivet.monitor.webinterface.StatusSiteSection.
15:03:58,514 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [/home/devel/Test3/conf/logback_HarvestJobManagerApplication.xml] at [file:/home/devel/Test3/conf/logback_HarvestJobManagerApplication.xml]
15:03:58,902 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - debug attribute not set
15:03:58,903 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
15:03:58,926 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [STDOUT]
15:03:58,985 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
15:03:59,209 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
15:03:59,238 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [FILE]
15:03:59,346 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@16208003 - No compression will be used
15:03:59,355 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@16208003 - Will use the pattern ./log/HarvestJobManagerApplication.%d{yyyy-MM-dd}.%i.log for the active file
15:03:59,373 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@698dc7 - The date pattern is 'yyyy-MM-dd' from file name pattern './log/HarvestJobManagerApplication.%d{yyyy-MM-dd}.%i.log'.
15:03:59,373 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@698dc7 - Roll-over at midnight.
15:03:59,396 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@698dc7 - Setting initial period to Tue Mar 09 15:03:59 CET 2021
15:03:59,396 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@698dc7 - SizeAndTimeBasedFNATP is deprecated. Use SizeAndTimeBasedRollingPolicy instead
15:03:59,396 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@698dc7 - For more information see http://logback.qos.ch/manual/appenders.html#SizeAndTimeBasedRollingPolicy
15:03:59,408 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
15:03:59,425 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - Active log file name: ./log/HarvestJobManagerApplication.log
15:03:59,425 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - File property is set to [./log/HarvestJobManagerApplication.log]
15:03:59,429 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [dk.netarkivet.monitor.logging.CachingSLF4JAppender]
15:04:00,803 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [MONITOR]
15:04:00,810 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to WARN
15:04:00,811 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[ROOT]
15:04:00,816 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [FILE] to Logger[ROOT]
15:04:00,816 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [MONITOR] to Logger[ROOT]
15:04:00,817 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [dk.netarkivet] to DEBUG
15:04:00,817 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
15:04:00,819 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@1d7741f - Registering current configuration as safe fallback point
2021-03-09 15:04:00.835 DEBUG dk.netarkivet.common.utils.Settings.invoke0 - Value found in loaded data: SB,KB
SLF4J: A number (1) of logging calls during the initialization phase have been intercepted and are
SLF4J: now being replayed. These are subject to the filtering rules of the underlying logging system.
SLF4J: See also http://www.slf4j.org/codes.html#replay
2021-03-09 15:04:00.906 INFO d.n.c.distribute.JMSConnectionSunMQ.<init> - Creating instance of dk.netarkivet.common.distribute.JMSConnectionSunMQ
2021-03-09 15:04:00.907 DEBUG d.n.common.distribute.JMSConnection.initConnection - Initializing a JMS connection dk.netarkivet.common.distribute.JMSConnectionSunMQ
2021-03-09 15:04:00.914 INFO d.n.c.distribute.JMSConnectionSunMQ.getConnectionFactory - Establishing SunMQ JMS Connection to 'kb-test-adm-001.kb.dk:7676'
2021-03-09 15:04:01.964 INFO d.n.h.datamodel.HarvestDBConnection.initDataSource - Connection pool initialized with the following values:
- minPoolSize=5
- maxPoolSize=20
- acquireIncrement=5
- maxStatements=0
- maxStatementsPerConnection=0
- idleConnTestPeriod=0
- idleConnTestQuery='null'
- idleConnTestOnCheckin=false
2021-03-09 15:04:02.315 INFO d.n.h.scheduler.JobDispatcher.<init> - Creating JobDispatcher
--------------------------------------------
STARTING MACHINE: devel@kb-test-har-003.kb.dk
Starting all applications on: kb-test-har-003.kb.dk
Starting linux application: HarvestControllerApplication_kblow001_h3
15:04:03,326 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [/home/devel/Test3/conf/logback_HarvestControllerApplication_kblow001_h3.xml] at [file:/home/devel/Test3/conf/logback_HarvestControllerApplication_kblow001_h3.xml]
15:04:03,521 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - debug attribute not set
15:04:03,524 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
15:04:03,532 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [STDOUT]
15:04:03,544 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
15:04:03,589 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
15:04:03,592 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [FILE]
15:04:03,613 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@501187768 - No compression will be used
15:04:03,614 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@501187768 - Will use the pattern ./log/HarvestControllerApplication_kblow001_h3.%d{yyyy-MM-dd}.%i.log for the active file
15:04:03,617 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@1139b2f3 - The date pattern is 'yyyy-MM-dd' from file name pattern './log/HarvestControllerApplication_kblow001_h3.%d{yyyy-MM-dd}.%i.log'.
15:04:03,617 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@1139b2f3 - Roll-over at midnight.
15:04:03,621 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@1139b2f3 - Setting initial period to Tue Mar 09 15:04:03 CET 2021
15:04:03,621 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@1139b2f3 - SizeAndTimeBasedFNATP is deprecated. Use SizeAndTimeBasedRollingPolicy instead
15:04:03,621 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@1139b2f3 - For more information see http://logback.qos.ch/manual/appenders.html#SizeAndTimeBasedRollingPolicy
15:04:03,623 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
15:04:03,625 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - Active log file name: ./log/HarvestControllerApplication_kblow001_h3.log
15:04:03,625 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - File property is set to [./log/HarvestControllerApplication_kblow001_h3.log]
15:04:03,626 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [dk.netarkivet.monitor.logging.CachingSLF4JAppender]
15:04:04,140 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [MONITOR]
15:04:04,151 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to WARN
15:04:04,151 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[ROOT]
15:04:04,152 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [FILE] to Logger[ROOT]
15:04:04,152 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [MONITOR] to Logger[ROOT]
15:04:04,152 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [dk.netarkivet] to DEBUG
15:04:04,152 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
15:04:04,153 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@7a69b07 - Registering current configuration as safe fallback point
2021-03-09 15:04:04.158 DEBUG dk.netarkivet.common.utils.Settings.invoke0 - Value found in loaded data: SB,KB
SLF4J: A number (1) of logging calls during the initialization phase have been intercepted and are
SLF4J: now being replayed. These are subject to the filtering rules of the underlying logging system.
SLF4J: See also http://www.slf4j.org/codes.html#replay
Starting dk.netarkivet.harvester.heritrix3.HarvestControllerServer
Version: 5.7-IIPCH3-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/cb5f59cb8465f2eea112989e4ed878851c79ff87)
2021-03-09 15:04:04.164 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Starting dk.netarkivet.harvester.heritrix3.HarvestControllerServer
Version: 5.7-IIPCH3-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/cb5f59cb8465f2eea112989e4ed878851c79ff87)
Java VM: 1.8.0_282
2021-03-09 15:04:04.165 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Java VM: 1.8.0_282
java.home: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.282.b08-1.el7_9.x86_64/jre
2021-03-09 15:04:04.166 INFO d.n.common.utils.ApplicationUtils.logAndPrint - java.home: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.282.b08-1.el7_9.x86_64/jre
Working dir: /home/devel/Test3
2021-03-09 15:04:04.166 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Working dir: /home/devel/Test3
2021-03-09 15:04:04.170 INFO d.n.common.utils.ApplicationUtils.startApp - Using settings files '/home/devel/Test3/conf/settings_HarvestControllerApplication_kblow001_h3.xml'
2021-03-09 15:04:04.297 INFO d.n.m.d.JMSMonitorRegistryClient.register - Registering this client for monitoring every 1 minutes, using hostname 'kb-test-har-003.kb.dk' and JMX/RMI ports 8150/8250
2021-03-09 15:04:04.297 INFO d.n.c.m.MBeanConnectorCreator.exposeJMXMBeanServer - Registered mbean server in registry on port 8150 communicating on port 8250 using password file 'conf/jmxremote.password'.
Service URL is service:jmx:rmi://kb-test-har-003.kb.dk:8250/jndi/rmi://kb-test-har-003.kb.dk:8150/jmxrmi
2021-03-09 15:04:04.309 INFO d.n.c.distribute.JMSConnectionSunMQ.<init> - Creating instance of dk.netarkivet.common.distribute.JMSConnectionSunMQ
2021-03-09 15:04:04.309 DEBUG d.n.common.distribute.JMSConnection.initConnection - Initializing a JMS connection dk.netarkivet.common.distribute.JMSConnectionSunMQ
2021-03-09 15:04:04.310 INFO d.n.c.distribute.JMSConnectionSunMQ.getConnectionFactory - Establishing SunMQ JMS Connection to 'kb-test-adm-001.kb.dk:7676'
dk.netarkivet.harvester.heritrix3.HarvestControllerServer Running
2021-03-09 15:04:04.321 INFO d.n.common.utils.ApplicationUtils.logAndPrint - dk.netarkivet.harvester.heritrix3.HarvestControllerServer Running
2021-03-09 15:04:04.333 INFO d.n.h.h.HarvestControllerServer.<init> - Starting HarvestControllerServer.
2021-03-09 15:04:04.334 INFO d.n.h.h.HarvestControllerServer.<init> - Bound to harvest channel 'LOWPRIORITY'
2021-03-09 15:04:04.334 INFO d.n.h.h.HarvestControllerServer.<init> - Serverdir: 'harvester_low'
2021-03-09 15:04:04.335 INFO d.n.h.h.HarvestControllerServer.<init> - Harvesting requires at least 400000000 bytes free.
2021-03-09 15:04:04.465 INFO d.n.c.d.bitrepository.HostNameUtils.getHostName - Local hostname (provided by getCanonicalHostName): kb-test-har-003.kb.dk
2021-03-09 15:04:06.022 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - BitmagArcRepositoryClient will timeout on each get request after 300000 milliseconds.
2021-03-09 15:04:06.022 DEBUG d.n.common.distribute.JMSConnection.setListener - Adding dk.netarkivet.archive.arcrepository.distribute.BitmagArcRepositoryClient@c5ee75e as listener to Test3_COMMON_THIS_REPOS_CLIENT_130_226_228_9_HCS_KBLOW001_H3
2021-03-09 15:04:06.033 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - BitmagArcRepositoryClient listens for replies on channel '[Queue 'Test3_COMMON_THIS_REPOS_CLIENT_130_226_228_9_HCS_KBLOW001_H3']'
2021-03-09 15:04:06.034 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - Getting bitmag config from settings.common.arcrepositoryClient.bitrepository.settingsDir=/home/devel/Test3/conf/bitmag_client_config
2021-03-09 15:04:06.035 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - Using 'Test3' as default collectionID
2021-03-09 15:04:06.041 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - Storing tempfiles in folder 'arcrepositoryTemp'
2021-03-09 15:04:06.119 DEBUG d.n.c.utils.BasicTwoWaySSLProvider.loadPrivateKey - X509CertificateHolder found
2021-03-09 15:04:06.121 DEBUG d.n.c.utils.BasicTwoWaySSLProvider.loadPrivateKey - PrivateKeyInfo found
2021-03-09 15:04:06.155 DEBUG d.n.h.h.HarvestControllerServer.<init> - Obtained JMS connection.
2021-03-09 15:04:06.156 INFO d.n.h.h.HarvestControllerServer.<init> - SEND_READY_DELAY used by HarvestControllerServer is 300
2021-03-09 15:04:06.157 INFO d.n.h.heritrix3.PostProcessing.processOldJobs - Looking for unprocessed crawldata in 'harvester_low'
2021-03-09 15:04:06.159 DEBUG d.n.common.distribute.JMSConnection.setListener - Adding dk.netarkivet.harvester.heritrix3.HarvestControllerServer@531c311e as listener to Test3_COMMON_HCHAN_VAL_RESP
2021-03-09 15:04:06.181 INFO d.n.h.h.HarvestControllerServer.<init> - Requested to check the validity of harvest channel 'LOWPRIORITY'
2021-03-09 15:04:06.290 INFO d.n.h.h.HarvestControllerServer.visit - Received message stating that channel 'LOWPRIORITY' is valid.
2021-03-09 15:04:06.292 INFO d.n.h.h.HarvestControllerServer.beginListeningIfSpaceAvailable - Starts to listen to new jobs on queue '[Queue 'Test3_COMMON_JOB_SNAPSHOT_LOWPRIORITY']'
2021-03-09 15:04:06.293 DEBUG d.n.common.distribute.JMSConnection.setListener - Adding dk.netarkivet.harvester.heritrix3.HarvestControllerServer@531c311e as listener to Test3_COMMON_JOB_SNAPSHOT_LOWPRIORITY
--------------------------------------------
STARTING MACHINE: devel@kb-test-har-005.kb.dk
Starting all applications on: kb-test-har-005.kb.dk
Starting linux application: HarvestControllerApplication_kbumbra_1
Starting linux application: HarvestControllerApplication_kbumbra_2
15:04:08,745 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [/home/devel/Test3/conf/logback_HarvestControllerApplication_kbumbra_1.xml] at [file:/home/devel/Test3/conf/logback_HarvestControllerApplication_kbumbra_1.xml]
15:04:09,110 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - debug attribute not set
15:04:09,113 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
15:04:09,126 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [STDOUT]
15:04:09,143 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
15:04:09,199 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
15:04:09,201 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [FILE]
15:04:09,228 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@501187768 - No compression will be used
15:04:09,229 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@501187768 - Will use the pattern ./log/HarvestControllerApplication_kbumbra_1.%d{yyyy-MM-dd}.%i.log for the active file
15:04:09,236 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@1139b2f3 - The date pattern is 'yyyy-MM-dd' from file name pattern './log/HarvestControllerApplication_kbumbra_1.%d{yyyy-MM-dd}.%i.log'.
15:04:09,236 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@1139b2f3 - Roll-over at midnight.
15:04:09,240 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@1139b2f3 - Setting initial period to Tue Mar 09 15:04:09 CET 2021
15:04:09,240 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@1139b2f3 - SizeAndTimeBasedFNATP is deprecated. Use SizeAndTimeBasedRollingPolicy instead
15:04:09,240 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@1139b2f3 - For more information see http://logback.qos.ch/manual/appenders.html#SizeAndTimeBasedRollingPolicy
15:04:09,242 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
15:04:09,244 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - Active log file name: ./log/HarvestControllerApplication_kbumbra_1.log
15:04:09,244 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - File property is set to [./log/HarvestControllerApplication_kbumbra_1.log]
15:04:09,245 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [dk.netarkivet.monitor.logging.CachingSLF4JAppender]
15:04:10,062 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [MONITOR]
15:04:10,064 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to WARN
15:04:10,065 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[ROOT]
15:04:10,065 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [FILE] to Logger[ROOT]
15:04:10,065 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [MONITOR] to Logger[ROOT]
15:04:10,066 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [dk.netarkivet] to DEBUG
15:04:10,066 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
15:04:10,066 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@7a69b07 - Registering current configuration as safe fallback point
2021-03-09 15:04:10.071 DEBUG dk.netarkivet.common.utils.Settings.invoke0 - Value found in loaded data: SB,KB
SLF4J: A number (1) of logging calls during the initialization phase have been intercepted and are
SLF4J: now being replayed. These are subject to the filtering rules of the underlying logging system.
SLF4J: See also http://www.slf4j.org/codes.html#replay
Starting dk.netarkivet.harvester.heritrix3.HarvestControllerServer
Version: 5.7-IIPCH3-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/cb5f59cb8465f2eea112989e4ed878851c79ff87)
2021-03-09 15:04:10.077 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Starting dk.netarkivet.harvester.heritrix3.HarvestControllerServer
Version: 5.7-IIPCH3-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/cb5f59cb8465f2eea112989e4ed878851c79ff87)
Java VM: 1.8.0_282
2021-03-09 15:04:10.078 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Java VM: 1.8.0_282
java.home: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.282.b08-1.el7_9.x86_64/jre
2021-03-09 15:04:10.078 INFO d.n.common.utils.ApplicationUtils.logAndPrint - java.home: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.282.b08-1.el7_9.x86_64/jre
Working dir: /home/devel/Test3
2021-03-09 15:04:10.078 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Working dir: /home/devel/Test3
2021-03-09 15:04:10.078 INFO d.n.common.utils.ApplicationUtils.startApp - Using settings files '/home/devel/Test3/conf/settings_HarvestControllerApplication_kbumbra_1.xml'
2021-03-09 15:04:10.274 INFO d.n.m.d.JMSMonitorRegistryClient.register - Registering this client for monitoring every 1 minutes, using hostname 'kb-test-har-005.kb.dk' and JMX/RMI ports 8150/8250
2021-03-09 15:04:10.274 INFO d.n.c.m.MBeanConnectorCreator.exposeJMXMBeanServer - Registered mbean server in registry on port 8150 communicating on port 8250 using password file 'conf/jmxremote.password'.
Service URL is service:jmx:rmi://kb-test-har-005.kb.dk:8250/jndi/rmi://kb-test-har-005.kb.dk:8150/jmxrmi
dk.netarkivet.harvester.heritrix3.HarvestControllerServer Running
2021-03-09 15:04:10.279 INFO d.n.common.utils.ApplicationUtils.logAndPrint - dk.netarkivet.harvester.heritrix3.HarvestControllerServer Running
2021-03-09 15:04:10.295 INFO d.n.c.distribute.JMSConnectionSunMQ.<init> - Creating instance of dk.netarkivet.common.distribute.JMSConnectionSunMQ
2021-03-09 15:04:10.295 DEBUG d.n.common.distribute.JMSConnection.initConnection - Initializing a JMS connection dk.netarkivet.common.distribute.JMSConnectionSunMQ
2021-03-09 15:04:10.295 INFO d.n.c.distribute.JMSConnectionSunMQ.getConnectionFactory - Establishing SunMQ JMS Connection to 'kb-test-adm-001.kb.dk:7676'
2021-03-09 15:04:10.301 INFO d.n.h.h.HarvestControllerServer.<init> - Starting HarvestControllerServer.
2021-03-09 15:04:10.301 INFO d.n.h.h.HarvestControllerServer.<init> - Bound to harvest channel 'UMBRA'
2021-03-09 15:04:10.301 INFO d.n.h.h.HarvestControllerServer.<init> - Serverdir: 'harvester_umbra_1'
2021-03-09 15:04:10.302 INFO d.n.h.h.HarvestControllerServer.<init> - Harvesting requires at least 400000000 bytes free.
2021-03-09 15:04:10.479 INFO d.n.c.d.bitrepository.HostNameUtils.getHostName - Local hostname (provided by getCanonicalHostName): kb-test-har-005.kb.dk
2021-03-09 15:04:12.595 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - BitmagArcRepositoryClient will timeout on each get request after 300000 milliseconds.
2021-03-09 15:04:12.596 DEBUG d.n.common.distribute.JMSConnection.setListener - Adding dk.netarkivet.archive.arcrepository.distribute.BitmagArcRepositoryClient@c5ee75e as listener to Test3_COMMON_THIS_REPOS_CLIENT_130_226_228_16_HCS_KBUMBRA_1
2021-03-09 15:04:12.675 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - BitmagArcRepositoryClient listens for replies on channel '[Queue 'Test3_COMMON_THIS_REPOS_CLIENT_130_226_228_16_HCS_KBUMBRA_1']'
2021-03-09 15:04:12.676 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - Getting bitmag config from settings.common.arcrepositoryClient.bitrepository.settingsDir=/home/devel/Test3/conf/bitmag_client_config
2021-03-09 15:04:12.677 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - Using 'Test3' as default collectionID
2021-03-09 15:04:12.681 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - Storing tempfiles in folder 'arcrepositoryTemp'
2021-03-09 15:04:12.779 DEBUG d.n.c.utils.BasicTwoWaySSLProvider.loadPrivateKey - X509CertificateHolder found
2021-03-09 15:04:12.780 DEBUG d.n.c.utils.BasicTwoWaySSLProvider.loadPrivateKey - PrivateKeyInfo found
2021-03-09 15:04:12.843 DEBUG d.n.h.h.HarvestControllerServer.<init> - Obtained JMS connection.
2021-03-09 15:04:12.845 INFO d.n.h.h.HarvestControllerServer.<init> - SEND_READY_DELAY used by HarvestControllerServer is 300
2021-03-09 15:04:12.846 INFO d.n.h.heritrix3.PostProcessing.processOldJobs - Looking for unprocessed crawldata in 'harvester_umbra_1'
2021-03-09 15:04:12.846 DEBUG d.n.common.distribute.JMSConnection.setListener - Adding dk.netarkivet.harvester.heritrix3.HarvestControllerServer@531c311e as listener to Test3_COMMON_HCHAN_VAL_RESP
2021-03-09 15:04:12.881 INFO d.n.h.h.HarvestControllerServer.<init> - Requested to check the validity of harvest channel 'UMBRA'
2021-03-09 15:04:12.900 INFO d.n.h.h.HarvestControllerServer.visit - Received message stating that channel 'UMBRA' is valid.
2021-03-09 15:04:12.901 INFO d.n.h.h.HarvestControllerServer.beginListeningIfSpaceAvailable - Starts to listen to new jobs on queue '[Queue 'Test3_COMMON_JOB_PARTIAL_UMBRA']'
2021-03-09 15:04:12.901 DEBUG d.n.common.distribute.JMSConnection.setListener - Adding dk.netarkivet.harvester.heritrix3.HarvestControllerServer@531c311e as listener to Test3_COMMON_JOB_PARTIAL_UMBRA
15:04:08,745 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [/home/devel/Test3/conf/logback_HarvestControllerApplication_kbumbra_2.xml] at [file:/home/devel/Test3/conf/logback_HarvestControllerApplication_kbumbra_2.xml]
15:04:09,191 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - debug attribute not set
15:04:09,194 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
15:04:09,201 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [STDOUT]
15:04:09,224 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
15:04:09,282 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
15:04:09,285 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [FILE]
15:04:09,322 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@501187768 - No compression will be used
15:04:09,323 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@501187768 - Will use the pattern ./log/HarvestControllerApplication_kbumbra_2.%d{yyyy-MM-dd}.%i.log for the active file
15:04:09,326 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@1139b2f3 - The date pattern is 'yyyy-MM-dd' from file name pattern './log/HarvestControllerApplication_kbumbra_2.%d{yyyy-MM-dd}.%i.log'.
15:04:09,326 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@1139b2f3 - Roll-over at midnight.
15:04:09,329 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@1139b2f3 - Setting initial period to Tue Mar 09 15:04:09 CET 2021
15:04:09,329 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@1139b2f3 - SizeAndTimeBasedFNATP is deprecated. Use SizeAndTimeBasedRollingPolicy instead
15:04:09,329 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@1139b2f3 - For more information see http://logback.qos.ch/manual/appenders.html#SizeAndTimeBasedRollingPolicy
15:04:09,335 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
15:04:09,337 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - Active log file name: ./log/HarvestControllerApplication_kbumbra_2.log
15:04:09,337 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - File property is set to [./log/HarvestControllerApplication_kbumbra_2.log]
15:04:09,338 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [dk.netarkivet.monitor.logging.CachingSLF4JAppender]
15:04:10,021 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [MONITOR]
15:04:10,022 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to WARN
15:04:10,023 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[ROOT]
15:04:10,023 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [FILE] to Logger[ROOT]
15:04:10,023 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [MONITOR] to Logger[ROOT]
15:04:10,024 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [dk.netarkivet] to DEBUG
15:04:10,024 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
15:04:10,024 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@7a69b07 - Registering current configuration as safe fallback point
2021-03-09 15:04:10.058 DEBUG dk.netarkivet.common.utils.Settings.invoke0 - Value found in loaded data: SB,KB
SLF4J: A number (1) of logging calls during the initialization phase have been intercepted and are
SLF4J: now being replayed. These are subject to the filtering rules of the underlying logging system.
SLF4J: See also http://www.slf4j.org/codes.html#replay
Starting dk.netarkivet.harvester.heritrix3.HarvestControllerServer
Version: 5.7-IIPCH3-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/cb5f59cb8465f2eea112989e4ed878851c79ff87)
2021-03-09 15:04:10.064 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Starting dk.netarkivet.harvester.heritrix3.HarvestControllerServer
Version: 5.7-IIPCH3-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/cb5f59cb8465f2eea112989e4ed878851c79ff87)
Java VM: 1.8.0_282
2021-03-09 15:04:10.064 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Java VM: 1.8.0_282
java.home: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.282.b08-1.el7_9.x86_64/jre
2021-03-09 15:04:10.083 INFO d.n.common.utils.ApplicationUtils.logAndPrint - java.home: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.282.b08-1.el7_9.x86_64/jre
Working dir: /home/devel/Test3
2021-03-09 15:04:10.084 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Working dir: /home/devel/Test3
2021-03-09 15:04:10.087 INFO d.n.common.utils.ApplicationUtils.startApp - Using settings files '/home/devel/Test3/conf/settings_HarvestControllerApplication_kbumbra_2.xml'
2021-03-09 15:04:10.284 INFO d.n.m.d.JMSMonitorRegistryClient.register - Registering this client for monitoring every 1 minutes, using hostname 'kb-test-har-005.kb.dk' and JMX/RMI ports 8155/8255
2021-03-09 15:04:10.285 INFO d.n.c.m.MBeanConnectorCreator.exposeJMXMBeanServer - Registered mbean server in registry on port 8155 communicating on port 8255 using password file 'conf/jmxremote.password'.
Service URL is service:jmx:rmi://kb-test-har-005.kb.dk:8255/jndi/rmi://kb-test-har-005.kb.dk:8155/jmxrmi
dk.netarkivet.harvester.heritrix3.HarvestControllerServer Running
2021-03-09 15:04:10.306 INFO d.n.common.utils.ApplicationUtils.logAndPrint - dk.netarkivet.harvester.heritrix3.HarvestControllerServer Running
2021-03-09 15:04:10.314 INFO d.n.c.distribute.JMSConnectionSunMQ.<init> - Creating instance of dk.netarkivet.common.distribute.JMSConnectionSunMQ
2021-03-09 15:04:10.315 DEBUG d.n.common.distribute.JMSConnection.initConnection - Initializing a JMS connection dk.netarkivet.common.distribute.JMSConnectionSunMQ
2021-03-09 15:04:10.315 INFO d.n.c.distribute.JMSConnectionSunMQ.getConnectionFactory - Establishing SunMQ JMS Connection to 'kb-test-adm-001.kb.dk:7676'
2021-03-09 15:04:10.367 INFO d.n.h.h.HarvestControllerServer.<init> - Starting HarvestControllerServer.
2021-03-09 15:04:10.367 INFO d.n.h.h.HarvestControllerServer.<init> - Bound to harvest channel 'UMBRA'
2021-03-09 15:04:10.367 INFO d.n.h.h.HarvestControllerServer.<init> - Serverdir: 'harvester_umbra_2'
2021-03-09 15:04:10.368 INFO d.n.h.h.HarvestControllerServer.<init> - Harvesting requires at least 400000000 bytes free.
2021-03-09 15:04:10.478 INFO d.n.c.d.bitrepository.HostNameUtils.getHostName - Local hostname (provided by getCanonicalHostName): kb-test-har-005.kb.dk
2021-03-09 15:04:12.520 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - BitmagArcRepositoryClient will timeout on each get request after 300000 milliseconds.
2021-03-09 15:04:12.521 DEBUG d.n.common.distribute.JMSConnection.setListener - Adding dk.netarkivet.archive.arcrepository.distribute.BitmagArcRepositoryClient@c5ee75e as listener to Test3_COMMON_THIS_REPOS_CLIENT_130_226_228_16_HCS_KBUMBRA_2
2021-03-09 15:04:12.586 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - BitmagArcRepositoryClient listens for replies on channel '[Queue 'Test3_COMMON_THIS_REPOS_CLIENT_130_226_228_16_HCS_KBUMBRA_2']'
2021-03-09 15:04:12.586 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - Getting bitmag config from settings.common.arcrepositoryClient.bitrepository.settingsDir=/home/devel/Test3/conf/bitmag_client_config
2021-03-09 15:04:12.587 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - Using 'Test3' as default collectionID
2021-03-09 15:04:12.591 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - Storing tempfiles in folder 'arcrepositoryTemp'
2021-03-09 15:04:12.673 DEBUG d.n.c.utils.BasicTwoWaySSLProvider.loadPrivateKey - X509CertificateHolder found
2021-03-09 15:04:12.674 DEBUG d.n.c.utils.BasicTwoWaySSLProvider.loadPrivateKey - PrivateKeyInfo found
2021-03-09 15:04:12.764 DEBUG d.n.h.h.HarvestControllerServer.<init> - Obtained JMS connection.
2021-03-09 15:04:12.765 INFO d.n.h.h.HarvestControllerServer.<init> - SEND_READY_DELAY used by HarvestControllerServer is 300
2021-03-09 15:04:12.765 INFO d.n.h.heritrix3.PostProcessing.processOldJobs - Looking for unprocessed crawldata in 'harvester_umbra_2'
2021-03-09 15:04:12.766 DEBUG d.n.common.distribute.JMSConnection.setListener - Adding dk.netarkivet.harvester.heritrix3.HarvestControllerServer@531c311e as listener to Test3_COMMON_HCHAN_VAL_RESP
2021-03-09 15:04:12.849 INFO d.n.h.h.HarvestControllerServer.<init> - Requested to check the validity of harvest channel 'UMBRA'
2021-03-09 15:04:12.887 INFO d.n.h.h.HarvestControllerServer.visit - Received message stating that channel 'UMBRA' is valid.
2021-03-09 15:04:12.888 INFO d.n.h.h.HarvestControllerServer.beginListeningIfSpaceAvailable - Starts to listen to new jobs on queue '[Queue 'Test3_COMMON_JOB_PARTIAL_UMBRA']'
2021-03-09 15:04:12.888 DEBUG d.n.common.distribute.JMSConnection.setListener - Adding dk.netarkivet.harvester.heritrix3.HarvestControllerServer@531c311e as listener to Test3_COMMON_JOB_PARTIAL_UMBRA
--------------------------------------------
STARTING MACHINE: devel@kb-test-har-004.kb.dk
Starting all applications on: kb-test-har-004.kb.dk
Starting linux application: HarvestControllerApplication_kblow002_h3
Starting linux application: HarvestControllerApplication_kbhigh_h3
15:04:14,596 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [/home/devel/Test3/conf/logback_HarvestControllerApplication_kbhigh_h3.xml] at [file:/home/devel/Test3/conf/logback_HarvestControllerApplication_kbhigh_h3.xml]
15:04:15,058 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - debug attribute not set
15:04:15,069 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
15:04:15,085 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [STDOUT]
15:04:15,114 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
15:04:15,189 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
15:04:15,193 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [FILE]
15:04:15,232 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@370440646 - No compression will be used
15:04:15,233 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@370440646 - Will use the pattern ./log/HarvestControllerApplication_kbhigh_h3.%d{yyyy-MM-dd}.%i.log for the active file
15:04:15,236 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@4313f5bc - The date pattern is 'yyyy-MM-dd' from file name pattern './log/HarvestControllerApplication_kbhigh_h3.%d{yyyy-MM-dd}.%i.log'.
15:04:15,236 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@4313f5bc - Roll-over at midnight.
15:04:15,242 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@4313f5bc - Setting initial period to Tue Mar 09 15:04:15 CET 2021
15:04:15,242 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@4313f5bc - SizeAndTimeBasedFNATP is deprecated. Use SizeAndTimeBasedRollingPolicy instead
15:04:15,242 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@4313f5bc - For more information see http://logback.qos.ch/manual/appenders.html#SizeAndTimeBasedRollingPolicy
15:04:15,248 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
15:04:15,253 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - Active log file name: ./log/HarvestControllerApplication_kbhigh_h3.log
15:04:15,253 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - File property is set to [./log/HarvestControllerApplication_kbhigh_h3.log]
15:04:15,254 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [dk.netarkivet.monitor.logging.CachingSLF4JAppender]
15:04:15,892 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [MONITOR]
15:04:15,901 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to WARN
15:04:15,901 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[ROOT]
15:04:15,902 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [FILE] to Logger[ROOT]
15:04:15,902 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [MONITOR] to Logger[ROOT]
15:04:15,902 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [dk.netarkivet] to DEBUG
15:04:15,902 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
15:04:15,903 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@7f010382 - Registering current configuration as safe fallback point
2021-03-09 15:04:15.913 DEBUG dk.netarkivet.common.utils.Settings.invoke0 - Value found in loaded data: SB,KB
SLF4J: A number (1) of logging calls during the initialization phase have been intercepted and are
SLF4J: now being replayed. These are subject to the filtering rules of the underlying logging system.
SLF4J: See also http://www.slf4j.org/codes.html#replay
Starting dk.netarkivet.harvester.heritrix3.HarvestControllerServer
Version: 5.7-IIPCH3-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/cb5f59cb8465f2eea112989e4ed878851c79ff87)
2021-03-09 15:04:15.924 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Starting dk.netarkivet.harvester.heritrix3.HarvestControllerServer
Version: 5.7-IIPCH3-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/cb5f59cb8465f2eea112989e4ed878851c79ff87)
Java VM: 1.8.0_25
2021-03-09 15:04:15.925 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Java VM: 1.8.0_25
java.home: /usr/local/java/jdk1.8.0_25_x64/jre
2021-03-09 15:04:15.925 INFO d.n.common.utils.ApplicationUtils.logAndPrint - java.home: /usr/local/java/jdk1.8.0_25_x64/jre
Working dir: /home/devel/Test3
2021-03-09 15:04:15.926 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Working dir: /home/devel/Test3
2021-03-09 15:04:15.926 INFO d.n.common.utils.ApplicationUtils.startApp - Using settings files '/home/devel/Test3/conf/settings_HarvestControllerApplication_kbhigh_h3.xml'
2021-03-09 15:04:16.051 INFO d.n.m.d.JMSMonitorRegistryClient.register - Registering this client for monitoring every 1 minutes, using hostname 'kb-test-har-004.kb.dk' and JMX/RMI ports 8152/8252
2021-03-09 15:04:16.052 INFO d.n.c.m.MBeanConnectorCreator.exposeJMXMBeanServer - Registered mbean server in registry on port 8152 communicating on port 8252 using password file 'conf/jmxremote.password'.
Service URL is service:jmx:rmi://kb-test-har-004.kb.dk:8252/jndi/rmi://kb-test-har-004.kb.dk:8152/jmxrmi
dk.netarkivet.harvester.heritrix3.HarvestControllerServer Running
2021-03-09 15:04:16.069 INFO d.n.common.utils.ApplicationUtils.logAndPrint - dk.netarkivet.harvester.heritrix3.HarvestControllerServer Running
2021-03-09 15:04:16.073 INFO d.n.c.distribute.JMSConnectionSunMQ.<init> - Creating instance of dk.netarkivet.common.distribute.JMSConnectionSunMQ
2021-03-09 15:04:16.079 DEBUG d.n.common.distribute.JMSConnection.initConnection - Initializing a JMS connection dk.netarkivet.common.distribute.JMSConnectionSunMQ
2021-03-09 15:04:16.079 INFO d.n.c.distribute.JMSConnectionSunMQ.getConnectionFactory - Establishing SunMQ JMS Connection to 'kb-test-adm-001.kb.dk:7676'
2021-03-09 15:04:16.091 INFO d.n.h.h.HarvestControllerServer.<init> - Starting HarvestControllerServer.
2021-03-09 15:04:16.097 INFO d.n.h.h.HarvestControllerServer.<init> - Bound to harvest channel 'HIGHPRIORITY'
2021-03-09 15:04:16.097 INFO d.n.h.h.HarvestControllerServer.<init> - Serverdir: 'harvester_high'
2021-03-09 15:04:16.098 INFO d.n.h.h.HarvestControllerServer.<init> - Harvesting requires at least 400000000 bytes free.
2021-03-09 15:04:16.249 INFO d.n.c.d.bitrepository.HostNameUtils.getHostName - Local hostname (provided by getCanonicalHostName): kb-test-har-004.kb.dk
15:04:14,636 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [/home/devel/Test3/conf/logback_HarvestControllerApplication_kblow002_h3.xml] at [file:/home/devel/Test3/conf/logback_HarvestControllerApplication_kblow002_h3.xml]
15:04:15,066 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - debug attribute not set
15:04:15,085 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
15:04:15,102 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [STDOUT]
15:04:15,123 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
15:04:15,189 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
15:04:15,193 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [FILE]
15:04:15,230 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@370440646 - No compression will be used
15:04:15,231 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@370440646 - Will use the pattern ./log/HarvestControllerApplication_kblow002_h3.%d{yyyy-MM-dd}.%i.log for the active file
15:04:15,234 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@4313f5bc - The date pattern is 'yyyy-MM-dd' from file name pattern './log/HarvestControllerApplication_kblow002_h3.%d{yyyy-MM-dd}.%i.log'.
15:04:15,234 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@4313f5bc - Roll-over at midnight.
15:04:15,239 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@4313f5bc - Setting initial period to Tue Mar 09 15:04:15 CET 2021
15:04:15,239 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@4313f5bc - SizeAndTimeBasedFNATP is deprecated. Use SizeAndTimeBasedRollingPolicy instead
15:04:15,239 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@4313f5bc - For more information see http://logback.qos.ch/manual/appenders.html#SizeAndTimeBasedRollingPolicy
15:04:15,241 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
15:04:15,244 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - Active log file name: ./log/HarvestControllerApplication_kblow002_h3.log
15:04:15,244 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - File property is set to [./log/HarvestControllerApplication_kblow002_h3.log]
15:04:15,245 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [dk.netarkivet.monitor.logging.CachingSLF4JAppender]
15:04:15,824 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [MONITOR]
15:04:15,825 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to WARN
15:04:15,825 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[ROOT]
15:04:15,836 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [FILE] to Logger[ROOT]
15:04:15,836 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [MONITOR] to Logger[ROOT]
15:04:15,836 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [dk.netarkivet] to DEBUG
15:04:15,836 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
15:04:15,837 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@7f010382 - Registering current configuration as safe fallback point
2021-03-09 15:04:15.848 DEBUG dk.netarkivet.common.utils.Settings.invoke0 - Value found in loaded data: SB,KB
SLF4J: A number (1) of logging calls during the initialization phase have been intercepted and are
SLF4J: now being replayed. These are subject to the filtering rules of the underlying logging system.
SLF4J: See also http://www.slf4j.org/codes.html#replay
Starting dk.netarkivet.harvester.heritrix3.HarvestControllerServer
Version: 5.7-IIPCH3-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/cb5f59cb8465f2eea112989e4ed878851c79ff87)
2021-03-09 15:04:15.858 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Starting dk.netarkivet.harvester.heritrix3.HarvestControllerServer
Version: 5.7-IIPCH3-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/cb5f59cb8465f2eea112989e4ed878851c79ff87)
Java VM: 1.8.0_25
2021-03-09 15:04:15.858 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Java VM: 1.8.0_25
java.home: /usr/local/java/jdk1.8.0_25_x64/jre
2021-03-09 15:04:15.859 INFO d.n.common.utils.ApplicationUtils.logAndPrint - java.home: /usr/local/java/jdk1.8.0_25_x64/jre
Working dir: /home/devel/Test3
2021-03-09 15:04:15.859 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Working dir: /home/devel/Test3
2021-03-09 15:04:15.872 INFO d.n.common.utils.ApplicationUtils.startApp - Using settings files '/home/devel/Test3/conf/settings_HarvestControllerApplication_kblow002_h3.xml'
2021-03-09 15:04:16.026 INFO d.n.m.d.JMSMonitorRegistryClient.register - Registering this client for monitoring every 1 minutes, using hostname 'kb-test-har-004.kb.dk' and JMX/RMI ports 8150/8250
2021-03-09 15:04:16.027 INFO d.n.c.m.MBeanConnectorCreator.exposeJMXMBeanServer - Registered mbean server in registry on port 8150 communicating on port 8250 using password file 'conf/jmxremote.password'.
Service URL is service:jmx:rmi://kb-test-har-004.kb.dk:8250/jndi/rmi://kb-test-har-004.kb.dk:8150/jmxrmi
dk.netarkivet.harvester.heritrix3.HarvestControllerServer Running
2021-03-09 15:04:16.043 INFO d.n.common.utils.ApplicationUtils.logAndPrint - dk.netarkivet.harvester.heritrix3.HarvestControllerServer Running
2021-03-09 15:04:16.054 INFO d.n.c.distribute.JMSConnectionSunMQ.<init> - Creating instance of dk.netarkivet.common.distribute.JMSConnectionSunMQ
2021-03-09 15:04:16.054 DEBUG d.n.common.distribute.JMSConnection.initConnection - Initializing a JMS connection dk.netarkivet.common.distribute.JMSConnectionSunMQ
2021-03-09 15:04:16.054 INFO d.n.c.distribute.JMSConnectionSunMQ.getConnectionFactory - Establishing SunMQ JMS Connection to 'kb-test-adm-001.kb.dk:7676'
2021-03-09 15:04:16.068 INFO d.n.h.h.HarvestControllerServer.<init> - Starting HarvestControllerServer.
2021-03-09 15:04:16.068 INFO d.n.h.h.HarvestControllerServer.<init> - Bound to harvest channel 'LOWPRIORITY'
2021-03-09 15:04:16.069 INFO d.n.h.h.HarvestControllerServer.<init> - Serverdir: 'harvester_low'
2021-03-09 15:04:16.069 INFO d.n.h.h.HarvestControllerServer.<init> - Harvesting requires at least 400000000 bytes free.
2021-03-09 15:04:16.265 INFO d.n.c.d.bitrepository.HostNameUtils.getHostName - Local hostname (provided by getCanonicalHostName): kb-test-har-004.kb.dk
--------------------------------------------
STARTING MACHINE: devel@kb-test-acs-001.kb.dk
Starting all applications on: kb-test-acs-001.kb.dk
Starting linux application: IndexServerApplication
Starting linux application: ViewerProxyApplication
15:04:19,868 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [/home/devel/Test3/conf/logback_IndexServerApplication.xml] at [file:/home/devel/Test3/conf/logback_IndexServerApplication.xml]
15:04:20,109 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - debug attribute not set
15:04:20,110 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
15:04:20,117 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [STDOUT]
15:04:20,134 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
15:04:20,214 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
15:04:20,225 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [FILE]
15:04:20,260 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@831236296 - No compression will be used
15:04:20,262 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@831236296 - Will use the pattern ./log/IndexServerApplication.%d{yyyy-MM-dd}.%i.log for the active file
15:04:20,264 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@6dbb137d - The date pattern is 'yyyy-MM-dd' from file name pattern './log/IndexServerApplication.%d{yyyy-MM-dd}.%i.log'.
15:04:20,264 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@6dbb137d - Roll-over at midnight.
15:04:20,269 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@6dbb137d - Setting initial period to Tue Mar 09 15:04:20 CET 2021
15:04:20,269 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@6dbb137d - SizeAndTimeBasedFNATP is deprecated. Use SizeAndTimeBasedRollingPolicy instead
15:04:20,269 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@6dbb137d - For more information see http://logback.qos.ch/manual/appenders.html#SizeAndTimeBasedRollingPolicy
15:04:20,271 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
15:04:20,273 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - Active log file name: ./log/IndexServerApplication.log
15:04:20,273 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - File property is set to [./log/IndexServerApplication.log]
15:04:20,274 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [dk.netarkivet.monitor.logging.CachingSLF4JAppender]
15:04:20,774 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [MONITOR]
15:04:20,776 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to WARN
15:04:20,776 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[ROOT]
15:04:20,777 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [FILE] to Logger[ROOT]
15:04:20,777 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [MONITOR] to Logger[ROOT]
15:04:20,777 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [dk.netarkivet] to DEBUG
15:04:20,777 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
15:04:20,778 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@3c9d0b9d - Registering current configuration as safe fallback point
2021-03-09 15:04:20.786 DEBUG dk.netarkivet.common.utils.Settings.invoke0 - Value found in loaded data: SB,KB
SLF4J: A number (1) of logging calls during the initialization phase have been intercepted and are
SLF4J: now being replayed. These are subject to the filtering rules of the underlying logging system.
SLF4J: See also http://www.slf4j.org/codes.html#replay
Starting dk.netarkivet.harvester.indexserver.IndexServer
Version: 5.7-IIPCH3-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/cb5f59cb8465f2eea112989e4ed878851c79ff87)
2021-03-09 15:04:20.792 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Starting dk.netarkivet.harvester.indexserver.IndexServer
Version: 5.7-IIPCH3-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/cb5f59cb8465f2eea112989e4ed878851c79ff87)
Java VM: 1.8.0_282
2021-03-09 15:04:20.796 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Java VM: 1.8.0_282
java.home: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.282.b08-1.el7_9.x86_64/jre
2021-03-09 15:04:20.796 INFO d.n.common.utils.ApplicationUtils.logAndPrint - java.home: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.282.b08-1.el7_9.x86_64/jre
Working dir: /home/devel/Test3
2021-03-09 15:04:20.797 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Working dir: /home/devel/Test3
2021-03-09 15:04:20.797 INFO d.n.common.utils.ApplicationUtils.startApp - Using settings files '/home/devel/Test3/conf/settings_IndexServerApplication.xml'
2021-03-09 15:04:20.965 INFO d.n.m.d.JMSMonitorRegistryClient.register - Registering this client for monitoring every 1 minutes, using hostname 'kb-test-acs-001.kb.dk' and JMX/RMI ports 8351/8451
2021-03-09 15:04:20.965 INFO d.n.c.m.MBeanConnectorCreator.exposeJMXMBeanServer - Registered mbean server in registry on port 8351 communicating on port 8451 using password file 'conf/jmxremote.password'.
Service URL is service:jmx:rmi://kb-test-acs-001.kb.dk:8451/jndi/rmi://kb-test-acs-001.kb.dk:8351/jmxrmi
dk.netarkivet.harvester.indexserver.IndexServer Running
2021-03-09 15:04:20.969 INFO d.n.common.utils.ApplicationUtils.logAndPrint - dk.netarkivet.harvester.indexserver.IndexServer Running
2021-03-09 15:04:20.979 INFO d.n.c.distribute.JMSConnectionSunMQ.<init> - Creating instance of dk.netarkivet.common.distribute.JMSConnectionSunMQ
2021-03-09 15:04:20.979 DEBUG d.n.common.distribute.JMSConnection.initConnection - Initializing a JMS connection dk.netarkivet.common.distribute.JMSConnectionSunMQ
2021-03-09 15:04:20.980 INFO d.n.c.distribute.JMSConnectionSunMQ.getConnectionFactory - Establishing SunMQ JMS Connection to 'kb-test-adm-001.kb.dk:7676'
2021-03-09 15:04:20.983 INFO d.n.h.indexserver.FileBasedCache.<init> - Metadata cache for 'cdxdata' uses directory '/home/devel/Test3/cache/cdxdata'
2021-03-09 15:04:20.995 INFO d.n.c.d.bitrepository.HostNameUtils.getHostName - Local hostname (provided by getCanonicalHostName): kb-test-acs-001.kb.dk
2021-03-09 15:04:22.822 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - BitmagArcRepositoryClient will timeout on each get request after 300000 milliseconds.
2021-03-09 15:04:22.823 DEBUG d.n.common.distribute.JMSConnection.setListener - Adding dk.netarkivet.archive.arcrepository.distribute.BitmagArcRepositoryClient@31dadd46 as listener to Test3_COMMON_THIS_REPOS_CLIENT_130_226_228_5_IS
2021-03-09 15:04:22.837 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - BitmagArcRepositoryClient listens for replies on channel '[Queue 'Test3_COMMON_THIS_REPOS_CLIENT_130_226_228_5_IS']'
2021-03-09 15:04:22.837 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - Getting bitmag config from settings.common.arcrepositoryClient.bitrepository.settingsDir=/home/devel/Test3/conf/bitmag_client_config
2021-03-09 15:04:22.838 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - Using 'Test3' as default collectionID
2021-03-09 15:04:22.849 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - Storing tempfiles in folder 'arcrepositoryTemp'
2021-03-09 15:04:22.935 DEBUG d.n.c.utils.BasicTwoWaySSLProvider.loadPrivateKey - X509CertificateHolder found
2021-03-09 15:04:22.936 DEBUG d.n.c.utils.BasicTwoWaySSLProvider.loadPrivateKey - PrivateKeyInfo found
2021-03-09 15:04:22.976 INFO d.n.h.indexserver.RawMetadataCache.<init> - Metadata cache for 'cdxdata' is fetching metadata with urls matching 'metadata://[^/]*/crawl/index/cdx.*' and mimetype matching 'application/x-cdx'. Migration of duplicate records is disabled
2021-03-09 15:04:22.976 INFO d.n.h.indexserver.FileBasedCache.<init> - Metadata cache for 'cdxindex' uses directory '/home/devel/Test3/cache/cdxindex'
2021-03-09 15:04:22.979 INFO d.n.h.indexserver.FileBasedCache.<init> - Metadata cache for 'crawllog' uses directory '/home/devel/Test3/cache/crawllog'
2021-03-09 15:04:22.980 INFO d.n.h.indexserver.RawMetadataCache.<init> - Metadata cache for 'crawllog' is fetching metadata with urls matching 'metadata://[^/]*/crawl/logs/crawl\.log.*' and mimetype matching 'text/plain'. Migration of duplicate records is disabled
2021-03-09 15:04:22.980 INFO d.n.h.indexserver.FileBasedCache.<init> - Metadata cache for 'dedupcrawllogindex' uses directory '/home/devel/Test3/cache/dedupcrawllogindex'
2021-03-09 15:04:22.981 INFO d.n.h.indexserver.FileBasedCache.<init> - Metadata cache for 'cdxdata' uses directory '/home/devel/Test3/cache/cdxdata'
2021-03-09 15:04:22.981 INFO d.n.h.indexserver.RawMetadataCache.<init> - Metadata cache for 'cdxdata' is fetching metadata with urls matching 'metadata://[^/]*/crawl/index/cdx.*' and mimetype matching 'application/x-cdx'. Migration of duplicate records is disabled
2021-03-09 15:04:22.982 INFO d.n.h.indexserver.FileBasedCache.<init> - Metadata cache for 'crawllog' uses directory '/home/devel/Test3/cache/crawllog'
2021-03-09 15:04:22.982 INFO d.n.h.indexserver.RawMetadataCache.<init> - Metadata cache for 'crawllog' is fetching metadata with urls matching 'metadata://[^/]*/crawl/logs/crawl\.log.*' and mimetype matching 'text/plain'. Migration of duplicate records is disabled
2021-03-09 15:04:22.982 INFO d.n.h.indexserver.FileBasedCache.<init> - Metadata cache for 'fullcrawllogindex' uses directory '/home/devel/Test3/cache/fullcrawllogindex'
2021-03-09 15:04:22.983 INFO d.n.h.indexserver.FileBasedCache.<init> - Metadata cache for 'cdxdata' uses directory '/home/devel/Test3/cache/cdxdata'
2021-03-09 15:04:22.984 INFO d.n.h.indexserver.RawMetadataCache.<init> - Metadata cache for 'cdxdata' is fetching metadata with urls matching 'metadata://[^/]*/crawl/index/cdx.*' and mimetype matching 'application/x-cdx'. Migration of duplicate records is disabled
2021-03-09 15:04:22.984 DEBUG d.n.h.indexserver.FileBasedCache.cache - Waiting to enter synchronization on /home/devel/Test3/cache/cdxindex/empty-cache.working
2021-03-09 15:04:22.985 DEBUG d.n.h.indexserver.FileBasedCache.cache - locking filechannel for file '/home/devel/Test3/cache/cdxindex/empty-cache.working' (thread = main)
2021-03-09 15:04:22.989 DEBUG d.n.common.utils.ProcessUtils.runProcess - Running external program: sort /home/devel/Test3/cache/cdxindex/empty-cache -o /home/devel/Test3/cache/cdxindex/empty-cache.unsorted with environment LANG=C
2021-03-09 15:04:23.001 DEBUG d.n.h.indexserver.FileBasedCache.cache - release lock on filechannel sun.nio.ch.FileChannelImpl@66fdec9
2021-03-09 15:04:23.001 INFO d.n.h.indexserver.FileBasedCache.getIndex - Generated index '/home/devel/Test3/cache/cdxindex/empty-cache' of id '[]', request was for '[]'
2021-03-09 15:04:23.002 DEBUG d.n.h.indexserver.FileBasedCache.cache - Waiting to enter synchronization on /home/devel/Test3/cache/dedupcrawllogindex/empty-cache.working
2021-03-09 15:04:23.002 DEBUG d.n.h.indexserver.FileBasedCache.cache - locking filechannel for file '/home/devel/Test3/cache/dedupcrawllogindex/empty-cache.working' (thread = main)
2021-03-09 15:04:23.002 INFO d.n.h.indexserver.CrawlLogIndexCache.prepareCombine - Starting to generate dedupcrawllogindex for the 0 jobs: []
2021-03-09 15:04:23.002 INFO d.n.h.indexserver.CrawlLogIndexCache.combine - Starting combine task #1. This combines a dataset with 0 crawl logs (thread = main)
2021-03-09 15:04:23.120 INFO d.n.h.indexserver.CrawlLogIndexCache.combine - Adding the final 0 subindices to main index. Forcing index to contain max 42 files (related to combine task #1)
2021-03-09 15:04:23.127 INFO d.n.h.indexserver.CrawlLogIndexCache.combine - Adding operation completed (combine task #1)!
2021-03-09 15:04:23.128 INFO d.n.h.indexserver.CrawlLogIndexCache.combine - Closed index (related to combine task #1
2021-03-09 15:04:23.128 INFO d.n.h.indexserver.CrawlLogIndexCache.combine - Gzip-compressing the individual 2 index files of combine task # 1
2021-03-09 15:04:23.132 INFO d.n.h.indexserver.CrawlLogIndexCache.combine - Completed combine task #1 that combined a dataset with 0 crawl logs (entries in combined index: 0) - compressed index has size 72 bytes
2021-03-09 15:04:23.133 DEBUG d.n.h.indexserver.FileBasedCache.cache - release lock on filechannel sun.nio.ch.FileChannelImpl@67ab1c47
2021-03-09 15:04:23.133 INFO d.n.h.indexserver.FileBasedCache.getIndex - Generated index '/home/devel/Test3/cache/dedupcrawllogindex/empty-cache' of id '[]', request was for '[]'
2021-03-09 15:04:23.134 DEBUG d.n.h.indexserver.FileBasedCache.cache - Waiting to enter synchronization on /home/devel/Test3/cache/fullcrawllogindex/empty-cache.working
2021-03-09 15:04:23.134 DEBUG d.n.h.indexserver.FileBasedCache.cache - locking filechannel for file '/home/devel/Test3/cache/fullcrawllogindex/empty-cache.working' (thread = main)
2021-03-09 15:04:23.134 INFO d.n.h.indexserver.CrawlLogIndexCache.prepareCombine - Starting to generate fullcrawllogindex for the 0 jobs: []
2021-03-09 15:04:23.134 INFO d.n.h.indexserver.CrawlLogIndexCache.combine - Starting combine task #1. This combines a dataset with 0 crawl logs (thread = main)
2021-03-09 15:04:23.136 INFO d.n.h.indexserver.CrawlLogIndexCache.combine - Adding the final 0 subindices to main index. Forcing index to contain max 42 files (related to combine task #1)
2021-03-09 15:04:23.139 INFO d.n.h.indexserver.CrawlLogIndexCache.combine - Adding operation completed (combine task #1)!
2021-03-09 15:04:23.139 INFO d.n.h.indexserver.CrawlLogIndexCache.combine - Closed index (related to combine task #1
2021-03-09 15:04:23.139 INFO d.n.h.indexserver.CrawlLogIndexCache.combine - Gzip-compressing the individual 2 index files of combine task # 1
2021-03-09 15:04:23.140 INFO d.n.h.indexserver.CrawlLogIndexCache.combine - Completed combine task #1 that combined a dataset with 0 crawl logs (entries in combined index: 0) - compressed index has size 72 bytes
2021-03-09 15:04:23.141 DEBUG d.n.h.indexserver.FileBasedCache.cache - release lock on filechannel sun.nio.ch.FileChannelImpl@30bcf3c1
2021-03-09 15:04:23.141 INFO d.n.h.indexserver.FileBasedCache.getIndex - Generated index '/home/devel/Test3/cache/fullcrawllogindex/empty-cache' of id '[]', request was for '[]'
2021-03-09 15:04:23.420 WARN o.a.hadoop.util.NativeCodeLoader.<clinit> - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2021-03-09 15:04:23.587 INFO d.n.h.i.d.IndexRequestServer.setHandler - Setting handler for RequestType: CDX
2021-03-09 15:04:23.588 INFO d.n.h.i.d.IndexRequestServer.setHandler - Setting handler for RequestType: DEDUP_CRAWL_LOG
2021-03-09 15:04:23.588 INFO d.n.h.i.d.IndexRequestServer.setHandler - Setting handler for RequestType: FULL_CRAWL_LOG
2021-03-09 15:04:23.589 INFO d.n.h.i.d.IndexRequestServer.restoreRequestsfromRequestDir - requestdir not found: creating request dir
2021-03-09 15:04:23.589 INFO d.n.h.i.d.IndexRequestServer.start - 0 indexing jobs in progress that was stored in requestdir: /home/devel/Test3/requests
2021-03-09 15:04:23.590 INFO d.n.h.i.d.IndexRequestServer.run - Enabling listening to the indexserver channel '[Queue 'Test3_COMMON_INDEX_SERVER']'
2021-03-09 15:04:23.590 DEBUG d.n.common.distribute.JMSConnection.setListener - Adding dk.netarkivet.harvester.indexserver.distribute.IndexRequestServer@612d987 as listener to Test3_COMMON_INDEX_SERVER
15:04:19,819 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [/home/devel/Test3/conf/logback_ViewerProxyApplication.xml] at [file:/home/devel/Test3/conf/logback_ViewerProxyApplication.xml]
15:04:20,126 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - debug attribute not set
15:04:20,127 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
15:04:20,142 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [STDOUT]
15:04:20,184 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
15:04:20,256 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
15:04:20,259 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [FILE]
15:04:20,301 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@831236296 - No compression will be used
15:04:20,302 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@831236296 - Will use the pattern ./log/ViewerProxyApplication.%d{yyyy-MM-dd}.%i.log for the active file
15:04:20,305 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@6dbb137d - The date pattern is 'yyyy-MM-dd' from file name pattern './log/ViewerProxyApplication.%d{yyyy-MM-dd}.%i.log'.
15:04:20,305 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@6dbb137d - Roll-over at midnight.
15:04:20,310 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@6dbb137d - Setting initial period to Tue Mar 09 15:04:20 CET 2021
15:04:20,310 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@6dbb137d - SizeAndTimeBasedFNATP is deprecated. Use SizeAndTimeBasedRollingPolicy instead
15:04:20,310 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@6dbb137d - For more information see http://logback.qos.ch/manual/appenders.html#SizeAndTimeBasedRollingPolicy
15:04:20,313 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
15:04:20,315 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - Active log file name: ./log/ViewerProxyApplication.log
15:04:20,315 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - File property is set to [./log/ViewerProxyApplication.log]
15:04:20,316 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [dk.netarkivet.monitor.logging.CachingSLF4JAppender]
15:04:20,874 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [MONITOR]
15:04:20,876 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to WARN
15:04:20,876 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[ROOT]
15:04:20,877 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [FILE] to Logger[ROOT]
15:04:20,877 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [MONITOR] to Logger[ROOT]
15:04:20,877 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [dk.netarkivet] to DEBUG
15:04:20,877 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
15:04:20,878 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@3c9d0b9d - Registering current configuration as safe fallback point
2021-03-09 15:04:20.889 DEBUG dk.netarkivet.common.utils.Settings.invoke0 - Value found in loaded data: SB,KB
SLF4J: A number (1) of logging calls during the initialization phase have been intercepted and are
SLF4J: now being replayed. These are subject to the filtering rules of the underlying logging system.
SLF4J: See also http://www.slf4j.org/codes.html#replay
Starting dk.netarkivet.viewerproxy.ViewerProxy
Version: 5.7-IIPCH3-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/cb5f59cb8465f2eea112989e4ed878851c79ff87)
2021-03-09 15:04:20.895 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Starting dk.netarkivet.viewerproxy.ViewerProxy
Version: 5.7-IIPCH3-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/cb5f59cb8465f2eea112989e4ed878851c79ff87)
Java VM: 1.8.0_282
2021-03-09 15:04:20.896 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Java VM: 1.8.0_282
java.home: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.282.b08-1.el7_9.x86_64/jre
2021-03-09 15:04:20.896 INFO d.n.common.utils.ApplicationUtils.logAndPrint - java.home: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.282.b08-1.el7_9.x86_64/jre
Working dir: /home/devel/Test3
2021-03-09 15:04:20.896 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Working dir: /home/devel/Test3
2021-03-09 15:04:20.897 INFO d.n.common.utils.ApplicationUtils.startApp - Using settings files '/home/devel/Test3/conf/settings_ViewerProxyApplication.xml'
2021-03-09 15:04:21.074 INFO d.n.m.d.JMSMonitorRegistryClient.register - Registering this client for monitoring every 1 minutes, using hostname 'kb-test-acs-001.kb.dk' and JMX/RMI ports 8350/8450
2021-03-09 15:04:21.075 INFO d.n.c.m.MBeanConnectorCreator.exposeJMXMBeanServer - Registered mbean server in registry on port 8350 communicating on port 8450 using password file 'conf/jmxremote.password'.
Service URL is service:jmx:rmi://kb-test-acs-001.kb.dk:8450/jndi/rmi://kb-test-acs-001.kb.dk:8350/jmxrmi
dk.netarkivet.viewerproxy.ViewerProxy Running
2021-03-09 15:04:21.077 INFO d.n.common.utils.ApplicationUtils.logAndPrint - dk.netarkivet.viewerproxy.ViewerProxy Running
2021-03-09 15:04:21.083 INFO d.n.c.distribute.JMSConnectionSunMQ.<init> - Creating instance of dk.netarkivet.common.distribute.JMSConnectionSunMQ
2021-03-09 15:04:21.083 DEBUG d.n.common.distribute.JMSConnection.initConnection - Initializing a JMS connection dk.netarkivet.common.distribute.JMSConnectionSunMQ
2021-03-09 15:04:21.084 INFO d.n.c.distribute.JMSConnectionSunMQ.getConnectionFactory - Establishing SunMQ JMS Connection to 'kb-test-adm-001.kb.dk:7676'
2021-03-09 15:04:21.090 INFO d.n.c.d.bitrepository.HostNameUtils.getHostName - Local hostname (provided by getCanonicalHostName): kb-test-acs-001.kb.dk
2021-03-09 15:04:22.709 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - BitmagArcRepositoryClient will timeout on each get request after 300000 milliseconds.
2021-03-09 15:04:22.710 DEBUG d.n.common.distribute.JMSConnection.setListener - Adding dk.netarkivet.archive.arcrepository.distribute.BitmagArcRepositoryClient@2766ca9d as listener to Test3_COMMON_THIS_REPOS_CLIENT_130_226_228_5_VP
2021-03-09 15:04:22.722 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - BitmagArcRepositoryClient listens for replies on channel '[Queue 'Test3_COMMON_THIS_REPOS_CLIENT_130_226_228_5_VP']'
2021-03-09 15:04:22.723 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - Getting bitmag config from settings.common.arcrepositoryClient.bitrepository.settingsDir=/home/devel/Test3/conf/bitmag_client_config
2021-03-09 15:04:22.723 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - Using 'Test3' as default collectionID
2021-03-09 15:04:22.728 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - Storing tempfiles in folder 'arcrepositoryTemp'
2021-03-09 15:04:22.855 DEBUG d.n.c.utils.BasicTwoWaySSLProvider.loadPrivateKey - X509CertificateHolder found
2021-03-09 15:04:22.865 DEBUG d.n.c.utils.BasicTwoWaySSLProvider.loadPrivateKey - PrivateKeyInfo found
2021-03-09 15:04:22.927 INFO d.n.h.indexserver.FileBasedCache.<init> - Metadata cache for 'FULL_CRAWL_LOG' uses directory '/home/devel/Test3/cache/FULL_CRAWL_LOG'
2021-03-09 15:04:22.949 INFO d.n.viewerproxy.ARCArchiveAccess.<init> - Constructed instance of ARCArchiveAccess with TryToLookupUriAsFtp: false
2021-03-09 15:04:22.972 INFO dk.netarkivet.viewerproxy.WebProxy.<init> - Starting viewerproxy jetty on port 8076
--------------------------------------------
STARTING MACHINE: devel@kb-test-way-001.kb.dk
Starting all applications on: kb-test-way-001.kb.dk
Starting linux application: WaybackIndexerApplication
Starting linux application: AggregatorApplication
15:04:26,885 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [/home/devel/Test3/conf/logback_AggregatorApplication.xml] at [file:/home/devel/Test3/conf/logback_AggregatorApplication.xml]
15:04:27,444 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - debug attribute not set
15:04:27,454 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
15:04:27,490 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [STDOUT]
15:04:27,553 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
15:04:27,769 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
15:04:27,794 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [FILE]
15:04:27,878 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@1948863195 - No compression will be used
15:04:27,881 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@1948863195 - Will use the pattern ./log/AggregatorApplication.%d{yyyy-MM-dd}.%i.log for the active file
15:04:27,893 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@70a9f84e - The date pattern is 'yyyy-MM-dd' from file name pattern './log/AggregatorApplication.%d{yyyy-MM-dd}.%i.log'.
15:04:27,893 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@70a9f84e - Roll-over at midnight.
15:04:27,913 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@70a9f84e - Setting initial period to Tue Mar 09 15:04:27 CET 2021
15:04:27,913 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@70a9f84e - SizeAndTimeBasedFNATP is deprecated. Use SizeAndTimeBasedRollingPolicy instead
15:04:27,913 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@70a9f84e - For more information see http://logback.qos.ch/manual/appenders.html#SizeAndTimeBasedRollingPolicy
15:04:27,918 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
15:04:27,929 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - Active log file name: ./log/AggregatorApplication.log
15:04:27,929 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - File property is set to [./log/AggregatorApplication.log]
15:04:27,930 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [dk.netarkivet.monitor.logging.CachingSLF4JAppender]
15:04:29,207 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [MONITOR]
15:04:29,213 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to WARN
15:04:29,213 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[ROOT]
15:04:29,226 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [FILE] to Logger[ROOT]
15:04:29,226 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [MONITOR] to Logger[ROOT]
15:04:29,232 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [dk.netarkivet] to DEBUG
15:04:29,232 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
15:04:29,232 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@130f889 - Registering current configuration as safe fallback point
2021-03-09 15:04:29.283 DEBUG dk.netarkivet.common.utils.Settings.invoke0 - Value found in loaded data: SB,KB
SLF4J: A number (1) of logging calls during the initialization phase have been intercepted and are
SLF4J: now being replayed. These are subject to the filtering rules of the underlying logging system.
SLF4J: See also http://www.slf4j.org/codes.html#replay
Starting dk.netarkivet.wayback.aggregator.AggregationWorker
Version: 5.7-IIPCH3-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/cb5f59cb8465f2eea112989e4ed878851c79ff87)
2021-03-09 15:04:29.306 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Starting dk.netarkivet.wayback.aggregator.AggregationWorker
Version: 5.7-IIPCH3-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/cb5f59cb8465f2eea112989e4ed878851c79ff87)
Java VM: 1.8.0_25
2021-03-09 15:04:29.309 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Java VM: 1.8.0_25
java.home: /usr/local/java.old/jdk1.8.0_25_x64/jre
2021-03-09 15:04:29.309 INFO d.n.common.utils.ApplicationUtils.logAndPrint - java.home: /usr/local/java.old/jdk1.8.0_25_x64/jre
Working dir: /home/devel/Test3
2021-03-09 15:04:29.309 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Working dir: /home/devel/Test3
2021-03-09 15:04:29.309 INFO d.n.common.utils.ApplicationUtils.startApp - Using settings files '/home/devel/Test3/conf/settings_AggregatorApplication.xml'
2021-03-09 15:04:29.582 INFO d.n.m.d.JMSMonitorRegistryClient.register - Registering this client for monitoring every 1 minutes, using hostname 'kb-test-way-001.kb.dk' and JMX/RMI ports 9150/9250
2021-03-09 15:04:29.583 INFO d.n.c.m.MBeanConnectorCreator.exposeJMXMBeanServer - Registered mbean server in registry on port 9150 communicating on port 9250 using password file 'conf/jmxremote.password'.
Service URL is service:jmx:rmi://kb-test-way-001.kb.dk:9250/jndi/rmi://kb-test-way-001.kb.dk:9150/jmxrmi
dk.netarkivet.wayback.aggregator.AggregationWorker Running
2021-03-09 15:04:29.609 INFO d.n.common.utils.ApplicationUtils.logAndPrint - dk.netarkivet.wayback.aggregator.AggregationWorker Running
2021-03-09 15:04:29.652 INFO d.n.c.distribute.JMSConnectionSunMQ.<init> - Creating instance of dk.netarkivet.common.distribute.JMSConnectionSunMQ
2021-03-09 15:04:29.658 DEBUG d.n.common.distribute.JMSConnection.initConnection - Initializing a JMS connection dk.netarkivet.common.distribute.JMSConnectionSunMQ
2021-03-09 15:04:29.659 INFO d.n.c.distribute.JMSConnectionSunMQ.getConnectionFactory - Establishing SunMQ JMS Connection to 'kb-test-adm-001.kb.dk:7676'
2021-03-09 15:04:29.670 DEBUG d.n.w.aggregator.AggregationWorker.runAggregation - No new raw index files found in '/home/devel/Test3/batchOutputDir' skipping aggregation
15:04:26,878 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [/home/devel/Test3/conf/logback_WaybackIndexerApplication.xml] at [file:/home/devel/Test3/conf/logback_WaybackIndexerApplication.xml]
15:04:27,793 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - debug attribute not set
15:04:27,805 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
15:04:27,826 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [STDOUT]
15:04:27,875 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
15:04:28,000 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
15:04:28,008 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [FILE]
15:04:28,084 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@111156771 - No compression will be used
15:04:28,085 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@111156771 - Will use the pattern ./log/WaybackIndexerApplication.%d{yyyy-MM-dd}.%i.log for the active file
15:04:28,094 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@5a955565 - The date pattern is 'yyyy-MM-dd' from file name pattern './log/WaybackIndexerApplication.%d{yyyy-MM-dd}.%i.log'.
15:04:28,094 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@5a955565 - Roll-over at midnight.
15:04:28,109 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@5a955565 - Setting initial period to Tue Mar 09 15:04:28 CET 2021
15:04:28,109 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@5a955565 - SizeAndTimeBasedFNATP is deprecated. Use SizeAndTimeBasedRollingPolicy instead
15:04:28,109 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@5a955565 - For more information see http://logback.qos.ch/manual/appenders.html#SizeAndTimeBasedRollingPolicy
15:04:28,117 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
15:04:28,136 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - Active log file name: ./log/WaybackIndexerApplication.log
15:04:28,136 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - File property is set to [./log/WaybackIndexerApplication.log]
15:04:28,137 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [dk.netarkivet.monitor.logging.CachingSLF4JAppender]
15:04:29,223 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [MONITOR]
15:04:29,233 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to WARN
15:04:29,233 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[ROOT]
15:04:29,235 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [FILE] to Logger[ROOT]
15:04:29,235 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [MONITOR] to Logger[ROOT]
15:04:29,235 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [dk.netarkivet] to DEBUG
15:04:29,235 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
15:04:29,236 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@6293abcc - Registering current configuration as safe fallback point
2021-03-09 15:04:29.282 DEBUG dk.netarkivet.common.utils.Settings.invoke0 - Value found in loaded data: SB,KB
SLF4J: A number (1) of logging calls during the initialization phase have been intercepted and are
SLF4J: now being replayed. These are subject to the filtering rules of the underlying logging system.
SLF4J: See also http://www.slf4j.org/codes.html#replay
Starting dk.netarkivet.wayback.indexer.WaybackIndexer
Version: 5.7-IIPCH3-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/cb5f59cb8465f2eea112989e4ed878851c79ff87)
2021-03-09 15:04:29.307 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Starting dk.netarkivet.wayback.indexer.WaybackIndexer
Version: 5.7-IIPCH3-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/cb5f59cb8465f2eea112989e4ed878851c79ff87)
Java VM: 1.8.0_25
2021-03-09 15:04:29.308 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Java VM: 1.8.0_25
java.home: /usr/local/java.old/jdk1.8.0_25_x64/jre
2021-03-09 15:04:29.308 INFO d.n.common.utils.ApplicationUtils.logAndPrint - java.home: /usr/local/java.old/jdk1.8.0_25_x64/jre
Working dir: /home/devel/Test3
2021-03-09 15:04:29.310 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Working dir: /home/devel/Test3
2021-03-09 15:04:29.310 INFO d.n.common.utils.ApplicationUtils.startApp - Using settings files '/home/devel/Test3/conf/settings_WaybackIndexerApplication.xml'
2021-03-09 15:04:29.576 INFO d.n.m.d.JMSMonitorRegistryClient.register - Registering this client for monitoring every 1 minutes, using hostname 'kb-test-way-001.kb.dk' and JMX/RMI ports 8151/8251
2021-03-09 15:04:29.580 INFO d.n.c.m.MBeanConnectorCreator.exposeJMXMBeanServer - Registered mbean server in registry on port 8151 communicating on port 8251 using password file 'conf/jmxremote.password'.
Service URL is service:jmx:rmi://kb-test-way-001.kb.dk:8251/jndi/rmi://kb-test-way-001.kb.dk:8151/jmxrmi
dk.netarkivet.wayback.indexer.WaybackIndexer Running
2021-03-09 15:04:29.585 INFO d.n.common.utils.ApplicationUtils.logAndPrint - dk.netarkivet.wayback.indexer.WaybackIndexer Running
2021-03-09 15:04:29.661 INFO d.n.c.distribute.JMSConnectionSunMQ.<init> - Creating instance of dk.netarkivet.common.distribute.JMSConnectionSunMQ
2021-03-09 15:04:29.664 DEBUG d.n.common.distribute.JMSConnection.initConnection - Initializing a JMS connection dk.netarkivet.common.distribute.JMSConnectionSunMQ
2021-03-09 15:04:29.664 INFO d.n.c.distribute.JMSConnectionSunMQ.getConnectionFactory - Establishing SunMQ JMS Connection to 'kb-test-adm-001.kb.dk:7676'
--------------------------------------------
--------------------------------------------
STARTING MACHINE: netarkdv@sb-test-har-001.statsbiblioteket.dk
Starting all applications on: sb-test-har-001.statsbiblioteket.dk
Starting linux application: HarvestControllerApplication_sbhigh_h3
15:04:30,729 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [/home/netarkdv/Test3/conf/logback_HarvestControllerApplication_sbhigh_h3.xml] at [file:/home/netarkdv/Test3/conf/logback_HarvestControllerApplication_sbhigh_h3.xml]
15:04:30,863 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - debug attribute not set
15:04:30,865 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
15:04:30,869 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [STDOUT]
15:04:30,877 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
15:04:30,918 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
15:04:30,921 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [FILE]
15:04:30,943 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@1907431275 - No compression will be used
15:04:30,944 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@1907431275 - Will use the pattern ./log/HarvestControllerApplication_sbhigh_h3.%d{yyyy-MM-dd}.%i.log for the active file
15:04:30,946 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@6193932a - The date pattern is 'yyyy-MM-dd' from file name pattern './log/HarvestControllerApplication_sbhigh_h3.%d{yyyy-MM-dd}.%i.log'.
15:04:30,946 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@6193932a - Roll-over at midnight.
15:04:30,950 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@6193932a - Setting initial period to Tue Mar 09 15:04:30 CET 2021
15:04:30,950 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@6193932a - SizeAndTimeBasedFNATP is deprecated. Use SizeAndTimeBasedRollingPolicy instead
15:04:30,950 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@6193932a - For more information see http://logback.qos.ch/manual/appenders.html#SizeAndTimeBasedRollingPolicy
15:04:30,952 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
15:04:30,954 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - Active log file name: ./log/HarvestControllerApplication_sbhigh_h3.log
15:04:30,954 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - File property is set to [./log/HarvestControllerApplication_sbhigh_h3.log]
15:04:30,957 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [dk.netarkivet.monitor.logging.CachingSLF4JAppender]
15:04:31,227 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [MONITOR]
15:04:31,228 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to WARN
15:04:31,228 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[ROOT]
15:04:31,228 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [FILE] to Logger[ROOT]
15:04:31,228 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [MONITOR] to Logger[ROOT]
15:04:31,229 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [dk.netarkivet] to DEBUG
15:04:31,229 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
15:04:31,229 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@647fd8ce - Registering current configuration as safe fallback point
2021-03-09 15:04:31.238 DEBUG dk.netarkivet.common.utils.Settings.invoke0 - Value found in loaded data: SB,KB
SLF4J: A number (1) of logging calls during the initialization phase have been intercepted and are
SLF4J: now being replayed. These are subject to the filtering rules of the underlying logging system.
SLF4J: See also http://www.slf4j.org/codes.html#replay
Starting dk.netarkivet.harvester.heritrix3.HarvestControllerServer
Version: 5.7-IIPCH3-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/cb5f59cb8465f2eea112989e4ed878851c79ff87)
2021-03-09 15:04:31.244 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Starting dk.netarkivet.harvester.heritrix3.HarvestControllerServer
Version: 5.7-IIPCH3-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/cb5f59cb8465f2eea112989e4ed878851c79ff87)
Java VM: 1.8.0_232
2021-03-09 15:04:31.244 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Java VM: 1.8.0_232
java.home: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.232.b09-0.el7_7.x86_64/jre
2021-03-09 15:04:31.245 INFO d.n.common.utils.ApplicationUtils.logAndPrint - java.home: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.232.b09-0.el7_7.x86_64/jre
Working dir: /home/netarkdv/Test3
2021-03-09 15:04:31.245 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Working dir: /home/netarkdv/Test3
2021-03-09 15:04:31.245 INFO d.n.common.utils.ApplicationUtils.startApp - Using settings files '/home/netarkdv/Test3/conf/settings_HarvestControllerApplication_sbhigh_h3.xml'
2021-03-09 15:04:31.389 INFO d.n.m.d.JMSMonitorRegistryClient.register - Registering this client for monitoring every 1 minutes, using hostname 'sb-test-har-001.statsbiblioteket.dk' and JMX/RMI ports 8150/8250
2021-03-09 15:04:31.390 INFO d.n.c.m.MBeanConnectorCreator.exposeJMXMBeanServer - Registered mbean server in registry on port 8150 communicating on port 8250 using password file 'conf/jmxremote.password'.
Service URL is service:jmx:rmi://sb-test-har-001.statsbiblioteket.dk:8250/jndi/rmi://sb-test-har-001.statsbiblioteket.dk:8150/jmxrmi
dk.netarkivet.harvester.heritrix3.HarvestControllerServer Running
2021-03-09 15:04:31.394 INFO d.n.common.utils.ApplicationUtils.logAndPrint - dk.netarkivet.harvester.heritrix3.HarvestControllerServer Running
2021-03-09 15:04:31.396 INFO d.n.c.distribute.JMSConnectionSunMQ.<init> - Creating instance of dk.netarkivet.common.distribute.JMSConnectionSunMQ
2021-03-09 15:04:31.396 DEBUG d.n.common.distribute.JMSConnection.initConnection - Initializing a JMS connection dk.netarkivet.common.distribute.JMSConnectionSunMQ
2021-03-09 15:04:31.396 INFO d.n.c.distribute.JMSConnectionSunMQ.getConnectionFactory - Establishing SunMQ JMS Connection to 'kb-test-adm-001.kb.dk:7676'
2021-03-09 15:04:31.414 INFO d.n.h.h.HarvestControllerServer.<init> - Starting HarvestControllerServer.
2021-03-09 15:04:31.414 INFO d.n.h.h.HarvestControllerServer.<init> - Bound to harvest channel 'HIGHPRIORITY'
2021-03-09 15:04:31.415 INFO d.n.h.h.HarvestControllerServer.<init> - Serverdir: 'harvester_high'
2021-03-09 15:04:31.416 INFO d.n.h.h.HarvestControllerServer.<init> - Harvesting requires at least 400000000 bytes free.
2021-03-09 15:04:31.526 INFO d.n.c.d.bitrepository.HostNameUtils.getHostName - Local hostname (provided by getCanonicalHostName): sb-test-har-001.statsbiblioteket.dk
2021-03-09 15:04:32.469 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - BitmagArcRepositoryClient will timeout on each get request after 300000 milliseconds.
2021-03-09 15:04:32.470 DEBUG d.n.common.distribute.JMSConnection.setListener - Adding dk.netarkivet.archive.arcrepository.distribute.BitmagArcRepositoryClient@68044f4 as listener to Test3_COMMON_THIS_REPOS_CLIENT_130_225_27_140_HCS_SBHIGH_H3
2021-03-09 15:04:32.487 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - BitmagArcRepositoryClient listens for replies on channel '[Queue 'Test3_COMMON_THIS_REPOS_CLIENT_130_225_27_140_HCS_SBHIGH_H3']'
2021-03-09 15:04:32.489 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - Getting bitmag config from settings.common.arcrepositoryClient.bitrepository.settingsDir=/home/netarkdv/Test3/conf/bitmag_client_config
2021-03-09 15:04:32.490 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - Using 'Test3' as default collectionID
2021-03-09 15:04:32.499 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - Storing tempfiles in folder 'arcrepositoryTemp'
2021-03-09 15:04:32.570 INFO d.n.c.utils.BasicTwoWaySSLProvider.loadPrivateKey - Key file '/home/devel/extra_configs/https_config/https_key.pem' with private key and certificate does not exist!
2021-03-09 15:04:32.592 DEBUG d.n.h.h.HarvestControllerServer.<init> - Obtained JMS connection.
2021-03-09 15:04:32.593 INFO d.n.h.h.HarvestControllerServer.<init> - SEND_READY_DELAY used by HarvestControllerServer is 300
2021-03-09 15:04:32.593 INFO d.n.h.heritrix3.PostProcessing.processOldJobs - Looking for unprocessed crawldata in 'harvester_high'
2021-03-09 15:04:32.594 DEBUG d.n.common.distribute.JMSConnection.setListener - Adding dk.netarkivet.harvester.heritrix3.HarvestControllerServer@63b1d4fa as listener to Test3_COMMON_HCHAN_VAL_RESP
2021-03-09 15:04:32.628 INFO d.n.h.h.HarvestControllerServer.<init> - Requested to check the validity of harvest channel 'HIGHPRIORITY'
2021-03-09 15:04:32.678 INFO d.n.h.h.HarvestControllerServer.visit - Received message stating that channel 'HIGHPRIORITY' is valid.
2021-03-09 15:04:32.681 INFO d.n.h.h.HarvestControllerServer.beginListeningIfSpaceAvailable - Starts to listen to new jobs on queue '[Queue 'Test3_COMMON_JOB_PARTIAL_HIGHPRIORITY']'
2021-03-09 15:04:32.681 DEBUG d.n.common.distribute.JMSConnection.setListener - Adding dk.netarkivet.harvester.heritrix3.HarvestControllerServer@63b1d4fa as listener to Test3_COMMON_JOB_PARTIAL_HIGHPRIORITY
--------------------------------------------
Completed the script all_test.sh
Finished: SUCCESS
Console Output
Help us localize this page