llection Test3
Delete content in collection=Test3
14:42:04.385 DEBUG o.b.c.settings.XMLFileSettingsLoader - Loading the settings file '/home/devel/ResetCollection/bitrepository-client/bin/../conf/RepositorySettings.xml'.
14:42:04.720 DEBUG org.bitrepository.common.JaxbHelper - Creating JAXBHelper based on schema from: jar:file:/home/devel/ResetCollection/bitrepository-client/lib/bitrepository-reference-settings-1.9.0.1.jar!/xsd/ReferenceSettings.xsd
14:42:04.750 DEBUG o.b.c.settings.XMLFileSettingsLoader - Loading the settings file '/home/devel/ResetCollection/bitrepository-client/bin/../conf/ReferenceSettings.xml'.
14:42:05.635 DEBUG o.b.p.security.BasicSecurityManager - X509CertificateHolder found
14:42:05.648 DEBUG o.b.p.security.BasicSecurityManager - Got something, that we don't (yet) recognize. Class: PEMKeyPair
14:42:05.649 DEBUG o.b.p.security.BasicSecurityManager - PrivateKeyInfo found
14:42:05.782 INFO o.b.p.activemq.ActiveMQMessageBus - Initializing ActiveMQMessageBus:'org.bitrepository.settings.repositorysettings.MessageBusConfiguration@4218d6a3[name=netarkiv-aar-bit-master-devel-01 broker, url=failover:(ssl://netarkiv-aar-bit-master-devel-01.statsbiblioteket.dk:61612?keepAlive=true)]'.
14:42:05.783 DEBUG org.bitrepository.common.JaxbHelper - Creating JAXBHelper based on schema from: jar:file:/home/devel/ResetCollection/bitrepository-client/lib/bitrepository-message-xml-xsd-30.jar!/xsd/BitRepositoryMessages.xsd
14:42:06.664 INFO o.a.a.t.failover.FailoverTransport - Successfully connected to ssl://netarkiv-aar-bit-master-devel-01.statsbiblioteket.dk:61612?keepAlive=true
14:42:06.771 DEBUG o.b.p.activemq.ActiveMQMessageBus - ActiveMQConnection initialized for 'org.bitrepository.settings.repositorysettings.MessageBusConfiguration@4218d6a3[name=netarkiv-aar-bit-master-devel-01 broker, url=failover:(ssl://netarkiv-aar-bit-master-devel-01.statsbiblioteket.dk:61612?keepAlive=true)]'.
14:42:06.774 INFO o.b.p.activemq.ActiveMQMessageBus - Settings component filter to: [kb-test-har-003.kb.dk-devel-31169]
14:42:06.777 DEBUG o.b.c.c.m.CollectionBasedConversationMediator - Initializing the CollectionBasedConversationMediator
14:42:06.777 DEBUG o.b.p.activemq.ActiveMQMessageBus - Adding listener 'org.bitrepository.client.conversation.mediator.CollectionBasedConversationMediator@2fd1433e' to destination: 'topic://netarkiv-devel-kb-test-har-003.kb.dk-devel-31169' on message-bus 'netarkiv-aar-bit-master-devel-01 broker'.
14:42:06.777 DEBUG o.b.p.activemq.ActiveMQMessageBus - Retrieving message consumer on destination 'topic://netarkiv-devel-kb-test-har-003.kb.dk-devel-31169' for listener 'org.bitrepository.client.conversation.mediator.CollectionBasedConversationMediator@2fd1433e'. Key: 'topic://netarkiv-devel-kb-test-har-003.kb.dk-devel-31169#802243390'.
14:42:06.778 DEBUG o.b.p.activemq.ActiveMQMessageBus - No consumer known. Creating new for key 'topic://netarkiv-devel-kb-test-har-003.kb.dk-devel-31169#802243390'.
Checksum: Count: FileID:
14:42:06.811 INFO o.b.a.g.ConversationBasedGetChecksumsClient - Requesting the checksums for file '' with the specifications 'org.bitrepository.bitrepositoryelements.ChecksumSpecTYPE@5403f35f[checksumType=MD5, checksumSalt=<null>(default), otherChecksumType=<null>(default)]null' with query [ContributorQuery{componentID='netarkiv-tape-devel-01, minTimestamp=Thu Jan 01 01:00:00 CET 1970, maxTimestamp=null, maxNumberOfResults=10000}, ContributorQuery{componentID='netarkiv-disk-devel-01, minTimestamp=Thu Jan 01 01:00:00 CET 1970, maxTimestamp=null, maxNumberOfResults=10000}, ContributorQuery{componentID='netarkiv-disk-devel-02, minTimestamp=Thu Jan 01 01:00:00 CET 1970, maxTimestamp=null, maxNumberOfResults=10000}, ContributorQuery{componentID='netarkiv-checksum-devel-01, minTimestamp=Thu Jan 01 01:00:00 CET 1970, maxTimestamp=null, maxNumberOfResults=10000}, ContributorQuery{componentID='netarkiv-kbh-disk-devel-01, minTimestamp=Thu Jan 01 01:00:00 CET 1970, maxTimestamp=null, maxNumberOfResults=10000}].
14:42:06.828 DEBUG org.bitrepository.common.JaxbHelper - Creating JAXBHelper based on schema from: jar:file:/home/devel/ResetCollection/bitrepository-client/lib/bitrepository-message-xml-xsd-30.jar!/versioning/ProtocolVersionDefinition.xsd
14:42:07.369 DEBUG o.b.p.m.l.GetChecksumsMessageLogger - Sent IdentifyPillarsForGetChecksumsRequest(31f9cb6c) collectionID Test3, destination topic://netarkiv-devel: FileIDs=AllFiles
14:42:07.369 DEBUG o.b.commandline.GetChecksumsCmd - Received event: 31f9cb6c: GET_CHECKSUMS: IDENTIFY_REQUEST_SENT: , Identifying pillars for GetChecksums
14:42:07.744 DEBUG o.b.p.m.l.GetChecksumsMessageLogger - Received IdentifyPillarsForGetChecksumsResponse(31f9cb6c) collectionID Test3, from netarkiv-kbh-disk-devel-01: IDENTIFICATION_POSITIVE(Checksum operation available.)
14:42:07.752 DEBUG o.b.commandline.GetChecksumsCmd - Received event: 31f9cb6c: GET_CHECKSUMS: COMPONENT_IDENTIFIED: ContributorID netarkiv-kbh-disk-devel-01
14:42:07.753 DEBUG o.b.c.c.s.ContributorResponseStatus - Received response from: netarkiv-kbh-disk-devel-01
14:42:07.753 DEBUG o.b.c.c.s.ContributorResponseStatus - Expected contributors: [netarkiv-disk-devel-02, netarkiv-checksum-devel-01, netarkiv-tape-devel-01, netarkiv-disk-devel-01, netarkiv-kbh-disk-devel-01], components that have not answered: [netarkiv-disk-devel-02, netarkiv-checksum-devel-01, netarkiv-tape-devel-01, netarkiv-disk-devel-01]
14:42:07.983 DEBUG o.b.p.m.l.GetChecksumsMessageLogger - Received IdentifyPillarsForGetChecksumsResponse(31f9cb6c) collectionID Test3, from netarkiv-tape-devel-01: IDENTIFICATION_POSITIVE(IDENTIFICATION_POSITIVE)
14:42:07.998 DEBUG o.b.commandline.GetChecksumsCmd - Received event: 31f9cb6c: GET_CHECKSUMS: COMPONENT_IDENTIFIED: ContributorID netarkiv-tape-devel-01
14:42:07.998 DEBUG o.b.c.c.s.ContributorResponseStatus - Received response from: netarkiv-tape-devel-01
14:42:07.999 DEBUG o.b.c.c.s.ContributorResponseStatus - Expected contributors: [netarkiv-disk-devel-02, netarkiv-checksum-devel-01, netarkiv-tape-devel-01, netarkiv-disk-devel-01, netarkiv-kbh-disk-devel-01], components that have not answered: [netarkiv-disk-devel-02, netarkiv-checksum-devel-01, netarkiv-disk-devel-01]
14:42:08.278 DEBUG o.b.p.m.l.GetChecksumsMessageLogger - Received IdentifyPillarsForGetChecksumsResponse(31f9cb6c) collectionID Test3, from netarkiv-disk-devel-01: IDENTIFICATION_POSITIVE(IDENTIFICATION_POSITIVE)
14:42:08.279 DEBUG o.b.commandline.GetChecksumsCmd - Received event: 31f9cb6c: GET_CHECKSUMS: COMPONENT_IDENTIFIED: ContributorID netarkiv-disk-devel-01
14:42:08.280 DEBUG o.b.c.c.s.ContributorResponseStatus - Received response from: netarkiv-disk-devel-01
14:42:08.280 DEBUG o.b.c.c.s.ContributorResponseStatus - Expected contributors: [netarkiv-disk-devel-02, netarkiv-checksum-devel-01, netarkiv-tape-devel-01, netarkiv-disk-devel-01, netarkiv-kbh-disk-devel-01], components that have not answered: [netarkiv-disk-devel-02, netarkiv-checksum-devel-01]
14:42:08.593 DEBUG o.b.p.m.l.GetChecksumsMessageLogger - Received IdentifyPillarsForGetChecksumsResponse(31f9cb6c) collectionID Test3, from netarkiv-checksum-devel-01: IDENTIFICATION_POSITIVE(Operation acknowledged and accepted.)
14:42:08.594 DEBUG o.b.commandline.GetChecksumsCmd - Received event: 31f9cb6c: GET_CHECKSUMS: COMPONENT_IDENTIFIED: ContributorID netarkiv-checksum-devel-01
14:42:08.594 DEBUG o.b.c.c.s.ContributorResponseStatus - Received response from: netarkiv-checksum-devel-01
14:42:08.594 DEBUG o.b.c.c.s.ContributorResponseStatus - Expected contributors: [netarkiv-disk-devel-02, netarkiv-checksum-devel-01, netarkiv-tape-devel-01, netarkiv-disk-devel-01, netarkiv-kbh-disk-devel-01], components that have not answered: [netarkiv-disk-devel-02]
14:42:16.826 DEBUG o.b.commandline.GetChecksumsCmd - Received event: 31f9cb6c: GET_CHECKSUMS: IDENTIFY_TIMEOUT: , Time has run out for looking up contributors
The following contributors didn't respond: [netarkiv-disk-devel-02]
14:42:16.827 DEBUG o.b.commandline.GetChecksumsCmd - Received event: 31f9cb6c: GET_CHECKSUMS: COMPONENT_FAILED: ContributorID netarkiv-disk-devel-02 ResponseCode: null, Timeout for identifying contributor
14:42:16.828 DEBUG o.b.commandline.GetChecksumsCmd - Received event: 31f9cb6c: GET_CHECKSUMS: IDENTIFICATION_COMPLETE: Contributors: [netarkiv-kbh-disk-devel-01, netarkiv-disk-devel-01, netarkiv-tape-devel-01, netarkiv-checksum-devel-01]
14:42:16.830 DEBUG o.b.commandline.GetChecksumsCmd - Received event: 31f9cb6c: GET_CHECKSUMS: REQUEST_SENT: ContributorID [netarkiv-checksum-devel-01, netarkiv-tape-devel-01, netarkiv-disk-devel-01, netarkiv-kbh-disk-devel-01], Sending GetFileIDsRequest's
14:42:17.232 DEBUG o.b.p.m.l.GetChecksumsMessageLogger - Sent GetChecksumsRequest(31f9cb6c) collectionID Test3, destination queue://netarkiv-tape-devel-01: FileIDs=AllFiles, ChecksumRequestForExistingFile=org.bitrepository.bitrepositoryelements.ChecksumSpecTYPE@5403f35f[checksumType=MD5, checksumSalt=<null>(default), otherChecksumType=<null>(default)], MaxNumberOfResults=10000, MinTimestamp=1970-01-01T01:00:00.000+01:00
14:42:17.508 DEBUG o.b.p.m.l.GetChecksumsMessageLogger - Sent GetChecksumsRequest(31f9cb6c) collectionID Test3, destination queue://netarkiv-disk-devel-01: FileIDs=AllFiles, ChecksumRequestForExistingFile=org.bitrepository.bitrepositoryelements.ChecksumSpecTYPE@5403f35f[checksumType=MD5, checksumSalt=<null>(default), otherChecksumType=<null>(default)], MaxNumberOfResults=10000, MinTimestamp=1970-01-01T01:00:00.000+01:00
14:42:17.985 DEBUG o.b.p.m.l.GetChecksumsMessageLogger - Received GetChecksumsProgressResponse(31f9cb6c) collectionID Test3, from netarkiv-tape-devel-01: OPERATION_ACCEPTED_PROGRESS(OPERATION_ACCEPTED_PROGRESS)
14:42:17.990 DEBUG o.b.commandline.GetChecksumsCmd - Received event: 31f9cb6c: GET_CHECKSUMS: PROGRESS: ContributorID netarkiv-tape-devel-01, OPERATION_ACCEPTED_PROGRESS
14:42:18.326 DEBUG o.b.p.m.l.GetChecksumsMessageLogger - Sent GetChecksumsRequest(31f9cb6c) collectionID Test3, destination topic://netarkiv-devel-netarkiv-checksum-devel-01-contributor: FileIDs=AllFiles, ChecksumRequestForExistingFile=org.bitrepository.bitrepositoryelements.ChecksumSpecTYPE@5403f35f[checksumType=MD5, checksumSalt=<null>(default), otherChecksumType=<null>(default)], MaxNumberOfResults=10000, MinTimestamp=1970-01-01T01:00:00.000+01:00
14:42:18.869 DEBUG o.b.p.m.l.GetChecksumsMessageLogger - Received GetChecksumsFinalResponse(31f9cb6c) collectionID Test3, from netarkiv-tape-devel-01: OPERATION_COMPLETED(OPERATION_COMPLETED), NumberOfChecksums=0, ChecksumRequestForExistingFile=org.bitrepository.bitrepositoryelements.ChecksumSpecTYPE@1034cee0[checksumType=MD5, checksumSalt=<null>(default), otherChecksumType=<null>(default)], PartialResult=false
14:42:18.877 DEBUG o.b.c.c.s.ContributorResponseStatus - Received response from: netarkiv-tape-devel-01
14:42:18.877 DEBUG o.b.c.c.s.ContributorResponseStatus - Expected contributors: [netarkiv-checksum-devel-01, netarkiv-tape-devel-01, netarkiv-disk-devel-01, netarkiv-kbh-disk-devel-01], components that have not answered: [netarkiv-checksum-devel-01, netarkiv-disk-devel-01, netarkiv-kbh-disk-devel-01]
14:42:19.027 DEBUG o.b.p.m.l.GetChecksumsMessageLogger - Sent GetChecksumsRequest(31f9cb6c) collectionID Test3, destination topic://netarkiv-devel.netarkiv-kbh-disk-devel-01: FileIDs=AllFiles, ChecksumRequestForExistingFile=org.bitrepository.bitrepositoryelements.ChecksumSpecTYPE@5403f35f[checksumType=MD5, checksumSalt=<null>(default), otherChecksumType=<null>(default)], MaxNumberOfResults=10000, MinTimestamp=1970-01-01T01:00:00.000+01:00
14:42:19.266 DEBUG o.b.p.m.l.GetChecksumsMessageLogger - Received GetChecksumsProgressResponse(31f9cb6c) collectionID Test3, from netarkiv-disk-devel-01: OPERATION_ACCEPTED_PROGRESS(OPERATION_ACCEPTED_PROGRESS)
14:42:19.267 DEBUG o.b.commandline.GetChecksumsCmd - Received event: 31f9cb6c: GET_CHECKSUMS: PROGRESS: ContributorID netarkiv-disk-devel-01, OPERATION_ACCEPTED_PROGRESS
14:42:19.488 DEBUG o.b.p.m.l.GetChecksumsMessageLogger - Received GetChecksumsFinalResponse(31f9cb6c) collectionID Test3, from netarkiv-disk-devel-01: OPERATION_COMPLETED(OPERATION_COMPLETED), NumberOfChecksums=0, ChecksumRequestForExistingFile=org.bitrepository.bitrepositoryelements.ChecksumSpecTYPE@101b9136[checksumType=MD5, checksumSalt=<null>(default), otherChecksumType=<null>(default)], PartialResult=false
14:42:19.496 DEBUG o.b.c.c.s.ContributorResponseStatus - Received response from: netarkiv-disk-devel-01
14:42:19.497 DEBUG o.b.c.c.s.ContributorResponseStatus - Expected contributors: [netarkiv-checksum-devel-01, netarkiv-tape-devel-01, netarkiv-disk-devel-01, netarkiv-kbh-disk-devel-01], components that have not answered: [netarkiv-checksum-devel-01, netarkiv-kbh-disk-devel-01]
14:42:19.813 DEBUG o.b.p.m.l.GetChecksumsMessageLogger - Received GetChecksumsProgressResponse(31f9cb6c) collectionID Test3, from netarkiv-checksum-devel-01: OPERATION_ACCEPTED_PROGRESS(Starting to calculate checksums.)
14:42:19.814 DEBUG o.b.commandline.GetChecksumsCmd - Received event: 31f9cb6c: GET_CHECKSUMS: PROGRESS: ContributorID netarkiv-checksum-devel-01, Starting to calculate checksums.
14:42:20.027 DEBUG o.b.p.m.l.GetChecksumsMessageLogger - Received GetChecksumsFinalResponse(31f9cb6c) collectionID Test3, from netarkiv-checksum-devel-01: OPERATION_COMPLETED, NumberOfChecksums=0, ChecksumRequestForExistingFile=org.bitrepository.bitrepositoryelements.ChecksumSpecTYPE@45ef3eb9[checksumType=MD5, checksumSalt=<null>(default), otherChecksumType=<null>(default)], PartialResult=false
14:42:20.028 DEBUG o.b.c.c.s.ContributorResponseStatus - Received response from: netarkiv-checksum-devel-01
14:42:20.028 DEBUG o.b.c.c.s.ContributorResponseStatus - Expected contributors: [netarkiv-checksum-devel-01, netarkiv-tape-devel-01, netarkiv-disk-devel-01, netarkiv-kbh-disk-devel-01], components that have not answered: [netarkiv-kbh-disk-devel-01]
14:42:20.257 DEBUG o.b.p.m.l.GetChecksumsMessageLogger - Received GetChecksumsFinalResponse(31f9cb6c) collectionID Test3, from netarkiv-kbh-disk-devel-01: OPERATION_COMPLETED(Completed checksum retrieval.), NumberOfChecksums=0, ChecksumRequestForExistingFile=org.bitrepository.bitrepositoryelements.ChecksumSpecTYPE@4068e606[checksumType=MD5, checksumSalt=<null>(default), otherChecksumType=<null>(default)], PartialResult=false
14:42:20.258 DEBUG o.b.c.c.s.ContributorResponseStatus - Received response from: netarkiv-kbh-disk-devel-01
14:42:20.258 DEBUG o.b.c.c.s.ContributorResponseStatus - Expected contributors: [netarkiv-checksum-devel-01, netarkiv-tape-devel-01, netarkiv-disk-devel-01, netarkiv-kbh-disk-devel-01], components that have not answered: []
Deleting file FileID: with checksum Checksum: in collection Test3 from pillars netarkiv-tape-devel-01 netarkiv-disk-devel-01 netarkiv-checksum-devel-01 netarkiv-kbh-disk-devel-01
14:42:21.431 DEBUG org.bitrepository.common.JaxbHelper - Creating JAXBHelper based on schema from: jar:file:/home/devel/ResetCollection/bitrepository-client/lib/bitrepository-repository-settings-xsd-12.jar!/xsd/RepositorySettings.xsd
14:42:21.568 DEBUG o.b.c.settings.XMLFileSettingsLoader - Loading the settings file '/home/devel/ResetCollection/bitrepository-client/bin/../conf/RepositorySettings.xml'.
14:42:21.861 DEBUG org.bitrepository.common.JaxbHelper - Creating JAXBHelper based on schema from: jar:file:/home/devel/ResetCollection/bitrepository-client/lib/bitrepository-reference-settings-1.9.0.1.jar!/xsd/ReferenceSettings.xsd
14:42:21.893 DEBUG o.b.c.settings.XMLFileSettingsLoader - Loading the settings file '/home/devel/ResetCollection/bitrepository-client/bin/../conf/ReferenceSettings.xml'.
14:42:22.579 DEBUG o.b.p.security.BasicSecurityManager - X509CertificateHolder found
14:42:22.594 DEBUG o.b.p.security.BasicSecurityManager - Got something, that we don't (yet) recognize. Class: PEMKeyPair
14:42:22.596 DEBUG o.b.p.security.BasicSecurityManager - PrivateKeyInfo found
14:42:22.776 INFO o.b.p.activemq.ActiveMQMessageBus - Initializing ActiveMQMessageBus:'org.bitrepository.settings.repositorysettings.MessageBusConfiguration@14cd1699[name=netarkiv-aar-bit-master-devel-01 broker, url=failover:(ssl://netarkiv-aar-bit-master-devel-01.statsbiblioteket.dk:61612?keepAlive=true)]'.
14:42:22.777 DEBUG org.bitrepository.common.JaxbHelper - Creating JAXBHelper based on schema from: jar:file:/home/devel/ResetCollection/bitrepository-client/lib/bitrepository-message-xml-xsd-30.jar!/xsd/BitRepositoryMessages.xsd
14:42:23.666 INFO o.a.a.t.failover.FailoverTransport - Successfully connected to ssl://netarkiv-aar-bit-master-devel-01.statsbiblioteket.dk:61612?keepAlive=true
14:42:23.745 DEBUG o.b.p.activemq.ActiveMQMessageBus - ActiveMQConnection initialized for 'org.bitrepository.settings.repositorysettings.MessageBusConfiguration@14cd1699[name=netarkiv-aar-bit-master-devel-01 broker, url=failover:(ssl://netarkiv-aar-bit-master-devel-01.statsbiblioteket.dk:61612?keepAlive=true)]'.
14:42:23.747 INFO o.b.p.activemq.ActiveMQMessageBus - Settings component filter to: [kb-test-har-003.kb.dk-devel-31206]
14:42:23.749 DEBUG o.b.c.c.m.CollectionBasedConversationMediator - Initializing the CollectionBasedConversationMediator
14:42:23.749 DEBUG o.b.p.activemq.ActiveMQMessageBus - Adding listener 'org.bitrepository.client.conversation.mediator.CollectionBasedConversationMediator@3514a4c0' to destination: 'topic://netarkiv-devel-kb-test-har-003.kb.dk-devel-31206' on message-bus 'netarkiv-aar-bit-master-devel-01 broker'.
14:42:23.749 DEBUG o.b.p.activemq.ActiveMQMessageBus - Retrieving message consumer on destination 'topic://netarkiv-devel-kb-test-har-003.kb.dk-devel-31206' for listener 'org.bitrepository.client.conversation.mediator.CollectionBasedConversationMediator@3514a4c0'. Key: 'topic://netarkiv-devel-kb-test-har-003.kb.dk-devel-31206#890545344'.
14:42:23.749 DEBUG o.b.p.activemq.ActiveMQMessageBus - No consumer known. Creating new for key 'topic://netarkiv-devel-kb-test-har-003.kb.dk-devel-31206#890545344'.
14:42:23.765 DEBUG o.b.commandline.DeleteFileCmd - Performing the DeleteFile operation.
java.lang.IllegalArgumentException: String hexString, 'Checksum:', must be an even number of characters.
at org.bitrepository.common.ArgumentValidator.checkTrue(ArgumentValidator.java:160)
at org.bitrepository.common.utils.Base16Utils.encodeBase16(Base16Utils.java:64)
at org.bitrepository.commandline.CommandLineClient.getChecksumDataForDeleteValidation(CommandLineClient.java:302)
at org.bitrepository.commandline.DeleteFileCmd.deleteTheFile(DeleteFileCmd.java:125)
at org.bitrepository.commandline.DeleteFileCmd.performOperation(DeleteFileCmd.java:107)
at org.bitrepository.commandline.CommandLineClient.runCommand(CommandLineClient.java:68)
at org.bitrepository.commandline.DeleteFileCmd.main(DeleteFileCmd.java:47)
xargs: ssh: exited with status 255; aborting
calling cleanup_hosts
running cleanup_hosts
Removing installation on sb-test-har-001.statsbiblioteket.dk
Removing installation on kb-test-har-003.kb.dk
Removing installation on kb-test-har-004.kb.dk
Removing installation on kb-test-har-005.kb.dk
Removing installation on kb-test-adm-001.kb.dk
Removing installation on kb-test-acs-001.kb.dk
Removing installation on kb-test-way-001.kb.dk
calling cleanup_linuxbarhosts
running cleanup_linuxbarhosts
Removing installation on sb-test-bar-001.statsbiblioteket.dk
Removing bitarchive on sb-test-bar-001.statsbiblioteket.dk
Calling cleanup_windowsbarhosts
running cleanup_windowsbarhosts
Removing C:\bitarkiv\Test3 on kb-test-bar-01.bitarkiv.kb.dk
The system cannot find the file specified.
Removing E:\bitarchive_1\Test3 on kb-test-bar-01.bitarkiv.kb.dk
The system cannot find the file specified.
Removing F:\bitarchive_2\Test3 on kb-test-bar-01.bitarkiv.kb.dk
The system cannot find the file specified.
Removing g:\bitarchive_3\Test3 on kb-test-bar-01.bitarkiv.kb.dk
The system cannot find the file specified.
TESTX=Test3
PORT=8076
MAILRECEIVERS=rbkr@kb.dk
VERSION=RBKR
DEPLOYCONF=deploy_config_bitmag.xml
H3ZIP=RBKR
BITMAGCONF=/home/devel/bitmag_test_config/client_config
Using bundler file /home/devel/nas_versions/bundler/NetarchiveSuite-heritrix3-bundler-RBKR.zip
Creating distribution files in /home/devel/prepared_software/Test3.
Will read source files using encoding 'UTF-8' (defaulted)
Branch in settings not found: common.warcRecordService
Branch in settings not found: common.arcrepositoryClient.bitrepository.getFileIDsMaxResults
Branch in settings not found: common.hadoop.kerberos
Branch in settings not found: common.hadoop.username
Branch in settings not found: common.hadoop.mapred.hadoopUberJar
Branch in settings not found: common.hadoop.mapred.inputFilesParentDir
Branch in settings not found: common.hadoop.mapred.cdxJob
Branch in settings not found: common.hadoop.mapred.metadataExtractionJob
Branch in settings not found: common.hadoop.mapred.metadataCDXExtractionJob
Branch in settings not found: common.hadoop.mapred.crawlLogExtractionJob
Branch in settings not found: common.fileResolver.baseUrl
Branch in settings not found: common.fileResolver.keyfile
Branch in settings not found: harvester.harvesting.heritrix3.bundle
Branch in settings not found: harvester.harvesting.heritrix3.certificate
Branch in settings not found: harvester.harvesting.heritrix3.bundle
Branch in settings not found: harvester.harvesting.heritrix3.certificate
Branch in settings not found: harvester.harvesting.umbra
Branch in settings not found: harvester.harvesting.heritrix3.bundle
Branch in settings not found: harvester.harvesting.heritrix3.certificate
Branch in settings not found: harvester.harvesting.umbra
Branch in settings not found: harvester.harvesting.heritrix3.bundle
Branch in settings not found: harvester.harvesting.heritrix3.bundle
Machine 'kb-test-adm-001.kb.dk' configured with encoding 'UTF-8' (defaulted)
Machine 'kb-test-har-003.kb.dk' configured with encoding 'UTF-8' (defaulted)
NetarchiveSuite-heritrix3-bundler.zip -> /home/devel/Test3/NetarchiveSuite-heritrix3-bundler.zip
/home/devel/quickstart-nas-5.0-h3/h3server.jks -> /home/devel/Test3/h3server.jks
Machine 'kb-test-har-005.kb.dk' configured with encoding 'UTF-8' (defaulted)
NetarchiveSuite-heritrix3-bundler.zip -> /home/devel/Test3/NetarchiveSuite-heritrix3-bundler.zip
/home/devel/quickstart-nas-5.0-h3/h3server.jks -> /home/devel/Test3/h3server.jks
Machine 'kb-test-har-004.kb.dk' configured with encoding 'UTF-8' (defaulted)
NetarchiveSuite-heritrix3-bundler.zip -> /home/devel/Test3/NetarchiveSuite-heritrix3-bundler.zip
Machine 'kb-test-acs-001.kb.dk' configured with encoding 'UTF-8' (defaulted)
Machine 'kb-test-way-001.kb.dk' configured with encoding 'UTF-8' (defaulted)
Machine 'sb-test-har-001.statsbiblioteket.dk' configured with encoding 'UTF-8' (defaulted)
/home/devel/prepared_software/Test3/NetarchiveSuite-heritrix3-bundler.zip -> /home/netarkdv/Test3/NetarchiveSuite-heritrix3-bundler.zip
14:43:01.787 [main] DEBUG dk.netarkivet.deploy.XmlStructure - Element baseDir is not a branch in the tree [archive, bitpreservation, baseDir]. Null returned
14:43:01.791 [main] DEBUG dk.netarkivet.deploy.XmlStructure - Element is not text. Null returned. Path given to getLeafValue archive
14:43:01.793 [main] DEBUG dk.netarkivet.deploy.XmlStructure - Element baseDir is not a branch in the tree [archive, bitpreservation, baseDir]. Null returned
14:43:01.793 [main] DEBUG dk.netarkivet.deploy.XmlStructure - Element is not text. Null returned. Path given to getLeafValue archive
14:43:01.793 [main] DEBUG dk.netarkivet.deploy.XmlStructure - Element baseDir is not a branch in the tree [archive, bitpreservation, baseDir]. Null returned
14:43:01.793 [main] DEBUG dk.netarkivet.deploy.XmlStructure - Element is not text. Null returned. Path given to getLeafValue archive
14:43:01.794 [main] DEBUG dk.netarkivet.deploy.XmlStructure - Element baseDir is not a branch in the tree [archive, bitpreservation, baseDir]. Null returned
14:43:01.794 [main] DEBUG dk.netarkivet.deploy.XmlStructure - Element is not text. Null returned. Path given to getLeafValue archive
14:43:01.794 [main] DEBUG dk.netarkivet.deploy.XmlStructure - Element baseDir is not a branch in the tree [archive, bitpreservation, baseDir]. Null returned
14:43:01.794 [main] DEBUG dk.netarkivet.deploy.XmlStructure - Element is not text. Null returned. Path given to getLeafValue archive
14:43:01.795 [main] DEBUG dk.netarkivet.deploy.XmlStructure - Element baseDir is not a branch in the tree [archive, bitpreservation, baseDir]. Null returned
14:43:01.795 [main] DEBUG dk.netarkivet.deploy.XmlStructure - Element is not text. Null returned. Path given to getLeafValue archive
14:43:01.884 [main] DEBUG dk.netarkivet.deploy.XmlStructure - Element baseDir is not a branch in the tree [archive, bitpreservation, baseDir]. Null returned
14:43:01.885 [main] DEBUG dk.netarkivet.deploy.XmlStructure - Element is not text. Null returned. Path given to getLeafValue archive
--------------------------------------------
INSTALLING TO MACHINE: devel@kb-test-adm-001.kb.dk
copying NetarchiveSuite-RBKR.zip to:kb-test-adm-001.kb.dk
deleting devel@kb-test-adm-001.kb.dk:/home/devel/Test3/lib
unzipping NetarchiveSuite-RBKR.zip at:kb-test-adm-001.kb.dk
Creating directories.
preparing for copying of settings and scripts
copying settings and scripts
make scripts executable
make password and access files readonly
--------------------------------------------
INSTALLING TO MACHINE: devel@kb-test-har-003.kb.dk
copying NetarchiveSuite-RBKR.zip to:kb-test-har-003.kb.dk
deleting devel@kb-test-har-003.kb.dk:/home/devel/Test3/lib
unzipping NetarchiveSuite-RBKR.zip at:kb-test-har-003.kb.dk
Creating directories.
preparing for copying of settings and scripts
copying settings and scripts
make scripts executable
make password and access files readonly
--------------------------------------------
INSTALLING TO MACHINE: devel@kb-test-har-005.kb.dk
copying NetarchiveSuite-RBKR.zip to:kb-test-har-005.kb.dk
deleting devel@kb-test-har-005.kb.dk:/home/devel/Test3/lib
unzipping NetarchiveSuite-RBKR.zip at:kb-test-har-005.kb.dk
Creating directories.
preparing for copying of settings and scripts
copying settings and scripts
make scripts executable
make password and access files readonly
--------------------------------------------
INSTALLING TO MACHINE: devel@kb-test-har-004.kb.dk
copying NetarchiveSuite-RBKR.zip to:kb-test-har-004.kb.dk
deleting devel@kb-test-har-004.kb.dk:/home/devel/Test3/lib
unzipping NetarchiveSuite-RBKR.zip at:kb-test-har-004.kb.dk
Creating directories.
preparing for copying of settings and scripts
copying settings and scripts
make scripts executable
make password and access files readonly
--------------------------------------------
INSTALLING TO MACHINE: devel@kb-test-acs-001.kb.dk
copying NetarchiveSuite-RBKR.zip to:kb-test-acs-001.kb.dk
deleting devel@kb-test-acs-001.kb.dk:/home/devel/Test3/lib
unzipping NetarchiveSuite-RBKR.zip at:kb-test-acs-001.kb.dk
Creating directories.
preparing for copying of settings and scripts
copying settings and scripts
make scripts executable
make password and access files readonly
--------------------------------------------
INSTALLING TO MACHINE: devel@kb-test-way-001.kb.dk
copying NetarchiveSuite-RBKR.zip to:kb-test-way-001.kb.dk
deleting devel@kb-test-way-001.kb.dk:/home/devel/Test3/lib
unzipping NetarchiveSuite-RBKR.zip at:kb-test-way-001.kb.dk
Creating directories.
preparing for copying of settings and scripts
copying settings and scripts
make scripts executable
make password and access files readonly
--------------------------------------------
--------------------------------------------
INSTALLING TO MACHINE: netarkdv@sb-test-har-001.statsbiblioteket.dk
copying NetarchiveSuite-RBKR.zip to:sb-test-har-001.statsbiblioteket.dk
deleting netarkdv@sb-test-har-001.statsbiblioteket.dk:/home/netarkdv/Test3/lib
unzipping NetarchiveSuite-RBKR.zip at:sb-test-har-001.statsbiblioteket.dk
Creating directories.
preparing for copying of settings and scripts
copying settings and scripts
make scripts executable
make password and access files readonly
--------------------------------------------
Initializing database for system Test3
+ host=devel@kb-test-adm-001
+ scp -p /home/devel/devel-config/scripts/db/setup_admindb.sh /home/devel/devel-config/scripts/db/setup_harvestdb.sh /home/devel/devel-config/scripts/db/teardown_admindb.sh /home/devel/devel-config/scripts/db/teardown_harvestdb.sh /home/devel/devel-config/scripts/../resources/harvestdb.testdata.sql /home/devel/devel-config/scripts/../resources/default_orderxml.cxml devel@kb-test-adm-001:bin/
+ ssh devel@kb-test-adm-001
Pseudo-terminal will not be allocated because stdin is not a terminal.
Destroying test3_admindb
DEBUG: transaction ID wrap limit is 2257721481, limited by database "test2056_harvestdb"
DROP DATABASE
Creating empty test3_admindb
DEBUG: transaction ID wrap limit is 2257721481, limited by database "test2056_harvestdb"
CREATE DATABASE
Initialising test3_admindb
CREATE TABLE
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "replica_pkey" for table "replica"
NOTICE: CREATE TABLE / UNIQUE will create implicit index "replica_replica_id_key" for table "replica"
NOTICE: CREATE TABLE / UNIQUE will create implicit index "replica_replica_name_key" for table "replica"
CREATE TABLE
CREATE SEQUENCE
ALTER TABLE
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "replicafileinfo_pkey" for table "replicafileinfo"
CREATE TABLE
CREATE SEQUENCE
ALTER TABLE
CREATE INDEX
CREATE INDEX
CREATE INDEX
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "file_pkey" for table "file"
CREATE TABLE
CREATE SEQUENCE
ALTER TABLE
CREATE INDEX
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "segment_pkey" for table "segment"
CREATE TABLE
CREATE SEQUENCE
ALTER TABLE
Destroying test3_harvestdb
DEBUG: transaction ID wrap limit is 2257721481, limited by database "test2056_harvestdb"
DROP DATABASE
DEBUG: transaction ID wrap limit is 2257721481, limited by database "test2056_harvestdb"
CREATE DATABASE
CREATE TABLE
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
GRANT
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "domains_pkey" for table "domains"
NOTICE: CREATE TABLE / UNIQUE will create implicit index "domains_name_key" for table "domains"
CREATE TABLE
CREATE SEQUENCE
ALTER TABLE
CREATE INDEX
CREATE INDEX
GRANT
GRANT
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "configurations_pkey" for table "configurations"
CREATE TABLE
CREATE SEQUENCE
ALTER TABLE
CREATE INDEX
CREATE INDEX
CREATE INDEX
GRANT
GRANT
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "config_passwords_pkey" for table "config_passwords"
CREATE TABLE
GRANT
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "config_seedlists_pkey" for table "config_seedlists"
CREATE TABLE
GRANT
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "seedlists_pkey" for table "seedlists"
CREATE TABLE
CREATE SEQUENCE
ALTER TABLE
CREATE INDEX
CREATE INDEX
GRANT
GRANT
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "passwords_pkey" for table "passwords"
CREATE TABLE
CREATE SEQUENCE
ALTER TABLE
CREATE INDEX
CREATE INDEX
GRANT
GRANT
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "ownerinfo_pkey" for table "ownerinfo"
CREATE TABLE
CREATE SEQUENCE
ALTER TABLE
CREATE INDEX
GRANT
GRANT
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "historyinfo_pkey" for table "historyinfo"
CREATE TABLE
CREATE SEQUENCE
ALTER TABLE
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
GRANT
GRANT
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "harvestdefinitions_pkey" for table "harvestdefinitions"
NOTICE: CREATE TABLE / UNIQUE will create implicit index "harvestdefinitions_name_key" for table "harvestdefinitions"
CREATE TABLE
CREATE INDEX
GRANT
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "fullharvests_pkey" for table "fullharvests"
CREATE TABLE
GRANT
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "partialharvests_pkey" for table "partialharvests"
CREATE TABLE
CREATE INDEX
GRANT
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "harvest_configs_pkey" for table "harvest_configs"
CREATE TABLE
GRANT
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "schedules_pkey" for table "schedules"
NOTICE: CREATE TABLE / UNIQUE will create implicit index "schedules_name_key" for table "schedules"
CREATE TABLE
CREATE SEQUENCE
ALTER TABLE
GRANT
GRANT
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "ordertemplates_pkey" for table "ordertemplates"
NOTICE: CREATE TABLE / UNIQUE will create implicit index "ordertemplates_name_key" for table "ordertemplates"
CREATE TABLE
CREATE SEQUENCE
ALTER TABLE
GRANT
GRANT
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "jobs_pkey" for table "jobs"
CREATE TABLE
CREATE INDEX
CREATE INDEX
GRANT
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "job_configs_pkey" for table "job_configs"
CREATE TABLE
CREATE INDEX
GRANT
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "global_crawler_trap_lists_pkey" for table "global_crawler_trap_lists"
NOTICE: CREATE TABLE / UNIQUE will create implicit index "global_crawler_trap_lists_name_key" for table "global_crawler_trap_lists"
CREATE TABLE
CREATE SEQUENCE
ALTER TABLE
CREATE INDEX
GRANT
GRANT
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "global_crawler_trap_expressions_pkey" for table "global_crawler_trap_expressions"
CREATE TABLE
CREATE SEQUENCE
ALTER TABLE
CREATE INDEX
CREATE INDEX
GRANT
GRANT
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "pkrunningjobshistory" for table "runningjobshistory"
CREATE TABLE
CREATE INDEX
CREATE INDEX
CREATE INDEX
GRANT
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "pkrunningjobsmonitor" for table "runningjobsmonitor"
CREATE TABLE
CREATE INDEX
CREATE INDEX
GRANT
NOTICE: CREATE TABLE / UNIQUE will create implicit index "pkfrontierreportlines" for table "frontierreportmonitor"
CREATE TABLE
GRANT
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "extendedfieldtype_pkey" for table "extendedfieldtype"
CREATE TABLE
CREATE SEQUENCE
ALTER TABLE
GRANT
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "extendedfield_pkey" for table "extendedfield"
CREATE TABLE
CREATE SEQUENCE
ALTER TABLE
GRANT
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "extendedfieldvalue_pkey" for table "extendedfieldvalue"
CREATE TABLE
CREATE SEQUENCE
ALTER TABLE
GRANT
INSERT 0 1
INSERT 0 1
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "harvestchannel_pkey" for table "harvestchannel"
NOTICE: CREATE TABLE / UNIQUE will create implicit index "harvestchannel_name_key" for table "harvestchannel"
CREATE TABLE
CREATE SEQUENCE
ALTER TABLE
CREATE INDEX
GRANT
GRANT
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
CREATE TABLE
NOTICE: ALTER TABLE / ADD PRIMARY KEY will create implicit index "eav_type_attribute_pkey" for table "eav_type_attribute"
ALTER TABLE
CREATE INDEX
CREATE SEQUENCE
CREATE TABLE
NOTICE: ALTER TABLE / ADD PRIMARY KEY will create implicit index "eav_attribute_pkey" for table "eav_attribute"
ALTER TABLE
CREATE INDEX
GRANT
GRANT
GRANT
INSERT 0 1
INSERT 0 1
INSERT 0 1
SET
SET
SET
SET
SET
SET
SET
SET
SET
setval
--------
17
(1 row)
setval
--------
17
(1 row)
setval
--------
17
(1 row)
setval
--------
5
(1 row)
setval
--------
4
(1 row)
setval
--------
17
(1 row)
14:44:45.216 [MLog-Init-Reporter] INFO com.mchange.v2.log.MLog - MLog clients using slf4j logging.
14:44:45.243 [MLog-Init-Reporter] DEBUG com.mchange.v2.log.MLog - Reading VM config for path list /com/mchange/v2/log/default-mchange-log.properties, /mchange-commons.properties, /c3p0.properties, hocon:/reference,/application,/c3p0,/, /mchange-log.properties, /
14:44:45.243 [MLog-Init-Reporter] DEBUG com.mchange.v2.log.MLog - The configuration file for resource identifier '/mchange-commons.properties' could not be found. Skipping.
14:44:45.243 [MLog-Init-Reporter] DEBUG com.mchange.v2.log.MLog - The configuration file for resource identifier '/c3p0.properties' could not be found. Skipping.
14:44:45.243 [MLog-Init-Reporter] DEBUG com.mchange.v2.log.MLog - The configuration file for resource identifier 'hocon:/reference,/application,/c3p0,/' could not be found. Skipping.
14:44:45.244 [MLog-Init-Reporter] DEBUG com.mchange.v2.log.MLog - The configuration file for resource identifier '/mchange-log.properties' could not be found. Skipping.
14:44:45.250 [main] DEBUG com.mchange.v2.cfg.MConfig - The configuration file for resource identifier '/mchange-commons.properties' could not be found. Skipping.
14:44:45.250 [main] DEBUG com.mchange.v2.cfg.MConfig - The configuration file for resource identifier '/mchange-log.properties' could not be found. Skipping.
14:44:45.250 [main] DEBUG com.mchange.v2.cfg.MConfig - The configuration file for resource identifier 'hocon:/reference,/application,/c3p0,/' could not be found. Skipping.
14:44:45.250 [main] DEBUG com.mchange.v2.cfg.MConfig - The configuration file for resource identifier '/c3p0.properties' could not be found. Skipping.
14:44:45.434 [main] INFO com.mchange.v2.c3p0.C3P0Registry - Initializing c3p0-0.9.5.4 [built 23-March-2019 23:00:48 -0700; debug? true; trace: 10]
14:44:45.495 [main] DEBUG com.mchange.v2.c3p0.management.DynamicPooledDataSourceManagerMBean - MBean: com.mchange.v2.c3p0:type=PooledDataSource,identityToken=10bdx52ag9ay327yki07t|1b0d3ed,name=10bdx52ag9ay327yki07t|1b0d3ed registered.
14:44:45.575 [main] DEBUG com.mchange.v2.c3p0.management.DynamicPooledDataSourceManagerMBean - MBean: com.mchange.v2.c3p0:type=PooledDataSource,identityToken=10bdx52ag9ay327yki07t|1b0d3ed,name=10bdx52ag9ay327yki07t|1b0d3ed unregistered, in order to be reregistered after update.
14:44:45.575 [main] DEBUG com.mchange.v2.c3p0.management.DynamicPooledDataSourceManagerMBean - MBean: com.mchange.v2.c3p0:type=PooledDataSource,identityToken=10bdx52ag9ay327yki07t|1b0d3ed,name=10bdx52ag9ay327yki07t|1b0d3ed registered.
14:44:45.595 [main] INFO dk.netarkivet.harvester.datamodel.HarvestDBConnection - Connection pool initialized with the following values:
- minPoolSize=5
- maxPoolSize=20
- acquireIncrement=5
- maxStatements=0
- maxStatementsPerConnection=0
- idleConnTestPeriod=0
- idleConnTestQuery='null'
- idleConnTestOnCheckin=false
14:44:45.688 [main] INFO com.mchange.v2.c3p0.impl.AbstractPoolBackedDataSource - Initializing c3p0 pool... com.mchange.v2.c3p0.ComboPooledDataSource [ acquireIncrement -> 5, acquireRetryAttempts -> 30, acquireRetryDelay -> 1000, autoCommitOnClose -> false, automaticTestTable -> null, breakAfterAcquireFailure -> false, checkoutTimeout -> 0, connectionCustomizerClassName -> null, connectionTesterClassName -> com.mchange.v2.c3p0.impl.DefaultConnectionTester, contextClassLoaderSource -> caller, dataSourceName -> 10bdx52ag9ay327yki07t|1b0d3ed, debugUnreturnedConnectionStackTraces -> false, description -> null, driverClass -> org.postgresql.Driver, extensions -> {}, factoryClassLocation -> null, forceIgnoreUnresolvedTransactions -> false, forceSynchronousCheckins -> false, forceUseNamedDriverClass -> false, identityToken -> 10bdx52ag9ay327yki07t|1b0d3ed, idleConnectionTestPeriod -> 0, initialPoolSize -> 3, jdbcUrl -> jdbc:postgresql://localhost:5432/test3_harvestdb, maxAdministrativeTaskTime -> 0, maxConnectionAge -> 3600, maxIdleTime -> 0, maxIdleTimeExcessConnections -> 0, maxPoolSize -> 20, maxStatements -> 0, maxStatementsPerConnection -> 0, minPoolSize -> 5, numHelperThreads -> 3, preferredTestQuery -> null, privilegeSpawnedThreads -> false, properties -> {user=******, password=******}, propertyCycle -> 0, statementCacheNumDeferredCloseThreads -> 0, testConnectionOnCheckin -> false, testConnectionOnCheckout -> false, unreturnedConnectionTimeout -> 0, userOverrides -> {}, usesTraditionalReflectiveProxies -> false ]
14:44:45.715 [main] DEBUG com.mchange.v2.cfg.MConfig - The configuration file for resource identifier '/mchange-commons.properties' could not be found. Skipping.
14:44:45.715 [main] DEBUG com.mchange.v2.cfg.MConfig - The configuration file for resource identifier '/mchange-log.properties' could not be found. Skipping.
14:44:45.715 [main] DEBUG com.mchange.v2.cfg.MConfig - The configuration file for resource identifier '/c3p0.properties' could not be found. Skipping.
14:44:45.715 [main] DEBUG com.mchange.v2.cfg.MConfig - The configuration file for resource identifier 'hocon:/reference,/application,/c3p0,/' could not be found. Skipping.
14:44:45.715 [main] WARN com.mchange.v2.resourcepool.BasicResourcePool - Bad pool size config, start 3 < min 5. Using 5 as start.
14:44:45.724 [main] DEBUG com.mchange.v2.resourcepool.BasicResourcePool - com.mchange.v2.resourcepool.BasicResourcePool@1507a04 config: [start -> 5; min -> 5; max -> 20; inc -> 5; num_acq_attempts -> 30; acq_attempt_delay -> 1000; check_idle_resources_delay -> 0; max_resource_age -> 3600000; max_idle_time -> 0; excess_max_idle_time -> 0; destroy_unreturned_resc_time -> 0; expiration_enforcement_delay -> 900000; break_on_acquisition_failure -> false; debug_store_checkout_exceptions -> false; force_synchronous_checkins -> false]
14:44:45.725 [main] DEBUG com.mchange.v2.c3p0.impl.C3P0PooledConnectionPoolManager - Created new pool for auth, username (masked): 'de******'.
14:44:45.726 [main] DEBUG com.mchange.v2.resourcepool.BasicResourcePool - acquire test -- pool size: 0; target_pool_size: 5; desired target? 1
14:44:45.745 [main] DEBUG com.mchange.v2.resourcepool.BasicResourcePool - awaitAvailable(): [unknown]
14:44:46.020 [main] DEBUG com.mchange.v2.c3p0.impl.NewProxyPreparedStatement - com.mchange.v2.c3p0.impl.NewProxyPreparedStatement@60704c [wrapping: SELECT version FROM schemaversions WHERE tablename = 'ordertemplates'] closed orphaned ResultSet: com.mchange.v2.c3p0.impl.NewProxyResultSet@1082e88 [wrapping: null]
14:44:46.033 [main] DEBUG com.mchange.v2.c3p0.impl.NewProxyPreparedStatement - com.mchange.v2.c3p0.impl.NewProxyPreparedStatement@5c96b [wrapping: SELECT COUNT(*) FROM ordertemplates WHERE name = 'default_orderxml'] closed orphaned ResultSet: com.mchange.v2.c3p0.impl.NewProxyResultSet@1e767f3 [wrapping: null]
14:44:46.055 [main] DEBUG com.mchange.v2.c3p0.impl.NewProxyPreparedStatement - com.mchange.v2.c3p0.impl.NewProxyPreparedStatement@e7dcfd [wrapping: SELECT COUNT(*) FROM ordertemplates WHERE name = 'default_orderxml'] closed orphaned ResultSet: com.mchange.v2.c3p0.impl.NewProxyResultSet@2114ba [wrapping: null]
14:44:46.074 [main] DEBUG com.mchange.v2.c3p0.impl.NewProxyConnection - com.mchange.v2.c3p0.impl.NewProxyConnection@1a9e7c8 [wrapping: org.postgresql.jdbc4.Jdbc4Connection@101d134]: close() called more than once.
The template 'default_orderxml' has now been updated.
+ set +x
--------------------------------------------
STARTING MACHINE: devel@kb-test-adm-001.kb.dk
Starting all applications on: kb-test-adm-001.kb.dk
Starting linux application: GUIApplication
Starting linux application: ArcRepositoryApplication
Starting linux application: HarvestJobManagerApplication
14:44:47,696 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [/home/devel/Test3/conf/logback_ArcRepositoryApplication.xml] at [file:/home/devel/Test3/conf/logback_ArcRepositoryApplication.xml]
14:44:48,500 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - debug attribute not set
14:44:48,506 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
14:44:48,542 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [STDOUT]
14:44:48,624 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
14:44:48,787 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
14:44:48,814 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [FILE]
14:44:48,917 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@28463284 - No compression will be used
14:44:48,918 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@28463284 - Will use the pattern ./log/ArcRepositoryApplication.%d{yyyy-MM-dd}.%i.log for the active file
14:44:48,931 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@3f3642 - The date pattern is 'yyyy-MM-dd' from file name pattern './log/ArcRepositoryApplication.%d{yyyy-MM-dd}.%i.log'.
14:44:48,931 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@3f3642 - Roll-over at midnight.
14:44:48,959 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@3f3642 - Setting initial period to Thu Mar 11 14:44:48 CET 2021
14:44:48,962 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@3f3642 - SizeAndTimeBasedFNATP is deprecated. Use SizeAndTimeBasedRollingPolicy instead
14:44:48,962 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@3f3642 - For more information see http://logback.qos.ch/manual/appenders.html#SizeAndTimeBasedRollingPolicy
14:44:48,968 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
14:44:48,979 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - Active log file name: ./log/ArcRepositoryApplication.log
14:44:48,979 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - File property is set to [./log/ArcRepositoryApplication.log]
14:44:48,985 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [dk.netarkivet.monitor.logging.CachingSLF4JAppender]
14:44:50,363 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [MONITOR]
14:44:50,380 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to WARN
14:44:50,380 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[ROOT]
14:44:50,381 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [FILE] to Logger[ROOT]
14:44:50,381 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [MONITOR] to Logger[ROOT]
14:44:50,382 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [dk.netarkivet] to DEBUG
14:44:50,382 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
14:44:50,383 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@c5cc4c - Registering current configuration as safe fallback point
2021-03-11 14:44:50.422 DEBUG dk.netarkivet.common.utils.Settings.invoke0 - Value found in loaded data: SB,KB
SLF4J: A number (1) of logging calls during the initialization phase have been intercepted and are
SLF4J: now being replayed. These are subject to the filtering rules of the underlying logging system.
SLF4J: See also http://www.slf4j.org/codes.html#replay
Starting dk.netarkivet.archive.arcrepository.ArcRepository
Version: 5.7-IIPCH3-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/19da50ffad622e72f8795a09e117baf44bb5256f)
2021-03-11 14:44:50.447 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Starting dk.netarkivet.archive.arcrepository.ArcRepository
Version: 5.7-IIPCH3-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/19da50ffad622e72f8795a09e117baf44bb5256f)
Java VM: 1.8.0_20
2021-03-11 14:44:50.460 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Java VM: 1.8.0_20
java.home: /usr/java/jdk1.8.0_20/jre
2021-03-11 14:44:50.461 INFO d.n.common.utils.ApplicationUtils.logAndPrint - java.home: /usr/java/jdk1.8.0_20/jre
Working dir: /home/devel/Test3
2021-03-11 14:44:50.462 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Working dir: /home/devel/Test3
2021-03-11 14:44:50.462 INFO d.n.common.utils.ApplicationUtils.startApp - Using settings files '/home/devel/Test3/conf/settings_ArcRepositoryApplication.xml'
2021-03-11 14:44:50.845 INFO d.n.m.d.JMSMonitorRegistryClient.register - Registering this client for monitoring every 1 minutes, using hostname 'kb-test-adm-001.kb.dk' and JMX/RMI ports 8151/8251
2021-03-11 14:44:50.846 INFO d.n.c.m.MBeanConnectorCreator.exposeJMXMBeanServer - Registered mbean server in registry on port 8151 communicating on port 8251 using password file 'conf/jmxremote.password'.
Service URL is service:jmx:rmi://kb-test-adm-001.kb.dk:8251/jndi/rmi://kb-test-adm-001.kb.dk:8151/jmxrmi
dk.netarkivet.archive.arcrepository.ArcRepository Running
2021-03-11 14:44:50.854 INFO d.n.common.utils.ApplicationUtils.logAndPrint - dk.netarkivet.archive.arcrepository.ArcRepository Running
2021-03-11 14:44:50.884 INFO d.n.c.distribute.JMSConnectionSunMQ.<init> - Creating instance of dk.netarkivet.common.distribute.JMSConnectionSunMQ
2021-03-11 14:44:50.885 DEBUG d.n.common.distribute.JMSConnection.initConnection - Initializing a JMS connection dk.netarkivet.common.distribute.JMSConnectionSunMQ
2021-03-11 14:44:50.885 INFO d.n.c.distribute.JMSConnectionSunMQ.getConnectionFactory - Establishing SunMQ JMS Connection to 'kb-test-adm-001.kb.dk:7676'
2021-03-11 14:44:51.636 INFO d.n.a.a.ArchiveDBConnection.initDataSource - Using jdbc url: jdbc:postgresql://localhost:5432/test3_admindb
2021-03-11 14:44:51.641 INFO d.n.a.a.ArchiveDBConnection.initDataSource - Connection pool initialized with the following values:
- minPoolSize=5
- maxPoolSize=10
- acquireIncrement=5
- maxStatements=0
- maxStatementsPerConnection=0
- idleConnTestPeriod=30
- idleConnTestQuery='SELECT COUNT(*) FROM replica'
- idleConnTestOnCheckin=false
2021-03-11 14:44:51.737 WARN c.m.v.resourcepool.BasicResourcePool.log - Bad pool size config, start 3 < min 5. Using 5 as start.
14:44:47,226 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [/home/devel/Test3/conf/logback_GUIApplication.xml] at [file:/home/devel/Test3/conf/logback_GUIApplication.xml]
14:44:47,715 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - debug attribute not set
14:44:47,716 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
14:44:47,726 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [STDOUT]
14:44:47,751 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
14:44:47,825 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
14:44:47,832 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [FILE]
14:44:47,880 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@12036246 - No compression will be used
14:44:47,882 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@12036246 - Will use the pattern ./log/GUIApplication.%d{yyyy-MM-dd}.%i.log for the active file
14:44:47,887 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@7b636c - The date pattern is 'yyyy-MM-dd' from file name pattern './log/GUIApplication.%d{yyyy-MM-dd}.%i.log'.
14:44:47,887 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@7b636c - Roll-over at midnight.
14:44:47,894 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@7b636c - Setting initial period to Thu Mar 11 14:44:47 CET 2021
14:44:47,895 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@7b636c - SizeAndTimeBasedFNATP is deprecated. Use SizeAndTimeBasedRollingPolicy instead
14:44:47,895 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@7b636c - For more information see http://logback.qos.ch/manual/appenders.html#SizeAndTimeBasedRollingPolicy
14:44:47,897 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
14:44:47,901 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - Active log file name: ./log/GUIApplication.log
14:44:47,901 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - File property is set to [./log/GUIApplication.log]
14:44:47,902 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [dk.netarkivet.monitor.logging.CachingSLF4JAppender]
14:44:48,680 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [MONITOR]
14:44:48,686 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to WARN
14:44:48,686 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[ROOT]
14:44:48,689 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [FILE] to Logger[ROOT]
14:44:48,689 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [MONITOR] to Logger[ROOT]
14:44:48,690 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [dk.netarkivet] to DEBUG
14:44:48,690 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
14:44:48,691 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@1dd4eda - Registering current configuration as safe fallback point
2021-03-11 14:44:48.708 DEBUG dk.netarkivet.common.utils.Settings.invoke0 - Value found in loaded data: SB,KB
SLF4J: A number (1) of logging calls during the initialization phase have been intercepted and are
SLF4J: now being replayed. These are subject to the filtering rules of the underlying logging system.
SLF4J: See also http://www.slf4j.org/codes.html#replay
Starting dk.netarkivet.common.webinterface.GUIWebServer
Version: 5.7-IIPCH3-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/19da50ffad622e72f8795a09e117baf44bb5256f)
2021-03-11 14:44:48.726 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Starting dk.netarkivet.common.webinterface.GUIWebServer
Version: 5.7-IIPCH3-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/19da50ffad622e72f8795a09e117baf44bb5256f)
Java VM: 1.8.0_20
2021-03-11 14:44:48.734 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Java VM: 1.8.0_20
java.home: /usr/java/jdk1.8.0_20/jre
2021-03-11 14:44:48.734 INFO d.n.common.utils.ApplicationUtils.logAndPrint - java.home: /usr/java/jdk1.8.0_20/jre
Working dir: /home/devel/Test3
2021-03-11 14:44:48.735 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Working dir: /home/devel/Test3
2021-03-11 14:44:48.736 INFO d.n.common.utils.ApplicationUtils.startApp - Using settings files '/home/devel/Test3/conf/settings_GUIApplication.xml'
2021-03-11 14:44:48.944 INFO d.n.m.d.JMSMonitorRegistryClient.register - Registering this client for monitoring every 1 minutes, using hostname 'kb-test-adm-001.kb.dk' and JMX/RMI ports 8150/8250
2021-03-11 14:44:48.944 INFO d.n.c.m.MBeanConnectorCreator.exposeJMXMBeanServer - Registered mbean server in registry on port 8150 communicating on port 8250 using password file 'conf/jmxremote.password'.
Service URL is service:jmx:rmi://kb-test-adm-001.kb.dk:8250/jndi/rmi://kb-test-adm-001.kb.dk:8150/jmxrmi
dk.netarkivet.common.webinterface.GUIWebServer Running
2021-03-11 14:44:48.948 INFO d.n.common.utils.ApplicationUtils.logAndPrint - dk.netarkivet.common.webinterface.GUIWebServer Running
2021-03-11 14:44:48.949 DEBUG dk.netarkivet.common.utils.Settings.debug - Value found in loaded data: webpages/HarvestDefinition.war,webpages/History.war,webpages/HarvestChannel.war,webpages/QA.war,webpages/Status.war
2021-03-11 14:44:48.949 DEBUG dk.netarkivet.common.utils.Settings.debug - Value found in loaded data: dk.netarkivet.harvester.webinterface.DefinitionsSiteSection,dk.netarkivet.harvester.webinterface.HistorySiteSection,dk.netarkivet.harvester.webinterface.HarvestChannelSiteSection,dk.netarkivet.viewerproxy.webinterface.QASiteSection,dk.netarkivet.monitor.webinterface.StatusSiteSection
2021-03-11 14:44:48.950 INFO d.n.common.webinterface.GUIWebServer.<init> - Starting webserver. Port: 8076 deployment directories: 'webpages/HarvestDefinition.war,webpages/History.war,webpages/HarvestChannel.war,webpages/QA.war,webpages/Status.war' classes: 'dk.netarkivet.harvester.webinterface.DefinitionsSiteSection,dk.netarkivet.harvester.webinterface.HistorySiteSection,dk.netarkivet.harvester.webinterface.HarvestChannelSiteSection,dk.netarkivet.viewerproxy.webinterface.QASiteSection,dk.netarkivet.monitor.webinterface.StatusSiteSection'
2021-03-11 14:44:48.989 INFO d.n.c.distribute.JMSConnectionSunMQ.<init> - Creating instance of dk.netarkivet.common.distribute.JMSConnectionSunMQ
2021-03-11 14:44:48.990 DEBUG d.n.common.distribute.JMSConnection.initConnection - Initializing a JMS connection dk.netarkivet.common.distribute.JMSConnectionSunMQ
2021-03-11 14:44:48.988 DEBUG d.n.common.webinterface.GUIWebServer.<init> - GUI using tempdir /home/devel/Test3/tmpdircommon
2021-03-11 14:44:48.991 DEBUG d.n.common.webinterface.GUIWebServer.<init> - GUI using basedir /home/devel/Test3
2021-03-11 14:44:48.990 INFO d.n.c.distribute.JMSConnectionSunMQ.getConnectionFactory - Establishing SunMQ JMS Connection to 'kb-test-adm-001.kb.dk:7676'
2021-03-11 14:44:48.992 INFO d.n.common.webinterface.GUIWebServer.<init> - Deleted existing tempdir '/home/devel/Test3/tmpdircommon'
2021-03-11 14:44:49.148 INFO d.n.common.webinterface.GUIWebServer.<init> - Scanning for taglibs is disabled as tomcat.util.scan.StandardJarScanFilter.jarsToSkip is unset.
2021-03-11 14:44:50.495 WARN o.a.hadoop.util.NativeCodeLoader.<clinit> - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2021-03-11 14:44:51.546 DEBUG dk.netarkivet.common.utils.Settings.debug - Value found in loaded data: dk.netarkivet.harvester.webinterface.DefinitionsSiteSection,dk.netarkivet.harvester.webinterface.HistorySiteSection,dk.netarkivet.harvester.webinterface.HarvestChannelSiteSection,dk.netarkivet.viewerproxy.webinterface.QASiteSection,dk.netarkivet.monitor.webinterface.StatusSiteSection
2021-03-11 14:44:51.553 DEBUG d.n.common.webinterface.SiteSection.getSections - Loading 5 site section(s).
2021-03-11 14:44:51.554 DEBUG d.n.common.webinterface.SiteSection.getSections - Loading site section dk.netarkivet.harvester.webinterface.DefinitionsSiteSection.
2021-03-11 14:44:51.555 DEBUG d.n.common.webinterface.SiteSection.getSections - Loading site section dk.netarkivet.harvester.webinterface.HistorySiteSection.
2021-03-11 14:44:51.557 DEBUG d.n.common.webinterface.SiteSection.getSections - Loading site section dk.netarkivet.harvester.webinterface.HarvestChannelSiteSection.
2021-03-11 14:44:51.558 DEBUG d.n.common.webinterface.SiteSection.getSections - Loading site section dk.netarkivet.viewerproxy.webinterface.QASiteSection.
2021-03-11 14:44:51.559 DEBUG d.n.common.webinterface.SiteSection.getSections - Loading site section dk.netarkivet.monitor.webinterface.StatusSiteSection.
14:44:47,851 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [/home/devel/Test3/conf/logback_HarvestJobManagerApplication.xml] at [file:/home/devel/Test3/conf/logback_HarvestJobManagerApplication.xml]
14:44:48,240 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - debug attribute not set
14:44:48,245 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
14:44:48,280 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [STDOUT]
14:44:48,340 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
14:44:48,570 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
14:44:48,596 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [FILE]
14:44:48,701 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@20420434 - No compression will be used
14:44:48,705 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@20420434 - Will use the pattern ./log/HarvestJobManagerApplication.%d{yyyy-MM-dd}.%i.log for the active file
14:44:48,712 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@1fe3806 - The date pattern is 'yyyy-MM-dd' from file name pattern './log/HarvestJobManagerApplication.%d{yyyy-MM-dd}.%i.log'.
14:44:48,712 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@1fe3806 - Roll-over at midnight.
14:44:48,727 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@1fe3806 - Setting initial period to Thu Mar 11 14:44:48 CET 2021
14:44:48,728 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@1fe3806 - SizeAndTimeBasedFNATP is deprecated. Use SizeAndTimeBasedRollingPolicy instead
14:44:48,728 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@1fe3806 - For more information see http://logback.qos.ch/manual/appenders.html#SizeAndTimeBasedRollingPolicy
14:44:48,733 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
14:44:48,744 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - Active log file name: ./log/HarvestJobManagerApplication.log
14:44:48,744 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - File property is set to [./log/HarvestJobManagerApplication.log]
14:44:48,745 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [dk.netarkivet.monitor.logging.CachingSLF4JAppender]
14:44:49,903 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [MONITOR]
14:44:49,909 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to WARN
14:44:49,910 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[ROOT]
14:44:49,911 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [FILE] to Logger[ROOT]
14:44:49,911 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [MONITOR] to Logger[ROOT]
14:44:49,912 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [dk.netarkivet] to DEBUG
14:44:49,912 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
14:44:49,913 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@171c76a - Registering current configuration as safe fallback point
2021-03-11 14:44:49.929 DEBUG dk.netarkivet.common.utils.Settings.invoke0 - Value found in loaded data: SB,KB
SLF4J: A number (1) of logging calls during the initialization phase have been intercepted and are
SLF4J: now being replayed. These are subject to the filtering rules of the underlying logging system.
SLF4J: See also http://www.slf4j.org/codes.html#replay
2021-03-11 14:44:49.975 INFO d.n.c.distribute.JMSConnectionSunMQ.<init> - Creating instance of dk.netarkivet.common.distribute.JMSConnectionSunMQ
2021-03-11 14:44:49.976 DEBUG d.n.common.distribute.JMSConnection.initConnection - Initializing a JMS connection dk.netarkivet.common.distribute.JMSConnectionSunMQ
2021-03-11 14:44:49.979 INFO d.n.c.distribute.JMSConnectionSunMQ.getConnectionFactory - Establishing SunMQ JMS Connection to 'kb-test-adm-001.kb.dk:7676'
2021-03-11 14:44:50.496 INFO d.n.h.datamodel.HarvestDBConnection.initDataSource - Connection pool initialized with the following values:
- minPoolSize=5
- maxPoolSize=20
- acquireIncrement=5
- maxStatements=0
- maxStatementsPerConnection=0
- idleConnTestPeriod=0
- idleConnTestQuery='null'
- idleConnTestOnCheckin=false
2021-03-11 14:44:50.585 WARN c.m.v.resourcepool.BasicResourcePool.log - Bad pool size config, start 3 < min 5. Using 5 as start.
2021-03-11 14:44:50.953 INFO d.n.h.scheduler.JobDispatcher.<init> - Creating JobDispatcher
Starting dk.netarkivet.harvester.scheduler.HarvestJobManager
Version: 5.7-IIPCH3-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/19da50ffad622e72f8795a09e117baf44bb5256f)
2021-03-11 14:44:51.075 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Starting dk.netarkivet.harvester.scheduler.HarvestJobManager
Version: 5.7-IIPCH3-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/19da50ffad622e72f8795a09e117baf44bb5256f)
2021-03-11 14:44:51.076 INFO d.n.common.utils.ApplicationUtils.startApp - Using settings files '/home/devel/Test3/conf/settings_HarvestJobManagerApplication.xml'
2021-03-11 14:44:51.369 INFO d.n.m.d.JMSMonitorRegistryClient.register - Registering this client for monitoring every 1 minutes, using hostname 'kb-test-adm-001.kb.dk' and JMX/RMI ports 9154/9254
2021-03-11 14:44:51.370 INFO d.n.c.m.MBeanConnectorCreator.exposeJMXMBeanServer - Registered mbean server in registry on port 9154 communicating on port 9254 using password file 'conf/jmxremote.password'.
Service URL is service:jmx:rmi://kb-test-adm-001.kb.dk:9254/jndi/rmi://kb-test-adm-001.kb.dk:9154/jmxrmi
2021-03-11 14:44:51.370 DEBUG d.n.c.lifecycle.LifeCycleComponent.start - Starting dk.netarkivet.harvester.scheduler.HarvestJobManager@66f1f5
2021-03-11 14:44:51.373 DEBUG d.n.common.distribute.JMSConnection.setListener - Adding dk.netarkivet.harvester.scheduler.HarvesterStatusReceiver@14df057 as listener to Test3_COMMON_HARVESTER_STATUS_TOPIC
2021-03-11 14:44:51.432 DEBUG d.n.common.distribute.JMSConnection.setListener - Adding dk.netarkivet.harvester.scheduler.HarvesterStatusReceiver@14df057 as listener to Test3_COMMON_HCHAN_VAL_REQ
2021-03-11 14:44:51.437 INFO d.n.h.s.HarvesterStatusReceiver.start - limitSubmittedJobsInQueue: true
2021-03-11 14:44:51.437 INFO d.n.h.s.HarvesterStatusReceiver.start - submittedJobsInQueueThreshold: 1
2021-03-11 14:44:51.456 INFO d.n.h.scheduler.HarvestJobGenerator.start - JobGeneratorTask set to run every 60 seconds
2021-03-11 14:44:51.456 DEBUG d.n.common.distribute.JMSConnection.setListener - Adding dk.netarkivet.harvester.scheduler.HarvestSchedulerMonitorServer@7e99d8 as listener to Test3_COMMON_THE_SCHED
dk.netarkivet.harvester.scheduler.HarvestJobManager Running
2021-03-11 14:44:51.482 INFO d.n.common.utils.ApplicationUtils.logAndPrint - dk.netarkivet.harvester.scheduler.HarvestJobManager Running
2021-03-11 14:44:51.515 INFO d.n.h.scheduler.JobSupervisor.rescheduleLeftOverJobs - 0 jobs has been resubmitted.
--------------------------------------------
STARTING MACHINE: devel@kb-test-har-003.kb.dk
Starting all applications on: kb-test-har-003.kb.dk
Starting linux application: HarvestControllerApplication_kblow001_h3
14:44:52,665 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [/home/devel/Test3/conf/logback_HarvestControllerApplication_kblow001_h3.xml] at [file:/home/devel/Test3/conf/logback_HarvestControllerApplication_kblow001_h3.xml]
14:44:52,874 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - debug attribute not set
14:44:52,878 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
14:44:52,886 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [STDOUT]
14:44:52,899 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
14:44:52,954 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
14:44:52,957 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [FILE]
14:44:52,978 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@1353070773 - No compression will be used
14:44:52,980 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@1353070773 - Will use the pattern ./log/HarvestControllerApplication_kblow001_h3.%d{yyyy-MM-dd}.%i.log for the active file
14:44:52,983 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@1817d444 - The date pattern is 'yyyy-MM-dd' from file name pattern './log/HarvestControllerApplication_kblow001_h3.%d{yyyy-MM-dd}.%i.log'.
14:44:52,983 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@1817d444 - Roll-over at midnight.
14:44:52,987 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@1817d444 - Setting initial period to Thu Mar 11 14:44:52 CET 2021
14:44:52,987 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@1817d444 - SizeAndTimeBasedFNATP is deprecated. Use SizeAndTimeBasedRollingPolicy instead
14:44:52,987 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@1817d444 - For more information see http://logback.qos.ch/manual/appenders.html#SizeAndTimeBasedRollingPolicy
14:44:52,989 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
14:44:52,991 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - Active log file name: ./log/HarvestControllerApplication_kblow001_h3.log
14:44:52,991 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - File property is set to [./log/HarvestControllerApplication_kblow001_h3.log]
14:44:52,992 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [dk.netarkivet.monitor.logging.CachingSLF4JAppender]
14:44:53,527 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [MONITOR]
14:44:53,530 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to WARN
14:44:53,530 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[ROOT]
14:44:53,531 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [FILE] to Logger[ROOT]
14:44:53,531 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [MONITOR] to Logger[ROOT]
14:44:53,531 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [dk.netarkivet] to DEBUG
14:44:53,531 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
14:44:53,532 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@6ca8564a - Registering current configuration as safe fallback point
2021-03-11 14:44:53.539 DEBUG dk.netarkivet.common.utils.Settings.invoke0 - Value found in loaded data: SB,KB
SLF4J: A number (1) of logging calls during the initialization phase have been intercepted and are
SLF4J: now being replayed. These are subject to the filtering rules of the underlying logging system.
SLF4J: See also http://www.slf4j.org/codes.html#replay
Starting dk.netarkivet.harvester.heritrix3.HarvestControllerServer
Version: 5.7-IIPCH3-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/19da50ffad622e72f8795a09e117baf44bb5256f)
2021-03-11 14:44:53.552 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Starting dk.netarkivet.harvester.heritrix3.HarvestControllerServer
Version: 5.7-IIPCH3-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/19da50ffad622e72f8795a09e117baf44bb5256f)
Java VM: 1.8.0_282
2021-03-11 14:44:53.565 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Java VM: 1.8.0_282
java.home: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.282.b08-1.el7_9.x86_64/jre
2021-03-11 14:44:53.565 INFO d.n.common.utils.ApplicationUtils.logAndPrint - java.home: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.282.b08-1.el7_9.x86_64/jre
Working dir: /home/devel/Test3
2021-03-11 14:44:53.566 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Working dir: /home/devel/Test3
2021-03-11 14:44:53.566 INFO d.n.common.utils.ApplicationUtils.startApp - Using settings files '/home/devel/Test3/conf/settings_HarvestControllerApplication_kblow001_h3.xml'
2021-03-11 14:44:53.692 INFO d.n.m.d.JMSMonitorRegistryClient.register - Registering this client for monitoring every 1 minutes, using hostname 'kb-test-har-003.kb.dk' and JMX/RMI ports 8150/8250
2021-03-11 14:44:53.692 INFO d.n.c.m.MBeanConnectorCreator.exposeJMXMBeanServer - Registered mbean server in registry on port 8150 communicating on port 8250 using password file 'conf/jmxremote.password'.
Service URL is service:jmx:rmi://kb-test-har-003.kb.dk:8250/jndi/rmi://kb-test-har-003.kb.dk:8150/jmxrmi
dk.netarkivet.harvester.heritrix3.HarvestControllerServer Running
2021-03-11 14:44:53.699 INFO d.n.common.utils.ApplicationUtils.logAndPrint - dk.netarkivet.harvester.heritrix3.HarvestControllerServer Running
2021-03-11 14:44:53.714 INFO d.n.c.distribute.JMSConnectionSunMQ.<init> - Creating instance of dk.netarkivet.common.distribute.JMSConnectionSunMQ
2021-03-11 14:44:53.715 DEBUG d.n.common.distribute.JMSConnection.initConnection - Initializing a JMS connection dk.netarkivet.common.distribute.JMSConnectionSunMQ
2021-03-11 14:44:53.715 INFO d.n.c.distribute.JMSConnectionSunMQ.getConnectionFactory - Establishing SunMQ JMS Connection to 'kb-test-adm-001.kb.dk:7676'
2021-03-11 14:44:53.720 INFO d.n.h.h.HarvestControllerServer.<init> - Starting HarvestControllerServer.
2021-03-11 14:44:53.721 INFO d.n.h.h.HarvestControllerServer.<init> - Bound to harvest channel 'LOWPRIORITY'
2021-03-11 14:44:53.721 INFO d.n.h.h.HarvestControllerServer.<init> - Serverdir: 'harvester_low'
2021-03-11 14:44:53.722 INFO d.n.h.h.HarvestControllerServer.<init> - Harvesting requires at least 400000000 bytes free.
2021-03-11 14:44:53.854 INFO d.n.c.d.bitrepository.HostNameUtils.getHostName - Local hostname (provided by getCanonicalHostName): kb-test-har-003.kb.dk
2021-03-11 14:44:55.729 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - BitmagArcRepositoryClient will timeout on each get request after 300000 milliseconds.
2021-03-11 14:44:55.730 DEBUG d.n.common.distribute.JMSConnection.setListener - Adding dk.netarkivet.archive.arcrepository.distribute.BitmagArcRepositoryClient@4e858e0a as listener to Test3_COMMON_THIS_REPOS_CLIENT_130_226_228_9_HCS_KBLOW001_H3
2021-03-11 14:44:55.738 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - BitmagArcRepositoryClient listens for replies on channel '[Queue 'Test3_COMMON_THIS_REPOS_CLIENT_130_226_228_9_HCS_KBLOW001_H3']'
2021-03-11 14:44:55.739 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - Getting bitmag config from settings.common.arcrepositoryClient.bitrepository.settingsDir=/home/devel/Test3/conf/bitmag_client_config
2021-03-11 14:44:55.739 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - Using 'Test3' as default collectionID
2021-03-11 14:44:55.744 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - Storing tempfiles in folder 'arcrepositoryTemp'
2021-03-11 14:44:55.803 DEBUG d.n.c.utils.BasicTwoWaySSLProvider.loadPrivateKey - X509CertificateHolder found
2021-03-11 14:44:55.804 DEBUG d.n.c.utils.BasicTwoWaySSLProvider.loadPrivateKey - PrivateKeyInfo found
2021-03-11 14:44:55.835 DEBUG d.n.h.h.HarvestControllerServer.<init> - Obtained JMS connection.
2021-03-11 14:44:55.836 INFO d.n.h.h.HarvestControllerServer.<init> - SEND_READY_DELAY used by HarvestControllerServer is 300
2021-03-11 14:44:55.837 INFO d.n.h.heritrix3.PostProcessing.processOldJobs - Looking for unprocessed crawldata in 'harvester_low'
2021-03-11 14:44:55.837 DEBUG d.n.common.distribute.JMSConnection.setListener - Adding dk.netarkivet.harvester.heritrix3.HarvestControllerServer@ea9b7c6 as listener to Test3_COMMON_HCHAN_VAL_RESP
2021-03-11 14:44:55.858 INFO d.n.h.h.HarvestControllerServer.<init> - Requested to check the validity of harvest channel 'LOWPRIORITY'
2021-03-11 14:44:55.917 INFO d.n.h.h.HarvestControllerServer.visit - Received message stating that channel 'LOWPRIORITY' is valid.
2021-03-11 14:44:55.919 INFO d.n.h.h.HarvestControllerServer.beginListeningIfSpaceAvailable - Starts to listen to new jobs on queue '[Queue 'Test3_COMMON_JOB_SNAPSHOT_LOWPRIORITY']'
2021-03-11 14:44:55.919 DEBUG d.n.common.distribute.JMSConnection.setListener - Adding dk.netarkivet.harvester.heritrix3.HarvestControllerServer@ea9b7c6 as listener to Test3_COMMON_JOB_SNAPSHOT_LOWPRIORITY
--------------------------------------------
STARTING MACHINE: devel@kb-test-har-005.kb.dk
Starting all applications on: kb-test-har-005.kb.dk
Starting linux application: HarvestControllerApplication_kbumbra_1
Starting linux application: HarvestControllerApplication_kbumbra_2
14:44:58,132 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [/home/devel/Test3/conf/logback_HarvestControllerApplication_kbumbra_1.xml] at [file:/home/devel/Test3/conf/logback_HarvestControllerApplication_kbumbra_1.xml]
14:44:58,503 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - debug attribute not set
14:44:58,506 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
14:44:58,513 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [STDOUT]
14:44:58,537 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
14:44:58,594 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
14:44:58,597 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [FILE]
14:44:58,626 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@1353070773 - No compression will be used
14:44:58,627 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@1353070773 - Will use the pattern ./log/HarvestControllerApplication_kbumbra_1.%d{yyyy-MM-dd}.%i.log for the active file
14:44:58,629 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@1817d444 - The date pattern is 'yyyy-MM-dd' from file name pattern './log/HarvestControllerApplication_kbumbra_1.%d{yyyy-MM-dd}.%i.log'.
14:44:58,629 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@1817d444 - Roll-over at midnight.
14:44:58,637 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@1817d444 - Setting initial period to Thu Mar 11 14:44:58 CET 2021
14:44:58,637 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@1817d444 - SizeAndTimeBasedFNATP is deprecated. Use SizeAndTimeBasedRollingPolicy instead
14:44:58,637 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@1817d444 - For more information see http://logback.qos.ch/manual/appenders.html#SizeAndTimeBasedRollingPolicy
14:44:58,639 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
14:44:58,641 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - Active log file name: ./log/HarvestControllerApplication_kbumbra_1.log
14:44:58,641 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - File property is set to [./log/HarvestControllerApplication_kbumbra_1.log]
14:44:58,642 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [dk.netarkivet.monitor.logging.CachingSLF4JAppender]
14:44:59,317 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [MONITOR]
14:44:59,319 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to WARN
14:44:59,319 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[ROOT]
14:44:59,320 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [FILE] to Logger[ROOT]
14:44:59,320 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [MONITOR] to Logger[ROOT]
14:44:59,320 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [dk.netarkivet] to DEBUG
14:44:59,320 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
14:44:59,321 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@6ca8564a - Registering current configuration as safe fallback point
2021-03-11 14:44:59.326 DEBUG dk.netarkivet.common.utils.Settings.invoke0 - Value found in loaded data: SB,KB
SLF4J: A number (1) of logging calls during the initialization phase have been intercepted and are
SLF4J: now being replayed. These are subject to the filtering rules of the underlying logging system.
SLF4J: See also http://www.slf4j.org/codes.html#replay
Starting dk.netarkivet.harvester.heritrix3.HarvestControllerServer
Version: 5.7-IIPCH3-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/19da50ffad622e72f8795a09e117baf44bb5256f)
2021-03-11 14:44:59.347 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Starting dk.netarkivet.harvester.heritrix3.HarvestControllerServer
Version: 5.7-IIPCH3-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/19da50ffad622e72f8795a09e117baf44bb5256f)
Java VM: 1.8.0_282
2021-03-11 14:44:59.348 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Java VM: 1.8.0_282
java.home: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.282.b08-1.el7_9.x86_64/jre
2021-03-11 14:44:59.348 INFO d.n.common.utils.ApplicationUtils.logAndPrint - java.home: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.282.b08-1.el7_9.x86_64/jre
Working dir: /home/devel/Test3
2021-03-11 14:44:59.348 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Working dir: /home/devel/Test3
2021-03-11 14:44:59.348 INFO d.n.common.utils.ApplicationUtils.startApp - Using settings files '/home/devel/Test3/conf/settings_HarvestControllerApplication_kbumbra_1.xml'
2021-03-11 14:44:59.541 INFO d.n.m.d.JMSMonitorRegistryClient.register - Registering this client for monitoring every 1 minutes, using hostname 'kb-test-har-005.kb.dk' and JMX/RMI ports 8150/8250
2021-03-11 14:44:59.542 INFO d.n.c.m.MBeanConnectorCreator.exposeJMXMBeanServer - Registered mbean server in registry on port 8150 communicating on port 8250 using password file 'conf/jmxremote.password'.
Service URL is service:jmx:rmi://kb-test-har-005.kb.dk:8250/jndi/rmi://kb-test-har-005.kb.dk:8150/jmxrmi
dk.netarkivet.harvester.heritrix3.HarvestControllerServer Running
2021-03-11 14:44:59.548 INFO d.n.common.utils.ApplicationUtils.logAndPrint - dk.netarkivet.harvester.heritrix3.HarvestControllerServer Running
2021-03-11 14:44:59.548 INFO d.n.c.distribute.JMSConnectionSunMQ.<init> - Creating instance of dk.netarkivet.common.distribute.JMSConnectionSunMQ
2021-03-11 14:44:59.548 DEBUG d.n.common.distribute.JMSConnection.initConnection - Initializing a JMS connection dk.netarkivet.common.distribute.JMSConnectionSunMQ
2021-03-11 14:44:59.548 INFO d.n.c.distribute.JMSConnectionSunMQ.getConnectionFactory - Establishing SunMQ JMS Connection to 'kb-test-adm-001.kb.dk:7676'
2021-03-11 14:44:59.559 INFO d.n.h.h.HarvestControllerServer.<init> - Starting HarvestControllerServer.
2021-03-11 14:44:59.559 INFO d.n.h.h.HarvestControllerServer.<init> - Bound to harvest channel 'UMBRA'
2021-03-11 14:44:59.559 INFO d.n.h.h.HarvestControllerServer.<init> - Serverdir: 'harvester_umbra_1'
2021-03-11 14:44:59.560 INFO d.n.h.h.HarvestControllerServer.<init> - Harvesting requires at least 400000000 bytes free.
2021-03-11 14:44:59.783 INFO d.n.c.d.bitrepository.HostNameUtils.getHostName - Local hostname (provided by getCanonicalHostName): kb-test-har-005.kb.dk
2021-03-11 14:45:01.794 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - BitmagArcRepositoryClient will timeout on each get request after 300000 milliseconds.
2021-03-11 14:45:01.795 DEBUG d.n.common.distribute.JMSConnection.setListener - Adding dk.netarkivet.archive.arcrepository.distribute.BitmagArcRepositoryClient@4e858e0a as listener to Test3_COMMON_THIS_REPOS_CLIENT_130_226_228_16_HCS_KBUMBRA_1
2021-03-11 14:45:01.833 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - BitmagArcRepositoryClient listens for replies on channel '[Queue 'Test3_COMMON_THIS_REPOS_CLIENT_130_226_228_16_HCS_KBUMBRA_1']'
2021-03-11 14:45:01.834 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - Getting bitmag config from settings.common.arcrepositoryClient.bitrepository.settingsDir=/home/devel/Test3/conf/bitmag_client_config
2021-03-11 14:45:01.834 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - Using 'Test3' as default collectionID
2021-03-11 14:45:01.838 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - Storing tempfiles in folder 'arcrepositoryTemp'
2021-03-11 14:45:01.972 DEBUG d.n.c.utils.BasicTwoWaySSLProvider.loadPrivateKey - X509CertificateHolder found
2021-03-11 14:45:01.986 DEBUG d.n.c.utils.BasicTwoWaySSLProvider.loadPrivateKey - PrivateKeyInfo found
2021-03-11 14:45:02.122 DEBUG d.n.h.h.HarvestControllerServer.<init> - Obtained JMS connection.
2021-03-11 14:45:02.123 INFO d.n.h.h.HarvestControllerServer.<init> - SEND_READY_DELAY used by HarvestControllerServer is 300
2021-03-11 14:45:02.123 INFO d.n.h.heritrix3.PostProcessing.processOldJobs - Looking for unprocessed crawldata in 'harvester_umbra_1'
2021-03-11 14:45:02.123 DEBUG d.n.common.distribute.JMSConnection.setListener - Adding dk.netarkivet.harvester.heritrix3.HarvestControllerServer@ea9b7c6 as listener to Test3_COMMON_HCHAN_VAL_RESP
2021-03-11 14:45:02.148 INFO d.n.h.h.HarvestControllerServer.<init> - Requested to check the validity of harvest channel 'UMBRA'
2021-03-11 14:45:02.164 INFO d.n.h.h.HarvestControllerServer.visit - Received message stating that channel 'UMBRA' is valid.
2021-03-11 14:45:02.165 INFO d.n.h.h.HarvestControllerServer.beginListeningIfSpaceAvailable - Starts to listen to new jobs on queue '[Queue 'Test3_COMMON_JOB_PARTIAL_UMBRA']'
2021-03-11 14:45:02.166 DEBUG d.n.common.distribute.JMSConnection.setListener - Adding dk.netarkivet.harvester.heritrix3.HarvestControllerServer@ea9b7c6 as listener to Test3_COMMON_JOB_PARTIAL_UMBRA
14:44:58,132 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [/home/devel/Test3/conf/logback_HarvestControllerApplication_kbumbra_2.xml] at [file:/home/devel/Test3/conf/logback_HarvestControllerApplication_kbumbra_2.xml]
14:44:58,528 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - debug attribute not set
14:44:58,541 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
14:44:58,547 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [STDOUT]
14:44:58,570 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
14:44:58,635 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
14:44:58,638 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [FILE]
14:44:58,666 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@1353070773 - No compression will be used
14:44:58,667 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@1353070773 - Will use the pattern ./log/HarvestControllerApplication_kbumbra_2.%d{yyyy-MM-dd}.%i.log for the active file
14:44:58,669 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@1817d444 - The date pattern is 'yyyy-MM-dd' from file name pattern './log/HarvestControllerApplication_kbumbra_2.%d{yyyy-MM-dd}.%i.log'.
14:44:58,669 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@1817d444 - Roll-over at midnight.
14:44:58,677 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@1817d444 - Setting initial period to Thu Mar 11 14:44:58 CET 2021
14:44:58,677 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@1817d444 - SizeAndTimeBasedFNATP is deprecated. Use SizeAndTimeBasedRollingPolicy instead
14:44:58,677 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@1817d444 - For more information see http://logback.qos.ch/manual/appenders.html#SizeAndTimeBasedRollingPolicy
14:44:58,679 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
14:44:58,681 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - Active log file name: ./log/HarvestControllerApplication_kbumbra_2.log
14:44:58,681 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - File property is set to [./log/HarvestControllerApplication_kbumbra_2.log]
14:44:58,682 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [dk.netarkivet.monitor.logging.CachingSLF4JAppender]
14:44:59,383 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [MONITOR]
14:44:59,385 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to WARN
14:44:59,385 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[ROOT]
14:44:59,389 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [FILE] to Logger[ROOT]
14:44:59,389 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [MONITOR] to Logger[ROOT]
14:44:59,389 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [dk.netarkivet] to DEBUG
14:44:59,389 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
14:44:59,390 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@6ca8564a - Registering current configuration as safe fallback point
2021-03-11 14:44:59.406 DEBUG dk.netarkivet.common.utils.Settings.invoke0 - Value found in loaded data: SB,KB
SLF4J: A number (1) of logging calls during the initialization phase have been intercepted and are
SLF4J: now being replayed. These are subject to the filtering rules of the underlying logging system.
SLF4J: See also http://www.slf4j.org/codes.html#replay
Starting dk.netarkivet.harvester.heritrix3.HarvestControllerServer
Version: 5.7-IIPCH3-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/19da50ffad622e72f8795a09e117baf44bb5256f)
2021-03-11 14:44:59.411 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Starting dk.netarkivet.harvester.heritrix3.HarvestControllerServer
Version: 5.7-IIPCH3-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/19da50ffad622e72f8795a09e117baf44bb5256f)
Java VM: 1.8.0_282
2021-03-11 14:44:59.412 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Java VM: 1.8.0_282
java.home: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.282.b08-1.el7_9.x86_64/jre
2021-03-11 14:44:59.412 INFO d.n.common.utils.ApplicationUtils.logAndPrint - java.home: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.282.b08-1.el7_9.x86_64/jre
Working dir: /home/devel/Test3
2021-03-11 14:44:59.412 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Working dir: /home/devel/Test3
2021-03-11 14:44:59.413 INFO d.n.common.utils.ApplicationUtils.startApp - Using settings files '/home/devel/Test3/conf/settings_HarvestControllerApplication_kbumbra_2.xml'
2021-03-11 14:44:59.572 INFO d.n.m.d.JMSMonitorRegistryClient.register - Registering this client for monitoring every 1 minutes, using hostname 'kb-test-har-005.kb.dk' and JMX/RMI ports 8155/8255
2021-03-11 14:44:59.572 INFO d.n.c.m.MBeanConnectorCreator.exposeJMXMBeanServer - Registered mbean server in registry on port 8155 communicating on port 8255 using password file 'conf/jmxremote.password'.
Service URL is service:jmx:rmi://kb-test-har-005.kb.dk:8255/jndi/rmi://kb-test-har-005.kb.dk:8155/jmxrmi
dk.netarkivet.harvester.heritrix3.HarvestControllerServer Running
2021-03-11 14:44:59.577 INFO d.n.common.utils.ApplicationUtils.logAndPrint - dk.netarkivet.harvester.heritrix3.HarvestControllerServer Running
2021-03-11 14:44:59.596 INFO d.n.c.distribute.JMSConnectionSunMQ.<init> - Creating instance of dk.netarkivet.common.distribute.JMSConnectionSunMQ
2021-03-11 14:44:59.597 DEBUG d.n.common.distribute.JMSConnection.initConnection - Initializing a JMS connection dk.netarkivet.common.distribute.JMSConnectionSunMQ
2021-03-11 14:44:59.597 INFO d.n.c.distribute.JMSConnectionSunMQ.getConnectionFactory - Establishing SunMQ JMS Connection to 'kb-test-adm-001.kb.dk:7676'
2021-03-11 14:44:59.607 INFO d.n.h.h.HarvestControllerServer.<init> - Starting HarvestControllerServer.
2021-03-11 14:44:59.608 INFO d.n.h.h.HarvestControllerServer.<init> - Bound to harvest channel 'UMBRA'
2021-03-11 14:44:59.608 INFO d.n.h.h.HarvestControllerServer.<init> - Serverdir: 'harvester_umbra_2'
2021-03-11 14:44:59.608 INFO d.n.h.h.HarvestControllerServer.<init> - Harvesting requires at least 400000000 bytes free.
2021-03-11 14:44:59.752 INFO d.n.c.d.bitrepository.HostNameUtils.getHostName - Local hostname (provided by getCanonicalHostName): kb-test-har-005.kb.dk
2021-03-11 14:45:01.915 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - BitmagArcRepositoryClient will timeout on each get request after 300000 milliseconds.
2021-03-11 14:45:01.929 DEBUG d.n.common.distribute.JMSConnection.setListener - Adding dk.netarkivet.archive.arcrepository.distribute.BitmagArcRepositoryClient@4e858e0a as listener to Test3_COMMON_THIS_REPOS_CLIENT_130_226_228_16_HCS_KBUMBRA_2
2021-03-11 14:45:01.937 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - BitmagArcRepositoryClient listens for replies on channel '[Queue 'Test3_COMMON_THIS_REPOS_CLIENT_130_226_228_16_HCS_KBUMBRA_2']'
2021-03-11 14:45:01.937 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - Getting bitmag config from settings.common.arcrepositoryClient.bitrepository.settingsDir=/home/devel/Test3/conf/bitmag_client_config
2021-03-11 14:45:01.938 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - Using 'Test3' as default collectionID
2021-03-11 14:45:01.941 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - Storing tempfiles in folder 'arcrepositoryTemp'
2021-03-11 14:45:02.049 DEBUG d.n.c.utils.BasicTwoWaySSLProvider.loadPrivateKey - X509CertificateHolder found
2021-03-11 14:45:02.050 DEBUG d.n.c.utils.BasicTwoWaySSLProvider.loadPrivateKey - PrivateKeyInfo found
2021-03-11 14:45:02.120 DEBUG d.n.h.h.HarvestControllerServer.<init> - Obtained JMS connection.
2021-03-11 14:45:02.121 INFO d.n.h.h.HarvestControllerServer.<init> - SEND_READY_DELAY used by HarvestControllerServer is 300
2021-03-11 14:45:02.121 INFO d.n.h.heritrix3.PostProcessing.processOldJobs - Looking for unprocessed crawldata in 'harvester_umbra_2'
2021-03-11 14:45:02.122 DEBUG d.n.common.distribute.JMSConnection.setListener - Adding dk.netarkivet.harvester.heritrix3.HarvestControllerServer@ea9b7c6 as listener to Test3_COMMON_HCHAN_VAL_RESP
2021-03-11 14:45:02.142 INFO d.n.h.h.HarvestControllerServer.<init> - Requested to check the validity of harvest channel 'UMBRA'
2021-03-11 14:45:02.157 INFO d.n.h.h.HarvestControllerServer.visit - Received message stating that channel 'UMBRA' is valid.
2021-03-11 14:45:02.158 INFO d.n.h.h.HarvestControllerServer.beginListeningIfSpaceAvailable - Starts to listen to new jobs on queue '[Queue 'Test3_COMMON_JOB_PARTIAL_UMBRA']'
2021-03-11 14:45:02.159 DEBUG d.n.common.distribute.JMSConnection.setListener - Adding dk.netarkivet.harvester.heritrix3.HarvestControllerServer@ea9b7c6 as listener to Test3_COMMON_JOB_PARTIAL_UMBRA
--------------------------------------------
STARTING MACHINE: devel@kb-test-har-004.kb.dk
Starting all applications on: kb-test-har-004.kb.dk
Starting linux application: HarvestControllerApplication_kblow002_h3
Starting linux application: HarvestControllerApplication_kbhigh_h3
14:45:03,866 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [/home/devel/Test3/conf/logback_HarvestControllerApplication_kbhigh_h3.xml] at [file:/home/devel/Test3/conf/logback_HarvestControllerApplication_kbhigh_h3.xml]
14:45:04,322 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - debug attribute not set
14:45:04,336 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
14:45:04,350 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [STDOUT]
14:45:04,376 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
14:45:04,479 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
14:45:04,487 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [FILE]
14:45:04,522 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@6750210 - No compression will be used
14:45:04,524 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@6750210 - Will use the pattern ./log/HarvestControllerApplication_kbhigh_h3.%d{yyyy-MM-dd}.%i.log for the active file
14:45:04,527 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@1f0f1111 - The date pattern is 'yyyy-MM-dd' from file name pattern './log/HarvestControllerApplication_kbhigh_h3.%d{yyyy-MM-dd}.%i.log'.
14:45:04,527 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@1f0f1111 - Roll-over at midnight.
14:45:04,534 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@1f0f1111 - Setting initial period to Thu Mar 11 14:45:04 CET 2021
14:45:04,535 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@1f0f1111 - SizeAndTimeBasedFNATP is deprecated. Use SizeAndTimeBasedRollingPolicy instead
14:45:04,535 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@1f0f1111 - For more information see http://logback.qos.ch/manual/appenders.html#SizeAndTimeBasedRollingPolicy
14:45:04,536 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
14:45:04,540 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - Active log file name: ./log/HarvestControllerApplication_kbhigh_h3.log
14:45:04,540 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - File property is set to [./log/HarvestControllerApplication_kbhigh_h3.log]
14:45:04,541 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [dk.netarkivet.monitor.logging.CachingSLF4JAppender]
14:45:05,318 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [MONITOR]
14:45:05,324 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to WARN
14:45:05,325 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[ROOT]
14:45:05,326 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [FILE] to Logger[ROOT]
14:45:05,326 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [MONITOR] to Logger[ROOT]
14:45:05,327 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [dk.netarkivet] to DEBUG
14:45:05,327 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
14:45:05,328 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@49c386c8 - Registering current configuration as safe fallback point
2021-03-11 14:45:05.335 DEBUG dk.netarkivet.common.utils.Settings.invoke0 - Value found in loaded data: SB,KB
SLF4J: A number (1) of logging calls during the initialization phase have been intercepted and are
SLF4J: now being replayed. These are subject to the filtering rules of the underlying logging system.
SLF4J: See also http://www.slf4j.org/codes.html#replay
Starting dk.netarkivet.harvester.heritrix3.HarvestControllerServer
Version: 5.7-IIPCH3-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/19da50ffad622e72f8795a09e117baf44bb5256f)
2021-03-11 14:45:05.344 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Starting dk.netarkivet.harvester.heritrix3.HarvestControllerServer
Version: 5.7-IIPCH3-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/19da50ffad622e72f8795a09e117baf44bb5256f)
Java VM: 1.8.0_25
2021-03-11 14:45:05.345 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Java VM: 1.8.0_25
java.home: /usr/local/java/jdk1.8.0_25_x64/jre
2021-03-11 14:45:05.346 INFO d.n.common.utils.ApplicationUtils.logAndPrint - java.home: /usr/local/java/jdk1.8.0_25_x64/jre
Working dir: /home/devel/Test3
2021-03-11 14:45:05.346 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Working dir: /home/devel/Test3
2021-03-11 14:45:05.352 INFO d.n.common.utils.ApplicationUtils.startApp - Using settings files '/home/devel/Test3/conf/settings_HarvestControllerApplication_kbhigh_h3.xml'
2021-03-11 14:45:05.512 INFO d.n.m.d.JMSMonitorRegistryClient.register - Registering this client for monitoring every 1 minutes, using hostname 'kb-test-har-004.kb.dk' and JMX/RMI ports 8152/8252
2021-03-11 14:45:05.513 INFO d.n.c.m.MBeanConnectorCreator.exposeJMXMBeanServer - Registered mbean server in registry on port 8152 communicating on port 8252 using password file 'conf/jmxremote.password'.
Service URL is service:jmx:rmi://kb-test-har-004.kb.dk:8252/jndi/rmi://kb-test-har-004.kb.dk:8152/jmxrmi
dk.netarkivet.harvester.heritrix3.HarvestControllerServer Running
2021-03-11 14:45:05.519 INFO d.n.common.utils.ApplicationUtils.logAndPrint - dk.netarkivet.harvester.heritrix3.HarvestControllerServer Running
2021-03-11 14:45:05.535 INFO d.n.c.distribute.JMSConnectionSunMQ.<init> - Creating instance of dk.netarkivet.common.distribute.JMSConnectionSunMQ
2021-03-11 14:45:05.535 DEBUG d.n.common.distribute.JMSConnection.initConnection - Initializing a JMS connection dk.netarkivet.common.distribute.JMSConnectionSunMQ
2021-03-11 14:45:05.536 INFO d.n.c.distribute.JMSConnectionSunMQ.getConnectionFactory - Establishing SunMQ JMS Connection to 'kb-test-adm-001.kb.dk:7676'
2021-03-11 14:45:05.552 INFO d.n.h.h.HarvestControllerServer.<init> - Starting HarvestControllerServer.
2021-03-11 14:45:05.553 INFO d.n.h.h.HarvestControllerServer.<init> - Bound to harvest channel 'HIGHPRIORITY'
2021-03-11 14:45:05.553 INFO d.n.h.h.HarvestControllerServer.<init> - Serverdir: 'harvester_high'
2021-03-11 14:45:05.554 INFO d.n.h.h.HarvestControllerServer.<init> - Harvesting requires at least 400000000 bytes free.
2021-03-11 14:45:05.736 INFO d.n.c.d.bitrepository.HostNameUtils.getHostName - Local hostname (provided by getCanonicalHostName): kb-test-har-004.kb.dk
14:45:03,877 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [/home/devel/Test3/conf/logback_HarvestControllerApplication_kblow002_h3.xml] at [file:/home/devel/Test3/conf/logback_HarvestControllerApplication_kblow002_h3.xml]
14:45:04,350 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - debug attribute not set
14:45:04,363 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
14:45:04,378 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [STDOUT]
14:45:04,399 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
14:45:04,483 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
14:45:04,487 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [FILE]
14:45:04,521 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@6750210 - No compression will be used
14:45:04,522 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@6750210 - Will use the pattern ./log/HarvestControllerApplication_kblow002_h3.%d{yyyy-MM-dd}.%i.log for the active file
14:45:04,524 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@1f0f1111 - The date pattern is 'yyyy-MM-dd' from file name pattern './log/HarvestControllerApplication_kblow002_h3.%d{yyyy-MM-dd}.%i.log'.
14:45:04,524 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@1f0f1111 - Roll-over at midnight.
14:45:04,529 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@1f0f1111 - Setting initial period to Thu Mar 11 14:45:04 CET 2021
14:45:04,529 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@1f0f1111 - SizeAndTimeBasedFNATP is deprecated. Use SizeAndTimeBasedRollingPolicy instead
14:45:04,529 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@1f0f1111 - For more information see http://logback.qos.ch/manual/appenders.html#SizeAndTimeBasedRollingPolicy
14:45:04,531 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
14:45:04,535 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - Active log file name: ./log/HarvestControllerApplication_kblow002_h3.log
14:45:04,535 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - File property is set to [./log/HarvestControllerApplication_kblow002_h3.log]
14:45:04,535 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [dk.netarkivet.monitor.logging.CachingSLF4JAppender]
14:45:05,098 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [MONITOR]
14:45:05,103 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to WARN
14:45:05,103 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[ROOT]
14:45:05,104 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [FILE] to Logger[ROOT]
14:45:05,104 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [MONITOR] to Logger[ROOT]
14:45:05,105 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [dk.netarkivet] to DEBUG
14:45:05,105 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
14:45:05,106 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@49c386c8 - Registering current configuration as safe fallback point
2021-03-11 14:45:05.111 DEBUG dk.netarkivet.common.utils.Settings.invoke0 - Value found in loaded data: SB,KB
SLF4J: A number (1) of logging calls during the initialization phase have been intercepted and are
SLF4J: now being replayed. These are subject to the filtering rules of the underlying logging system.
SLF4J: See also http://www.slf4j.org/codes.html#replay
Starting dk.netarkivet.harvester.heritrix3.HarvestControllerServer
Version: 5.7-IIPCH3-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/19da50ffad622e72f8795a09e117baf44bb5256f)
2021-03-11 14:45:05.117 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Starting dk.netarkivet.harvester.heritrix3.HarvestControllerServer
Version: 5.7-IIPCH3-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/19da50ffad622e72f8795a09e117baf44bb5256f)
Java VM: 1.8.0_25
2021-03-11 14:45:05.118 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Java VM: 1.8.0_25
java.home: /usr/local/java/jdk1.8.0_25_x64/jre
2021-03-11 14:45:05.118 INFO d.n.common.utils.ApplicationUtils.logAndPrint - java.home: /usr/local/java/jdk1.8.0_25_x64/jre
Working dir: /home/devel/Test3
2021-03-11 14:45:05.118 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Working dir: /home/devel/Test3
2021-03-11 14:45:05.123 INFO d.n.common.utils.ApplicationUtils.startApp - Using settings files '/home/devel/Test3/conf/settings_HarvestControllerApplication_kblow002_h3.xml'
2021-03-11 14:45:05.413 INFO d.n.m.d.JMSMonitorRegistryClient.register - Registering this client for monitoring every 1 minutes, using hostname 'kb-test-har-004.kb.dk' and JMX/RMI ports 8150/8250
2021-03-11 14:45:05.413 INFO d.n.c.m.MBeanConnectorCreator.exposeJMXMBeanServer - Registered mbean server in registry on port 8150 communicating on port 8250 using password file 'conf/jmxremote.password'.
Service URL is service:jmx:rmi://kb-test-har-004.kb.dk:8250/jndi/rmi://kb-test-har-004.kb.dk:8150/jmxrmi
dk.netarkivet.harvester.heritrix3.HarvestControllerServer Running
2021-03-11 14:45:05.424 INFO d.n.common.utils.ApplicationUtils.logAndPrint - dk.netarkivet.harvester.heritrix3.HarvestControllerServer Running
2021-03-11 14:45:05.431 INFO d.n.c.distribute.JMSConnectionSunMQ.<init> - Creating instance of dk.netarkivet.common.distribute.JMSConnectionSunMQ
2021-03-11 14:45:05.431 DEBUG d.n.common.distribute.JMSConnection.initConnection - Initializing a JMS connection dk.netarkivet.common.distribute.JMSConnectionSunMQ
2021-03-11 14:45:05.432 INFO d.n.c.distribute.JMSConnectionSunMQ.getConnectionFactory - Establishing SunMQ JMS Connection to 'kb-test-adm-001.kb.dk:7676'
2021-03-11 14:45:05.451 INFO d.n.h.h.HarvestControllerServer.<init> - Starting HarvestControllerServer.
2021-03-11 14:45:05.451 INFO d.n.h.h.HarvestControllerServer.<init> - Bound to harvest channel 'LOWPRIORITY'
2021-03-11 14:45:05.452 INFO d.n.h.h.HarvestControllerServer.<init> - Serverdir: 'harvester_low'
2021-03-11 14:45:05.452 INFO d.n.h.h.HarvestControllerServer.<init> - Harvesting requires at least 400000000 bytes free.
2021-03-11 14:45:05.668 INFO d.n.c.d.bitrepository.HostNameUtils.getHostName - Local hostname (provided by getCanonicalHostName): kb-test-har-004.kb.dk
--------------------------------------------
STARTING MACHINE: devel@kb-test-acs-001.kb.dk
Starting all applications on: kb-test-acs-001.kb.dk
Starting linux application: IndexServerApplication
Starting linux application: ViewerProxyApplication
14:45:09,139 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [/home/devel/Test3/conf/logback_IndexServerApplication.xml] at [file:/home/devel/Test3/conf/logback_IndexServerApplication.xml]
14:45:09,535 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - debug attribute not set
14:45:09,535 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
14:45:09,542 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [STDOUT]
14:45:09,558 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
14:45:09,646 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
14:45:09,651 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [FILE]
14:45:09,695 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@789653861 - No compression will be used
14:45:09,696 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@789653861 - Will use the pattern ./log/IndexServerApplication.%d{yyyy-MM-dd}.%i.log for the active file
14:45:09,700 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@1a04f701 - The date pattern is 'yyyy-MM-dd' from file name pattern './log/IndexServerApplication.%d{yyyy-MM-dd}.%i.log'.
14:45:09,700 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@1a04f701 - Roll-over at midnight.
14:45:09,711 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@1a04f701 - Setting initial period to Thu Mar 11 14:45:09 CET 2021
14:45:09,712 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@1a04f701 - SizeAndTimeBasedFNATP is deprecated. Use SizeAndTimeBasedRollingPolicy instead
14:45:09,712 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@1a04f701 - For more information see http://logback.qos.ch/manual/appenders.html#SizeAndTimeBasedRollingPolicy
14:45:09,714 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
14:45:09,718 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - Active log file name: ./log/IndexServerApplication.log
14:45:09,718 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - File property is set to [./log/IndexServerApplication.log]
14:45:09,720 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [dk.netarkivet.monitor.logging.CachingSLF4JAppender]
14:45:10,518 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [MONITOR]
14:45:10,520 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to WARN
14:45:10,520 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[ROOT]
14:45:10,521 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [FILE] to Logger[ROOT]
14:45:10,521 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [MONITOR] to Logger[ROOT]
14:45:10,522 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [dk.netarkivet] to DEBUG
14:45:10,522 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
14:45:10,522 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@4e91d63f - Registering current configuration as safe fallback point
2021-03-11 14:45:10.528 DEBUG dk.netarkivet.common.utils.Settings.invoke0 - Value found in loaded data: SB,KB
SLF4J: A number (1) of logging calls during the initialization phase have been intercepted and are
SLF4J: now being replayed. These are subject to the filtering rules of the underlying logging system.
SLF4J: See also http://www.slf4j.org/codes.html#replay
Starting dk.netarkivet.harvester.indexserver.IndexServer
Version: 5.7-IIPCH3-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/19da50ffad622e72f8795a09e117baf44bb5256f)
2021-03-11 14:45:10.535 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Starting dk.netarkivet.harvester.indexserver.IndexServer
Version: 5.7-IIPCH3-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/19da50ffad622e72f8795a09e117baf44bb5256f)
Java VM: 1.8.0_282
2021-03-11 14:45:10.535 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Java VM: 1.8.0_282
java.home: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.282.b08-1.el7_9.x86_64/jre
2021-03-11 14:45:10.536 INFO d.n.common.utils.ApplicationUtils.logAndPrint - java.home: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.282.b08-1.el7_9.x86_64/jre
Working dir: /home/devel/Test3
2021-03-11 14:45:10.536 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Working dir: /home/devel/Test3
2021-03-11 14:45:10.536 INFO d.n.common.utils.ApplicationUtils.startApp - Using settings files '/home/devel/Test3/conf/settings_IndexServerApplication.xml'
2021-03-11 14:45:10.729 INFO d.n.m.d.JMSMonitorRegistryClient.register - Registering this client for monitoring every 1 minutes, using hostname 'kb-test-acs-001.kb.dk' and JMX/RMI ports 8351/8451
2021-03-11 14:45:10.730 INFO d.n.c.m.MBeanConnectorCreator.exposeJMXMBeanServer - Registered mbean server in registry on port 8351 communicating on port 8451 using password file 'conf/jmxremote.password'.
Service URL is service:jmx:rmi://kb-test-acs-001.kb.dk:8451/jndi/rmi://kb-test-acs-001.kb.dk:8351/jmxrmi
dk.netarkivet.harvester.indexserver.IndexServer Running
2021-03-11 14:45:10.736 INFO d.n.common.utils.ApplicationUtils.logAndPrint - dk.netarkivet.harvester.indexserver.IndexServer Running
2021-03-11 14:45:10.739 INFO d.n.c.distribute.JMSConnectionSunMQ.<init> - Creating instance of dk.netarkivet.common.distribute.JMSConnectionSunMQ
2021-03-11 14:45:10.740 DEBUG d.n.common.distribute.JMSConnection.initConnection - Initializing a JMS connection dk.netarkivet.common.distribute.JMSConnectionSunMQ
2021-03-11 14:45:10.740 INFO d.n.c.distribute.JMSConnectionSunMQ.getConnectionFactory - Establishing SunMQ JMS Connection to 'kb-test-adm-001.kb.dk:7676'
2021-03-11 14:45:10.754 INFO d.n.h.indexserver.FileBasedCache.<init> - Metadata cache for 'cdxdata' uses directory '/home/devel/Test3/cache/cdxdata'
2021-03-11 14:45:10.781 INFO d.n.c.d.bitrepository.HostNameUtils.getHostName - Local hostname (provided by getCanonicalHostName): kb-test-acs-001.kb.dk
2021-03-11 14:45:12.787 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - BitmagArcRepositoryClient will timeout on each get request after 300000 milliseconds.
2021-03-11 14:45:12.787 DEBUG d.n.common.distribute.JMSConnection.setListener - Adding dk.netarkivet.archive.arcrepository.distribute.BitmagArcRepositoryClient@7e6ef134 as listener to Test3_COMMON_THIS_REPOS_CLIENT_130_226_228_5_IS
2021-03-11 14:45:12.800 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - BitmagArcRepositoryClient listens for replies on channel '[Queue 'Test3_COMMON_THIS_REPOS_CLIENT_130_226_228_5_IS']'
2021-03-11 14:45:12.801 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - Getting bitmag config from settings.common.arcrepositoryClient.bitrepository.settingsDir=/home/devel/Test3/conf/bitmag_client_config
2021-03-11 14:45:12.801 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - Using 'Test3' as default collectionID
2021-03-11 14:45:12.806 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - Storing tempfiles in folder 'arcrepositoryTemp'
2021-03-11 14:45:12.901 DEBUG d.n.c.utils.BasicTwoWaySSLProvider.loadPrivateKey - X509CertificateHolder found
2021-03-11 14:45:12.902 DEBUG d.n.c.utils.BasicTwoWaySSLProvider.loadPrivateKey - PrivateKeyInfo found
2021-03-11 14:45:12.954 INFO d.n.h.indexserver.RawMetadataCache.<init> - Metadata cache for 'cdxdata' is fetching metadata with urls matching 'metadata://[^/]*/crawl/index/cdx.*' and mimetype matching 'application/x-cdx'. Migration of duplicate records is disabled
2021-03-11 14:45:12.955 INFO d.n.h.indexserver.FileBasedCache.<init> - Metadata cache for 'cdxindex' uses directory '/home/devel/Test3/cache/cdxindex'
2021-03-11 14:45:12.958 INFO d.n.h.indexserver.FileBasedCache.<init> - Metadata cache for 'crawllog' uses directory '/home/devel/Test3/cache/crawllog'
2021-03-11 14:45:12.959 INFO d.n.h.indexserver.RawMetadataCache.<init> - Metadata cache for 'crawllog' is fetching metadata with urls matching 'metadata://[^/]*/crawl/logs/crawl\.log.*' and mimetype matching 'text/plain'. Migration of duplicate records is disabled
2021-03-11 14:45:12.959 INFO d.n.h.indexserver.FileBasedCache.<init> - Metadata cache for 'dedupcrawllogindex' uses directory '/home/devel/Test3/cache/dedupcrawllogindex'
2021-03-11 14:45:12.960 INFO d.n.h.indexserver.FileBasedCache.<init> - Metadata cache for 'cdxdata' uses directory '/home/devel/Test3/cache/cdxdata'
2021-03-11 14:45:12.960 INFO d.n.h.indexserver.RawMetadataCache.<init> - Metadata cache for 'cdxdata' is fetching metadata with urls matching 'metadata://[^/]*/crawl/index/cdx.*' and mimetype matching 'application/x-cdx'. Migration of duplicate records is disabled
2021-03-11 14:45:12.961 INFO d.n.h.indexserver.FileBasedCache.<init> - Metadata cache for 'crawllog' uses directory '/home/devel/Test3/cache/crawllog'
2021-03-11 14:45:12.961 INFO d.n.h.indexserver.RawMetadataCache.<init> - Metadata cache for 'crawllog' is fetching metadata with urls matching 'metadata://[^/]*/crawl/logs/crawl\.log.*' and mimetype matching 'text/plain'. Migration of duplicate records is disabled
2021-03-11 14:45:12.961 INFO d.n.h.indexserver.FileBasedCache.<init> - Metadata cache for 'fullcrawllogindex' uses directory '/home/devel/Test3/cache/fullcrawllogindex'
2021-03-11 14:45:12.962 INFO d.n.h.indexserver.FileBasedCache.<init> - Metadata cache for 'cdxdata' uses directory '/home/devel/Test3/cache/cdxdata'
2021-03-11 14:45:12.962 INFO d.n.h.indexserver.RawMetadataCache.<init> - Metadata cache for 'cdxdata' is fetching metadata with urls matching 'metadata://[^/]*/crawl/index/cdx.*' and mimetype matching 'application/x-cdx'. Migration of duplicate records is disabled
2021-03-11 14:45:12.967 DEBUG d.n.h.indexserver.FileBasedCache.cache - Waiting to enter synchronization on /home/devel/Test3/cache/cdxindex/empty-cache.working
2021-03-11 14:45:12.967 DEBUG d.n.h.indexserver.FileBasedCache.cache - locking filechannel for file '/home/devel/Test3/cache/cdxindex/empty-cache.working' (thread = main)
2021-03-11 14:45:12.970 DEBUG d.n.common.utils.ProcessUtils.runProcess - Running external program: sort /home/devel/Test3/cache/cdxindex/empty-cache -o /home/devel/Test3/cache/cdxindex/empty-cache.unsorted with environment LANG=C
2021-03-11 14:45:12.982 DEBUG d.n.h.indexserver.FileBasedCache.cache - release lock on filechannel sun.nio.ch.FileChannelImpl@52851b44
2021-03-11 14:45:12.982 INFO d.n.h.indexserver.FileBasedCache.getIndex - Generated index '/home/devel/Test3/cache/cdxindex/empty-cache' of id '[]', request was for '[]'
2021-03-11 14:45:12.983 DEBUG d.n.h.indexserver.FileBasedCache.cache - Waiting to enter synchronization on /home/devel/Test3/cache/dedupcrawllogindex/empty-cache.working
2021-03-11 14:45:12.983 DEBUG d.n.h.indexserver.FileBasedCache.cache - locking filechannel for file '/home/devel/Test3/cache/dedupcrawllogindex/empty-cache.working' (thread = main)
2021-03-11 14:45:12.983 INFO d.n.h.indexserver.CrawlLogIndexCache.prepareCombine - Starting to generate dedupcrawllogindex for the 0 jobs: []
2021-03-11 14:45:12.983 INFO d.n.h.indexserver.CrawlLogIndexCache.combine - Starting combine task #1. This combines a dataset with 0 crawl logs (thread = main)
2021-03-11 14:45:13.103 INFO d.n.h.indexserver.CrawlLogIndexCache.combine - Adding the final 0 subindices to main index. Forcing index to contain max 42 files (related to combine task #1)
2021-03-11 14:45:13.110 INFO d.n.h.indexserver.CrawlLogIndexCache.combine - Adding operation completed (combine task #1)!
2021-03-11 14:45:13.111 INFO d.n.h.indexserver.CrawlLogIndexCache.combine - Closed index (related to combine task #1
2021-03-11 14:45:13.111 INFO d.n.h.indexserver.CrawlLogIndexCache.combine - Gzip-compressing the individual 2 index files of combine task # 1
2021-03-11 14:45:13.115 INFO d.n.h.indexserver.CrawlLogIndexCache.combine - Completed combine task #1 that combined a dataset with 0 crawl logs (entries in combined index: 0) - compressed index has size 72 bytes
2021-03-11 14:45:13.115 DEBUG d.n.h.indexserver.FileBasedCache.cache - release lock on filechannel sun.nio.ch.FileChannelImpl@15cafec7
2021-03-11 14:45:13.115 INFO d.n.h.indexserver.FileBasedCache.getIndex - Generated index '/home/devel/Test3/cache/dedupcrawllogindex/empty-cache' of id '[]', request was for '[]'
2021-03-11 14:45:13.116 DEBUG d.n.h.indexserver.FileBasedCache.cache - Waiting to enter synchronization on /home/devel/Test3/cache/fullcrawllogindex/empty-cache.working
2021-03-11 14:45:13.116 DEBUG d.n.h.indexserver.FileBasedCache.cache - locking filechannel for file '/home/devel/Test3/cache/fullcrawllogindex/empty-cache.working' (thread = main)
2021-03-11 14:45:13.116 INFO d.n.h.indexserver.CrawlLogIndexCache.prepareCombine - Starting to generate fullcrawllogindex for the 0 jobs: []
2021-03-11 14:45:13.116 INFO d.n.h.indexserver.CrawlLogIndexCache.combine - Starting combine task #1. This combines a dataset with 0 crawl logs (thread = main)
2021-03-11 14:45:13.118 INFO d.n.h.indexserver.CrawlLogIndexCache.combine - Adding the final 0 subindices to main index. Forcing index to contain max 42 files (related to combine task #1)
2021-03-11 14:45:13.121 INFO d.n.h.indexserver.CrawlLogIndexCache.combine - Adding operation completed (combine task #1)!
2021-03-11 14:45:13.121 INFO d.n.h.indexserver.CrawlLogIndexCache.combine - Closed index (related to combine task #1
2021-03-11 14:45:13.121 INFO d.n.h.indexserver.CrawlLogIndexCache.combine - Gzip-compressing the individual 2 index files of combine task # 1
2021-03-11 14:45:13.122 INFO d.n.h.indexserver.CrawlLogIndexCache.combine - Completed combine task #1 that combined a dataset with 0 crawl logs (entries in combined index: 0) - compressed index has size 72 bytes
2021-03-11 14:45:13.123 DEBUG d.n.h.indexserver.FileBasedCache.cache - release lock on filechannel sun.nio.ch.FileChannelImpl@cb191ca
2021-03-11 14:45:13.123 INFO d.n.h.indexserver.FileBasedCache.getIndex - Generated index '/home/devel/Test3/cache/fullcrawllogindex/empty-cache' of id '[]', request was for '[]'
14:45:09,139 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [/home/devel/Test3/conf/logback_ViewerProxyApplication.xml] at [file:/home/devel/Test3/conf/logback_ViewerProxyApplication.xml]
14:45:09,532 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - debug attribute not set
14:45:09,533 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
14:45:09,540 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [STDOUT]
14:45:09,557 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
14:45:09,649 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
14:45:09,656 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [FILE]
14:45:09,708 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@789653861 - No compression will be used
14:45:09,710 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@789653861 - Will use the pattern ./log/ViewerProxyApplication.%d{yyyy-MM-dd}.%i.log for the active file
14:45:09,716 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@1a04f701 - The date pattern is 'yyyy-MM-dd' from file name pattern './log/ViewerProxyApplication.%d{yyyy-MM-dd}.%i.log'.
14:45:09,716 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@1a04f701 - Roll-over at midnight.
14:45:09,723 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@1a04f701 - Setting initial period to Thu Mar 11 14:45:09 CET 2021
14:45:09,723 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@1a04f701 - SizeAndTimeBasedFNATP is deprecated. Use SizeAndTimeBasedRollingPolicy instead
14:45:09,723 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@1a04f701 - For more information see http://logback.qos.ch/manual/appenders.html#SizeAndTimeBasedRollingPolicy
14:45:09,727 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
14:45:09,732 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - Active log file name: ./log/ViewerProxyApplication.log
14:45:09,732 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - File property is set to [./log/ViewerProxyApplication.log]
14:45:09,734 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [dk.netarkivet.monitor.logging.CachingSLF4JAppender]
14:45:10,449 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [MONITOR]
14:45:10,451 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to WARN
14:45:10,451 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[ROOT]
14:45:10,467 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [FILE] to Logger[ROOT]
14:45:10,467 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [MONITOR] to Logger[ROOT]
14:45:10,467 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [dk.netarkivet] to DEBUG
14:45:10,467 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
14:45:10,468 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@4e91d63f - Registering current configuration as safe fallback point
2021-03-11 14:45:10.473 DEBUG dk.netarkivet.common.utils.Settings.invoke0 - Value found in loaded data: SB,KB
SLF4J: A number (1) of logging calls during the initialization phase have been intercepted and are
SLF4J: now being replayed. These are subject to the filtering rules of the underlying logging system.
SLF4J: See also http://www.slf4j.org/codes.html#replay
Starting dk.netarkivet.viewerproxy.ViewerProxy
Version: 5.7-IIPCH3-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/19da50ffad622e72f8795a09e117baf44bb5256f)
2021-03-11 14:45:10.488 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Starting dk.netarkivet.viewerproxy.ViewerProxy
Version: 5.7-IIPCH3-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/19da50ffad622e72f8795a09e117baf44bb5256f)
Java VM: 1.8.0_282
2021-03-11 14:45:10.488 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Java VM: 1.8.0_282
java.home: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.282.b08-1.el7_9.x86_64/jre
2021-03-11 14:45:10.489 INFO d.n.common.utils.ApplicationUtils.logAndPrint - java.home: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.282.b08-1.el7_9.x86_64/jre
Working dir: /home/devel/Test3
2021-03-11 14:45:10.489 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Working dir: /home/devel/Test3
2021-03-11 14:45:10.489 INFO d.n.common.utils.ApplicationUtils.startApp - Using settings files '/home/devel/Test3/conf/settings_ViewerProxyApplication.xml'
2021-03-11 14:45:10.730 INFO d.n.m.d.JMSMonitorRegistryClient.register - Registering this client for monitoring every 1 minutes, using hostname 'kb-test-acs-001.kb.dk' and JMX/RMI ports 8350/8450
2021-03-11 14:45:10.731 INFO d.n.c.m.MBeanConnectorCreator.exposeJMXMBeanServer - Registered mbean server in registry on port 8350 communicating on port 8450 using password file 'conf/jmxremote.password'.
Service URL is service:jmx:rmi://kb-test-acs-001.kb.dk:8450/jndi/rmi://kb-test-acs-001.kb.dk:8350/jmxrmi
dk.netarkivet.viewerproxy.ViewerProxy Running
2021-03-11 14:45:10.739 INFO d.n.common.utils.ApplicationUtils.logAndPrint - dk.netarkivet.viewerproxy.ViewerProxy Running
2021-03-11 14:45:10.746 INFO d.n.c.distribute.JMSConnectionSunMQ.<init> - Creating instance of dk.netarkivet.common.distribute.JMSConnectionSunMQ
2021-03-11 14:45:10.747 DEBUG d.n.common.distribute.JMSConnection.initConnection - Initializing a JMS connection dk.netarkivet.common.distribute.JMSConnectionSunMQ
2021-03-11 14:45:10.747 INFO d.n.c.distribute.JMSConnectionSunMQ.getConnectionFactory - Establishing SunMQ JMS Connection to 'kb-test-adm-001.kb.dk:7676'
2021-03-11 14:45:10.773 INFO d.n.c.d.bitrepository.HostNameUtils.getHostName - Local hostname (provided by getCanonicalHostName): kb-test-acs-001.kb.dk
2021-03-11 14:45:12.765 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - BitmagArcRepositoryClient will timeout on each get request after 300000 milliseconds.
2021-03-11 14:45:12.765 DEBUG d.n.common.distribute.JMSConnection.setListener - Adding dk.netarkivet.archive.arcrepository.distribute.BitmagArcRepositoryClient@3fc9504b as listener to Test3_COMMON_THIS_REPOS_CLIENT_130_226_228_5_VP
2021-03-11 14:45:12.788 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - BitmagArcRepositoryClient listens for replies on channel '[Queue 'Test3_COMMON_THIS_REPOS_CLIENT_130_226_228_5_VP']'
2021-03-11 14:45:12.789 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - Getting bitmag config from settings.common.arcrepositoryClient.bitrepository.settingsDir=/home/devel/Test3/conf/bitmag_client_config
2021-03-11 14:45:12.789 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - Using 'Test3' as default collectionID
2021-03-11 14:45:12.794 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - Storing tempfiles in folder 'arcrepositoryTemp'
2021-03-11 14:45:12.892 DEBUG d.n.c.utils.BasicTwoWaySSLProvider.loadPrivateKey - X509CertificateHolder found
2021-03-11 14:45:12.894 DEBUG d.n.c.utils.BasicTwoWaySSLProvider.loadPrivateKey - PrivateKeyInfo found
2021-03-11 14:45:12.938 INFO d.n.h.indexserver.FileBasedCache.<init> - Metadata cache for 'FULL_CRAWL_LOG' uses directory '/home/devel/Test3/cache/FULL_CRAWL_LOG'
2021-03-11 14:45:12.951 INFO d.n.viewerproxy.ARCArchiveAccess.<init> - Constructed instance of ARCArchiveAccess with TryToLookupUriAsFtp: false
2021-03-11 14:45:12.985 INFO dk.netarkivet.viewerproxy.WebProxy.<init> - Starting viewerproxy jetty on port 8076
--------------------------------------------
STARTING MACHINE: devel@kb-test-way-001.kb.dk
Starting all applications on: kb-test-way-001.kb.dk
Starting linux application: WaybackIndexerApplication
Starting linux application: AggregatorApplication
14:45:15,708 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [/home/devel/Test3/conf/logback_AggregatorApplication.xml] at [file:/home/devel/Test3/conf/logback_AggregatorApplication.xml]
14:45:16,375 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - debug attribute not set
14:45:16,379 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
14:45:16,393 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [STDOUT]
14:45:16,451 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
14:45:16,558 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
14:45:16,576 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [FILE]
14:45:16,683 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@793315160 - No compression will be used
14:45:16,684 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@793315160 - Will use the pattern ./log/AggregatorApplication.%d{yyyy-MM-dd}.%i.log for the active file
14:45:16,702 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@101df177 - The date pattern is 'yyyy-MM-dd' from file name pattern './log/AggregatorApplication.%d{yyyy-MM-dd}.%i.log'.
14:45:16,702 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@101df177 - Roll-over at midnight.
14:45:16,719 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@101df177 - Setting initial period to Thu Mar 11 14:45:16 CET 2021
14:45:16,719 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@101df177 - SizeAndTimeBasedFNATP is deprecated. Use SizeAndTimeBasedRollingPolicy instead
14:45:16,719 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@101df177 - For more information see http://logback.qos.ch/manual/appenders.html#SizeAndTimeBasedRollingPolicy
14:45:16,729 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
14:45:16,739 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - Active log file name: ./log/AggregatorApplication.log
14:45:16,744 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - File property is set to [./log/AggregatorApplication.log]
14:45:16,745 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [dk.netarkivet.monitor.logging.CachingSLF4JAppender]
14:45:18,435 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [MONITOR]
14:45:18,448 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to WARN
14:45:18,448 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[ROOT]
14:45:18,449 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [FILE] to Logger[ROOT]
14:45:18,449 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [MONITOR] to Logger[ROOT]
14:45:18,449 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [dk.netarkivet] to DEBUG
14:45:18,449 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
14:45:18,455 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@166fa74d - Registering current configuration as safe fallback point
2021-03-11 14:45:18.477 DEBUG dk.netarkivet.common.utils.Settings.invoke0 - Value found in loaded data: SB,KB
SLF4J: A number (1) of logging calls during the initialization phase have been intercepted and are
SLF4J: now being replayed. These are subject to the filtering rules of the underlying logging system.
SLF4J: See also http://www.slf4j.org/codes.html#replay
Starting dk.netarkivet.wayback.aggregator.AggregationWorker
Version: 5.7-IIPCH3-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/19da50ffad622e72f8795a09e117baf44bb5256f)
2021-03-11 14:45:18.535 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Starting dk.netarkivet.wayback.aggregator.AggregationWorker
Version: 5.7-IIPCH3-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/19da50ffad622e72f8795a09e117baf44bb5256f)
Java VM: 1.8.0_25
2021-03-11 14:45:18.535 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Java VM: 1.8.0_25
java.home: /usr/local/java.old/jdk1.8.0_25_x64/jre
2021-03-11 14:45:18.536 INFO d.n.common.utils.ApplicationUtils.logAndPrint - java.home: /usr/local/java.old/jdk1.8.0_25_x64/jre
Working dir: /home/devel/Test3
2021-03-11 14:45:18.540 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Working dir: /home/devel/Test3
2021-03-11 14:45:18.541 INFO d.n.common.utils.ApplicationUtils.startApp - Using settings files '/home/devel/Test3/conf/settings_AggregatorApplication.xml'
14:45:15,711 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [/home/devel/Test3/conf/logback_WaybackIndexerApplication.xml] at [file:/home/devel/Test3/conf/logback_WaybackIndexerApplication.xml]
14:45:16,263 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - debug attribute not set
14:45:16,264 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
14:45:16,299 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [STDOUT]
14:45:16,365 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
14:45:16,554 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
14:45:16,574 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [FILE]
14:45:16,679 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@2143437117 - No compression will be used
14:45:16,680 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@2143437117 - Will use the pattern ./log/WaybackIndexerApplication.%d{yyyy-MM-dd}.%i.log for the active file
14:45:16,688 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@f8c1ddd - The date pattern is 'yyyy-MM-dd' from file name pattern './log/WaybackIndexerApplication.%d{yyyy-MM-dd}.%i.log'.
14:45:16,688 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@f8c1ddd - Roll-over at midnight.
14:45:16,709 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@f8c1ddd - Setting initial period to Thu Mar 11 14:45:16 CET 2021
14:45:16,709 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@f8c1ddd - SizeAndTimeBasedFNATP is deprecated. Use SizeAndTimeBasedRollingPolicy instead
14:45:16,709 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@f8c1ddd - For more information see http://logback.qos.ch/manual/appenders.html#SizeAndTimeBasedRollingPolicy
14:45:16,715 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
14:45:16,727 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - Active log file name: ./log/WaybackIndexerApplication.log
14:45:16,727 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - File property is set to [./log/WaybackIndexerApplication.log]
14:45:16,732 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [dk.netarkivet.monitor.logging.CachingSLF4JAppender]
14:45:18,471 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [MONITOR]
14:45:18,481 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to WARN
14:45:18,481 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[ROOT]
14:45:18,482 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [FILE] to Logger[ROOT]
14:45:18,482 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [MONITOR] to Logger[ROOT]
14:45:18,482 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [dk.netarkivet] to DEBUG
14:45:18,482 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
14:45:18,488 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@70be0a2b - Registering current configuration as safe fallback point
2021-03-11 14:45:18.508 DEBUG dk.netarkivet.common.utils.Settings.invoke0 - Value found in loaded data: SB,KB
SLF4J: A number (1) of logging calls during the initialization phase have been intercepted and are
SLF4J: now being replayed. These are subject to the filtering rules of the underlying logging system.
SLF4J: See also http://www.slf4j.org/codes.html#replay
Starting dk.netarkivet.wayback.indexer.WaybackIndexer
Version: 5.7-IIPCH3-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/19da50ffad622e72f8795a09e117baf44bb5256f)
2021-03-11 14:45:18.536 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Starting dk.netarkivet.wayback.indexer.WaybackIndexer
Version: 5.7-IIPCH3-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/19da50ffad622e72f8795a09e117baf44bb5256f)
Java VM: 1.8.0_25
2021-03-11 14:45:18.537 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Java VM: 1.8.0_25
java.home: /usr/local/java.old/jdk1.8.0_25_x64/jre
2021-03-11 14:45:18.537 INFO d.n.common.utils.ApplicationUtils.logAndPrint - java.home: /usr/local/java.old/jdk1.8.0_25_x64/jre
Working dir: /home/devel/Test3
2021-03-11 14:45:18.539 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Working dir: /home/devel/Test3
2021-03-11 14:45:18.539 INFO d.n.common.utils.ApplicationUtils.startApp - Using settings files '/home/devel/Test3/conf/settings_WaybackIndexerApplication.xml'
--------------------------------------------
--------------------------------------------
STARTING MACHINE: netarkdv@sb-test-har-001.statsbiblioteket.dk
Starting all applications on: sb-test-har-001.statsbiblioteket.dk
Starting linux application: HarvestControllerApplication_sbhigh_h3
14:45:19,874 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [/home/netarkdv/Test3/conf/logback_HarvestControllerApplication_sbhigh_h3.xml] at [file:/home/netarkdv/Test3/conf/logback_HarvestControllerApplication_sbhigh_h3.xml]
14:45:20,058 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - debug attribute not set
14:45:20,061 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
14:45:20,068 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [STDOUT]
14:45:20,078 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
14:45:20,116 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
14:45:20,118 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [FILE]
14:45:20,133 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@536765369 - No compression will be used
14:45:20,134 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@536765369 - Will use the pattern ./log/HarvestControllerApplication_sbhigh_h3.%d{yyyy-MM-dd}.%i.log for the active file
14:45:20,137 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@51e5fc98 - The date pattern is 'yyyy-MM-dd' from file name pattern './log/HarvestControllerApplication_sbhigh_h3.%d{yyyy-MM-dd}.%i.log'.
14:45:20,137 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@51e5fc98 - Roll-over at midnight.
14:45:20,141 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@51e5fc98 - Setting initial period to Thu Mar 11 14:45:20 CET 2021
14:45:20,141 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@51e5fc98 - SizeAndTimeBasedFNATP is deprecated. Use SizeAndTimeBasedRollingPolicy instead
14:45:20,141 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@51e5fc98 - For more information see http://logback.qos.ch/manual/appenders.html#SizeAndTimeBasedRollingPolicy
14:45:20,144 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
14:45:20,146 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - Active log file name: ./log/HarvestControllerApplication_sbhigh_h3.log
14:45:20,146 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - File property is set to [./log/HarvestControllerApplication_sbhigh_h3.log]
14:45:20,148 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [dk.netarkivet.monitor.logging.CachingSLF4JAppender]
14:45:20,405 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [MONITOR]
14:45:20,406 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to WARN
14:45:20,406 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[ROOT]
14:45:20,407 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [FILE] to Logger[ROOT]
14:45:20,407 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [MONITOR] to Logger[ROOT]
14:45:20,408 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [dk.netarkivet] to DEBUG
14:45:20,408 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
14:45:20,409 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@7c469c48 - Registering current configuration as safe fallback point
2021-03-11 14:45:20.412 DEBUG dk.netarkivet.common.utils.Settings.invoke0 - Value found in loaded data: SB,KB
SLF4J: A number (1) of logging calls during the initialization phase have been intercepted and are
SLF4J: now being replayed. These are subject to the filtering rules of the underlying logging system.
SLF4J: See also http://www.slf4j.org/codes.html#replay
Starting dk.netarkivet.harvester.heritrix3.HarvestControllerServer
Version: 5.7-IIPCH3-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/19da50ffad622e72f8795a09e117baf44bb5256f)
2021-03-11 14:45:20.416 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Starting dk.netarkivet.harvester.heritrix3.HarvestControllerServer
Version: 5.7-IIPCH3-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/19da50ffad622e72f8795a09e117baf44bb5256f)
Java VM: 1.8.0_232
2021-03-11 14:45:20.416 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Java VM: 1.8.0_232
java.home: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.232.b09-0.el7_7.x86_64/jre
2021-03-11 14:45:20.416 INFO d.n.common.utils.ApplicationUtils.logAndPrint - java.home: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.232.b09-0.el7_7.x86_64/jre
Working dir: /home/netarkdv/Test3
2021-03-11 14:45:20.416 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Working dir: /home/netarkdv/Test3
2021-03-11 14:45:20.417 INFO d.n.common.utils.ApplicationUtils.startApp - Using settings files '/home/netarkdv/Test3/conf/settings_HarvestControllerApplication_sbhigh_h3.xml'
2021-03-11 14:45:20.577 INFO d.n.m.d.JMSMonitorRegistryClient.register - Registering this client for monitoring every 1 minutes, using hostname 'sb-test-har-001.statsbiblioteket.dk' and JMX/RMI ports 8150/8250
2021-03-11 14:45:20.577 INFO d.n.c.m.MBeanConnectorCreator.exposeJMXMBeanServer - Registered mbean server in registry on port 8150 communicating on port 8250 using password file 'conf/jmxremote.password'.
Service URL is service:jmx:rmi://sb-test-har-001.statsbiblioteket.dk:8250/jndi/rmi://sb-test-har-001.statsbiblioteket.dk:8150/jmxrmi
dk.netarkivet.harvester.heritrix3.HarvestControllerServer Running
2021-03-11 14:45:20.583 INFO d.n.common.utils.ApplicationUtils.logAndPrint - dk.netarkivet.harvester.heritrix3.HarvestControllerServer Running
2021-03-11 14:45:20.599 INFO d.n.c.distribute.JMSConnectionSunMQ.<init> - Creating instance of dk.netarkivet.common.distribute.JMSConnectionSunMQ
2021-03-11 14:45:20.599 DEBUG d.n.common.distribute.JMSConnection.initConnection - Initializing a JMS connection dk.netarkivet.common.distribute.JMSConnectionSunMQ
2021-03-11 14:45:20.600 INFO d.n.c.distribute.JMSConnectionSunMQ.getConnectionFactory - Establishing SunMQ JMS Connection to 'kb-test-adm-001.kb.dk:7676'
2021-03-11 14:45:20.604 INFO d.n.h.h.HarvestControllerServer.<init> - Starting HarvestControllerServer.
2021-03-11 14:45:20.613 INFO d.n.h.h.HarvestControllerServer.<init> - Bound to harvest channel 'HIGHPRIORITY'
2021-03-11 14:45:20.615 INFO d.n.h.h.HarvestControllerServer.<init> - Serverdir: 'harvester_high'
2021-03-11 14:45:20.615 INFO d.n.h.h.HarvestControllerServer.<init> - Harvesting requires at least 400000000 bytes free.
2021-03-11 14:45:20.755 INFO d.n.c.d.bitrepository.HostNameUtils.getHostName - Local hostname (provided by getCanonicalHostName): sb-test-har-001.statsbiblioteket.dk
2021-03-11 14:45:21.983 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - BitmagArcRepositoryClient will timeout on each get request after 300000 milliseconds.
2021-03-11 14:45:21.985 DEBUG d.n.common.distribute.JMSConnection.setListener - Adding dk.netarkivet.archive.arcrepository.distribute.BitmagArcRepositoryClient@640f11a1 as listener to Test3_COMMON_THIS_REPOS_CLIENT_130_225_27_140_HCS_SBHIGH_H3
2021-03-11 14:45:22.005 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - BitmagArcRepositoryClient listens for replies on channel '[Queue 'Test3_COMMON_THIS_REPOS_CLIENT_130_225_27_140_HCS_SBHIGH_H3']'
2021-03-11 14:45:22.007 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - Getting bitmag config from settings.common.arcrepositoryClient.bitrepository.settingsDir=/home/netarkdv/Test3/conf/bitmag_client_config
2021-03-11 14:45:22.007 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - Using 'Test3' as default collectionID
2021-03-11 14:45:22.014 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - Storing tempfiles in folder 'arcrepositoryTemp'
2021-03-11 14:45:22.089 INFO d.n.c.utils.BasicTwoWaySSLProvider.loadPrivateKey - Key file '/home/devel/extra_configs/https_config/https_key.pem' with private key and certificate does not exist!
2021-03-11 14:45:22.112 DEBUG d.n.h.h.HarvestControllerServer.<init> - Obtained JMS connection.
2021-03-11 14:45:22.112 INFO d.n.h.h.HarvestControllerServer.<init> - SEND_READY_DELAY used by HarvestControllerServer is 300
2021-03-11 14:45:22.118 INFO d.n.h.heritrix3.PostProcessing.processOldJobs - Looking for unprocessed crawldata in 'harvester_high'
2021-03-11 14:45:22.119 DEBUG d.n.common.distribute.JMSConnection.setListener - Adding dk.netarkivet.harvester.heritrix3.HarvestControllerServer@3e5499cc as listener to Test3_COMMON_HCHAN_VAL_RESP
2021-03-11 14:45:22.167 INFO d.n.h.h.HarvestControllerServer.<init> - Requested to check the validity of harvest channel 'HIGHPRIORITY'
2021-03-11 14:45:22.206 INFO d.n.h.h.HarvestControllerServer.visit - Received message stating that channel 'HIGHPRIORITY' is valid.
2021-03-11 14:45:22.209 INFO d.n.h.h.HarvestControllerServer.beginListeningIfSpaceAvailable - Starts to listen to new jobs on queue '[Queue 'Test3_COMMON_JOB_PARTIAL_HIGHPRIORITY']'
2021-03-11 14:45:22.210 DEBUG d.n.common.distribute.JMSConnection.setListener - Adding dk.netarkivet.harvester.heritrix3.HarvestControllerServer@3e5499cc as listener to Test3_COMMON_JOB_PARTIAL_HIGHPRIORITY
--------------------------------------------
Completed the script all_test.sh
Finished: SUCCESS
Console Output
Help us localize this page