p.activemq.ActiveMQMessageBus - Retrieving message consumer on destination 'topic://netarkiv-devel-kb-test-har-003.kb.dk-devel-2712' for listener 'org.bitrepository.client.conversation.mediator.CollectionBasedConversationMediator@4d5b6aac'. Key: 'topic://netarkiv-devel-kb-test-har-003.kb.dk-devel-2712#1297836716'.
08:33:14.733 DEBUG o.b.p.activemq.ActiveMQMessageBus - No consumer known. Creating new for key 'topic://netarkiv-devel-kb-test-har-003.kb.dk-devel-2712#1297836716'.
08:33:14.759 DEBUG o.b.commandline.DeleteFileCmd - Performing the DeleteFile operation.
08:33:14.765 DEBUG o.b.commandline.DeleteFileCmd - Initiating the DeleteFile conversation.
08:33:14.767 INFO o.b.m.d.ConversationBasedDeleteFileClient - Requesting the deletion of the file '1-1-20220708124645914-00000-kb-test-har-004.kb.dk.warc.gz' from the pillar 'netarkiv-disk-devel-01' with checksum 'org.bitrepository.bitrepositoryelements.ChecksumDataForFileTYPE@156b88f5[checksumSpec=org.bitrepository.bitrepositoryelements.ChecksumSpecTYPE@3bf9ce3e[checksumType=MD5, checksumSalt=<null>(default), otherChecksumType=<null>(default)], checksumValue={108,-116,71,105,33,-87,78,-3,-1,-126,16,-83,-32,87,-81,109}, calculationTimestamp=2022-07-11T08:33:14.761+02:00]', while requested checksum 'null'. And the audit trail information 'null'.
08:33:14.789 DEBUG org.bitrepository.common.JaxbHelper - Creating JAXBHelper based on schema from: jar:file:/home/devel/ResetCollection/bitrepository-client/lib/bitrepository-message-xml-xsd-30.jar!/versioning/ProtocolVersionDefinition.xsd
08:33:15.385 DEBUG o.b.p.m.l.DeleteFileMessageLogger - Sent IdentifyPillarsForDeleteFileRequest(5f39d1a9) collectionID Test6, to netarkiv-disk-devel-01, destination topic://netarkiv-devel: FileID=1-1-20220708124645914-00000-kb-test-har-004.kb.dk.warc.gz
08:33:15.386 DEBUG o.b.commandline.DeleteFileCmd - Received event: 5f39d1a9: DELETE_FILE for file 1-1-20220708124645914-00000-kb-test-har-004.kb.dk.warc.gz: IDENTIFY_REQUEST_SENT: , Identifying pillars for delete file
08:33:15.693 DEBUG o.b.p.m.l.DeleteFileMessageLogger - Received IdentifyPillarsForDeleteFileResponse(5f39d1a9) collectionID Test6, from netarkiv-kbh-disk-devel-01: IDENTIFICATION_POSITIVE(File 1-1-20220708124645914-00000-kb-test-har-004.kb.dk.warc.gz is in collection and can be deleted.)
08:33:15.707 DEBUG o.b.c.c.ConversationEventMonitor - 5f39d1a9: DELETE_FILE for file 1-1-20220708124645914-00000-kb-test-har-004.kb.dk.warc.gz:Ignoring message from irrelevant component netarkiv-kbh-disk-devel-01
08:33:15.883 DEBUG o.b.p.m.l.DeleteFileMessageLogger - Received IdentifyPillarsForDeleteFileResponse(5f39d1a9) collectionID Test6, from netarkiv-disk-devel-01: IDENTIFICATION_POSITIVE(IDENTIFICATION_POSITIVE)
08:33:15.901 DEBUG o.b.commandline.DeleteFileCmd - Received event: 5f39d1a9: DELETE_FILE for file 1-1-20220708124645914-00000-kb-test-har-004.kb.dk.warc.gz: COMPONENT_IDENTIFIED: ContributorID netarkiv-disk-devel-01
08:33:15.902 DEBUG o.b.c.c.s.ContributorResponseStatus - Received response from: netarkiv-disk-devel-01
08:33:15.902 DEBUG o.b.c.c.s.ContributorResponseStatus - Expected contributors: [netarkiv-disk-devel-01], components that have not answered: []
08:33:15.902 DEBUG o.b.commandline.DeleteFileCmd - Received event: 5f39d1a9: DELETE_FILE for file 1-1-20220708124645914-00000-kb-test-har-004.kb.dk.warc.gz: IDENTIFICATION_COMPLETE: Contributors: [netarkiv-disk-devel-01]
08:33:15.903 DEBUG o.b.commandline.DeleteFileCmd - Received event: 5f39d1a9: DELETE_FILE for file 1-1-20220708124645914-00000-kb-test-har-004.kb.dk.warc.gz: REQUEST_SENT: ContributorID [netarkiv-disk-devel-01], Sending request for deleting file
08:33:16.362 DEBUG o.b.p.m.l.DeleteFileMessageLogger - Received IdentifyPillarsForDeleteFileResponse(5f39d1a9) collectionID Test6, from netarkiv-tape-devel-01: FILE_NOT_FOUND_FAILURE(Cannot serve request, file not on store)
08:33:16.432 DEBUG o.b.p.m.l.DeleteFileMessageLogger - Sent DeleteFileRequest(5f39d1a9) collectionID Test6, to netarkiv-disk-devel-01, destination queue://netarkiv-disk-devel-01: FileID=1-1-20220708124645914-00000-kb-test-har-004.kb.dk.warc.gz, ChecksumDataForExistingFile=org.bitrepository.bitrepositoryelements.ChecksumDataForFileTYPE@156b88f5[checksumSpec=org.bitrepository.bitrepositoryelements.ChecksumSpecTYPE@3bf9ce3e[checksumType=MD5, checksumSalt=<null>(default), otherChecksumType=<null>(default)], checksumValue={108,-116,71,105,33,-87,78,-3,-1,-126,16,-83,-32,87,-81,109}, calculationTimestamp=2022-07-11T08:33:14.761+02:00]
08:33:16.432 DEBUG o.b.c.c.ConversationEventMonitor - 5f39d1a9: DELETE_FILE for file 1-1-20220708124645914-00000-kb-test-har-004.kb.dk.warc.gz:Ignoring message from irrelevant component netarkiv-tape-devel-01
08:33:16.980 DEBUG o.b.p.m.l.DeleteFileMessageLogger - Received DeleteFileProgressResponse(5f39d1a9) collectionID Test6, from netarkiv-disk-devel-01: OPERATION_ACCEPTED_PROGRESS(OPERATION_ACCEPTED_PROGRESS)
08:33:16.987 DEBUG o.b.commandline.DeleteFileCmd - Received event: 5f39d1a9: DELETE_FILE for file 1-1-20220708124645914-00000-kb-test-har-004.kb.dk.warc.gz: PROGRESS: ContributorID netarkiv-disk-devel-01, OPERATION_ACCEPTED_PROGRESS
08:33:17.292 DEBUG o.b.p.m.l.DeleteFileMessageLogger - Received DeleteFileFinalResponse(5f39d1a9) collectionID Test6, from netarkiv-disk-devel-01: OPERATION_COMPLETED(OPERATION_COMPLETED)
08:33:17.304 DEBUG o.b.c.c.s.ContributorResponseStatus - Received response from: netarkiv-disk-devel-01
08:33:17.304 DEBUG o.b.c.c.s.ContributorResponseStatus - Expected contributors: [netarkiv-disk-devel-01], components that have not answered: []
08:33:17.304 DEBUG o.b.commandline.DeleteFileCmd - Complete: 5f39d1a9: DELETE_FILE for file 1-1-20220708124645914-00000-kb-test-har-004.kb.dk.warc.gz: COMPLETE:
08:33:17.304 INFO o.b.commandline.DeleteFileCmd - Results of the DeleteFile operation for the file '1-1-20220708124645914-00000-kb-test-har-004.kb.dk.warc.gz':
08:33:17.304 DEBUG o.b.commandline.DeleteFileCmd - 5f39d1a9: DELETE_FILE for file 1-1-20220708124645914-00000-kb-test-har-004.kb.dk.warc.gz: COMPLETE:
08:33:18.441 DEBUG org.bitrepository.common.JaxbHelper - Creating JAXBHelper based on schema from: jar:file:/home/devel/ResetCollection/bitrepository-client/lib/bitrepository-repository-settings-xsd-12.jar!/xsd/RepositorySettings.xsd
08:33:18.570 DEBUG o.b.c.settings.XMLFileSettingsLoader - Loading the settings file '/home/devel/ResetCollection/bitrepository-client/bin/../conf/RepositorySettings.xml'.
08:33:18.887 DEBUG org.bitrepository.common.JaxbHelper - Creating JAXBHelper based on schema from: jar:file:/home/devel/ResetCollection/bitrepository-client/lib/bitrepository-reference-settings-1.9.0.1.jar!/xsd/ReferenceSettings.xsd
08:33:18.971 DEBUG o.b.c.settings.XMLFileSettingsLoader - Loading the settings file '/home/devel/ResetCollection/bitrepository-client/bin/../conf/ReferenceSettings.xml'.
08:33:19.981 DEBUG o.b.p.security.BasicSecurityManager - X509CertificateHolder found
08:33:20.008 DEBUG o.b.p.security.BasicSecurityManager - Got something, that we don't (yet) recognize. Class: PEMKeyPair
08:33:20.008 DEBUG o.b.p.security.BasicSecurityManager - PrivateKeyInfo found
08:33:20.283 INFO o.b.p.activemq.ActiveMQMessageBus - Initializing ActiveMQMessageBus:'org.bitrepository.settings.repositorysettings.MessageBusConfiguration@13c9d689[name=netarkiv-aar-bit-master-devel-01 broker, url=failover:(ssl://netarkiv-aar-bit-master-devel-01.statsbiblioteket.dk:61612?keepAlive=true)]'.
08:33:20.284 DEBUG org.bitrepository.common.JaxbHelper - Creating JAXBHelper based on schema from: jar:file:/home/devel/ResetCollection/bitrepository-client/lib/bitrepository-message-xml-xsd-30.jar!/xsd/BitRepositoryMessages.xsd
08:33:21.079 INFO o.a.a.t.failover.FailoverTransport - Successfully connected to ssl://netarkiv-aar-bit-master-devel-01.statsbiblioteket.dk:61612?keepAlive=true
08:33:21.178 DEBUG o.b.p.activemq.ActiveMQMessageBus - ActiveMQConnection initialized for 'org.bitrepository.settings.repositorysettings.MessageBusConfiguration@13c9d689[name=netarkiv-aar-bit-master-devel-01 broker, url=failover:(ssl://netarkiv-aar-bit-master-devel-01.statsbiblioteket.dk:61612?keepAlive=true)]'.
08:33:21.181 INFO o.b.p.activemq.ActiveMQMessageBus - Settings component filter to: [kb-test-har-003.kb.dk-devel-2792]
08:33:21.184 DEBUG o.b.c.c.m.CollectionBasedConversationMediator - Initializing the CollectionBasedConversationMediator
08:33:21.185 DEBUG o.b.p.activemq.ActiveMQMessageBus - Adding listener 'org.bitrepository.client.conversation.mediator.CollectionBasedConversationMediator@4a7f959b' to destination: 'topic://netarkiv-devel-kb-test-har-003.kb.dk-devel-2792' on message-bus 'netarkiv-aar-bit-master-devel-01 broker'.
08:33:21.185 DEBUG o.b.p.activemq.ActiveMQMessageBus - Retrieving message consumer on destination 'topic://netarkiv-devel-kb-test-har-003.kb.dk-devel-2792' for listener 'org.bitrepository.client.conversation.mediator.CollectionBasedConversationMediator@4a7f959b'. Key: 'topic://netarkiv-devel-kb-test-har-003.kb.dk-devel-2792#1249875355'.
08:33:21.186 DEBUG o.b.p.activemq.ActiveMQMessageBus - No consumer known. Creating new for key 'topic://netarkiv-devel-kb-test-har-003.kb.dk-devel-2792#1249875355'.
08:33:21.206 DEBUG o.b.commandline.DeleteFileCmd - Performing the DeleteFile operation.
08:33:21.219 DEBUG o.b.commandline.DeleteFileCmd - Initiating the DeleteFile conversation.
08:33:21.221 INFO o.b.m.d.ConversationBasedDeleteFileClient - Requesting the deletion of the file '1-1-20220708124645914-00000-kb-test-har-004.kb.dk.warc.gz' from the pillar 'netarkiv-checksum-devel-01' with checksum 'org.bitrepository.bitrepositoryelements.ChecksumDataForFileTYPE@16610890[checksumSpec=org.bitrepository.bitrepositoryelements.ChecksumSpecTYPE@71def8f8[checksumType=MD5, checksumSalt=<null>(default), otherChecksumType=<null>(default)], checksumValue={108,-116,71,105,33,-87,78,-3,-1,-126,16,-83,-32,87,-81,109}, calculationTimestamp=2022-07-11T08:33:21.209+02:00]', while requested checksum 'null'. And the audit trail information 'null'.
08:33:21.244 DEBUG org.bitrepository.common.JaxbHelper - Creating JAXBHelper based on schema from: jar:file:/home/devel/ResetCollection/bitrepository-client/lib/bitrepository-message-xml-xsd-30.jar!/versioning/ProtocolVersionDefinition.xsd
08:33:21.884 DEBUG o.b.p.m.l.DeleteFileMessageLogger - Sent IdentifyPillarsForDeleteFileRequest(de12f848) collectionID Test6, to netarkiv-checksum-devel-01, destination topic://netarkiv-devel: FileID=1-1-20220708124645914-00000-kb-test-har-004.kb.dk.warc.gz
08:33:21.885 DEBUG o.b.commandline.DeleteFileCmd - Received event: de12f848: DELETE_FILE for file 1-1-20220708124645914-00000-kb-test-har-004.kb.dk.warc.gz: IDENTIFY_REQUEST_SENT: , Identifying pillars for delete file
08:33:22.416 DEBUG o.b.p.m.l.DeleteFileMessageLogger - Received IdentifyPillarsForDeleteFileResponse(de12f848) collectionID Test6, from netarkiv-kbh-disk-devel-01: IDENTIFICATION_POSITIVE(File 1-1-20220708124645914-00000-kb-test-har-004.kb.dk.warc.gz is in collection and can be deleted.)
08:33:22.460 DEBUG o.b.c.c.ConversationEventMonitor - de12f848: DELETE_FILE for file 1-1-20220708124645914-00000-kb-test-har-004.kb.dk.warc.gz:Ignoring message from irrelevant component netarkiv-kbh-disk-devel-01
08:33:22.879 DEBUG o.b.p.m.l.DeleteFileMessageLogger - Received IdentifyPillarsForDeleteFileResponse(de12f848) collectionID Test6, from netarkiv-checksum-devel-01: IDENTIFICATION_POSITIVE(Operation acknowledged and accepted.)
08:33:22.880 DEBUG o.b.commandline.DeleteFileCmd - Received event: de12f848: DELETE_FILE for file 1-1-20220708124645914-00000-kb-test-har-004.kb.dk.warc.gz: COMPONENT_IDENTIFIED: ContributorID netarkiv-checksum-devel-01
08:33:22.880 DEBUG o.b.c.c.s.ContributorResponseStatus - Received response from: netarkiv-checksum-devel-01
08:33:22.880 DEBUG o.b.c.c.s.ContributorResponseStatus - Expected contributors: [netarkiv-checksum-devel-01], components that have not answered: []
08:33:22.881 DEBUG o.b.commandline.DeleteFileCmd - Received event: de12f848: DELETE_FILE for file 1-1-20220708124645914-00000-kb-test-har-004.kb.dk.warc.gz: IDENTIFICATION_COMPLETE: Contributors: [netarkiv-checksum-devel-01]
08:33:22.881 DEBUG o.b.commandline.DeleteFileCmd - Received event: de12f848: DELETE_FILE for file 1-1-20220708124645914-00000-kb-test-har-004.kb.dk.warc.gz: REQUEST_SENT: ContributorID [netarkiv-checksum-devel-01], Sending request for deleting file
08:33:23.240 DEBUG o.b.p.m.l.DeleteFileMessageLogger - Received IdentifyPillarsForDeleteFileResponse(de12f848) collectionID Test6, from netarkiv-disk-devel-01: FILE_NOT_FOUND_FAILURE(Cannot serve request, file not on store)
08:33:23.792 DEBUG o.b.p.m.l.DeleteFileMessageLogger - Sent DeleteFileRequest(de12f848) collectionID Test6, to netarkiv-checksum-devel-01, destination topic://netarkiv-devel-netarkiv-checksum-devel-01-contributor: FileID=1-1-20220708124645914-00000-kb-test-har-004.kb.dk.warc.gz, ChecksumDataForExistingFile=org.bitrepository.bitrepositoryelements.ChecksumDataForFileTYPE@16610890[checksumSpec=org.bitrepository.bitrepositoryelements.ChecksumSpecTYPE@71def8f8[checksumType=MD5, checksumSalt=<null>(default), otherChecksumType=<null>(default)], checksumValue={108,-116,71,105,33,-87,78,-3,-1,-126,16,-83,-32,87,-81,109}, calculationTimestamp=2022-07-11T08:33:21.209+02:00]
08:33:23.798 DEBUG o.b.c.c.ConversationEventMonitor - de12f848: DELETE_FILE for file 1-1-20220708124645914-00000-kb-test-har-004.kb.dk.warc.gz:Ignoring message from irrelevant component netarkiv-disk-devel-01
08:33:23.804 DEBUG o.b.p.m.l.DeleteFileMessageLogger - Received IdentifyPillarsForDeleteFileResponse(de12f848) collectionID Test6, from netarkiv-tape-devel-01: FILE_NOT_FOUND_FAILURE(Cannot serve request, file not on store)
08:33:23.815 DEBUG o.b.c.c.ConversationEventMonitor - de12f848: DELETE_FILE for file 1-1-20220708124645914-00000-kb-test-har-004.kb.dk.warc.gz:Ignoring message from irrelevant component netarkiv-tape-devel-01
08:33:24.187 DEBUG o.b.p.m.l.DeleteFileMessageLogger - Received DeleteFileProgressResponse(de12f848) collectionID Test6, from netarkiv-checksum-devel-01: OPERATION_ACCEPTED_PROGRESS
08:33:24.194 DEBUG o.b.commandline.DeleteFileCmd - Received event: de12f848: DELETE_FILE for file 1-1-20220708124645914-00000-kb-test-har-004.kb.dk.warc.gz: PROGRESS: ContributorID netarkiv-checksum-devel-01
08:33:24.591 DEBUG o.b.p.m.l.DeleteFileMessageLogger - Received DeleteFileFinalResponse(de12f848) collectionID Test6, from netarkiv-checksum-devel-01: OPERATION_COMPLETED
08:33:24.608 DEBUG o.b.c.c.s.ContributorResponseStatus - Received response from: netarkiv-checksum-devel-01
08:33:24.608 DEBUG o.b.c.c.s.ContributorResponseStatus - Expected contributors: [netarkiv-checksum-devel-01], components that have not answered: []
08:33:24.608 DEBUG o.b.commandline.DeleteFileCmd - Complete: de12f848: DELETE_FILE for file 1-1-20220708124645914-00000-kb-test-har-004.kb.dk.warc.gz: COMPLETE:
08:33:24.613 INFO o.b.commandline.DeleteFileCmd - Results of the DeleteFile operation for the file '1-1-20220708124645914-00000-kb-test-har-004.kb.dk.warc.gz':
08:33:24.613 DEBUG o.b.commandline.DeleteFileCmd - de12f848: DELETE_FILE for file 1-1-20220708124645914-00000-kb-test-har-004.kb.dk.warc.gz: COMPLETE:
08:33:26.121 DEBUG org.bitrepository.common.JaxbHelper - Creating JAXBHelper based on schema from: jar:file:/home/devel/ResetCollection/bitrepository-client/lib/bitrepository-repository-settings-xsd-12.jar!/xsd/RepositorySettings.xsd
08:33:26.238 DEBUG o.b.c.settings.XMLFileSettingsLoader - Loading the settings file '/home/devel/ResetCollection/bitrepository-client/bin/../conf/RepositorySettings.xml'.
08:33:26.456 DEBUG org.bitrepository.common.JaxbHelper - Creating JAXBHelper based on schema from: jar:file:/home/devel/ResetCollection/bitrepository-client/lib/bitrepository-reference-settings-1.9.0.1.jar!/xsd/ReferenceSettings.xsd
08:33:26.488 DEBUG o.b.c.settings.XMLFileSettingsLoader - Loading the settings file '/home/devel/ResetCollection/bitrepository-client/bin/../conf/ReferenceSettings.xml'.
08:33:27.033 DEBUG o.b.p.security.BasicSecurityManager - X509CertificateHolder found
08:33:27.049 DEBUG o.b.p.security.BasicSecurityManager - Got something, that we don't (yet) recognize. Class: PEMKeyPair
08:33:27.049 DEBUG o.b.p.security.BasicSecurityManager - PrivateKeyInfo found
08:33:27.253 INFO o.b.p.activemq.ActiveMQMessageBus - Initializing ActiveMQMessageBus:'org.bitrepository.settings.repositorysettings.MessageBusConfiguration@13c9d689[name=netarkiv-aar-bit-master-devel-01 broker, url=failover:(ssl://netarkiv-aar-bit-master-devel-01.statsbiblioteket.dk:61612?keepAlive=true)]'.
08:33:27.254 DEBUG org.bitrepository.common.JaxbHelper - Creating JAXBHelper based on schema from: jar:file:/home/devel/ResetCollection/bitrepository-client/lib/bitrepository-message-xml-xsd-30.jar!/xsd/BitRepositoryMessages.xsd
08:33:28.267 INFO o.a.a.t.failover.FailoverTransport - Successfully connected to ssl://netarkiv-aar-bit-master-devel-01.statsbiblioteket.dk:61612?keepAlive=true
08:33:28.379 DEBUG o.b.p.activemq.ActiveMQMessageBus - ActiveMQConnection initialized for 'org.bitrepository.settings.repositorysettings.MessageBusConfiguration@13c9d689[name=netarkiv-aar-bit-master-devel-01 broker, url=failover:(ssl://netarkiv-aar-bit-master-devel-01.statsbiblioteket.dk:61612?keepAlive=true)]'.
08:33:28.380 INFO o.b.p.activemq.ActiveMQMessageBus - Settings component filter to: [kb-test-har-003.kb.dk-devel-2880]
08:33:28.391 DEBUG o.b.c.c.m.CollectionBasedConversationMediator - Initializing the CollectionBasedConversationMediator
08:33:28.391 DEBUG o.b.p.activemq.ActiveMQMessageBus - Adding listener 'org.bitrepository.client.conversation.mediator.CollectionBasedConversationMediator@4d5b6aac' to destination: 'topic://netarkiv-devel-kb-test-har-003.kb.dk-devel-2880' on message-bus 'netarkiv-aar-bit-master-devel-01 broker'.
08:33:28.392 DEBUG o.b.p.activemq.ActiveMQMessageBus - Retrieving message consumer on destination 'topic://netarkiv-devel-kb-test-har-003.kb.dk-devel-2880' for listener 'org.bitrepository.client.conversation.mediator.CollectionBasedConversationMediator@4d5b6aac'. Key: 'topic://netarkiv-devel-kb-test-har-003.kb.dk-devel-2880#1297836716'.
08:33:28.392 DEBUG o.b.p.activemq.ActiveMQMessageBus - No consumer known. Creating new for key 'topic://netarkiv-devel-kb-test-har-003.kb.dk-devel-2880#1297836716'.
08:33:28.407 DEBUG o.b.commandline.DeleteFileCmd - Performing the DeleteFile operation.
08:33:28.412 DEBUG o.b.commandline.DeleteFileCmd - Initiating the DeleteFile conversation.
08:33:28.413 INFO o.b.m.d.ConversationBasedDeleteFileClient - Requesting the deletion of the file '1-1-20220708124645914-00000-kb-test-har-004.kb.dk.warc.gz' from the pillar 'netarkiv-kbh-disk-devel-01' with checksum 'org.bitrepository.bitrepositoryelements.ChecksumDataForFileTYPE@156b88f5[checksumSpec=org.bitrepository.bitrepositoryelements.ChecksumSpecTYPE@3bf9ce3e[checksumType=MD5, checksumSalt=<null>(default), otherChecksumType=<null>(default)], checksumValue={108,-116,71,105,33,-87,78,-3,-1,-126,16,-83,-32,87,-81,109}, calculationTimestamp=2022-07-11T08:33:28.408+02:00]', while requested checksum 'null'. And the audit trail information 'null'.
08:33:28.430 DEBUG org.bitrepository.common.JaxbHelper - Creating JAXBHelper based on schema from: jar:file:/home/devel/ResetCollection/bitrepository-client/lib/bitrepository-message-xml-xsd-30.jar!/versioning/ProtocolVersionDefinition.xsd
08:33:29.043 DEBUG o.b.p.m.l.DeleteFileMessageLogger - Sent IdentifyPillarsForDeleteFileRequest(cb39a236) collectionID Test6, to netarkiv-kbh-disk-devel-01, destination topic://netarkiv-devel: FileID=1-1-20220708124645914-00000-kb-test-har-004.kb.dk.warc.gz
08:33:29.043 DEBUG o.b.commandline.DeleteFileCmd - Received event: cb39a236: DELETE_FILE for file 1-1-20220708124645914-00000-kb-test-har-004.kb.dk.warc.gz: IDENTIFY_REQUEST_SENT: , Identifying pillars for delete file
08:33:29.549 DEBUG o.b.p.m.l.DeleteFileMessageLogger - Received IdentifyPillarsForDeleteFileResponse(cb39a236) collectionID Test6, from netarkiv-kbh-disk-devel-01: IDENTIFICATION_POSITIVE(File 1-1-20220708124645914-00000-kb-test-har-004.kb.dk.warc.gz is in collection and can be deleted.)
08:33:29.582 DEBUG o.b.commandline.DeleteFileCmd - Received event: cb39a236: DELETE_FILE for file 1-1-20220708124645914-00000-kb-test-har-004.kb.dk.warc.gz: COMPONENT_IDENTIFIED: ContributorID netarkiv-kbh-disk-devel-01
08:33:29.583 DEBUG o.b.c.c.s.ContributorResponseStatus - Received response from: netarkiv-kbh-disk-devel-01
08:33:29.583 DEBUG o.b.c.c.s.ContributorResponseStatus - Expected contributors: [netarkiv-kbh-disk-devel-01], components that have not answered: []
08:33:29.583 DEBUG o.b.commandline.DeleteFileCmd - Received event: cb39a236: DELETE_FILE for file 1-1-20220708124645914-00000-kb-test-har-004.kb.dk.warc.gz: IDENTIFICATION_COMPLETE: Contributors: [netarkiv-kbh-disk-devel-01]
08:33:29.584 DEBUG o.b.commandline.DeleteFileCmd - Received event: cb39a236: DELETE_FILE for file 1-1-20220708124645914-00000-kb-test-har-004.kb.dk.warc.gz: REQUEST_SENT: ContributorID [netarkiv-kbh-disk-devel-01], Sending request for deleting file
08:33:30.003 DEBUG o.b.p.m.l.DeleteFileMessageLogger - Received IdentifyPillarsForDeleteFileResponse(cb39a236) collectionID Test6, from netarkiv-disk-devel-01: FILE_NOT_FOUND_FAILURE(Cannot serve request, file not on store)
08:33:30.361 DEBUG o.b.p.m.l.DeleteFileMessageLogger - Sent DeleteFileRequest(cb39a236) collectionID Test6, to netarkiv-kbh-disk-devel-01, destination topic://netarkiv-devel.netarkiv-kbh-disk-devel-01: FileID=1-1-20220708124645914-00000-kb-test-har-004.kb.dk.warc.gz, ChecksumDataForExistingFile=org.bitrepository.bitrepositoryelements.ChecksumDataForFileTYPE@156b88f5[checksumSpec=org.bitrepository.bitrepositoryelements.ChecksumSpecTYPE@3bf9ce3e[checksumType=MD5, checksumSalt=<null>(default), otherChecksumType=<null>(default)], checksumValue={108,-116,71,105,33,-87,78,-3,-1,-126,16,-83,-32,87,-81,109}, calculationTimestamp=2022-07-11T08:33:28.408+02:00]
08:33:30.362 DEBUG o.b.c.c.ConversationEventMonitor - cb39a236: DELETE_FILE for file 1-1-20220708124645914-00000-kb-test-har-004.kb.dk.warc.gz:Ignoring message from irrelevant component netarkiv-disk-devel-01
08:33:30.401 DEBUG o.b.p.m.l.DeleteFileMessageLogger - Received IdentifyPillarsForDeleteFileResponse(cb39a236) collectionID Test6, from netarkiv-tape-devel-01: FILE_NOT_FOUND_FAILURE(Cannot serve request, file not on store)
08:33:30.412 DEBUG o.b.c.c.ConversationEventMonitor - cb39a236: DELETE_FILE for file 1-1-20220708124645914-00000-kb-test-har-004.kb.dk.warc.gz:Ignoring message from irrelevant component netarkiv-tape-devel-01
08:33:30.591 DEBUG o.b.p.m.l.DeleteFileMessageLogger - Received DeleteFileProgressResponse(cb39a236) collectionID Test6, from netarkiv-kbh-disk-devel-01: OPERATION_ACCEPTED_PROGRESS(Deleting..)
08:33:30.599 DEBUG o.b.commandline.DeleteFileCmd - Received event: cb39a236: DELETE_FILE for file 1-1-20220708124645914-00000-kb-test-har-004.kb.dk.warc.gz: PROGRESS: ContributorID netarkiv-kbh-disk-devel-01, Deleting..
08:33:30.960 DEBUG o.b.p.m.l.DeleteFileMessageLogger - Received DeleteFileFinalResponse(cb39a236) collectionID Test6, from netarkiv-kbh-disk-devel-01: OPERATION_COMPLETED(1-1-20220708124645914-00000-kb-test-har-004.kb.dk.warc.gz deleted.)
08:33:30.970 DEBUG o.b.c.c.s.ContributorResponseStatus - Received response from: netarkiv-kbh-disk-devel-01
08:33:30.970 DEBUG o.b.c.c.s.ContributorResponseStatus - Expected contributors: [netarkiv-kbh-disk-devel-01], components that have not answered: []
08:33:30.970 DEBUG o.b.commandline.DeleteFileCmd - Complete: cb39a236: DELETE_FILE for file 1-1-20220708124645914-00000-kb-test-har-004.kb.dk.warc.gz: COMPLETE:
08:33:30.971 INFO o.b.commandline.DeleteFileCmd - Results of the DeleteFile operation for the file '1-1-20220708124645914-00000-kb-test-har-004.kb.dk.warc.gz':
08:33:30.971 DEBUG o.b.commandline.DeleteFileCmd - cb39a236: DELETE_FILE for file 1-1-20220708124645914-00000-kb-test-har-004.kb.dk.warc.gz: COMPLETE:
calling cleanup_hosts
running cleanup_hosts
Removing installation on sb-test-har-001.statsbiblioteket.dk
Removing installation on kb-test-har-003.kb.dk
Removing installation on kb-test-har-004.kb.dk
Removing installation on kb-test-har-005.kb.dk
Removing installation on kb-test-adm-001.kb.dk
Removing installation on kb-test-acs-001.kb.dk
Removing installation on kb-test-way-001.kb.dk
calling cleanup_linuxbarhosts
running cleanup_linuxbarhosts
Removing installation on sb-test-bar-001.statsbiblioteket.dk
Removing bitarchive on sb-test-bar-001.statsbiblioteket.dk
Calling cleanup_windowsbarhosts
running cleanup_windowsbarhosts
Removing C:\bitarkiv\Test6 on kb-test-bar-01.bitarkiv.kb.dk
ssh: connect to host kb-test-bar-01.bitarkiv.kb.dk port 22: Connection timed out
Removing E:\bitarchive_1\Test6 on kb-test-bar-01.bitarkiv.kb.dk
ssh: connect to host kb-test-bar-01.bitarkiv.kb.dk port 22: Connection timed out
Removing F:\bitarchive_2\Test6 on kb-test-bar-01.bitarkiv.kb.dk
Removing g:\bitarchive_3\Test6 on kb-test-bar-01.bitarkiv.kb.dk
ssh: connect to host kb-test-bar-01.bitarkiv.kb.dk port 22: Connection timed out
ssh: connect to host kb-test-bar-01.bitarkiv.kb.dk port 22: Connection timed out
Removing C:\Users\ba-devel\Test6 on kb-test-bar-01.bitarkiv.kb.dk
ssh: connect to host kb-test-bar-01.bitarkiv.kb.dk port 22: Connection timed out
TESTX=Test6
PORT=8078
MAILRECEIVERS=csr@kb.dk
VERSION=CSR
DEPLOYCONF=deploy_config_bitmag.xml
H3ZIP=CSR
BITMAGCONF=/home/devel/bitmag_test_config/client_config
Using bundler file /home/devel/nas_versions/bundler/NetarchiveSuite-heritrix3-bundler-CSR.zip
Creating distribution files in /home/devel/prepared_software/Test6.
Will read source files using encoding 'UTF-8' (defaulted)
Branch in settings not found: common.warcRecordService
Branch in settings not found: common.arcrepositoryClient.bitrepository.getFileIDsMaxResults
Branch in settings not found: common.hadoop.kerberos
Branch in settings not found: common.hadoop.username
Branch in settings not found: common.hadoop.mapred.hadoopUberJar
Branch in settings not found: common.hadoop.mapred.inputFilesParentDir
Branch in settings not found: common.hadoop.mapred.cdxJob
Branch in settings not found: common.hadoop.mapred.metadataExtractionJob
Branch in settings not found: common.hadoop.mapred.metadataCDXExtractionJob
Branch in settings not found: common.hadoop.mapred.crawlLogExtractionJob
Branch in settings not found: common.fileResolver.class
Branch in settings not found: common.fileResolver.baseUrl
Branch in settings not found: common.fileResolver.keyfile
Branch in settings not found: common.webinterface.metadata_cache_dir
Branch in settings not found: harvester.harvesting.heritrix3.bundle
Branch in settings not found: harvester.harvesting.heritrix3.certificate
Branch in settings not found: harvester.harvesting.heritrix3.bundle
Branch in settings not found: harvester.harvesting.heritrix3.certificate
Branch in settings not found: harvester.harvesting.umbra
Branch in settings not found: harvester.harvesting.heritrix3.bundle
Branch in settings not found: harvester.harvesting.heritrix3.certificate
Branch in settings not found: harvester.harvesting.umbra
Branch in settings not found: harvester.harvesting.heritrix3.bundle
Branch in settings not found: harvester.harvesting.heritrix3.bundle
Machine 'kb-test-adm-001.kb.dk' configured with encoding 'UTF-8' (defaulted)
Machine 'kb-test-har-003.kb.dk' configured with encoding 'UTF-8' (defaulted)
NetarchiveSuite-heritrix3-bundler.zip -> /home/devel/Test6/NetarchiveSuite-heritrix3-bundler.zip
/home/devel/quickstart-nas-5.0-h3/h3server.jks -> /home/devel/Test6/h3server.jks
Machine 'kb-test-har-005.kb.dk' configured with encoding 'UTF-8' (defaulted)
NetarchiveSuite-heritrix3-bundler.zip -> /home/devel/Test6/NetarchiveSuite-heritrix3-bundler.zip
/home/devel/quickstart-nas-5.0-h3/h3server.jks -> /home/devel/Test6/h3server.jks
Machine 'kb-test-har-004.kb.dk' configured with encoding 'UTF-8' (defaulted)
NetarchiveSuite-heritrix3-bundler.zip -> /home/devel/Test6/NetarchiveSuite-heritrix3-bundler.zip
Machine 'kb-test-acs-001.kb.dk' configured with encoding 'UTF-8' (defaulted)
Machine 'kb-test-way-001.kb.dk' configured with encoding 'UTF-8' (defaulted)
Machine 'sb-test-har-001.statsbiblioteket.dk' configured with encoding 'UTF-8' (defaulted)
/home/devel/prepared_software/Test6/NetarchiveSuite-heritrix3-bundler.zip -> /home/netarkdv/Test6/NetarchiveSuite-heritrix3-bundler.zip
08:39:20.158 [main] DEBUG dk.netarkivet.deploy.XmlStructure - Element baseDir is not a branch in the tree [archive, bitpreservation, baseDir]. Null returned
08:39:20.168 [main] DEBUG dk.netarkivet.deploy.XmlStructure - Element is not text. Null returned. Path given to getLeafValue archive
08:39:20.173 [main] DEBUG dk.netarkivet.deploy.XmlStructure - Element baseDir is not a branch in the tree [archive, bitpreservation, baseDir]. Null returned
08:39:20.173 [main] DEBUG dk.netarkivet.deploy.XmlStructure - Element is not text. Null returned. Path given to getLeafValue archive
08:39:20.175 [main] DEBUG dk.netarkivet.deploy.XmlStructure - Element baseDir is not a branch in the tree [archive, bitpreservation, baseDir]. Null returned
08:39:20.176 [main] DEBUG dk.netarkivet.deploy.XmlStructure - Element is not text. Null returned. Path given to getLeafValue archive
08:39:20.177 [main] DEBUG dk.netarkivet.deploy.XmlStructure - Element baseDir is not a branch in the tree [archive, bitpreservation, baseDir]. Null returned
08:39:20.177 [main] DEBUG dk.netarkivet.deploy.XmlStructure - Element is not text. Null returned. Path given to getLeafValue archive
08:39:20.179 [main] DEBUG dk.netarkivet.deploy.XmlStructure - Element baseDir is not a branch in the tree [archive, bitpreservation, baseDir]. Null returned
08:39:20.179 [main] DEBUG dk.netarkivet.deploy.XmlStructure - Element is not text. Null returned. Path given to getLeafValue archive
08:39:20.181 [main] DEBUG dk.netarkivet.deploy.XmlStructure - Element baseDir is not a branch in the tree [archive, bitpreservation, baseDir]. Null returned
08:39:20.182 [main] DEBUG dk.netarkivet.deploy.XmlStructure - Element is not text. Null returned. Path given to getLeafValue archive
08:39:20.316 [main] DEBUG dk.netarkivet.deploy.XmlStructure - Element baseDir is not a branch in the tree [archive, bitpreservation, baseDir]. Null returned
08:39:20.317 [main] DEBUG dk.netarkivet.deploy.XmlStructure - Element is not text. Null returned. Path given to getLeafValue archive
--------------------------------------------
INSTALLING TO MACHINE: devel@kb-test-adm-001.kb.dk
copying NetarchiveSuite-CSR.zip to:kb-test-adm-001.kb.dk
deleting devel@kb-test-adm-001.kb.dk:/home/devel/Test6/lib
unzipping NetarchiveSuite-CSR.zip at:kb-test-adm-001.kb.dk
Creating directories.
preparing for copying of settings and scripts
copying settings and scripts
make scripts executable
make password and access files readonly
--------------------------------------------
INSTALLING TO MACHINE: devel@kb-test-har-003.kb.dk
copying NetarchiveSuite-CSR.zip to:kb-test-har-003.kb.dk
deleting devel@kb-test-har-003.kb.dk:/home/devel/Test6/lib
unzipping NetarchiveSuite-CSR.zip at:kb-test-har-003.kb.dk
Creating directories.
preparing for copying of settings and scripts
copying settings and scripts
make scripts executable
make password and access files readonly
--------------------------------------------
INSTALLING TO MACHINE: devel@kb-test-har-005.kb.dk
copying NetarchiveSuite-CSR.zip to:kb-test-har-005.kb.dk
deleting devel@kb-test-har-005.kb.dk:/home/devel/Test6/lib
unzipping NetarchiveSuite-CSR.zip at:kb-test-har-005.kb.dk
Creating directories.
preparing for copying of settings and scripts
copying settings and scripts
make scripts executable
make password and access files readonly
--------------------------------------------
INSTALLING TO MACHINE: devel@kb-test-har-004.kb.dk
copying NetarchiveSuite-CSR.zip to:kb-test-har-004.kb.dk
deleting devel@kb-test-har-004.kb.dk:/home/devel/Test6/lib
unzipping NetarchiveSuite-CSR.zip at:kb-test-har-004.kb.dk
Creating directories.
preparing for copying of settings and scripts
copying settings and scripts
make scripts executable
make password and access files readonly
--------------------------------------------
INSTALLING TO MACHINE: devel@kb-test-acs-001.kb.dk
copying NetarchiveSuite-CSR.zip to:kb-test-acs-001.kb.dk
deleting devel@kb-test-acs-001.kb.dk:/home/devel/Test6/lib
unzipping NetarchiveSuite-CSR.zip at:kb-test-acs-001.kb.dk
Creating directories.
preparing for copying of settings and scripts
copying settings and scripts
make scripts executable
make password and access files readonly
--------------------------------------------
INSTALLING TO MACHINE: devel@kb-test-way-001.kb.dk
copying NetarchiveSuite-CSR.zip to:kb-test-way-001.kb.dk
deleting devel@kb-test-way-001.kb.dk:/home/devel/Test6/lib
unzipping NetarchiveSuite-CSR.zip at:kb-test-way-001.kb.dk
Creating directories.
preparing for copying of settings and scripts
copying settings and scripts
make scripts executable
make password and access files readonly
--------------------------------------------
--------------------------------------------
INSTALLING TO MACHINE: netarkdv@sb-test-har-001.statsbiblioteket.dk
copying NetarchiveSuite-CSR.zip to:sb-test-har-001.statsbiblioteket.dk
deleting netarkdv@sb-test-har-001.statsbiblioteket.dk:/home/netarkdv/Test6/lib
unzipping NetarchiveSuite-CSR.zip at:sb-test-har-001.statsbiblioteket.dk
Creating directories.
preparing for copying of settings and scripts
copying settings and scripts
make scripts executable
make password and access files readonly
--------------------------------------------
Initializing database for system Test6
+ host=devel@kb-test-adm-001
+ scp -p /home/devel/devel-config/scripts/db/setup_admindb.sh /home/devel/devel-config/scripts/db/setup_harvestdb.sh /home/devel/devel-config/scripts/db/teardown_admindb.sh /home/devel/devel-config/scripts/db/teardown_harvestdb.sh /home/devel/devel-config/scripts/../resources/harvestdb.testdata.sql /home/devel/devel-config/scripts/../resources/default_orderxml.cxml devel@kb-test-adm-001:bin/
+ ssh devel@kb-test-adm-001
Pseudo-terminal will not be allocated because stdin is not a terminal.
Dette system må kun tilgås af brugere autoriserede af Det Kgl. Bibliotek.
This system is only to be used by users authorized by
the Royal Danish Library.
Destroying test6_admindb
DROP DATABASE
Creating empty test6_admindb
CREATE DATABASE
Initialising test6_admindb
CREATE TABLE
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
CREATE TABLE
CREATE SEQUENCE
ALTER TABLE
CREATE TABLE
CREATE SEQUENCE
ALTER TABLE
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE TABLE
CREATE SEQUENCE
ALTER TABLE
CREATE INDEX
CREATE TABLE
CREATE SEQUENCE
ALTER TABLE
Destroying test6_harvestdb
DROP DATABASE
CREATE DATABASE
CREATE TABLE
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
GRANT
CREATE TABLE
CREATE SEQUENCE
ALTER TABLE
CREATE INDEX
CREATE INDEX
GRANT
GRANT
CREATE TABLE
CREATE SEQUENCE
ALTER TABLE
CREATE INDEX
CREATE INDEX
CREATE INDEX
GRANT
GRANT
CREATE TABLE
GRANT
CREATE TABLE
GRANT
CREATE TABLE
CREATE SEQUENCE
ALTER TABLE
CREATE INDEX
CREATE INDEX
GRANT
GRANT
CREATE TABLE
CREATE SEQUENCE
ALTER TABLE
CREATE INDEX
CREATE INDEX
GRANT
GRANT
CREATE TABLE
CREATE SEQUENCE
ALTER TABLE
CREATE INDEX
GRANT
GRANT
CREATE TABLE
CREATE SEQUENCE
ALTER TABLE
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
GRANT
GRANT
CREATE TABLE
CREATE INDEX
GRANT
CREATE TABLE
GRANT
CREATE TABLE
CREATE INDEX
GRANT
CREATE TABLE
GRANT
CREATE TABLE
CREATE SEQUENCE
ALTER TABLE
GRANT
GRANT
CREATE TABLE
CREATE SEQUENCE
ALTER TABLE
GRANT
GRANT
CREATE TABLE
CREATE INDEX
CREATE INDEX
GRANT
CREATE TABLE
CREATE INDEX
GRANT
CREATE TABLE
CREATE SEQUENCE
ALTER TABLE
CREATE INDEX
GRANT
GRANT
CREATE TABLE
CREATE SEQUENCE
ALTER TABLE
CREATE INDEX
CREATE INDEX
GRANT
GRANT
CREATE TABLE
CREATE INDEX
CREATE INDEX
CREATE INDEX
GRANT
CREATE TABLE
CREATE INDEX
CREATE INDEX
GRANT
CREATE TABLE
GRANT
CREATE TABLE
CREATE SEQUENCE
ALTER TABLE
GRANT
CREATE TABLE
CREATE SEQUENCE
ALTER TABLE
GRANT
CREATE TABLE
CREATE SEQUENCE
ALTER TABLE
GRANT
INSERT 0 1
INSERT 0 1
CREATE TABLE
CREATE SEQUENCE
ALTER TABLE
CREATE INDEX
GRANT
GRANT
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE INDEX
CREATE SEQUENCE
CREATE TABLE
ALTER TABLE
CREATE INDEX
GRANT
GRANT
GRANT
INSERT 0 1
INSERT 0 1
INSERT 0 1
SET
SET
SET
SET
SET
SET
SET
SET
SET
COPY 16
COPY 16
setval
--------
17
(1 row)
COPY 16
setval
--------
17
(1 row)
COPY 3
setval
--------
17
(1 row)
COPY 2
setval
--------
5
(1 row)
COPY 4
setval
--------
4
(1 row)
COPY 25
COPY 16
setval
--------
17
(1 row)
08:41:09.483 [MLog-Init-Reporter] INFO com.mchange.v2.log.MLog - MLog clients using slf4j logging.
08:41:09.510 [MLog-Init-Reporter] DEBUG com.mchange.v2.log.MLog - Reading VM config for path list /com/mchange/v2/log/default-mchange-log.properties, /mchange-commons.properties, /c3p0.properties, hocon:/reference,/application,/c3p0,/, /mchange-log.properties, /
08:41:09.510 [MLog-Init-Reporter] DEBUG com.mchange.v2.log.MLog - The configuration file for resource identifier '/mchange-commons.properties' could not be found. Skipping.
08:41:09.510 [MLog-Init-Reporter] DEBUG com.mchange.v2.log.MLog - The configuration file for resource identifier '/c3p0.properties' could not be found. Skipping.
08:41:09.511 [MLog-Init-Reporter] DEBUG com.mchange.v2.log.MLog - The configuration file for resource identifier 'hocon:/reference,/application,/c3p0,/' could not be found. Skipping.
08:41:09.511 [MLog-Init-Reporter] DEBUG com.mchange.v2.log.MLog - The configuration file for resource identifier '/mchange-log.properties' could not be found. Skipping.
08:41:09.517 [main] DEBUG com.mchange.v2.cfg.MConfig - The configuration file for resource identifier '/mchange-commons.properties' could not be found. Skipping.
08:41:09.517 [main] DEBUG com.mchange.v2.cfg.MConfig - The configuration file for resource identifier '/mchange-log.properties' could not be found. Skipping.
08:41:09.517 [main] DEBUG com.mchange.v2.cfg.MConfig - The configuration file for resource identifier 'hocon:/reference,/application,/c3p0,/' could not be found. Skipping.
08:41:09.517 [main] DEBUG com.mchange.v2.cfg.MConfig - The configuration file for resource identifier '/c3p0.properties' could not be found. Skipping.
08:41:09.832 [main] INFO com.mchange.v2.c3p0.C3P0Registry - Initializing c3p0-0.9.5.4 [built 23-March-2019 23:00:48 -0700; debug? true; trace: 10]
08:41:09.913 [main] DEBUG com.mchange.v2.c3p0.management.DynamicPooledDataSourceManagerMBean - MBean: com.mchange.v2.c3p0:type=PooledDataSource,identityToken=10bdx52ap1th8p1g94ymvt|b9afc07,name=10bdx52ap1th8p1g94ymvt|b9afc07 registered.
08:41:09.970 [main] DEBUG com.mchange.v2.c3p0.management.DynamicPooledDataSourceManagerMBean - MBean: com.mchange.v2.c3p0:type=PooledDataSource,identityToken=10bdx52ap1th8p1g94ymvt|b9afc07,name=10bdx52ap1th8p1g94ymvt|b9afc07 unregistered, in order to be reregistered after update.
08:41:09.977 [main] DEBUG com.mchange.v2.c3p0.management.DynamicPooledDataSourceManagerMBean - MBean: com.mchange.v2.c3p0:type=PooledDataSource,identityToken=10bdx52ap1th8p1g94ymvt|b9afc07,name=10bdx52ap1th8p1g94ymvt|b9afc07 registered.
08:41:10.007 [main] INFO dk.netarkivet.harvester.datamodel.HarvestDBConnection - Connection pool initialized with the following values:
- minPoolSize=5
- maxPoolSize=20
- acquireIncrement=5
- maxStatements=0
- maxStatementsPerConnection=0
- idleConnTestPeriod=0
- idleConnTestQuery='null'
- idleConnTestOnCheckin=false
08:41:10.077 [main] INFO com.mchange.v2.c3p0.impl.AbstractPoolBackedDataSource - Initializing c3p0 pool... com.mchange.v2.c3p0.ComboPooledDataSource [ acquireIncrement -> 5, acquireRetryAttempts -> 30, acquireRetryDelay -> 1000, autoCommitOnClose -> false, automaticTestTable -> null, breakAfterAcquireFailure -> false, checkoutTimeout -> 0, connectionCustomizerClassName -> null, connectionTesterClassName -> com.mchange.v2.c3p0.impl.DefaultConnectionTester, contextClassLoaderSource -> caller, dataSourceName -> 10bdx52ap1th8p1g94ymvt|b9afc07, debugUnreturnedConnectionStackTraces -> false, description -> null, driverClass -> org.postgresql.Driver, extensions -> {}, factoryClassLocation -> null, forceIgnoreUnresolvedTransactions -> false, forceSynchronousCheckins -> false, forceUseNamedDriverClass -> false, identityToken -> 10bdx52ap1th8p1g94ymvt|b9afc07, idleConnectionTestPeriod -> 0, initialPoolSize -> 3, jdbcUrl -> jdbc:postgresql://localhost:5432/test6_harvestdb, maxAdministrativeTaskTime -> 0, maxConnectionAge -> 3600, maxIdleTime -> 0, maxIdleTimeExcessConnections -> 0, maxPoolSize -> 20, maxStatements -> 0, maxStatementsPerConnection -> 0, minPoolSize -> 5, numHelperThreads -> 3, preferredTestQuery -> null, privilegeSpawnedThreads -> false, properties -> {user=******, password=******}, propertyCycle -> 0, statementCacheNumDeferredCloseThreads -> 0, testConnectionOnCheckin -> false, testConnectionOnCheckout -> false, unreturnedConnectionTimeout -> 0, userOverrides -> {}, usesTraditionalReflectiveProxies -> false ]
08:41:10.095 [main] DEBUG com.mchange.v2.cfg.MConfig - The configuration file for resource identifier '/mchange-commons.properties' could not be found. Skipping.
08:41:10.095 [main] DEBUG com.mchange.v2.cfg.MConfig - The configuration file for resource identifier '/mchange-log.properties' could not be found. Skipping.
08:41:10.095 [main] DEBUG com.mchange.v2.cfg.MConfig - The configuration file for resource identifier '/c3p0.properties' could not be found. Skipping.
08:41:10.095 [main] DEBUG com.mchange.v2.cfg.MConfig - The configuration file for resource identifier 'hocon:/reference,/application,/c3p0,/' could not be found. Skipping.
08:41:10.095 [main] WARN com.mchange.v2.resourcepool.BasicResourcePool - Bad pool size config, start 3 < min 5. Using 5 as start.
08:41:10.120 [main] DEBUG com.mchange.v2.resourcepool.BasicResourcePool - com.mchange.v2.resourcepool.BasicResourcePool@1f0f1111 config: [start -> 5; min -> 5; max -> 20; inc -> 5; num_acq_attempts -> 30; acq_attempt_delay -> 1000; check_idle_resources_delay -> 0; max_resource_age -> 3600000; max_idle_time -> 0; excess_max_idle_time -> 0; destroy_unreturned_resc_time -> 0; expiration_enforcement_delay -> 900000; break_on_acquisition_failure -> false; debug_store_checkout_exceptions -> false; force_synchronous_checkins -> false]
08:41:10.120 [main] DEBUG com.mchange.v2.c3p0.impl.C3P0PooledConnectionPoolManager - Created new pool for auth, username (masked): 'de******'.
08:41:10.120 [main] DEBUG com.mchange.v2.resourcepool.BasicResourcePool - acquire test -- pool size: 0; target_pool_size: 5; desired target? 1
08:41:10.120 [main] DEBUG com.mchange.v2.resourcepool.BasicResourcePool - awaitAvailable(): [unknown]
08:41:10.296 [main] DEBUG com.mchange.v2.c3p0.impl.NewProxyPreparedStatement - com.mchange.v2.c3p0.impl.NewProxyPreparedStatement@17046283 [wrapping: SELECT version FROM schemaversions WHERE tablename = 'ordertemplates'] closed orphaned ResultSet: com.mchange.v2.c3p0.impl.NewProxyResultSet@70e8f8e [wrapping: null]
08:41:10.306 [main] DEBUG com.mchange.v2.c3p0.impl.NewProxyPreparedStatement - com.mchange.v2.c3p0.impl.NewProxyPreparedStatement@10e41621 [wrapping: SELECT COUNT(*) FROM ordertemplates WHERE name = 'default_orderxml'] closed orphaned ResultSet: com.mchange.v2.c3p0.impl.NewProxyResultSet@19bb07ed [wrapping: null]
08:41:10.328 [main] DEBUG com.mchange.v2.c3p0.impl.NewProxyPreparedStatement - com.mchange.v2.c3p0.impl.NewProxyPreparedStatement@6a28ffa4 [wrapping: SELECT COUNT(*) FROM ordertemplates WHERE name = 'default_orderxml'] closed orphaned ResultSet: com.mchange.v2.c3p0.impl.NewProxyResultSet@9353778 [wrapping: null]
08:41:10.336 [main] DEBUG com.mchange.v2.c3p0.impl.NewProxyConnection - com.mchange.v2.c3p0.impl.NewProxyConnection@21de60b4 [wrapping: org.postgresql.jdbc.PgConnection@c267ef4]: close() called more than once.
The template 'default_orderxml' has now been updated.
+ set +x
--------------------------------------------
STARTING MACHINE: devel@kb-test-adm-001.kb.dk
Starting all applications on: kb-test-adm-001.kb.dk
Starting linux application: GUIApplication
Starting linux application: ArcRepositoryApplication
Starting linux application: HarvestJobManagerApplication
08:41:11,973 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [/home/devel/Test6/conf/logback_ArcRepositoryApplication.xml] at [file:/home/devel/Test6/conf/logback_ArcRepositoryApplication.xml]
08:41:12,613 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - debug attribute not set
08:41:12,616 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
08:41:12,634 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [STDOUT]
08:41:12,661 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
08:41:12,901 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
08:41:12,957 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [FILE]
08:41:13,022 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@1254344205 - No compression will be used
08:41:13,024 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@1254344205 - Will use the pattern ./log/ArcRepositoryApplication.%d{yyyy-MM-dd}.%i.log for the active file
08:41:13,030 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@4facf68f - The date pattern is 'yyyy-MM-dd' from file name pattern './log/ArcRepositoryApplication.%d{yyyy-MM-dd}.%i.log'.
08:41:13,030 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@4facf68f - Roll-over at midnight.
08:41:13,037 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@4facf68f - Setting initial period to Mon Jul 11 08:41:13 CEST 2022
08:41:13,037 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@4facf68f - SizeAndTimeBasedFNATP is deprecated. Use SizeAndTimeBasedRollingPolicy instead
08:41:13,037 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@4facf68f - For more information see http://logback.qos.ch/manual/appenders.html#SizeAndTimeBasedRollingPolicy
08:41:13,039 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
08:41:13,043 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - Active log file name: ./log/ArcRepositoryApplication.log
08:41:13,043 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - File property is set to [./log/ArcRepositoryApplication.log]
08:41:13,044 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [dk.netarkivet.monitor.logging.CachingSLF4JAppender]
08:41:14,576 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [MONITOR]
08:41:14,589 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to WARN
08:41:14,589 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[ROOT]
08:41:14,591 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [FILE] to Logger[ROOT]
08:41:14,591 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [MONITOR] to Logger[ROOT]
08:41:14,594 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [dk.netarkivet] to DEBUG
08:41:14,594 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
08:41:14,597 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@76508ed1 - Registering current configuration as safe fallback point
2022-07-11 08:41:14.614 DEBUG dk.netarkivet.common.utils.Settings.invoke0 - Value found in loaded data: SB,KB
SLF4J: A number (1) of logging calls during the initialization phase have been intercepted and are
SLF4J: now being replayed. These are subject to the filtering rules of the underlying logging system.
SLF4J: See also http://www.slf4j.org/codes.html#replay
Starting dk.netarkivet.archive.arcrepository.ArcRepository
Version: 7.4-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/5605ede33d8e9feaccf41bc5ab16c9b6dc7a8800)
2022-07-11 08:41:14.648 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Starting dk.netarkivet.archive.arcrepository.ArcRepository
Version: 7.4-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/5605ede33d8e9feaccf41bc5ab16c9b6dc7a8800)
Java VM: 1.8.0_322
2022-07-11 08:41:14.654 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Java VM: 1.8.0_322
java.home: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.322.b06-2.el8_5.x86_64/jre
2022-07-11 08:41:14.660 INFO d.n.common.utils.ApplicationUtils.logAndPrint - java.home: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.322.b06-2.el8_5.x86_64/jre
Working dir: /home/devel/Test6
2022-07-11 08:41:14.661 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Working dir: /home/devel/Test6
2022-07-11 08:41:14.665 INFO d.n.common.utils.ApplicationUtils.startApp - Using settings files '/home/devel/Test6/conf/settings_ArcRepositoryApplication.xml'
2022-07-11 08:41:15.059 INFO d.n.m.d.JMSMonitorRegistryClient.register - Registering this client for monitoring every 1 minutes, using hostname 'kb-test-adm-001.kb.dk' and JMX/RMI ports 8171/8271
2022-07-11 08:41:15.059 INFO d.n.c.m.MBeanConnectorCreator.exposeJMXMBeanServer - Registered mbean server in registry on port 8171 communicating on port 8271 using password file 'conf/jmxremote.password'.
Service URL is service:jmx:rmi://kb-test-adm-001.kb.dk:8271/jndi/rmi://kb-test-adm-001.kb.dk:8171/jmxrmi
dk.netarkivet.archive.arcrepository.ArcRepository Running
2022-07-11 08:41:15.069 INFO d.n.common.utils.ApplicationUtils.logAndPrint - dk.netarkivet.archive.arcrepository.ArcRepository Running
2022-07-11 08:41:15.119 INFO d.n.c.distribute.JMSConnectionSunMQ.<init> - Creating instance of dk.netarkivet.common.distribute.JMSConnectionSunMQ
2022-07-11 08:41:15.125 DEBUG d.n.common.distribute.JMSConnection.initConnection - Initializing a JMS connection dk.netarkivet.common.distribute.JMSConnectionSunMQ
2022-07-11 08:41:15.125 INFO d.n.c.distribute.JMSConnectionSunMQ.getConnectionFactory - Establishing SunMQ JMS Connection to 'kb-test-adm-001.kb.dk:7676'
2022-07-11 08:41:15.721 INFO d.n.a.a.ArchiveDBConnection.initDataSource - Using jdbc url: jdbc:postgresql://localhost:5432/test6_admindb
2022-07-11 08:41:15.737 INFO d.n.a.a.ArchiveDBConnection.initDataSource - Connection pool initialized with the following values:
- minPoolSize=5
- maxPoolSize=10
- acquireIncrement=5
- maxStatements=0
- maxStatementsPerConnection=0
- idleConnTestPeriod=30
- idleConnTestQuery='SELECT COUNT(*) FROM replica'
- idleConnTestOnCheckin=false
2022-07-11 08:41:15.984 WARN c.m.v.resourcepool.BasicResourcePool.log - Bad pool size config, start 3 < min 5. Using 5 as start.
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/devel/Test6/lib/logback-classic-1.2.3.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/devel/Test6/lib/slf4j-log4j12-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
08:41:12,698 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [/home/devel/Test6/conf/logback_GUIApplication.xml] at [file:/home/devel/Test6/conf/logback_GUIApplication.xml]
08:41:13,276 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - debug attribute not set
08:41:13,282 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
08:41:13,306 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [STDOUT]
08:41:13,347 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
08:41:13,521 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
08:41:13,570 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [FILE]
08:41:13,658 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@798981583 - No compression will be used
08:41:13,663 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@798981583 - Will use the pattern ./log/GUIApplication.%d{yyyy-MM-dd}.%i.log for the active file
08:41:13,675 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@747ddf94 - The date pattern is 'yyyy-MM-dd' from file name pattern './log/GUIApplication.%d{yyyy-MM-dd}.%i.log'.
08:41:13,675 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@747ddf94 - Roll-over at midnight.
08:41:13,684 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@747ddf94 - Setting initial period to Mon Jul 11 08:41:13 CEST 2022
08:41:13,685 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@747ddf94 - SizeAndTimeBasedFNATP is deprecated. Use SizeAndTimeBasedRollingPolicy instead
08:41:13,685 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@747ddf94 - For more information see http://logback.qos.ch/manual/appenders.html#SizeAndTimeBasedRollingPolicy
08:41:13,691 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
08:41:13,706 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - Active log file name: ./log/GUIApplication.log
08:41:13,706 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - File property is set to [./log/GUIApplication.log]
08:41:13,709 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [dk.netarkivet.monitor.logging.CachingSLF4JAppender]
08:41:15,317 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [MONITOR]
08:41:15,323 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to WARN
08:41:15,323 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[ROOT]
08:41:15,328 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [FILE] to Logger[ROOT]
08:41:15,328 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [MONITOR] to Logger[ROOT]
08:41:15,334 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [dk.netarkivet] to DEBUG
08:41:15,334 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
08:41:15,334 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@35e2d654 - Registering current configuration as safe fallback point
SLF4J: Actual binding is of type [ch.qos.logback.classic.util.ContextSelectorStaticBinder]
2022-07-11 08:41:15.354 DEBUG dk.netarkivet.common.utils.Settings.invoke0 - Value found in loaded data: SB,KB
SLF4J: A number (1) of logging calls during the initialization phase have been intercepted and are
SLF4J: now being replayed. These are subject to the filtering rules of the underlying logging system.
SLF4J: See also http://www.slf4j.org/codes.html#replay
Starting dk.netarkivet.common.webinterface.GUIWebServer
Version: 7.4-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/5605ede33d8e9feaccf41bc5ab16c9b6dc7a8800)
2022-07-11 08:41:15.399 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Starting dk.netarkivet.common.webinterface.GUIWebServer
Version: 7.4-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/5605ede33d8e9feaccf41bc5ab16c9b6dc7a8800)
Java VM: 1.8.0_322
2022-07-11 08:41:15.441 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Java VM: 1.8.0_322
java.home: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.322.b06-2.el8_5.x86_64/jre
2022-07-11 08:41:15.449 INFO d.n.common.utils.ApplicationUtils.logAndPrint - java.home: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.322.b06-2.el8_5.x86_64/jre
Working dir: /home/devel/Test6
2022-07-11 08:41:15.450 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Working dir: /home/devel/Test6
2022-07-11 08:41:15.452 INFO d.n.common.utils.ApplicationUtils.startApp - Using settings files '/home/devel/Test6/conf/settings_GUIApplication.xml'
2022-07-11 08:41:15.970 INFO d.n.m.d.JMSMonitorRegistryClient.register - Registering this client for monitoring every 1 minutes, using hostname 'kb-test-adm-001.kb.dk' and JMX/RMI ports 8170/8270
2022-07-11 08:41:15.970 INFO d.n.c.m.MBeanConnectorCreator.exposeJMXMBeanServer - Registered mbean server in registry on port 8170 communicating on port 8270 using password file 'conf/jmxremote.password'.
Service URL is service:jmx:rmi://kb-test-adm-001.kb.dk:8270/jndi/rmi://kb-test-adm-001.kb.dk:8170/jmxrmi
dk.netarkivet.common.webinterface.GUIWebServer Running
2022-07-11 08:41:15.971 INFO d.n.common.utils.ApplicationUtils.logAndPrint - dk.netarkivet.common.webinterface.GUIWebServer Running
2022-07-11 08:41:15.972 DEBUG dk.netarkivet.common.utils.Settings.debug - Value found in loaded data: webpages/HarvestDefinition.war,webpages/History.war,webpages/HarvestChannel.war,webpages/QA.war,webpages/Status.war
2022-07-11 08:41:15.972 DEBUG dk.netarkivet.common.utils.Settings.debug - Value found in loaded data: dk.netarkivet.harvester.webinterface.DefinitionsSiteSection,dk.netarkivet.harvester.webinterface.HistorySiteSection,dk.netarkivet.harvester.webinterface.HarvestChannelSiteSection,dk.netarkivet.viewerproxy.webinterface.QASiteSection,dk.netarkivet.monitor.webinterface.StatusSiteSection
2022-07-11 08:41:15.972 INFO d.n.common.webinterface.GUIWebServer.<init> - Starting webserver. Port: 8078 deployment directories: 'webpages/HarvestDefinition.war,webpages/History.war,webpages/HarvestChannel.war,webpages/QA.war,webpages/Status.war' classes: 'dk.netarkivet.harvester.webinterface.DefinitionsSiteSection,dk.netarkivet.harvester.webinterface.HistorySiteSection,dk.netarkivet.harvester.webinterface.HarvestChannelSiteSection,dk.netarkivet.viewerproxy.webinterface.QASiteSection,dk.netarkivet.monitor.webinterface.StatusSiteSection'
2022-07-11 08:41:15.999 DEBUG d.n.common.webinterface.GUIWebServer.<init> - GUI using tempdir /home/devel/Test6/tmpdircommon
2022-07-11 08:41:15.999 DEBUG d.n.common.webinterface.GUIWebServer.<init> - GUI using basedir /home/devel/Test6
2022-07-11 08:41:16.007 INFO d.n.c.distribute.JMSConnectionSunMQ.<init> - Creating instance of dk.netarkivet.common.distribute.JMSConnectionSunMQ
2022-07-11 08:41:16.007 INFO d.n.common.webinterface.GUIWebServer.<init> - Deleted existing tempdir '/home/devel/Test6/tmpdircommon'
2022-07-11 08:41:16.009 DEBUG d.n.common.distribute.JMSConnection.initConnection - Initializing a JMS connection dk.netarkivet.common.distribute.JMSConnectionSunMQ
2022-07-11 08:41:16.030 INFO d.n.c.distribute.JMSConnectionSunMQ.getConnectionFactory - Establishing SunMQ JMS Connection to 'kb-test-adm-001.kb.dk:7676'
08:41:12,072 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [/home/devel/Test6/conf/logback_HarvestJobManagerApplication.xml] at [file:/home/devel/Test6/conf/logback_HarvestJobManagerApplication.xml]
08:41:12,376 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - debug attribute not set
08:41:12,382 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
08:41:12,426 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [STDOUT]
08:41:12,472 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
08:41:12,685 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
08:41:12,698 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [FILE]
08:41:12,786 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@1742920067 - No compression will be used
08:41:12,789 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@1742920067 - Will use the pattern ./log/HarvestJobManagerApplication.%d{yyyy-MM-dd}.%i.log for the active file
08:41:12,795 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@5d47c63f - The date pattern is 'yyyy-MM-dd' from file name pattern './log/HarvestJobManagerApplication.%d{yyyy-MM-dd}.%i.log'.
08:41:12,795 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@5d47c63f - Roll-over at midnight.
08:41:12,812 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@5d47c63f - Setting initial period to Mon Jul 11 08:41:12 CEST 2022
08:41:12,813 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@5d47c63f - SizeAndTimeBasedFNATP is deprecated. Use SizeAndTimeBasedRollingPolicy instead
08:41:12,813 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@5d47c63f - For more information see http://logback.qos.ch/manual/appenders.html#SizeAndTimeBasedRollingPolicy
08:41:12,818 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
08:41:12,825 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - Active log file name: ./log/HarvestJobManagerApplication.log
08:41:12,825 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - File property is set to [./log/HarvestJobManagerApplication.log]
08:41:12,826 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [dk.netarkivet.monitor.logging.CachingSLF4JAppender]
08:41:14,447 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [MONITOR]
08:41:14,464 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to WARN
08:41:14,464 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[ROOT]
08:41:14,467 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [FILE] to Logger[ROOT]
08:41:14,467 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [MONITOR] to Logger[ROOT]
08:41:14,468 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [dk.netarkivet] to DEBUG
08:41:14,468 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
08:41:14,472 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@5ea434c8 - Registering current configuration as safe fallback point
2022-07-11 08:41:14.501 DEBUG dk.netarkivet.common.utils.Settings.invoke0 - Value found in loaded data: SB,KB
SLF4J: A number (1) of logging calls during the initialization phase have been intercepted and are
SLF4J: now being replayed. These are subject to the filtering rules of the underlying logging system.
SLF4J: See also http://www.slf4j.org/codes.html#replay
2022-07-11 08:41:14.576 INFO d.n.c.distribute.JMSConnectionSunMQ.<init> - Creating instance of dk.netarkivet.common.distribute.JMSConnectionSunMQ
2022-07-11 08:41:14.582 DEBUG d.n.common.distribute.JMSConnection.initConnection - Initializing a JMS connection dk.netarkivet.common.distribute.JMSConnectionSunMQ
2022-07-11 08:41:14.583 INFO d.n.c.distribute.JMSConnectionSunMQ.getConnectionFactory - Establishing SunMQ JMS Connection to 'kb-test-adm-001.kb.dk:7676'
2022-07-11 08:41:15.565 INFO d.n.h.datamodel.HarvestDBConnection.initDataSource - Connection pool initialized with the following values:
- minPoolSize=5
- maxPoolSize=20
- acquireIncrement=5
- maxStatements=0
- maxStatementsPerConnection=0
- idleConnTestPeriod=0
- idleConnTestQuery='null'
- idleConnTestOnCheckin=false
2022-07-11 08:41:15.688 WARN c.m.v.resourcepool.BasicResourcePool.log - Bad pool size config, start 3 < min 5. Using 5 as start.
2022-07-11 08:41:16.053 INFO d.n.h.scheduler.JobDispatcher.<init> - Creating JobDispatcher
--------------------------------------------
STARTING MACHINE: devel@kb-test-har-003.kb.dk
Starting all applications on: kb-test-har-003.kb.dk
Starting linux application: HarvestControllerApplication_kblow001_h3
08:41:16,982 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [/home/devel/Test6/conf/logback_HarvestControllerApplication_kblow001_h3.xml] at [file:/home/devel/Test6/conf/logback_HarvestControllerApplication_kblow001_h3.xml]
08:41:17,204 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - debug attribute not set
08:41:17,206 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
08:41:17,213 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [STDOUT]
08:41:17,224 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
08:41:17,268 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
08:41:17,273 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [FILE]
08:41:17,292 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@858952163 - No compression will be used
08:41:17,293 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@858952163 - Will use the pattern ./log/HarvestControllerApplication_kblow001_h3.%d{yyyy-MM-dd}.%i.log for the active file
08:41:17,296 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@479d31f3 - The date pattern is 'yyyy-MM-dd' from file name pattern './log/HarvestControllerApplication_kblow001_h3.%d{yyyy-MM-dd}.%i.log'.
08:41:17,296 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@479d31f3 - Roll-over at midnight.
08:41:17,300 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@479d31f3 - Setting initial period to Mon Jul 11 08:41:17 CEST 2022
08:41:17,300 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@479d31f3 - SizeAndTimeBasedFNATP is deprecated. Use SizeAndTimeBasedRollingPolicy instead
08:41:17,300 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@479d31f3 - For more information see http://logback.qos.ch/manual/appenders.html#SizeAndTimeBasedRollingPolicy
08:41:17,302 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
08:41:17,304 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - Active log file name: ./log/HarvestControllerApplication_kblow001_h3.log
08:41:17,304 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - File property is set to [./log/HarvestControllerApplication_kblow001_h3.log]
08:41:17,305 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [dk.netarkivet.monitor.logging.CachingSLF4JAppender]
08:41:18,056 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [MONITOR]
08:41:18,058 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to WARN
08:41:18,058 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[ROOT]
08:41:18,059 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [FILE] to Logger[ROOT]
08:41:18,059 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [MONITOR] to Logger[ROOT]
08:41:18,059 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [dk.netarkivet] to DEBUG
08:41:18,059 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
08:41:18,060 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@40ef3420 - Registering current configuration as safe fallback point
2022-07-11 08:41:18.064 DEBUG dk.netarkivet.common.utils.Settings.invoke0 - Value found in loaded data: SB,KB
SLF4J: A number (1) of logging calls during the initialization phase have been intercepted and are
SLF4J: now being replayed. These are subject to the filtering rules of the underlying logging system.
SLF4J: See also http://www.slf4j.org/codes.html#replay
Starting dk.netarkivet.harvester.heritrix3.HarvestControllerServer
Version: 7.4-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/5605ede33d8e9feaccf41bc5ab16c9b6dc7a8800)
2022-07-11 08:41:18.070 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Starting dk.netarkivet.harvester.heritrix3.HarvestControllerServer
Version: 7.4-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/5605ede33d8e9feaccf41bc5ab16c9b6dc7a8800)
Java VM: 1.8.0_332
2022-07-11 08:41:18.071 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Java VM: 1.8.0_332
java.home: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.332.b09-1.el7_9.x86_64/jre
2022-07-11 08:41:18.071 INFO d.n.common.utils.ApplicationUtils.logAndPrint - java.home: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.332.b09-1.el7_9.x86_64/jre
Working dir: /home/devel/Test6
2022-07-11 08:41:18.071 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Working dir: /home/devel/Test6
2022-07-11 08:41:18.075 INFO d.n.common.utils.ApplicationUtils.startApp - Using settings files '/home/devel/Test6/conf/settings_HarvestControllerApplication_kblow001_h3.xml'
2022-07-11 08:41:18.206 INFO d.n.m.d.JMSMonitorRegistryClient.register - Registering this client for monitoring every 1 minutes, using hostname 'kb-test-har-003.kb.dk' and JMX/RMI ports 8170/8270
2022-07-11 08:41:18.207 INFO d.n.c.m.MBeanConnectorCreator.exposeJMXMBeanServer - Registered mbean server in registry on port 8170 communicating on port 8270 using password file 'conf/jmxremote.password'.
Service URL is service:jmx:rmi://kb-test-har-003.kb.dk:8270/jndi/rmi://kb-test-har-003.kb.dk:8170/jmxrmi
dk.netarkivet.harvester.heritrix3.HarvestControllerServer Running
2022-07-11 08:41:18.212 INFO d.n.common.utils.ApplicationUtils.logAndPrint - dk.netarkivet.harvester.heritrix3.HarvestControllerServer Running
2022-07-11 08:41:18.212 INFO d.n.c.distribute.JMSConnectionSunMQ.<init> - Creating instance of dk.netarkivet.common.distribute.JMSConnectionSunMQ
2022-07-11 08:41:18.213 DEBUG d.n.common.distribute.JMSConnection.initConnection - Initializing a JMS connection dk.netarkivet.common.distribute.JMSConnectionSunMQ
2022-07-11 08:41:18.213 INFO d.n.c.distribute.JMSConnectionSunMQ.getConnectionFactory - Establishing SunMQ JMS Connection to 'kb-test-adm-001.kb.dk:7676'
2022-07-11 08:41:18.221 INFO d.n.h.h.HarvestControllerServer.<init> - Starting HarvestControllerServer.
2022-07-11 08:41:18.221 INFO d.n.h.h.HarvestControllerServer.<init> - Bound to harvest channel 'LOWPRIORITY'
2022-07-11 08:41:18.222 INFO d.n.h.h.HarvestControllerServer.<init> - Serverdir: 'harvester_low'
2022-07-11 08:41:18.222 INFO d.n.h.h.HarvestControllerServer.<init> - Harvesting requires at least 400000000 bytes free.
2022-07-11 08:41:18.350 DEBUG d.n.c.d.bitrepository.BitmagUtils.initialize - Bitrepository settings will be loaded from /home/devel/extra_configs/bitmag_config.
2022-07-11 08:41:18.351 DEBUG d.n.c.d.bitrepository.BitmagUtils.initialize - Path to Bitrepository settings is /home/devel/extra_configs/bitmag_config.
2022-07-11 08:41:18.361 INFO d.n.c.d.bitrepository.HostNameUtils.getHostName - Local hostname (provided by getCanonicalHostName): kb-test-har-003.kb.dk
2022-07-11 08:41:19.909 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - BitmagArcRepositoryClient will timeout on each get request after 300000 milliseconds.
2022-07-11 08:41:19.910 DEBUG d.n.common.distribute.JMSConnection.setListener - Adding dk.netarkivet.archive.arcrepository.distribute.BitmagArcRepositoryClient@5c3b6c6e as listener to Test6_COMMON_THIS_REPOS_CLIENT_130_226_228_9_HCS_KBLOW001_H3
2022-07-11 08:41:19.917 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - BitmagArcRepositoryClient listens for replies on channel '[Queue 'Test6_COMMON_THIS_REPOS_CLIENT_130_226_228_9_HCS_KBLOW001_H3']'
2022-07-11 08:41:19.918 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - Getting bitmag config from settings.common.arcrepositoryClient.bitrepository.settingsDir=/home/devel/extra_configs/bitmag_config
2022-07-11 08:41:19.918 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - Using 'Test6' as default collectionID
2022-07-11 08:41:19.923 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - Storing tempfiles in folder 'arcrepositoryTemp'
2022-07-11 08:41:19.932 INFO d.n.common.utils.HttpsClientBuilder.<init> - Creating Aggressive retry handler for https queries.
2022-07-11 08:41:20.067 INFO d.n.c.utils.BasicTwoWaySSLProvider.loadPrivateKey - Key file '/home/devel/extra_configs/https_config/https_key.pem' with private key and certificate does not exist!
2022-07-11 08:41:20.117 DEBUG d.n.h.h.HarvestControllerServer.<init> - Obtained JMS connection.
2022-07-11 08:41:20.119 INFO d.n.h.h.HarvestControllerServer.<init> - SEND_READY_DELAY used by HarvestControllerServer is 300
2022-07-11 08:41:20.120 INFO d.n.h.heritrix3.PostProcessing.processOldJobs - Looking for unprocessed crawldata in 'harvester_low'
2022-07-11 08:41:20.120 DEBUG d.n.common.distribute.JMSConnection.setListener - Adding dk.netarkivet.harvester.heritrix3.HarvestControllerServer@7c56e013 as listener to Test6_COMMON_HCHAN_VAL_RESP
2022-07-11 08:41:20.135 INFO d.n.h.h.HarvestControllerServer.visit - Received message stating that channel 'LOWPRIORITY' is valid.
2022-07-11 08:41:20.138 INFO d.n.h.h.HarvestControllerServer.beginListeningIfSpaceAvailable - Starts to listen to new jobs on queue '[Queue 'Test6_COMMON_JOB_SNAPSHOT_LOWPRIORITY']'
2022-07-11 08:41:20.139 DEBUG d.n.common.distribute.JMSConnection.setListener - Adding dk.netarkivet.harvester.heritrix3.HarvestControllerServer@7c56e013 as listener to Test6_COMMON_JOB_SNAPSHOT_LOWPRIORITY
2022-07-11 08:41:20.156 INFO d.n.h.h.HarvestControllerServer.<init> - Requested to check the validity of harvest channel 'LOWPRIORITY'
--------------------------------------------
STARTING MACHINE: devel@kb-test-har-005.kb.dk
Starting all applications on: kb-test-har-005.kb.dk
Starting linux application: HarvestControllerApplication_browser_1
Starting linux application: HarvestControllerApplication_browser_2
08:41:23,064 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [/home/devel/Test6/conf/logback_HarvestControllerApplication_browser_1.xml] at [file:/home/devel/Test6/conf/logback_HarvestControllerApplication_browser_1.xml]
08:41:23,543 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - debug attribute not set
08:41:23,546 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
08:41:23,553 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [STDOUT]
08:41:23,592 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
08:41:23,708 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
08:41:23,726 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [FILE]
08:41:23,780 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@447981768 - No compression will be used
08:41:23,783 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@447981768 - Will use the pattern ./log/HarvestControllerApplication_browser_1.%d{yyyy-MM-dd}.%i.log for the active file
08:41:23,789 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@43195e57 - The date pattern is 'yyyy-MM-dd' from file name pattern './log/HarvestControllerApplication_browser_1.%d{yyyy-MM-dd}.%i.log'.
08:41:23,789 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@43195e57 - Roll-over at midnight.
08:41:23,798 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@43195e57 - Setting initial period to Mon Jul 11 08:41:23 CEST 2022
08:41:23,798 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@43195e57 - SizeAndTimeBasedFNATP is deprecated. Use SizeAndTimeBasedRollingPolicy instead
08:41:23,798 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@43195e57 - For more information see http://logback.qos.ch/manual/appenders.html#SizeAndTimeBasedRollingPolicy
08:41:23,801 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
08:41:23,810 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - Active log file name: ./log/HarvestControllerApplication_browser_1.log
08:41:23,810 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - File property is set to [./log/HarvestControllerApplication_browser_1.log]
08:41:23,812 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [dk.netarkivet.monitor.logging.CachingSLF4JAppender]
08:41:24,753 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [MONITOR]
08:41:24,755 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to WARN
08:41:24,755 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[ROOT]
08:41:24,791 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [FILE] to Logger[ROOT]
08:41:24,791 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [MONITOR] to Logger[ROOT]
08:41:24,791 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [dk.netarkivet] to DEBUG
08:41:24,791 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
08:41:24,792 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@333291e3 - Registering current configuration as safe fallback point
2022-07-11 08:41:24.797 DEBUG dk.netarkivet.common.utils.Settings.invoke0 - Value found in loaded data: SB,KB
SLF4J: A number (1) of logging calls during the initialization phase have been intercepted and are
SLF4J: now being replayed. These are subject to the filtering rules of the underlying logging system.
SLF4J: See also http://www.slf4j.org/codes.html#replay
Starting dk.netarkivet.harvester.heritrix3.HarvestControllerServer
Version: 7.4-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/5605ede33d8e9feaccf41bc5ab16c9b6dc7a8800)
2022-07-11 08:41:24.806 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Starting dk.netarkivet.harvester.heritrix3.HarvestControllerServer
Version: 7.4-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/5605ede33d8e9feaccf41bc5ab16c9b6dc7a8800)
Java VM: 1.8.0_312
2022-07-11 08:41:24.806 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Java VM: 1.8.0_312
java.home: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.312.b07-1.el7_9.x86_64/jre
2022-07-11 08:41:24.807 INFO d.n.common.utils.ApplicationUtils.logAndPrint - java.home: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.312.b07-1.el7_9.x86_64/jre
Working dir: /home/devel/Test6
2022-07-11 08:41:24.807 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Working dir: /home/devel/Test6
2022-07-11 08:41:24.807 INFO d.n.common.utils.ApplicationUtils.startApp - Using settings files '/home/devel/Test6/conf/settings_HarvestControllerApplication_browser_1.xml'
2022-07-11 08:41:25.056 INFO d.n.m.d.JMSMonitorRegistryClient.register - Registering this client for monitoring every 1 minutes, using hostname 'kb-test-har-005.kb.dk' and JMX/RMI ports 8170/8270
2022-07-11 08:41:25.056 INFO d.n.c.m.MBeanConnectorCreator.exposeJMXMBeanServer - Registered mbean server in registry on port 8170 communicating on port 8270 using password file 'conf/jmxremote.password'.
Service URL is service:jmx:rmi://kb-test-har-005.kb.dk:8270/jndi/rmi://kb-test-har-005.kb.dk:8170/jmxrmi
dk.netarkivet.harvester.heritrix3.HarvestControllerServer Running
2022-07-11 08:41:25.062 INFO d.n.common.utils.ApplicationUtils.logAndPrint - dk.netarkivet.harvester.heritrix3.HarvestControllerServer Running
2022-07-11 08:41:25.089 INFO d.n.h.h.HarvestControllerServer.<init> - Starting HarvestControllerServer.
2022-07-11 08:41:25.090 INFO d.n.h.h.HarvestControllerServer.<init> - Bound to harvest channel 'BROWSER'
2022-07-11 08:41:25.090 INFO d.n.h.h.HarvestControllerServer.<init> - Serverdir: 'harvester_browser_1'
2022-07-11 08:41:25.090 INFO d.n.h.h.HarvestControllerServer.<init> - Harvesting requires at least 400000000 bytes free.
2022-07-11 08:41:25.098 INFO d.n.c.distribute.JMSConnectionSunMQ.<init> - Creating instance of dk.netarkivet.common.distribute.JMSConnectionSunMQ
2022-07-11 08:41:25.099 DEBUG d.n.common.distribute.JMSConnection.initConnection - Initializing a JMS connection dk.netarkivet.common.distribute.JMSConnectionSunMQ
2022-07-11 08:41:25.099 INFO d.n.c.distribute.JMSConnectionSunMQ.getConnectionFactory - Establishing SunMQ JMS Connection to 'kb-test-adm-001.kb.dk:7676'
2022-07-11 08:41:25.297 DEBUG d.n.c.d.bitrepository.BitmagUtils.initialize - Bitrepository settings will be loaded from /home/devel/extra_configs/bitmag_config.
2022-07-11 08:41:25.298 DEBUG d.n.c.d.bitrepository.BitmagUtils.initialize - Path to Bitrepository settings is /home/devel/extra_configs/bitmag_config.
2022-07-11 08:41:25.339 INFO d.n.c.d.bitrepository.HostNameUtils.getHostName - Local hostname (provided by getCanonicalHostName): kb-test-har-005.kb.dk
08:41:23,063 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [/home/devel/Test6/conf/logback_HarvestControllerApplication_browser_2.xml] at [file:/home/devel/Test6/conf/logback_HarvestControllerApplication_browser_2.xml]
08:41:23,562 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - debug attribute not set
08:41:23,566 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
08:41:23,572 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [STDOUT]
08:41:23,592 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
08:41:23,706 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
08:41:23,726 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [FILE]
08:41:23,780 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@447981768 - No compression will be used
08:41:23,781 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@447981768 - Will use the pattern ./log/HarvestControllerApplication_browser_2.%d{yyyy-MM-dd}.%i.log for the active file
08:41:23,785 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@43195e57 - The date pattern is 'yyyy-MM-dd' from file name pattern './log/HarvestControllerApplication_browser_2.%d{yyyy-MM-dd}.%i.log'.
08:41:23,785 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@43195e57 - Roll-over at midnight.
08:41:23,797 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@43195e57 - Setting initial period to Mon Jul 11 08:41:23 CEST 2022
08:41:23,797 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@43195e57 - SizeAndTimeBasedFNATP is deprecated. Use SizeAndTimeBasedRollingPolicy instead
08:41:23,797 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@43195e57 - For more information see http://logback.qos.ch/manual/appenders.html#SizeAndTimeBasedRollingPolicy
08:41:23,800 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
08:41:23,803 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - Active log file name: ./log/HarvestControllerApplication_browser_2.log
08:41:23,803 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - File property is set to [./log/HarvestControllerApplication_browser_2.log]
08:41:23,808 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [dk.netarkivet.monitor.logging.CachingSLF4JAppender]
08:41:24,778 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [MONITOR]
08:41:24,780 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to WARN
08:41:24,780 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[ROOT]
08:41:24,781 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [FILE] to Logger[ROOT]
08:41:24,781 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [MONITOR] to Logger[ROOT]
08:41:24,781 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [dk.netarkivet] to DEBUG
08:41:24,781 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
08:41:24,782 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@333291e3 - Registering current configuration as safe fallback point
2022-07-11 08:41:24.800 DEBUG dk.netarkivet.common.utils.Settings.invoke0 - Value found in loaded data: SB,KB
SLF4J: A number (1) of logging calls during the initialization phase have been intercepted and are
SLF4J: now being replayed. These are subject to the filtering rules of the underlying logging system.
SLF4J: See also http://www.slf4j.org/codes.html#replay
Starting dk.netarkivet.harvester.heritrix3.HarvestControllerServer
Version: 7.4-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/5605ede33d8e9feaccf41bc5ab16c9b6dc7a8800)
2022-07-11 08:41:24.807 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Starting dk.netarkivet.harvester.heritrix3.HarvestControllerServer
Version: 7.4-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/5605ede33d8e9feaccf41bc5ab16c9b6dc7a8800)
Java VM: 1.8.0_312
2022-07-11 08:41:24.807 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Java VM: 1.8.0_312
java.home: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.312.b07-1.el7_9.x86_64/jre
2022-07-11 08:41:24.808 INFO d.n.common.utils.ApplicationUtils.logAndPrint - java.home: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.312.b07-1.el7_9.x86_64/jre
Working dir: /home/devel/Test6
2022-07-11 08:41:24.808 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Working dir: /home/devel/Test6
2022-07-11 08:41:24.808 INFO d.n.common.utils.ApplicationUtils.startApp - Using settings files '/home/devel/Test6/conf/settings_HarvestControllerApplication_browser_2.xml'
2022-07-11 08:41:25.050 INFO d.n.m.d.JMSMonitorRegistryClient.register - Registering this client for monitoring every 1 minutes, using hostname 'kb-test-har-005.kb.dk' and JMX/RMI ports 8175/8275
2022-07-11 08:41:25.050 INFO d.n.c.m.MBeanConnectorCreator.exposeJMXMBeanServer - Registered mbean server in registry on port 8175 communicating on port 8275 using password file 'conf/jmxremote.password'.
Service URL is service:jmx:rmi://kb-test-har-005.kb.dk:8275/jndi/rmi://kb-test-har-005.kb.dk:8175/jmxrmi
dk.netarkivet.harvester.heritrix3.HarvestControllerServer Running
2022-07-11 08:41:25.056 INFO d.n.common.utils.ApplicationUtils.logAndPrint - dk.netarkivet.harvester.heritrix3.HarvestControllerServer Running
2022-07-11 08:41:25.081 INFO d.n.h.h.HarvestControllerServer.<init> - Starting HarvestControllerServer.
2022-07-11 08:41:25.082 INFO d.n.h.h.HarvestControllerServer.<init> - Bound to harvest channel 'BROWSER'
2022-07-11 08:41:25.082 INFO d.n.h.h.HarvestControllerServer.<init> - Serverdir: 'harvester_browser_2'
2022-07-11 08:41:25.083 INFO d.n.h.h.HarvestControllerServer.<init> - Harvesting requires at least 400000000 bytes free.
2022-07-11 08:41:25.091 INFO d.n.c.distribute.JMSConnectionSunMQ.<init> - Creating instance of dk.netarkivet.common.distribute.JMSConnectionSunMQ
2022-07-11 08:41:25.092 DEBUG d.n.common.distribute.JMSConnection.initConnection - Initializing a JMS connection dk.netarkivet.common.distribute.JMSConnectionSunMQ
2022-07-11 08:41:25.092 INFO d.n.c.distribute.JMSConnectionSunMQ.getConnectionFactory - Establishing SunMQ JMS Connection to 'kb-test-adm-001.kb.dk:7676'
2022-07-11 08:41:25.291 DEBUG d.n.c.d.bitrepository.BitmagUtils.initialize - Bitrepository settings will be loaded from /home/devel/extra_configs/bitmag_config.
2022-07-11 08:41:25.292 DEBUG d.n.c.d.bitrepository.BitmagUtils.initialize - Path to Bitrepository settings is /home/devel/extra_configs/bitmag_config.
2022-07-11 08:41:25.314 INFO d.n.c.d.bitrepository.HostNameUtils.getHostName - Local hostname (provided by getCanonicalHostName): kb-test-har-005.kb.dk
--------------------------------------------
STARTING MACHINE: devel@kb-test-har-004.kb.dk
Starting all applications on: kb-test-har-004.kb.dk
Starting linux application: HarvestControllerApplication_kblow002_h3
Starting linux application: HarvestControllerApplication_kbhigh_h3
08:41:29,227 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [/home/devel/Test6/conf/logback_HarvestControllerApplication_kbhigh_h3.xml] at [file:/home/devel/Test6/conf/logback_HarvestControllerApplication_kbhigh_h3.xml]
08:41:29,857 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - debug attribute not set
08:41:29,872 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
08:41:29,893 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [STDOUT]
08:41:29,930 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
08:41:30,111 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
08:41:30,138 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [FILE]
08:41:30,242 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@1704629915 - No compression will be used
08:41:30,245 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@1704629915 - Will use the pattern ./log/HarvestControllerApplication_kbhigh_h3.%d{yyyy-MM-dd}.%i.log for the active file
08:41:30,258 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@76908cc0 - The date pattern is 'yyyy-MM-dd' from file name pattern './log/HarvestControllerApplication_kbhigh_h3.%d{yyyy-MM-dd}.%i.log'.
08:41:30,258 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@76908cc0 - Roll-over at midnight.
08:41:30,276 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@76908cc0 - Setting initial period to Mon Jul 11 08:41:30 CEST 2022
08:41:30,278 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@76908cc0 - SizeAndTimeBasedFNATP is deprecated. Use SizeAndTimeBasedRollingPolicy instead
08:41:30,278 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@76908cc0 - For more information see http://logback.qos.ch/manual/appenders.html#SizeAndTimeBasedRollingPolicy
08:41:30,282 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
08:41:30,297 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - Active log file name: ./log/HarvestControllerApplication_kbhigh_h3.log
08:41:30,297 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - File property is set to [./log/HarvestControllerApplication_kbhigh_h3.log]
08:41:30,301 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [dk.netarkivet.monitor.logging.CachingSLF4JAppender]
08:41:31,446 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [MONITOR]
08:41:31,458 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to WARN
08:41:31,458 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[ROOT]
08:41:31,460 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [FILE] to Logger[ROOT]
08:41:31,460 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [MONITOR] to Logger[ROOT]
08:41:31,461 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [dk.netarkivet] to DEBUG
08:41:31,461 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
08:41:31,463 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@2473d930 - Registering current configuration as safe fallback point
2022-07-11 08:41:31.493 DEBUG dk.netarkivet.common.utils.Settings.invoke0 - Value found in loaded data: SB,KB
SLF4J: A number (1) of logging calls during the initialization phase have been intercepted and are
SLF4J: now being replayed. These are subject to the filtering rules of the underlying logging system.
SLF4J: See also http://www.slf4j.org/codes.html#replay
Starting dk.netarkivet.harvester.heritrix3.HarvestControllerServer
Version: 7.4-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/5605ede33d8e9feaccf41bc5ab16c9b6dc7a8800)
2022-07-11 08:41:31.521 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Starting dk.netarkivet.harvester.heritrix3.HarvestControllerServer
Version: 7.4-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/5605ede33d8e9feaccf41bc5ab16c9b6dc7a8800)
Java VM: 1.8.0_25
2022-07-11 08:41:31.523 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Java VM: 1.8.0_25
java.home: /usr/local/java/jdk1.8.0_25_x64/jre
2022-07-11 08:41:31.525 INFO d.n.common.utils.ApplicationUtils.logAndPrint - java.home: /usr/local/java/jdk1.8.0_25_x64/jre
Working dir: /home/devel/Test6
2022-07-11 08:41:31.527 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Working dir: /home/devel/Test6
2022-07-11 08:41:31.529 INFO d.n.common.utils.ApplicationUtils.startApp - Using settings files '/home/devel/Test6/conf/settings_HarvestControllerApplication_kbhigh_h3.xml'
2022-07-11 08:41:31.853 INFO d.n.m.d.JMSMonitorRegistryClient.register - Registering this client for monitoring every 1 minutes, using hostname 'kb-test-har-004.kb.dk' and JMX/RMI ports 8172/8272
2022-07-11 08:41:31.853 INFO d.n.c.m.MBeanConnectorCreator.exposeJMXMBeanServer - Registered mbean server in registry on port 8172 communicating on port 8272 using password file 'conf/jmxremote.password'.
Service URL is service:jmx:rmi://kb-test-har-004.kb.dk:8272/jndi/rmi://kb-test-har-004.kb.dk:8172/jmxrmi
dk.netarkivet.harvester.heritrix3.HarvestControllerServer Running
2022-07-11 08:41:31.860 INFO d.n.common.utils.ApplicationUtils.logAndPrint - dk.netarkivet.harvester.heritrix3.HarvestControllerServer Running
2022-07-11 08:41:31.891 INFO d.n.c.distribute.JMSConnectionSunMQ.<init> - Creating instance of dk.netarkivet.common.distribute.JMSConnectionSunMQ
2022-07-11 08:41:31.891 DEBUG d.n.common.distribute.JMSConnection.initConnection - Initializing a JMS connection dk.netarkivet.common.distribute.JMSConnectionSunMQ
2022-07-11 08:41:31.892 INFO d.n.c.distribute.JMSConnectionSunMQ.getConnectionFactory - Establishing SunMQ JMS Connection to 'kb-test-adm-001.kb.dk:7676'
2022-07-11 08:41:31.909 INFO d.n.h.h.HarvestControllerServer.<init> - Starting HarvestControllerServer.
2022-07-11 08:41:31.912 INFO d.n.h.h.HarvestControllerServer.<init> - Bound to harvest channel 'HIGHPRIORITY'
2022-07-11 08:41:31.913 INFO d.n.h.h.HarvestControllerServer.<init> - Serverdir: 'harvester_high'
2022-07-11 08:41:31.913 INFO d.n.h.h.HarvestControllerServer.<init> - Harvesting requires at least 400000000 bytes free.
2022-07-11 08:41:32.220 DEBUG d.n.c.d.bitrepository.BitmagUtils.initialize - Bitrepository settings will be loaded from /home/devel/extra_configs/bitmag_config.
2022-07-11 08:41:32.261 DEBUG d.n.c.d.bitrepository.BitmagUtils.initialize - Path to Bitrepository settings is /home/devel/extra_configs/bitmag_config.
2022-07-11 08:41:32.277 INFO d.n.c.d.bitrepository.HostNameUtils.getHostName - Local hostname (provided by getCanonicalHostName): kb-test-har-004.kb.dk
08:41:29,226 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [/home/devel/Test6/conf/logback_HarvestControllerApplication_kblow002_h3.xml] at [file:/home/devel/Test6/conf/logback_HarvestControllerApplication_kblow002_h3.xml]
08:41:30,001 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - debug attribute not set
08:41:30,009 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
08:41:30,032 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [STDOUT]
08:41:30,060 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
08:41:30,144 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
08:41:30,149 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [FILE]
08:41:30,216 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@1704629915 - No compression will be used
08:41:30,222 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@1704629915 - Will use the pattern ./log/HarvestControllerApplication_kblow002_h3.%d{yyyy-MM-dd}.%i.log for the active file
08:41:30,226 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@76908cc0 - The date pattern is 'yyyy-MM-dd' from file name pattern './log/HarvestControllerApplication_kblow002_h3.%d{yyyy-MM-dd}.%i.log'.
08:41:30,226 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@76908cc0 - Roll-over at midnight.
08:41:30,239 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@76908cc0 - Setting initial period to Mon Jul 11 08:41:30 CEST 2022
08:41:30,240 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@76908cc0 - SizeAndTimeBasedFNATP is deprecated. Use SizeAndTimeBasedRollingPolicy instead
08:41:30,240 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@76908cc0 - For more information see http://logback.qos.ch/manual/appenders.html#SizeAndTimeBasedRollingPolicy
08:41:30,245 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
08:41:30,254 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - Active log file name: ./log/HarvestControllerApplication_kblow002_h3.log
08:41:30,254 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - File property is set to [./log/HarvestControllerApplication_kblow002_h3.log]
08:41:30,255 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [dk.netarkivet.monitor.logging.CachingSLF4JAppender]
08:41:31,427 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [MONITOR]
08:41:31,441 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to WARN
08:41:31,441 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[ROOT]
08:41:31,445 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [FILE] to Logger[ROOT]
08:41:31,445 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [MONITOR] to Logger[ROOT]
08:41:31,447 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [dk.netarkivet] to DEBUG
08:41:31,447 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
08:41:31,451 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@2473d930 - Registering current configuration as safe fallback point
2022-07-11 08:41:31.475 DEBUG dk.netarkivet.common.utils.Settings.invoke0 - Value found in loaded data: SB,KB
SLF4J: A number (1) of logging calls during the initialization phase have been intercepted and are
SLF4J: now being replayed. These are subject to the filtering rules of the underlying logging system.
SLF4J: See also http://www.slf4j.org/codes.html#replay
Starting dk.netarkivet.harvester.heritrix3.HarvestControllerServer
Version: 7.4-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/5605ede33d8e9feaccf41bc5ab16c9b6dc7a8800)
2022-07-11 08:41:31.509 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Starting dk.netarkivet.harvester.heritrix3.HarvestControllerServer
Version: 7.4-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/5605ede33d8e9feaccf41bc5ab16c9b6dc7a8800)
Java VM: 1.8.0_25
2022-07-11 08:41:31.514 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Java VM: 1.8.0_25
java.home: /usr/local/java/jdk1.8.0_25_x64/jre
2022-07-11 08:41:31.515 INFO d.n.common.utils.ApplicationUtils.logAndPrint - java.home: /usr/local/java/jdk1.8.0_25_x64/jre
Working dir: /home/devel/Test6
2022-07-11 08:41:31.518 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Working dir: /home/devel/Test6
2022-07-11 08:41:31.519 INFO d.n.common.utils.ApplicationUtils.startApp - Using settings files '/home/devel/Test6/conf/settings_HarvestControllerApplication_kblow002_h3.xml'
2022-07-11 08:41:31.845 INFO d.n.m.d.JMSMonitorRegistryClient.register - Registering this client for monitoring every 1 minutes, using hostname 'kb-test-har-004.kb.dk' and JMX/RMI ports 8170/8270
2022-07-11 08:41:31.846 INFO d.n.c.m.MBeanConnectorCreator.exposeJMXMBeanServer - Registered mbean server in registry on port 8170 communicating on port 8270 using password file 'conf/jmxremote.password'.
Service URL is service:jmx:rmi://kb-test-har-004.kb.dk:8270/jndi/rmi://kb-test-har-004.kb.dk:8170/jmxrmi
dk.netarkivet.harvester.heritrix3.HarvestControllerServer Running
2022-07-11 08:41:31.873 INFO d.n.common.utils.ApplicationUtils.logAndPrint - dk.netarkivet.harvester.heritrix3.HarvestControllerServer Running
2022-07-11 08:41:31.903 INFO d.n.c.distribute.JMSConnectionSunMQ.<init> - Creating instance of dk.netarkivet.common.distribute.JMSConnectionSunMQ
2022-07-11 08:41:31.904 DEBUG d.n.common.distribute.JMSConnection.initConnection - Initializing a JMS connection dk.netarkivet.common.distribute.JMSConnectionSunMQ
2022-07-11 08:41:31.904 INFO d.n.c.distribute.JMSConnectionSunMQ.getConnectionFactory - Establishing SunMQ JMS Connection to 'kb-test-adm-001.kb.dk:7676'
2022-07-11 08:41:31.920 INFO d.n.h.h.HarvestControllerServer.<init> - Starting HarvestControllerServer.
2022-07-11 08:41:31.920 INFO d.n.h.h.HarvestControllerServer.<init> - Bound to harvest channel 'LOWPRIORITY'
2022-07-11 08:41:31.920 INFO d.n.h.h.HarvestControllerServer.<init> - Serverdir: 'harvester_low'
2022-07-11 08:41:31.921 INFO d.n.h.h.HarvestControllerServer.<init> - Harvesting requires at least 400000000 bytes free.
2022-07-11 08:41:32.220 DEBUG d.n.c.d.bitrepository.BitmagUtils.initialize - Bitrepository settings will be loaded from /home/devel/extra_configs/bitmag_config.
2022-07-11 08:41:32.262 DEBUG d.n.c.d.bitrepository.BitmagUtils.initialize - Path to Bitrepository settings is /home/devel/extra_configs/bitmag_config.
2022-07-11 08:41:32.279 INFO d.n.c.d.bitrepository.HostNameUtils.getHostName - Local hostname (provided by getCanonicalHostName): kb-test-har-004.kb.dk
--------------------------------------------
STARTING MACHINE: devel@kb-test-acs-001.kb.dk
Starting all applications on: kb-test-acs-001.kb.dk
Starting linux application: IndexServerApplication
Starting linux application: ViewerProxyApplication
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/devel/Test6/lib/logback-classic-1.2.3.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/devel/Test6/lib/slf4j-log4j12-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
08:41:34,279 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [/home/devel/Test6/conf/logback_IndexServerApplication.xml] at [file:/home/devel/Test6/conf/logback_IndexServerApplication.xml]
08:41:34,603 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - debug attribute not set
08:41:34,606 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
08:41:34,615 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [STDOUT]
08:41:34,629 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
08:41:34,684 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
08:41:34,689 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [FILE]
08:41:34,720 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@29183965 - No compression will be used
08:41:34,721 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@29183965 - Will use the pattern ./log/IndexServerApplication.%d{yyyy-MM-dd}.%i.log for the active file
08:41:34,724 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@55183b20 - The date pattern is 'yyyy-MM-dd' from file name pattern './log/IndexServerApplication.%d{yyyy-MM-dd}.%i.log'.
08:41:34,724 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@55183b20 - Roll-over at midnight.
08:41:34,728 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@55183b20 - Setting initial period to Mon Jul 11 08:41:34 CEST 2022
08:41:34,728 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@55183b20 - SizeAndTimeBasedFNATP is deprecated. Use SizeAndTimeBasedRollingPolicy instead
08:41:34,728 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@55183b20 - For more information see http://logback.qos.ch/manual/appenders.html#SizeAndTimeBasedRollingPolicy
08:41:34,730 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
08:41:34,733 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - Active log file name: ./log/IndexServerApplication.log
08:41:34,733 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - File property is set to [./log/IndexServerApplication.log]
08:41:34,734 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [dk.netarkivet.monitor.logging.CachingSLF4JAppender]
08:41:35,513 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [MONITOR]
08:41:35,515 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to WARN
08:41:35,515 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[ROOT]
08:41:35,516 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [FILE] to Logger[ROOT]
08:41:35,517 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [MONITOR] to Logger[ROOT]
08:41:35,517 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [dk.netarkivet] to DEBUG
08:41:35,517 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
08:41:35,518 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@4f83df68 - Registering current configuration as safe fallback point
SLF4J: Actual binding is of type [ch.qos.logback.classic.util.ContextSelectorStaticBinder]
2022-07-11 08:41:35.524 DEBUG dk.netarkivet.common.utils.Settings.invoke0 - Value found in loaded data: SB,KB
SLF4J: A number (1) of logging calls during the initialization phase have been intercepted and are
SLF4J: now being replayed. These are subject to the filtering rules of the underlying logging system.
SLF4J: See also http://www.slf4j.org/codes.html#replay
Starting dk.netarkivet.harvester.indexserver.IndexServer
Version: 7.4-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/5605ede33d8e9feaccf41bc5ab16c9b6dc7a8800)
2022-07-11 08:41:35.540 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Starting dk.netarkivet.harvester.indexserver.IndexServer
Version: 7.4-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/5605ede33d8e9feaccf41bc5ab16c9b6dc7a8800)
Java VM: 1.8.0_332
2022-07-11 08:41:35.541 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Java VM: 1.8.0_332
java.home: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.332.b09-1.el7_9.x86_64/jre
2022-07-11 08:41:35.541 INFO d.n.common.utils.ApplicationUtils.logAndPrint - java.home: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.332.b09-1.el7_9.x86_64/jre
Working dir: /home/devel/Test6
2022-07-11 08:41:35.541 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Working dir: /home/devel/Test6
2022-07-11 08:41:35.542 INFO d.n.common.utils.ApplicationUtils.startApp - Using settings files '/home/devel/Test6/conf/settings_IndexServerApplication.xml'
2022-07-11 08:41:35.696 INFO d.n.m.d.JMSMonitorRegistryClient.register - Registering this client for monitoring every 1 minutes, using hostname 'kb-test-acs-001.kb.dk' and JMX/RMI ports 8371/8471
2022-07-11 08:41:35.696 INFO d.n.c.m.MBeanConnectorCreator.exposeJMXMBeanServer - Registered mbean server in registry on port 8371 communicating on port 8471 using password file 'conf/jmxremote.password'.
Service URL is service:jmx:rmi://kb-test-acs-001.kb.dk:8471/jndi/rmi://kb-test-acs-001.kb.dk:8371/jmxrmi
dk.netarkivet.harvester.indexserver.IndexServer Running
2022-07-11 08:41:35.701 INFO d.n.common.utils.ApplicationUtils.logAndPrint - dk.netarkivet.harvester.indexserver.IndexServer Running
2022-07-11 08:41:35.706 INFO d.n.h.indexserver.FileBasedCache.<init> - Metadata cache for 'cdxdata' uses directory '/home/devel/Test6/cache/cdxdata'
2022-07-11 08:41:35.708 INFO d.n.c.distribute.JMSConnectionSunMQ.<init> - Creating instance of dk.netarkivet.common.distribute.JMSConnectionSunMQ
2022-07-11 08:41:35.708 DEBUG d.n.common.distribute.JMSConnection.initConnection - Initializing a JMS connection dk.netarkivet.common.distribute.JMSConnectionSunMQ
2022-07-11 08:41:35.709 INFO d.n.c.distribute.JMSConnectionSunMQ.getConnectionFactory - Establishing SunMQ JMS Connection to 'kb-test-adm-001.kb.dk:7676'
2022-07-11 08:41:35.715 DEBUG d.n.c.d.bitrepository.BitmagUtils.initialize - Bitrepository settings will be loaded from /home/devel/extra_configs/bitmag_config.
2022-07-11 08:41:35.715 DEBUG d.n.c.d.bitrepository.BitmagUtils.initialize - Path to Bitrepository settings is /home/devel/extra_configs/bitmag_config.
2022-07-11 08:41:35.720 INFO d.n.c.d.bitrepository.HostNameUtils.getHostName - Local hostname (provided by getCanonicalHostName): kb-test-acs-001.kb.dk
2022-07-11 08:41:37.542 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - BitmagArcRepositoryClient will timeout on each get request after 300000 milliseconds.
2022-07-11 08:41:37.542 DEBUG d.n.common.distribute.JMSConnection.setListener - Adding dk.netarkivet.archive.arcrepository.distribute.BitmagArcRepositoryClient@4fc5e095 as listener to Test6_COMMON_THIS_REPOS_CLIENT_130_226_228_5_IS
2022-07-11 08:41:37.564 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - BitmagArcRepositoryClient listens for replies on channel '[Queue 'Test6_COMMON_THIS_REPOS_CLIENT_130_226_228_5_IS']'
2022-07-11 08:41:37.565 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - Getting bitmag config from settings.common.arcrepositoryClient.bitrepository.settingsDir=/home/devel/extra_configs/bitmag_config
2022-07-11 08:41:37.565 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - Using 'Test6' as default collectionID
2022-07-11 08:41:37.572 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - Storing tempfiles in folder 'arcrepositoryTemp'
2022-07-11 08:41:37.602 INFO d.n.common.utils.HttpsClientBuilder.<init> - Creating Aggressive retry handler for https queries.
2022-07-11 08:41:37.728 DEBUG d.n.c.utils.BasicTwoWaySSLProvider.loadPrivateKey - X509CertificateHolder found
2022-07-11 08:41:37.729 DEBUG d.n.c.utils.BasicTwoWaySSLProvider.loadPrivateKey - PrivateKeyInfo found
2022-07-11 08:41:37.797 INFO d.n.h.indexserver.RawMetadataCache.<init> - Metadata cache for 'cdxdata' is fetching metadata with urls matching 'metadata://[^/]*/crawl/index/cdx.*' and mimetype matching 'application/x-cdx'. Migration of duplicate records is disabled
2022-07-11 08:41:37.797 INFO d.n.h.indexserver.FileBasedCache.<init> - Metadata cache for 'cdxindex' uses directory '/home/devel/Test6/cache/cdxindex'
2022-07-11 08:41:37.812 INFO d.n.h.indexserver.FileBasedCache.<init> - Metadata cache for 'crawllog' uses directory '/home/devel/Test6/cache/crawllog'
2022-07-11 08:41:37.812 INFO d.n.h.indexserver.RawMetadataCache.<init> - Metadata cache for 'crawllog' is fetching metadata with urls matching 'metadata://[^/]*/crawl/logs/crawl\.log.*' and mimetype matching 'text/plain'. Migration of duplicate records is disabled
2022-07-11 08:41:37.813 INFO d.n.h.indexserver.FileBasedCache.<init> - Metadata cache for 'dedupcrawllogindex' uses directory '/home/devel/Test6/cache/dedupcrawllogindex'
2022-07-11 08:41:37.813 INFO d.n.h.indexserver.FileBasedCache.<init> - Metadata cache for 'cdxdata' uses directory '/home/devel/Test6/cache/cdxdata'
2022-07-11 08:41:37.814 INFO d.n.h.indexserver.RawMetadataCache.<init> - Metadata cache for 'cdxdata' is fetching metadata with urls matching 'metadata://[^/]*/crawl/index/cdx.*' and mimetype matching 'application/x-cdx'. Migration of duplicate records is disabled
2022-07-11 08:41:37.814 INFO d.n.h.indexserver.FileBasedCache.<init> - Metadata cache for 'crawllog' uses directory '/home/devel/Test6/cache/crawllog'
2022-07-11 08:41:37.815 INFO d.n.h.indexserver.RawMetadataCache.<init> - Metadata cache for 'crawllog' is fetching metadata with urls matching 'metadata://[^/]*/crawl/logs/crawl\.log.*' and mimetype matching 'text/plain'. Migration of duplicate records is disabled
2022-07-11 08:41:37.815 INFO d.n.h.indexserver.FileBasedCache.<init> - Metadata cache for 'fullcrawllogindex' uses directory '/home/devel/Test6/cache/fullcrawllogindex'
2022-07-11 08:41:37.816 INFO d.n.h.indexserver.FileBasedCache.<init> - Metadata cache for 'cdxdata' uses directory '/home/devel/Test6/cache/cdxdata'
2022-07-11 08:41:37.816 INFO d.n.h.indexserver.RawMetadataCache.<init> - Metadata cache for 'cdxdata' is fetching metadata with urls matching 'metadata://[^/]*/crawl/index/cdx.*' and mimetype matching 'application/x-cdx'. Migration of duplicate records is disabled
2022-07-11 08:41:37.817 DEBUG d.n.h.indexserver.FileBasedCache.cache - Waiting to enter synchronization on /home/devel/Test6/cache/cdxindex/empty-cache.working
2022-07-11 08:41:37.817 DEBUG d.n.h.indexserver.FileBasedCache.cache - locking filechannel for file '/home/devel/Test6/cache/cdxindex/empty-cache.working' (thread = main)
2022-07-11 08:41:37.822 DEBUG d.n.common.utils.ProcessUtils.runProcess - Running external program: sort /home/devel/Test6/cache/cdxindex/empty-cache -o /home/devel/Test6/cache/cdxindex/empty-cache.unsorted with environment LANG=C
2022-07-11 08:41:37.835 DEBUG d.n.h.indexserver.FileBasedCache.cache - release lock on filechannel sun.nio.ch.FileChannelImpl@18f20260
2022-07-11 08:41:37.835 INFO d.n.h.indexserver.FileBasedCache.getIndex - Generated index '/home/devel/Test6/cache/cdxindex/empty-cache' of id '[]', request was for '[]'
2022-07-11 08:41:37.836 DEBUG d.n.h.indexserver.FileBasedCache.cache - Waiting to enter synchronization on /home/devel/Test6/cache/dedupcrawllogindex/empty-cache.working
2022-07-11 08:41:37.836 DEBUG d.n.h.indexserver.FileBasedCache.cache - locking filechannel for file '/home/devel/Test6/cache/dedupcrawllogindex/empty-cache.working' (thread = main)
2022-07-11 08:41:37.836 INFO d.n.h.indexserver.CrawlLogIndexCache.prepareCombine - Starting to generate dedupcrawllogindex for the 0 jobs: []
2022-07-11 08:41:37.836 INFO d.n.h.indexserver.CrawlLogIndexCache.combine - Starting combine task #1. This combines a dataset with 0 crawl logs (thread = main)
2022-07-11 08:41:37.957 INFO d.n.h.indexserver.CrawlLogIndexCache.combine - Adding the final 0 subindices to main index. Forcing index to contain max 42 files (related to combine task #1)
2022-07-11 08:41:37.965 INFO d.n.h.indexserver.CrawlLogIndexCache.combine - Adding operation completed (combine task #1)!
2022-07-11 08:41:37.966 INFO d.n.h.indexserver.CrawlLogIndexCache.combine - Closed index (related to combine task #1
2022-07-11 08:41:37.966 INFO d.n.h.indexserver.CrawlLogIndexCache.combine - Gzip-compressing the individual 2 index files of combine task # 1
2022-07-11 08:41:37.970 INFO d.n.h.indexserver.CrawlLogIndexCache.combine - Completed combine task #1 that combined a dataset with 0 crawl logs (entries in combined index: 0) - compressed index has size 72 bytes
2022-07-11 08:41:37.970 DEBUG d.n.h.indexserver.FileBasedCache.cache - release lock on filechannel sun.nio.ch.FileChannelImpl@55e7a35c
2022-07-11 08:41:37.971 INFO d.n.h.indexserver.FileBasedCache.getIndex - Generated index '/home/devel/Test6/cache/dedupcrawllogindex/empty-cache' of id '[]', request was for '[]'
2022-07-11 08:41:37.971 DEBUG d.n.h.indexserver.FileBasedCache.cache - Waiting to enter synchronization on /home/devel/Test6/cache/fullcrawllogindex/empty-cache.working
2022-07-11 08:41:37.971 DEBUG d.n.h.indexserver.FileBasedCache.cache - locking filechannel for file '/home/devel/Test6/cache/fullcrawllogindex/empty-cache.working' (thread = main)
2022-07-11 08:41:37.972 INFO d.n.h.indexserver.CrawlLogIndexCache.prepareCombine - Starting to generate fullcrawllogindex for the 0 jobs: []
2022-07-11 08:41:37.972 INFO d.n.h.indexserver.CrawlLogIndexCache.combine - Starting combine task #1. This combines a dataset with 0 crawl logs (thread = main)
2022-07-11 08:41:37.974 INFO d.n.h.indexserver.CrawlLogIndexCache.combine - Adding the final 0 subindices to main index. Forcing index to contain max 42 files (related to combine task #1)
2022-07-11 08:41:37.976 INFO d.n.h.indexserver.CrawlLogIndexCache.combine - Adding operation completed (combine task #1)!
2022-07-11 08:41:37.976 INFO d.n.h.indexserver.CrawlLogIndexCache.combine - Closed index (related to combine task #1
2022-07-11 08:41:37.977 INFO d.n.h.indexserver.CrawlLogIndexCache.combine - Gzip-compressing the individual 2 index files of combine task # 1
2022-07-11 08:41:37.978 INFO d.n.h.indexserver.CrawlLogIndexCache.combine - Completed combine task #1 that combined a dataset with 0 crawl logs (entries in combined index: 0) - compressed index has size 72 bytes
2022-07-11 08:41:37.978 DEBUG d.n.h.indexserver.FileBasedCache.cache - release lock on filechannel sun.nio.ch.FileChannelImpl@5922ae77
2022-07-11 08:41:37.978 INFO d.n.h.indexserver.FileBasedCache.getIndex - Generated index '/home/devel/Test6/cache/fullcrawllogindex/empty-cache' of id '[]', request was for '[]'
2022-07-11 08:41:38.007 INFO d.n.c.utils.hadoop.HadoopJobUtils.doKerberosLogin - Kerberos: /home/devel/extra_configs/kerberos_config/krb5.conf, /home/devel/extra_configs/kerberos_config/nat-nas-devel.keytab, nat-nas-devel@KBHPC.KB.DK
2022-07-11 08:41:38.232 WARN o.a.hadoop.util.NativeCodeLoader.<clinit> - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
08:41:34,115 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [/home/devel/Test6/conf/logback_ViewerProxyApplication.xml] at [file:/home/devel/Test6/conf/logback_ViewerProxyApplication.xml]
08:41:34,432 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - debug attribute not set
08:41:34,434 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
08:41:34,442 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [STDOUT]
08:41:34,493 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
08:41:34,630 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
08:41:34,649 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [FILE]
08:41:34,697 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@117009527 - No compression will be used
08:41:34,702 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@117009527 - Will use the pattern ./log/ViewerProxyApplication.%d{yyyy-MM-dd}.%i.log for the active file
08:41:34,706 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@be64738 - The date pattern is 'yyyy-MM-dd' from file name pattern './log/ViewerProxyApplication.%d{yyyy-MM-dd}.%i.log'.
08:41:34,706 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@be64738 - Roll-over at midnight.
08:41:34,719 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@be64738 - Setting initial period to Mon Jul 11 08:41:34 CEST 2022
08:41:34,719 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@be64738 - SizeAndTimeBasedFNATP is deprecated. Use SizeAndTimeBasedRollingPolicy instead
08:41:34,719 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@be64738 - For more information see http://logback.qos.ch/manual/appenders.html#SizeAndTimeBasedRollingPolicy
08:41:34,721 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
08:41:34,727 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - Active log file name: ./log/ViewerProxyApplication.log
08:41:34,727 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - File property is set to [./log/ViewerProxyApplication.log]
08:41:34,728 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [dk.netarkivet.monitor.logging.CachingSLF4JAppender]
08:41:35,456 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [MONITOR]
08:41:35,458 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to WARN
08:41:35,458 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[ROOT]
08:41:35,459 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [FILE] to Logger[ROOT]
08:41:35,459 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [MONITOR] to Logger[ROOT]
08:41:35,459 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [dk.netarkivet] to DEBUG
08:41:35,459 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
08:41:35,460 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@3ba9ad43 - Registering current configuration as safe fallback point
2022-07-11 08:41:35.472 DEBUG dk.netarkivet.common.utils.Settings.invoke0 - Value found in loaded data: SB,KB
SLF4J: A number (1) of logging calls during the initialization phase have been intercepted and are
SLF4J: now being replayed. These are subject to the filtering rules of the underlying logging system.
SLF4J: See also http://www.slf4j.org/codes.html#replay
Starting dk.netarkivet.viewerproxy.ViewerProxy
Version: 7.4-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/5605ede33d8e9feaccf41bc5ab16c9b6dc7a8800)
2022-07-11 08:41:35.485 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Starting dk.netarkivet.viewerproxy.ViewerProxy
Version: 7.4-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/5605ede33d8e9feaccf41bc5ab16c9b6dc7a8800)
Java VM: 1.8.0_332
2022-07-11 08:41:35.486 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Java VM: 1.8.0_332
java.home: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.332.b09-1.el7_9.x86_64/jre
2022-07-11 08:41:35.486 INFO d.n.common.utils.ApplicationUtils.logAndPrint - java.home: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.332.b09-1.el7_9.x86_64/jre
Working dir: /home/devel/Test6
2022-07-11 08:41:35.490 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Working dir: /home/devel/Test6
2022-07-11 08:41:35.490 INFO d.n.common.utils.ApplicationUtils.startApp - Using settings files '/home/devel/Test6/conf/settings_ViewerProxyApplication.xml'
2022-07-11 08:41:35.717 INFO d.n.m.d.JMSMonitorRegistryClient.register - Registering this client for monitoring every 1 minutes, using hostname 'kb-test-acs-001.kb.dk' and JMX/RMI ports 8370/8470
2022-07-11 08:41:35.717 INFO d.n.c.m.MBeanConnectorCreator.exposeJMXMBeanServer - Registered mbean server in registry on port 8370 communicating on port 8470 using password file 'conf/jmxremote.password'.
Service URL is service:jmx:rmi://kb-test-acs-001.kb.dk:8470/jndi/rmi://kb-test-acs-001.kb.dk:8370/jmxrmi
dk.netarkivet.viewerproxy.ViewerProxy Running
2022-07-11 08:41:35.720 INFO d.n.common.utils.ApplicationUtils.logAndPrint - dk.netarkivet.viewerproxy.ViewerProxy Running
2022-07-11 08:41:35.730 INFO d.n.c.distribute.JMSConnectionSunMQ.<init> - Creating instance of dk.netarkivet.common.distribute.JMSConnectionSunMQ
2022-07-11 08:41:35.731 DEBUG d.n.common.distribute.JMSConnection.initConnection - Initializing a JMS connection dk.netarkivet.common.distribute.JMSConnectionSunMQ
2022-07-11 08:41:35.731 INFO d.n.c.distribute.JMSConnectionSunMQ.getConnectionFactory - Establishing SunMQ JMS Connection to 'kb-test-adm-001.kb.dk:7676'
2022-07-11 08:41:35.742 DEBUG d.n.c.d.bitrepository.BitmagUtils.initialize - Bitrepository settings will be loaded from /home/devel/extra_configs/bitmag_config.
2022-07-11 08:41:35.742 DEBUG d.n.c.d.bitrepository.BitmagUtils.initialize - Path to Bitrepository settings is /home/devel/extra_configs/bitmag_config.
2022-07-11 08:41:35.747 INFO d.n.c.d.bitrepository.HostNameUtils.getHostName - Local hostname (provided by getCanonicalHostName): kb-test-acs-001.kb.dk
2022-07-11 08:41:37.536 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - BitmagArcRepositoryClient will timeout on each get request after 300000 milliseconds.
2022-07-11 08:41:37.537 DEBUG d.n.common.distribute.JMSConnection.setListener - Adding dk.netarkivet.archive.arcrepository.distribute.BitmagArcRepositoryClient@793138bd as listener to Test6_COMMON_THIS_REPOS_CLIENT_130_226_228_5_VP
2022-07-11 08:41:37.559 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - BitmagArcRepositoryClient listens for replies on channel '[Queue 'Test6_COMMON_THIS_REPOS_CLIENT_130_226_228_5_VP']'
2022-07-11 08:41:37.560 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - Getting bitmag config from settings.common.arcrepositoryClient.bitrepository.settingsDir=/home/devel/extra_configs/bitmag_config
2022-07-11 08:41:37.561 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - Using 'Test6' as default collectionID
2022-07-11 08:41:37.581 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - Storing tempfiles in folder 'arcrepositoryTemp'
2022-07-11 08:41:37.590 INFO d.n.common.utils.HttpsClientBuilder.<init> - Creating Aggressive retry handler for https queries.
2022-07-11 08:41:37.688 DEBUG d.n.c.utils.BasicTwoWaySSLProvider.loadPrivateKey - X509CertificateHolder found
2022-07-11 08:41:37.690 DEBUG d.n.c.utils.BasicTwoWaySSLProvider.loadPrivateKey - PrivateKeyInfo found
2022-07-11 08:41:37.740 INFO d.n.h.indexserver.FileBasedCache.<init> - Metadata cache for 'FULL_CRAWL_LOG' uses directory '/home/devel/Test6/cache/FULL_CRAWL_LOG'
2022-07-11 08:41:37.757 INFO d.n.viewerproxy.ARCArchiveAccess.<init> - Constructed instance of ARCArchiveAccess with TryToLookupUriAsFtp: false
2022-07-11 08:41:37.803 INFO dk.netarkivet.viewerproxy.WebProxy.<init> - Starting viewerproxy jetty on port 8078
--------------------------------------------
STARTING MACHINE: devel@kb-test-way-001.kb.dk
Starting all applications on: kb-test-way-001.kb.dk
Starting linux application: WaybackIndexerApplication
Starting linux application: AggregatorApplication
08:41:40,758 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [/home/devel/Test6/conf/logback_AggregatorApplication.xml] at [file:/home/devel/Test6/conf/logback_AggregatorApplication.xml]
08:41:41,142 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - debug attribute not set
08:41:41,151 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
08:41:41,173 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [STDOUT]
08:41:41,223 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
08:41:41,397 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
08:41:41,424 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [FILE]
08:41:41,507 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@477289012 - No compression will be used
08:41:41,512 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@477289012 - Will use the pattern ./log/AggregatorApplication.%d{yyyy-MM-dd}.%i.log for the active file
08:41:41,522 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@6b0c2d26 - The date pattern is 'yyyy-MM-dd' from file name pattern './log/AggregatorApplication.%d{yyyy-MM-dd}.%i.log'.
08:41:41,522 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@6b0c2d26 - Roll-over at midnight.
08:41:41,555 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@6b0c2d26 - Setting initial period to Mon Jul 11 08:41:41 CEST 2022
08:41:41,555 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@6b0c2d26 - SizeAndTimeBasedFNATP is deprecated. Use SizeAndTimeBasedRollingPolicy instead
08:41:41,555 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@6b0c2d26 - For more information see http://logback.qos.ch/manual/appenders.html#SizeAndTimeBasedRollingPolicy
08:41:41,560 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
08:41:41,569 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - Active log file name: ./log/AggregatorApplication.log
08:41:41,569 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - File property is set to [./log/AggregatorApplication.log]
08:41:41,572 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [dk.netarkivet.monitor.logging.CachingSLF4JAppender]
08:41:42,897 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [MONITOR]
08:41:42,906 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to WARN
08:41:42,906 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[ROOT]
08:41:42,913 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [FILE] to Logger[ROOT]
08:41:42,913 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [MONITOR] to Logger[ROOT]
08:41:42,914 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [dk.netarkivet] to DEBUG
08:41:42,914 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
08:41:42,917 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@3d3fcdb0 - Registering current configuration as safe fallback point
2022-07-11 08:41:42.938 DEBUG dk.netarkivet.common.utils.Settings.invoke0 - Value found in loaded data: SB,KB
SLF4J: A number (1) of logging calls during the initialization phase have been intercepted and are
SLF4J: now being replayed. These are subject to the filtering rules of the underlying logging system.
SLF4J: See also http://www.slf4j.org/codes.html#replay
Starting dk.netarkivet.wayback.aggregator.AggregationWorker
Version: 7.4-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/5605ede33d8e9feaccf41bc5ab16c9b6dc7a8800)
2022-07-11 08:41:42.956 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Starting dk.netarkivet.wayback.aggregator.AggregationWorker
Version: 7.4-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/5605ede33d8e9feaccf41bc5ab16c9b6dc7a8800)
Java VM: 1.8.0_25
2022-07-11 08:41:42.962 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Java VM: 1.8.0_25
java.home: /usr/local/java.old/jdk1.8.0_25_x64/jre
2022-07-11 08:41:42.963 INFO d.n.common.utils.ApplicationUtils.logAndPrint - java.home: /usr/local/java.old/jdk1.8.0_25_x64/jre
Working dir: /home/devel/Test6
2022-07-11 08:41:42.964 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Working dir: /home/devel/Test6
2022-07-11 08:41:42.966 INFO d.n.common.utils.ApplicationUtils.startApp - Using settings files '/home/devel/Test6/conf/settings_AggregatorApplication.xml'
2022-07-11 08:41:43.211 INFO d.n.m.d.JMSMonitorRegistryClient.register - Registering this client for monitoring every 1 minutes, using hostname 'kb-test-way-001.kb.dk' and JMX/RMI ports 9170/9270
2022-07-11 08:41:43.211 INFO d.n.c.m.MBeanConnectorCreator.exposeJMXMBeanServer - Registered mbean server in registry on port 9170 communicating on port 9270 using password file 'conf/jmxremote.password'.
Service URL is service:jmx:rmi://kb-test-way-001.kb.dk:9270/jndi/rmi://kb-test-way-001.kb.dk:9170/jmxrmi
dk.netarkivet.wayback.aggregator.AggregationWorker Running
2022-07-11 08:41:43.214 INFO d.n.common.utils.ApplicationUtils.logAndPrint - dk.netarkivet.wayback.aggregator.AggregationWorker Running
2022-07-11 08:41:43.253 INFO d.n.c.distribute.JMSConnectionSunMQ.<init> - Creating instance of dk.netarkivet.common.distribute.JMSConnectionSunMQ
2022-07-11 08:41:43.256 DEBUG d.n.common.distribute.JMSConnection.initConnection - Initializing a JMS connection dk.netarkivet.common.distribute.JMSConnectionSunMQ
2022-07-11 08:41:43.256 INFO d.n.c.distribute.JMSConnectionSunMQ.getConnectionFactory - Establishing SunMQ JMS Connection to 'kb-test-adm-001.kb.dk:7676'
2022-07-11 08:41:43.284 DEBUG d.n.w.aggregator.AggregationWorker.runAggregation - No new raw index files found in '/home/devel/Test6/batchOutputDir' skipping aggregation
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/devel/Test6/lib/logback-classic-1.2.3.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/devel/Test6/lib/slf4j-log4j12-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
08:41:40,919 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [/home/devel/Test6/conf/logback_WaybackIndexerApplication.xml] at [file:/home/devel/Test6/conf/logback_WaybackIndexerApplication.xml]
08:41:41,523 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - debug attribute not set
08:41:41,534 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
08:41:41,563 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [STDOUT]
08:41:41,597 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
08:41:41,740 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
08:41:41,761 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [FILE]
08:41:41,846 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@1201484275 - No compression will be used
08:41:41,852 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@1201484275 - Will use the pattern ./log/WaybackIndexerApplication.%d{yyyy-MM-dd}.%i.log for the active file
08:41:41,857 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@40ef3420 - The date pattern is 'yyyy-MM-dd' from file name pattern './log/WaybackIndexerApplication.%d{yyyy-MM-dd}.%i.log'.
08:41:41,857 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@40ef3420 - Roll-over at midnight.
08:41:41,864 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@40ef3420 - Setting initial period to Mon Jul 11 08:41:41 CEST 2022
08:41:41,865 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@40ef3420 - SizeAndTimeBasedFNATP is deprecated. Use SizeAndTimeBasedRollingPolicy instead
08:41:41,865 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@40ef3420 - For more information see http://logback.qos.ch/manual/appenders.html#SizeAndTimeBasedRollingPolicy
08:41:41,869 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
08:41:41,876 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - Active log file name: ./log/WaybackIndexerApplication.log
08:41:41,876 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - File property is set to [./log/WaybackIndexerApplication.log]
08:41:41,878 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [dk.netarkivet.monitor.logging.CachingSLF4JAppender]
08:41:42,860 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [MONITOR]
08:41:42,874 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to WARN
08:41:42,874 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[ROOT]
08:41:42,874 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [FILE] to Logger[ROOT]
08:41:42,874 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [MONITOR] to Logger[ROOT]
08:41:42,875 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [dk.netarkivet] to DEBUG
08:41:42,875 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
08:41:42,875 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@498d318c - Registering current configuration as safe fallback point
SLF4J: Actual binding is of type [ch.qos.logback.classic.util.ContextSelectorStaticBinder]
2022-07-11 08:41:42.896 DEBUG dk.netarkivet.common.utils.Settings.invoke0 - Value found in loaded data: SB,KB
SLF4J: A number (1) of logging calls during the initialization phase have been intercepted and are
SLF4J: now being replayed. These are subject to the filtering rules of the underlying logging system.
SLF4J: See also http://www.slf4j.org/codes.html#replay
Starting dk.netarkivet.wayback.indexer.WaybackIndexer
Version: 7.4-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/5605ede33d8e9feaccf41bc5ab16c9b6dc7a8800)
2022-07-11 08:41:42.934 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Starting dk.netarkivet.wayback.indexer.WaybackIndexer
Version: 7.4-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/5605ede33d8e9feaccf41bc5ab16c9b6dc7a8800)
Java VM: 1.8.0_25
2022-07-11 08:41:42.934 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Java VM: 1.8.0_25
java.home: /usr/local/java.old/jdk1.8.0_25_x64/jre
2022-07-11 08:41:42.935 INFO d.n.common.utils.ApplicationUtils.logAndPrint - java.home: /usr/local/java.old/jdk1.8.0_25_x64/jre
Working dir: /home/devel/Test6
2022-07-11 08:41:42.935 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Working dir: /home/devel/Test6
2022-07-11 08:41:42.935 INFO d.n.common.utils.ApplicationUtils.startApp - Using settings files '/home/devel/Test6/conf/settings_WaybackIndexerApplication.xml'
2022-07-11 08:41:43.220 INFO d.n.m.d.JMSMonitorRegistryClient.register - Registering this client for monitoring every 1 minutes, using hostname 'kb-test-way-001.kb.dk' and JMX/RMI ports 8171/8271
2022-07-11 08:41:43.221 INFO d.n.c.m.MBeanConnectorCreator.exposeJMXMBeanServer - Registered mbean server in registry on port 8171 communicating on port 8271 using password file 'conf/jmxremote.password'.
Service URL is service:jmx:rmi://kb-test-way-001.kb.dk:8271/jndi/rmi://kb-test-way-001.kb.dk:8171/jmxrmi
dk.netarkivet.wayback.indexer.WaybackIndexer Running
2022-07-11 08:41:43.229 INFO d.n.common.utils.ApplicationUtils.logAndPrint - dk.netarkivet.wayback.indexer.WaybackIndexer Running
2022-07-11 08:41:43.255 INFO d.n.c.distribute.JMSConnectionSunMQ.<init> - Creating instance of dk.netarkivet.common.distribute.JMSConnectionSunMQ
2022-07-11 08:41:43.262 DEBUG d.n.common.distribute.JMSConnection.initConnection - Initializing a JMS connection dk.netarkivet.common.distribute.JMSConnectionSunMQ
2022-07-11 08:41:43.262 INFO d.n.c.distribute.JMSConnectionSunMQ.getConnectionFactory - Establishing SunMQ JMS Connection to 'kb-test-adm-001.kb.dk:7676'
2022-07-11 08:41:43.274 DEBUG d.n.c.d.bitrepository.BitmagUtils.initialize - Bitrepository settings will be loaded from /home/devel/extra_configs/bitmag_config.
2022-07-11 08:41:43.332 DEBUG d.n.c.d.bitrepository.BitmagUtils.initialize - Path to Bitrepository settings is /home/devel/extra_configs/bitmag_config.
2022-07-11 08:41:43.346 INFO d.n.c.d.bitrepository.HostNameUtils.getHostName - Local hostname (provided by getCanonicalHostName): kb-test-way-001.kb.dk
--------------------------------------------
--------------------------------------------
STARTING MACHINE: netarkdv@sb-test-har-001.statsbiblioteket.dk
Starting all applications on: sb-test-har-001.statsbiblioteket.dk
Starting linux application: HarvestControllerApplication_sbhigh_h3
08:41:44,363 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [/home/netarkdv/Test6/conf/logback_HarvestControllerApplication_sbhigh_h3.xml] at [file:/home/netarkdv/Test6/conf/logback_HarvestControllerApplication_sbhigh_h3.xml]
08:41:44,496 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - debug attribute not set
08:41:44,498 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
08:41:44,503 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [STDOUT]
08:41:44,512 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
08:41:44,549 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
08:41:44,552 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [FILE]
08:41:44,567 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@858952163 - No compression will be used
08:41:44,568 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@858952163 - Will use the pattern ./log/HarvestControllerApplication_sbhigh_h3.%d{yyyy-MM-dd}.%i.log for the active file
08:41:44,569 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@479d31f3 - The date pattern is 'yyyy-MM-dd' from file name pattern './log/HarvestControllerApplication_sbhigh_h3.%d{yyyy-MM-dd}.%i.log'.
08:41:44,569 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@479d31f3 - Roll-over at midnight.
08:41:44,572 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@479d31f3 - Setting initial period to Mon Jul 11 08:41:44 CEST 2022
08:41:44,572 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@479d31f3 - SizeAndTimeBasedFNATP is deprecated. Use SizeAndTimeBasedRollingPolicy instead
08:41:44,572 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@479d31f3 - For more information see http://logback.qos.ch/manual/appenders.html#SizeAndTimeBasedRollingPolicy
08:41:44,573 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
08:41:44,575 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - Active log file name: ./log/HarvestControllerApplication_sbhigh_h3.log
08:41:44,575 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - File property is set to [./log/HarvestControllerApplication_sbhigh_h3.log]
08:41:44,575 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [dk.netarkivet.monitor.logging.CachingSLF4JAppender]
08:41:44,803 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [MONITOR]
08:41:44,805 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to WARN
08:41:44,805 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[ROOT]
08:41:44,805 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [FILE] to Logger[ROOT]
08:41:44,805 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [MONITOR] to Logger[ROOT]
08:41:44,806 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [dk.netarkivet] to DEBUG
08:41:44,806 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
08:41:44,806 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@40ef3420 - Registering current configuration as safe fallback point
2022-07-11 08:41:44.809 DEBUG dk.netarkivet.common.utils.Settings.invoke0 - Value found in loaded data: SB,KB
SLF4J: A number (1) of logging calls during the initialization phase have been intercepted and are
SLF4J: now being replayed. These are subject to the filtering rules of the underlying logging system.
SLF4J: See also http://www.slf4j.org/codes.html#replay
Starting dk.netarkivet.harvester.heritrix3.HarvestControllerServer
Version: 7.4-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/5605ede33d8e9feaccf41bc5ab16c9b6dc7a8800)
2022-07-11 08:41:44.813 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Starting dk.netarkivet.harvester.heritrix3.HarvestControllerServer
Version: 7.4-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/5605ede33d8e9feaccf41bc5ab16c9b6dc7a8800)
Java VM: 1.8.0_332
2022-07-11 08:41:44.814 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Java VM: 1.8.0_332
java.home: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.332.b09-2.el8_6.x86_64/jre
2022-07-11 08:41:44.814 INFO d.n.common.utils.ApplicationUtils.logAndPrint - java.home: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.332.b09-2.el8_6.x86_64/jre
Working dir: /home/netarkdv/Test6
2022-07-11 08:41:44.814 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Working dir: /home/netarkdv/Test6
2022-07-11 08:41:44.814 INFO d.n.common.utils.ApplicationUtils.startApp - Using settings files '/home/netarkdv/Test6/conf/settings_HarvestControllerApplication_sbhigh_h3.xml'
2022-07-11 08:41:44.899 INFO d.n.m.d.JMSMonitorRegistryClient.register - Registering this client for monitoring every 1 minutes, using hostname 'sb-test-har-001.statsbiblioteket.dk' and JMX/RMI ports 8170/8270
2022-07-11 08:41:44.900 INFO d.n.c.m.MBeanConnectorCreator.exposeJMXMBeanServer - Registered mbean server in registry on port 8170 communicating on port 8270 using password file 'conf/jmxremote.password'.
Service URL is service:jmx:rmi://sb-test-har-001.statsbiblioteket.dk:8270/jndi/rmi://sb-test-har-001.statsbiblioteket.dk:8170/jmxrmi
dk.netarkivet.harvester.heritrix3.HarvestControllerServer Running
2022-07-11 08:41:44.904 INFO d.n.common.utils.ApplicationUtils.logAndPrint - dk.netarkivet.harvester.heritrix3.HarvestControllerServer Running
2022-07-11 08:41:44.911 INFO d.n.h.h.HarvestControllerServer.<init> - Starting HarvestControllerServer.
2022-07-11 08:41:44.911 INFO d.n.h.h.HarvestControllerServer.<init> - Bound to harvest channel 'HIGHPRIORITY'
2022-07-11 08:41:44.911 INFO d.n.h.h.HarvestControllerServer.<init> - Serverdir: 'harvester_high'
2022-07-11 08:41:44.911 INFO d.n.h.h.HarvestControllerServer.<init> - Harvesting requires at least 400000000 bytes free.
2022-07-11 08:41:44.915 INFO d.n.c.distribute.JMSConnectionSunMQ.<init> - Creating instance of dk.netarkivet.common.distribute.JMSConnectionSunMQ
2022-07-11 08:41:44.915 DEBUG d.n.common.distribute.JMSConnection.initConnection - Initializing a JMS connection dk.netarkivet.common.distribute.JMSConnectionSunMQ
2022-07-11 08:41:44.915 INFO d.n.c.distribute.JMSConnectionSunMQ.getConnectionFactory - Establishing SunMQ JMS Connection to 'kb-test-adm-001.kb.dk:7676'
2022-07-11 08:41:45.034 DEBUG d.n.c.d.bitrepository.BitmagUtils.initialize - Bitrepository settings will be loaded from /home/netarkdv/extra_configs/bitmag_config.
2022-07-11 08:41:45.042 DEBUG d.n.c.d.bitrepository.BitmagUtils.initialize - Path to Bitrepository settings is /home/netarkdv/extra_configs/bitmag_config.
2022-07-11 08:41:45.046 INFO d.n.c.d.bitrepository.HostNameUtils.getHostName - Local hostname (provided by getCanonicalHostName): sb-test-har-001.statsbiblioteket.dk
2022-07-11 08:41:45.900 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - BitmagArcRepositoryClient will timeout on each get request after 300000 milliseconds.
2022-07-11 08:41:45.901 DEBUG d.n.common.distribute.JMSConnection.setListener - Adding dk.netarkivet.archive.arcrepository.distribute.BitmagArcRepositoryClient@1c7fd41f as listener to Test6_COMMON_THIS_REPOS_CLIENT_130_225_27_140_HCS_SBHIGH_H3
2022-07-11 08:41:45.932 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - BitmagArcRepositoryClient listens for replies on channel '[Queue 'Test6_COMMON_THIS_REPOS_CLIENT_130_225_27_140_HCS_SBHIGH_H3']'
2022-07-11 08:41:45.934 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - Getting bitmag config from settings.common.arcrepositoryClient.bitrepository.settingsDir=/home/netarkdv/extra_configs/bitmag_config
2022-07-11 08:41:45.935 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - Using 'Test6' as default collectionID
2022-07-11 08:41:45.938 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - Storing tempfiles in folder 'arcrepositoryTemp'
2022-07-11 08:41:45.944 INFO d.n.common.utils.HttpsClientBuilder.<init> - Creating Aggressive retry handler for https queries.
2022-07-11 08:41:45.972 INFO d.n.c.utils.BasicTwoWaySSLProvider.loadPrivateKey - Key file '/home/devel/extra_configs/https_config/https_key.pem' with private key and certificate does not exist!
2022-07-11 08:41:45.989 DEBUG d.n.h.h.HarvestControllerServer.<init> - Obtained JMS connection.
2022-07-11 08:41:45.990 INFO d.n.h.h.HarvestControllerServer.<init> - SEND_READY_DELAY used by HarvestControllerServer is 300
2022-07-11 08:41:45.991 INFO d.n.h.heritrix3.PostProcessing.processOldJobs - Looking for unprocessed crawldata in 'harvester_high'
2022-07-11 08:41:45.991 DEBUG d.n.common.distribute.JMSConnection.setListener - Adding dk.netarkivet.harvester.heritrix3.HarvestControllerServer@1de5f0ef as listener to Test6_COMMON_HCHAN_VAL_RESP
2022-07-11 08:41:46.040 INFO d.n.h.h.HarvestControllerServer.<init> - Requested to check the validity of harvest channel 'HIGHPRIORITY'
2022-07-11 08:41:46.067 INFO d.n.h.h.HarvestControllerServer.visit - Received message stating that channel 'HIGHPRIORITY' is valid.
2022-07-11 08:41:46.068 INFO d.n.h.h.HarvestControllerServer.beginListeningIfSpaceAvailable - Starts to listen to new jobs on queue '[Queue 'Test6_COMMON_JOB_PARTIAL_HIGHPRIORITY']'
2022-07-11 08:41:46.068 DEBUG d.n.common.distribute.JMSConnection.setListener - Adding dk.netarkivet.harvester.heritrix3.HarvestControllerServer@1de5f0ef as listener to Test6_COMMON_JOB_PARTIAL_HIGHPRIORITY
--------------------------------------------
Completed the script all_test.sh
Finished: SUCCESS
Console Output
Help us localize this page