ased on schema from: jar:file:/home/devel/ResetCollection/bitrepository-client/lib/bitrepository-repository-settings-xsd-12.jar!/xsd/RepositorySettings.xsd
15:00:38.476 DEBUG o.b.c.settings.XMLFileSettingsLoader - Loading the settings file '/home/devel/ResetCollection/bitrepository-client/bin/../conf/RepositorySettings.xml'.
15:00:38.770 DEBUG org.bitrepository.common.JaxbHelper - Creating JAXBHelper based on schema from: jar:file:/home/devel/ResetCollection/bitrepository-client/lib/bitrepository-reference-settings-1.9.0.1.jar!/xsd/ReferenceSettings.xsd
15:00:38.821 DEBUG o.b.c.settings.XMLFileSettingsLoader - Loading the settings file '/home/devel/ResetCollection/bitrepository-client/bin/../conf/ReferenceSettings.xml'.
15:00:39.526 DEBUG o.b.p.security.BasicSecurityManager - X509CertificateHolder found
15:00:39.552 DEBUG o.b.p.security.BasicSecurityManager - Got something, that we don't (yet) recognize. Class: PEMKeyPair
15:00:39.553 DEBUG o.b.p.security.BasicSecurityManager - PrivateKeyInfo found
15:00:39.759 INFO o.b.p.activemq.ActiveMQMessageBus - Initializing ActiveMQMessageBus:'org.bitrepository.settings.repositorysettings.MessageBusConfiguration@3754a4bf[name=netarkiv-aar-bit-master-devel-01 broker, url=failover:(ssl://netarkiv-aar-bit-master-devel-01.statsbiblioteket.dk:61612?keepAlive=true)]'.
15:00:39.759 DEBUG org.bitrepository.common.JaxbHelper - Creating JAXBHelper based on schema from: jar:file:/home/devel/ResetCollection/bitrepository-client/lib/bitrepository-message-xml-xsd-30.jar!/xsd/BitRepositoryMessages.xsd
15:00:40.737 INFO o.a.a.t.failover.FailoverTransport - Successfully connected to ssl://netarkiv-aar-bit-master-devel-01.statsbiblioteket.dk:61612?keepAlive=true
15:00:40.832 DEBUG o.b.p.activemq.ActiveMQMessageBus - ActiveMQConnection initialized for 'org.bitrepository.settings.repositorysettings.MessageBusConfiguration@3754a4bf[name=netarkiv-aar-bit-master-devel-01 broker, url=failover:(ssl://netarkiv-aar-bit-master-devel-01.statsbiblioteket.dk:61612?keepAlive=true)]'.
15:00:40.834 INFO o.b.p.activemq.ActiveMQMessageBus - Settings component filter to: [kb-test-har-003.kb.dk-devel-29769]
15:00:40.837 DEBUG o.b.c.c.m.CollectionBasedConversationMediator - Initializing the CollectionBasedConversationMediator
15:00:40.837 DEBUG o.b.p.activemq.ActiveMQMessageBus - Adding listener 'org.bitrepository.client.conversation.mediator.CollectionBasedConversationMediator@429bffaa' to destination: 'topic://netarkiv-devel-kb-test-har-003.kb.dk-devel-29769' on message-bus 'netarkiv-aar-bit-master-devel-01 broker'.
15:00:40.837 DEBUG o.b.p.activemq.ActiveMQMessageBus - Retrieving message consumer on destination 'topic://netarkiv-devel-kb-test-har-003.kb.dk-devel-29769' for listener 'org.bitrepository.client.conversation.mediator.CollectionBasedConversationMediator@429bffaa'. Key: 'topic://netarkiv-devel-kb-test-har-003.kb.dk-devel-29769#1117519786'.
15:00:40.838 DEBUG o.b.p.activemq.ActiveMQMessageBus - No consumer known. Creating new for key 'topic://netarkiv-devel-kb-test-har-003.kb.dk-devel-29769#1117519786'.
15:00:40.854 DEBUG o.b.commandline.DeleteFileCmd - Performing the DeleteFile operation.
15:00:40.865 DEBUG o.b.commandline.DeleteFileCmd - Initiating the DeleteFile conversation.
15:00:40.868 INFO o.b.m.d.ConversationBasedDeleteFileClient - Requesting the deletion of the file '1-1-20220125121709634-00000-kb-test-har-003.kb.dk.warc.gz' from the pillar 'netarkiv-disk-devel-01' with checksum 'org.bitrepository.bitrepositoryelements.ChecksumDataForFileTYPE@71def8f8[checksumSpec=org.bitrepository.bitrepositoryelements.ChecksumSpecTYPE@383bfa16[checksumType=MD5, checksumSalt=<null>(default), otherChecksumType=<null>(default)], checksumValue={-128,83,23,93,110,33,-22,-18,6,13,-83,-18,115,105,-111,48}, calculationTimestamp=2022-01-25T15:00:40.857+01:00]', while requested checksum 'null'. And the audit trail information 'null'.
15:00:40.885 DEBUG org.bitrepository.common.JaxbHelper - Creating JAXBHelper based on schema from: jar:file:/home/devel/ResetCollection/bitrepository-client/lib/bitrepository-message-xml-xsd-30.jar!/versioning/ProtocolVersionDefinition.xsd
15:00:41.426 DEBUG o.b.p.m.l.DeleteFileMessageLogger - Sent IdentifyPillarsForDeleteFileRequest(2e50864a) collectionID Test6, to netarkiv-disk-devel-01, destination topic://netarkiv-devel: FileID=1-1-20220125121709634-00000-kb-test-har-003.kb.dk.warc.gz
15:00:41.427 DEBUG o.b.commandline.DeleteFileCmd - Received event: 2e50864a: DELETE_FILE for file 1-1-20220125121709634-00000-kb-test-har-003.kb.dk.warc.gz: IDENTIFY_REQUEST_SENT: , Identifying pillars for delete file
15:00:41.821 DEBUG o.b.p.m.l.DeleteFileMessageLogger - Received IdentifyPillarsForDeleteFileResponse(2e50864a) collectionID Test6, from netarkiv-kbh-disk-devel-01: IDENTIFICATION_POSITIVE(File 1-1-20220125121709634-00000-kb-test-har-003.kb.dk.warc.gz is in collection and can be deleted.)
15:00:41.837 DEBUG o.b.c.c.ConversationEventMonitor - 2e50864a: DELETE_FILE for file 1-1-20220125121709634-00000-kb-test-har-003.kb.dk.warc.gz:Ignoring message from irrelevant component netarkiv-kbh-disk-devel-01
15:00:42.159 DEBUG o.b.p.m.l.DeleteFileMessageLogger - Received IdentifyPillarsForDeleteFileResponse(2e50864a) collectionID Test6, from netarkiv-tape-devel-01: FILE_NOT_FOUND_FAILURE(Cannot serve request, file not on store)
15:00:42.172 DEBUG o.b.c.c.ConversationEventMonitor - 2e50864a: DELETE_FILE for file 1-1-20220125121709634-00000-kb-test-har-003.kb.dk.warc.gz:Ignoring message from irrelevant component netarkiv-tape-devel-01
15:00:42.425 DEBUG o.b.p.m.l.DeleteFileMessageLogger - Received IdentifyPillarsForDeleteFileResponse(2e50864a) collectionID Test6, from netarkiv-disk-devel-01: IDENTIFICATION_POSITIVE(IDENTIFICATION_POSITIVE)
15:00:42.426 DEBUG o.b.commandline.DeleteFileCmd - Received event: 2e50864a: DELETE_FILE for file 1-1-20220125121709634-00000-kb-test-har-003.kb.dk.warc.gz: COMPONENT_IDENTIFIED: ContributorID netarkiv-disk-devel-01
15:00:42.427 DEBUG o.b.c.c.s.ContributorResponseStatus - Received response from: netarkiv-disk-devel-01
15:00:42.427 DEBUG o.b.c.c.s.ContributorResponseStatus - Expected contributors: [netarkiv-disk-devel-01], components that have not answered: []
15:00:42.427 DEBUG o.b.commandline.DeleteFileCmd - Received event: 2e50864a: DELETE_FILE for file 1-1-20220125121709634-00000-kb-test-har-003.kb.dk.warc.gz: IDENTIFICATION_COMPLETE: Contributors: [netarkiv-disk-devel-01]
15:00:42.428 DEBUG o.b.commandline.DeleteFileCmd - Received event: 2e50864a: DELETE_FILE for file 1-1-20220125121709634-00000-kb-test-har-003.kb.dk.warc.gz: REQUEST_SENT: ContributorID [netarkiv-disk-devel-01], Sending request for deleting file
15:00:42.709 DEBUG o.b.p.m.l.DeleteFileMessageLogger - Sent DeleteFileRequest(2e50864a) collectionID Test6, to netarkiv-disk-devel-01, destination queue://netarkiv-disk-devel-01: FileID=1-1-20220125121709634-00000-kb-test-har-003.kb.dk.warc.gz, ChecksumDataForExistingFile=org.bitrepository.bitrepositoryelements.ChecksumDataForFileTYPE@71def8f8[checksumSpec=org.bitrepository.bitrepositoryelements.ChecksumSpecTYPE@383bfa16[checksumType=MD5, checksumSalt=<null>(default), otherChecksumType=<null>(default)], checksumValue={-128,83,23,93,110,33,-22,-18,6,13,-83,-18,115,105,-111,48}, calculationTimestamp=2022-01-25T15:00:40.857+01:00]
15:00:43.303 DEBUG o.b.p.m.l.DeleteFileMessageLogger - Received DeleteFileProgressResponse(2e50864a) collectionID Test6, from netarkiv-disk-devel-01: OPERATION_ACCEPTED_PROGRESS(OPERATION_ACCEPTED_PROGRESS)
15:00:43.333 DEBUG o.b.commandline.DeleteFileCmd - Received event: 2e50864a: DELETE_FILE for file 1-1-20220125121709634-00000-kb-test-har-003.kb.dk.warc.gz: PROGRESS: ContributorID netarkiv-disk-devel-01, OPERATION_ACCEPTED_PROGRESS
15:00:43.580 DEBUG o.b.p.m.l.DeleteFileMessageLogger - Received DeleteFileFinalResponse(2e50864a) collectionID Test6, from netarkiv-disk-devel-01: OPERATION_COMPLETED(OPERATION_COMPLETED)
15:00:43.588 DEBUG o.b.c.c.s.ContributorResponseStatus - Received response from: netarkiv-disk-devel-01
15:00:43.588 DEBUG o.b.c.c.s.ContributorResponseStatus - Expected contributors: [netarkiv-disk-devel-01], components that have not answered: []
15:00:43.588 DEBUG o.b.commandline.DeleteFileCmd - Complete: 2e50864a: DELETE_FILE for file 1-1-20220125121709634-00000-kb-test-har-003.kb.dk.warc.gz: COMPLETE:
15:00:43.588 INFO o.b.commandline.DeleteFileCmd - Results of the DeleteFile operation for the file '1-1-20220125121709634-00000-kb-test-har-003.kb.dk.warc.gz':
15:00:43.588 DEBUG o.b.commandline.DeleteFileCmd - 2e50864a: DELETE_FILE for file 1-1-20220125121709634-00000-kb-test-har-003.kb.dk.warc.gz: COMPLETE:
15:00:44.818 DEBUG org.bitrepository.common.JaxbHelper - Creating JAXBHelper based on schema from: jar:file:/home/devel/ResetCollection/bitrepository-client/lib/bitrepository-repository-settings-xsd-12.jar!/xsd/RepositorySettings.xsd
15:00:44.914 DEBUG o.b.c.settings.XMLFileSettingsLoader - Loading the settings file '/home/devel/ResetCollection/bitrepository-client/bin/../conf/RepositorySettings.xml'.
15:00:45.258 DEBUG org.bitrepository.common.JaxbHelper - Creating JAXBHelper based on schema from: jar:file:/home/devel/ResetCollection/bitrepository-client/lib/bitrepository-reference-settings-1.9.0.1.jar!/xsd/ReferenceSettings.xsd
15:00:45.313 DEBUG o.b.c.settings.XMLFileSettingsLoader - Loading the settings file '/home/devel/ResetCollection/bitrepository-client/bin/../conf/ReferenceSettings.xml'.
15:00:46.052 DEBUG o.b.p.security.BasicSecurityManager - X509CertificateHolder found
15:00:46.069 DEBUG o.b.p.security.BasicSecurityManager - Got something, that we don't (yet) recognize. Class: PEMKeyPair
15:00:46.069 DEBUG o.b.p.security.BasicSecurityManager - PrivateKeyInfo found
15:00:46.249 INFO o.b.p.activemq.ActiveMQMessageBus - Initializing ActiveMQMessageBus:'org.bitrepository.settings.repositorysettings.MessageBusConfiguration@13c9d689[name=netarkiv-aar-bit-master-devel-01 broker, url=failover:(ssl://netarkiv-aar-bit-master-devel-01.statsbiblioteket.dk:61612?keepAlive=true)]'.
15:00:46.250 DEBUG org.bitrepository.common.JaxbHelper - Creating JAXBHelper based on schema from: jar:file:/home/devel/ResetCollection/bitrepository-client/lib/bitrepository-message-xml-xsd-30.jar!/xsd/BitRepositoryMessages.xsd
15:00:47.260 INFO o.a.a.t.failover.FailoverTransport - Successfully connected to ssl://netarkiv-aar-bit-master-devel-01.statsbiblioteket.dk:61612?keepAlive=true
15:00:47.384 DEBUG o.b.p.activemq.ActiveMQMessageBus - ActiveMQConnection initialized for 'org.bitrepository.settings.repositorysettings.MessageBusConfiguration@13c9d689[name=netarkiv-aar-bit-master-devel-01 broker, url=failover:(ssl://netarkiv-aar-bit-master-devel-01.statsbiblioteket.dk:61612?keepAlive=true)]'.
15:00:47.387 INFO o.b.p.activemq.ActiveMQMessageBus - Settings component filter to: [kb-test-har-003.kb.dk-devel-29808]
15:00:47.390 DEBUG o.b.c.c.m.CollectionBasedConversationMediator - Initializing the CollectionBasedConversationMediator
15:00:47.390 DEBUG o.b.p.activemq.ActiveMQMessageBus - Adding listener 'org.bitrepository.client.conversation.mediator.CollectionBasedConversationMediator@4d5b6aac' to destination: 'topic://netarkiv-devel-kb-test-har-003.kb.dk-devel-29808' on message-bus 'netarkiv-aar-bit-master-devel-01 broker'.
15:00:47.391 DEBUG o.b.p.activemq.ActiveMQMessageBus - Retrieving message consumer on destination 'topic://netarkiv-devel-kb-test-har-003.kb.dk-devel-29808' for listener 'org.bitrepository.client.conversation.mediator.CollectionBasedConversationMediator@4d5b6aac'. Key: 'topic://netarkiv-devel-kb-test-har-003.kb.dk-devel-29808#1297836716'.
15:00:47.391 DEBUG o.b.p.activemq.ActiveMQMessageBus - No consumer known. Creating new for key 'topic://netarkiv-devel-kb-test-har-003.kb.dk-devel-29808#1297836716'.
15:00:47.410 DEBUG o.b.commandline.DeleteFileCmd - Performing the DeleteFile operation.
15:00:47.422 DEBUG o.b.commandline.DeleteFileCmd - Initiating the DeleteFile conversation.
15:00:47.426 INFO o.b.m.d.ConversationBasedDeleteFileClient - Requesting the deletion of the file '1-1-20220125121709634-00000-kb-test-har-003.kb.dk.warc.gz' from the pillar 'netarkiv-checksum-devel-01' with checksum 'org.bitrepository.bitrepositoryelements.ChecksumDataForFileTYPE@156b88f5[checksumSpec=org.bitrepository.bitrepositoryelements.ChecksumSpecTYPE@3bf9ce3e[checksumType=MD5, checksumSalt=<null>(default), otherChecksumType=<null>(default)], checksumValue={-128,83,23,93,110,33,-22,-18,6,13,-83,-18,115,105,-111,48}, calculationTimestamp=2022-01-25T15:00:47.413+01:00]', while requested checksum 'null'. And the audit trail information 'null'.
15:00:47.444 DEBUG org.bitrepository.common.JaxbHelper - Creating JAXBHelper based on schema from: jar:file:/home/devel/ResetCollection/bitrepository-client/lib/bitrepository-message-xml-xsd-30.jar!/versioning/ProtocolVersionDefinition.xsd
15:00:48.097 DEBUG o.b.p.m.l.DeleteFileMessageLogger - Sent IdentifyPillarsForDeleteFileRequest(1edf095d) collectionID Test6, to netarkiv-checksum-devel-01, destination topic://netarkiv-devel: FileID=1-1-20220125121709634-00000-kb-test-har-003.kb.dk.warc.gz
15:00:48.099 DEBUG o.b.commandline.DeleteFileCmd - Received event: 1edf095d: DELETE_FILE for file 1-1-20220125121709634-00000-kb-test-har-003.kb.dk.warc.gz: IDENTIFY_REQUEST_SENT: , Identifying pillars for delete file
15:00:48.473 DEBUG o.b.p.m.l.DeleteFileMessageLogger - Received IdentifyPillarsForDeleteFileResponse(1edf095d) collectionID Test6, from netarkiv-kbh-disk-devel-01: IDENTIFICATION_POSITIVE(File 1-1-20220125121709634-00000-kb-test-har-003.kb.dk.warc.gz is in collection and can be deleted.)
15:00:48.489 DEBUG o.b.c.c.ConversationEventMonitor - 1edf095d: DELETE_FILE for file 1-1-20220125121709634-00000-kb-test-har-003.kb.dk.warc.gz:Ignoring message from irrelevant component netarkiv-kbh-disk-devel-01
15:00:48.756 DEBUG o.b.p.m.l.DeleteFileMessageLogger - Received IdentifyPillarsForDeleteFileResponse(1edf095d) collectionID Test6, from netarkiv-checksum-devel-01: IDENTIFICATION_POSITIVE(Operation acknowledged and accepted.)
15:00:48.762 DEBUG o.b.commandline.DeleteFileCmd - Received event: 1edf095d: DELETE_FILE for file 1-1-20220125121709634-00000-kb-test-har-003.kb.dk.warc.gz: COMPONENT_IDENTIFIED: ContributorID netarkiv-checksum-devel-01
15:00:48.762 DEBUG o.b.c.c.s.ContributorResponseStatus - Received response from: netarkiv-checksum-devel-01
15:00:48.762 DEBUG o.b.c.c.s.ContributorResponseStatus - Expected contributors: [netarkiv-checksum-devel-01], components that have not answered: []
15:00:48.763 DEBUG o.b.commandline.DeleteFileCmd - Received event: 1edf095d: DELETE_FILE for file 1-1-20220125121709634-00000-kb-test-har-003.kb.dk.warc.gz: IDENTIFICATION_COMPLETE: Contributors: [netarkiv-checksum-devel-01]
15:00:48.763 DEBUG o.b.commandline.DeleteFileCmd - Received event: 1edf095d: DELETE_FILE for file 1-1-20220125121709634-00000-kb-test-har-003.kb.dk.warc.gz: REQUEST_SENT: ContributorID [netarkiv-checksum-devel-01], Sending request for deleting file
15:00:49.448 DEBUG o.b.p.m.l.DeleteFileMessageLogger - Received IdentifyPillarsForDeleteFileResponse(1edf095d) collectionID Test6, from netarkiv-tape-devel-01: FILE_NOT_FOUND_FAILURE(Cannot serve request, file not on store)
15:00:49.616 DEBUG o.b.p.m.l.DeleteFileMessageLogger - Sent DeleteFileRequest(1edf095d) collectionID Test6, to netarkiv-checksum-devel-01, destination topic://netarkiv-devel-netarkiv-checksum-devel-01-contributor: FileID=1-1-20220125121709634-00000-kb-test-har-003.kb.dk.warc.gz, ChecksumDataForExistingFile=org.bitrepository.bitrepositoryelements.ChecksumDataForFileTYPE@156b88f5[checksumSpec=org.bitrepository.bitrepositoryelements.ChecksumSpecTYPE@3bf9ce3e[checksumType=MD5, checksumSalt=<null>(default), otherChecksumType=<null>(default)], checksumValue={-128,83,23,93,110,33,-22,-18,6,13,-83,-18,115,105,-111,48}, calculationTimestamp=2022-01-25T15:00:47.413+01:00]
15:00:49.616 DEBUG o.b.c.c.ConversationEventMonitor - 1edf095d: DELETE_FILE for file 1-1-20220125121709634-00000-kb-test-har-003.kb.dk.warc.gz:Ignoring message from irrelevant component netarkiv-tape-devel-01
15:00:49.816 DEBUG o.b.p.m.l.DeleteFileMessageLogger - Received IdentifyPillarsForDeleteFileResponse(1edf095d) collectionID Test6, from netarkiv-disk-devel-01: FILE_NOT_FOUND_FAILURE(Cannot serve request, file not on store)
15:00:49.825 DEBUG o.b.c.c.ConversationEventMonitor - 1edf095d: DELETE_FILE for file 1-1-20220125121709634-00000-kb-test-har-003.kb.dk.warc.gz:Ignoring message from irrelevant component netarkiv-disk-devel-01
15:00:50.084 DEBUG o.b.p.m.l.DeleteFileMessageLogger - Received DeleteFileProgressResponse(1edf095d) collectionID Test6, from netarkiv-checksum-devel-01: OPERATION_ACCEPTED_PROGRESS
15:00:50.084 DEBUG o.b.commandline.DeleteFileCmd - Received event: 1edf095d: DELETE_FILE for file 1-1-20220125121709634-00000-kb-test-har-003.kb.dk.warc.gz: PROGRESS: ContributorID netarkiv-checksum-devel-01
15:00:50.368 DEBUG o.b.p.m.l.DeleteFileMessageLogger - Received DeleteFileFinalResponse(1edf095d) collectionID Test6, from netarkiv-checksum-devel-01: OPERATION_COMPLETED
15:00:50.371 DEBUG o.b.c.c.s.ContributorResponseStatus - Received response from: netarkiv-checksum-devel-01
15:00:50.371 DEBUG o.b.c.c.s.ContributorResponseStatus - Expected contributors: [netarkiv-checksum-devel-01], components that have not answered: []
15:00:50.371 DEBUG o.b.commandline.DeleteFileCmd - Complete: 1edf095d: DELETE_FILE for file 1-1-20220125121709634-00000-kb-test-har-003.kb.dk.warc.gz: COMPLETE:
15:00:50.371 INFO o.b.commandline.DeleteFileCmd - Results of the DeleteFile operation for the file '1-1-20220125121709634-00000-kb-test-har-003.kb.dk.warc.gz':
15:00:50.372 DEBUG o.b.commandline.DeleteFileCmd - 1edf095d: DELETE_FILE for file 1-1-20220125121709634-00000-kb-test-har-003.kb.dk.warc.gz: COMPLETE:
15:00:51.689 DEBUG org.bitrepository.common.JaxbHelper - Creating JAXBHelper based on schema from: jar:file:/home/devel/ResetCollection/bitrepository-client/lib/bitrepository-repository-settings-xsd-12.jar!/xsd/RepositorySettings.xsd
15:00:51.797 DEBUG o.b.c.settings.XMLFileSettingsLoader - Loading the settings file '/home/devel/ResetCollection/bitrepository-client/bin/../conf/RepositorySettings.xml'.
15:00:52.150 DEBUG org.bitrepository.common.JaxbHelper - Creating JAXBHelper based on schema from: jar:file:/home/devel/ResetCollection/bitrepository-client/lib/bitrepository-reference-settings-1.9.0.1.jar!/xsd/ReferenceSettings.xsd
15:00:52.197 DEBUG o.b.c.settings.XMLFileSettingsLoader - Loading the settings file '/home/devel/ResetCollection/bitrepository-client/bin/../conf/ReferenceSettings.xml'.
15:00:52.973 DEBUG o.b.p.security.BasicSecurityManager - X509CertificateHolder found
15:00:52.992 DEBUG o.b.p.security.BasicSecurityManager - Got something, that we don't (yet) recognize. Class: PEMKeyPair
15:00:52.993 DEBUG o.b.p.security.BasicSecurityManager - PrivateKeyInfo found
15:00:53.147 INFO o.b.p.activemq.ActiveMQMessageBus - Initializing ActiveMQMessageBus:'org.bitrepository.settings.repositorysettings.MessageBusConfiguration@62379589[name=netarkiv-aar-bit-master-devel-01 broker, url=failover:(ssl://netarkiv-aar-bit-master-devel-01.statsbiblioteket.dk:61612?keepAlive=true)]'.
15:00:53.148 DEBUG org.bitrepository.common.JaxbHelper - Creating JAXBHelper based on schema from: jar:file:/home/devel/ResetCollection/bitrepository-client/lib/bitrepository-message-xml-xsd-30.jar!/xsd/BitRepositoryMessages.xsd
15:00:54.011 INFO o.a.a.t.failover.FailoverTransport - Successfully connected to ssl://netarkiv-aar-bit-master-devel-01.statsbiblioteket.dk:61612?keepAlive=true
15:00:54.098 DEBUG o.b.p.activemq.ActiveMQMessageBus - ActiveMQConnection initialized for 'org.bitrepository.settings.repositorysettings.MessageBusConfiguration@62379589[name=netarkiv-aar-bit-master-devel-01 broker, url=failover:(ssl://netarkiv-aar-bit-master-devel-01.statsbiblioteket.dk:61612?keepAlive=true)]'.
15:00:54.101 INFO o.b.p.activemq.ActiveMQMessageBus - Settings component filter to: [kb-test-har-003.kb.dk-devel-29848]
15:00:54.104 DEBUG o.b.c.c.m.CollectionBasedConversationMediator - Initializing the CollectionBasedConversationMediator
15:00:54.104 DEBUG o.b.p.activemq.ActiveMQMessageBus - Adding listener 'org.bitrepository.client.conversation.mediator.CollectionBasedConversationMediator@4a7f959b' to destination: 'topic://netarkiv-devel-kb-test-har-003.kb.dk-devel-29848' on message-bus 'netarkiv-aar-bit-master-devel-01 broker'.
15:00:54.104 DEBUG o.b.p.activemq.ActiveMQMessageBus - Retrieving message consumer on destination 'topic://netarkiv-devel-kb-test-har-003.kb.dk-devel-29848' for listener 'org.bitrepository.client.conversation.mediator.CollectionBasedConversationMediator@4a7f959b'. Key: 'topic://netarkiv-devel-kb-test-har-003.kb.dk-devel-29848#1249875355'.
15:00:54.105 DEBUG o.b.p.activemq.ActiveMQMessageBus - No consumer known. Creating new for key 'topic://netarkiv-devel-kb-test-har-003.kb.dk-devel-29848#1249875355'.
15:00:54.119 DEBUG o.b.commandline.DeleteFileCmd - Performing the DeleteFile operation.
15:00:54.129 DEBUG o.b.commandline.DeleteFileCmd - Initiating the DeleteFile conversation.
15:00:54.131 INFO o.b.m.d.ConversationBasedDeleteFileClient - Requesting the deletion of the file '1-1-20220125121709634-00000-kb-test-har-003.kb.dk.warc.gz' from the pillar 'netarkiv-kbh-disk-devel-01' with checksum 'org.bitrepository.bitrepositoryelements.ChecksumDataForFileTYPE@16610890[checksumSpec=org.bitrepository.bitrepositoryelements.ChecksumSpecTYPE@71def8f8[checksumType=MD5, checksumSalt=<null>(default), otherChecksumType=<null>(default)], checksumValue={-128,83,23,93,110,33,-22,-18,6,13,-83,-18,115,105,-111,48}, calculationTimestamp=2022-01-25T15:00:54.122+01:00]', while requested checksum 'null'. And the audit trail information 'null'.
15:00:54.151 DEBUG org.bitrepository.common.JaxbHelper - Creating JAXBHelper based on schema from: jar:file:/home/devel/ResetCollection/bitrepository-client/lib/bitrepository-message-xml-xsd-30.jar!/versioning/ProtocolVersionDefinition.xsd
15:00:54.854 DEBUG o.b.p.m.l.DeleteFileMessageLogger - Sent IdentifyPillarsForDeleteFileRequest(87153d89) collectionID Test6, to netarkiv-kbh-disk-devel-01, destination topic://netarkiv-devel: FileID=1-1-20220125121709634-00000-kb-test-har-003.kb.dk.warc.gz
15:00:54.855 DEBUG o.b.commandline.DeleteFileCmd - Received event: 87153d89: DELETE_FILE for file 1-1-20220125121709634-00000-kb-test-har-003.kb.dk.warc.gz: IDENTIFY_REQUEST_SENT: , Identifying pillars for delete file
15:00:55.232 DEBUG o.b.p.m.l.DeleteFileMessageLogger - Received IdentifyPillarsForDeleteFileResponse(87153d89) collectionID Test6, from netarkiv-kbh-disk-devel-01: IDENTIFICATION_POSITIVE(File 1-1-20220125121709634-00000-kb-test-har-003.kb.dk.warc.gz is in collection and can be deleted.)
15:00:55.246 DEBUG o.b.commandline.DeleteFileCmd - Received event: 87153d89: DELETE_FILE for file 1-1-20220125121709634-00000-kb-test-har-003.kb.dk.warc.gz: COMPONENT_IDENTIFIED: ContributorID netarkiv-kbh-disk-devel-01
15:00:55.246 DEBUG o.b.c.c.s.ContributorResponseStatus - Received response from: netarkiv-kbh-disk-devel-01
15:00:55.246 DEBUG o.b.c.c.s.ContributorResponseStatus - Expected contributors: [netarkiv-kbh-disk-devel-01], components that have not answered: []
15:00:55.249 DEBUG o.b.commandline.DeleteFileCmd - Received event: 87153d89: DELETE_FILE for file 1-1-20220125121709634-00000-kb-test-har-003.kb.dk.warc.gz: IDENTIFICATION_COMPLETE: Contributors: [netarkiv-kbh-disk-devel-01]
15:00:55.249 DEBUG o.b.commandline.DeleteFileCmd - Received event: 87153d89: DELETE_FILE for file 1-1-20220125121709634-00000-kb-test-har-003.kb.dk.warc.gz: REQUEST_SENT: ContributorID [netarkiv-kbh-disk-devel-01], Sending request for deleting file
15:00:55.610 DEBUG o.b.p.m.l.DeleteFileMessageLogger - Received IdentifyPillarsForDeleteFileResponse(87153d89) collectionID Test6, from netarkiv-tape-devel-01: FILE_NOT_FOUND_FAILURE(Cannot serve request, file not on store)
15:00:56.215 DEBUG o.b.p.m.l.DeleteFileMessageLogger - Received IdentifyPillarsForDeleteFileResponse(87153d89) collectionID Test6, from netarkiv-disk-devel-01: FILE_NOT_FOUND_FAILURE(Cannot serve request, file not on store)
15:00:56.313 DEBUG o.b.p.m.l.DeleteFileMessageLogger - Sent DeleteFileRequest(87153d89) collectionID Test6, to netarkiv-kbh-disk-devel-01, destination topic://netarkiv-devel.netarkiv-kbh-disk-devel-01: FileID=1-1-20220125121709634-00000-kb-test-har-003.kb.dk.warc.gz, ChecksumDataForExistingFile=org.bitrepository.bitrepositoryelements.ChecksumDataForFileTYPE@16610890[checksumSpec=org.bitrepository.bitrepositoryelements.ChecksumSpecTYPE@71def8f8[checksumType=MD5, checksumSalt=<null>(default), otherChecksumType=<null>(default)], checksumValue={-128,83,23,93,110,33,-22,-18,6,13,-83,-18,115,105,-111,48}, calculationTimestamp=2022-01-25T15:00:54.122+01:00]
15:00:56.315 DEBUG o.b.c.c.ConversationEventMonitor - 87153d89: DELETE_FILE for file 1-1-20220125121709634-00000-kb-test-har-003.kb.dk.warc.gz:Ignoring message from irrelevant component netarkiv-disk-devel-01
15:00:56.316 DEBUG o.b.c.c.ConversationEventMonitor - 87153d89: DELETE_FILE for file 1-1-20220125121709634-00000-kb-test-har-003.kb.dk.warc.gz:Ignoring message from irrelevant component netarkiv-tape-devel-01
15:00:56.789 DEBUG o.b.p.m.l.DeleteFileMessageLogger - Received DeleteFileProgressResponse(87153d89) collectionID Test6, from netarkiv-kbh-disk-devel-01: OPERATION_ACCEPTED_PROGRESS(Deleting..)
15:00:56.791 DEBUG o.b.commandline.DeleteFileCmd - Received event: 87153d89: DELETE_FILE for file 1-1-20220125121709634-00000-kb-test-har-003.kb.dk.warc.gz: PROGRESS: ContributorID netarkiv-kbh-disk-devel-01, Deleting..
15:00:57.162 DEBUG o.b.p.m.l.DeleteFileMessageLogger - Received DeleteFileFinalResponse(87153d89) collectionID Test6, from netarkiv-kbh-disk-devel-01: OPERATION_COMPLETED(1-1-20220125121709634-00000-kb-test-har-003.kb.dk.warc.gz deleted.)
15:00:57.162 DEBUG o.b.c.c.s.ContributorResponseStatus - Received response from: netarkiv-kbh-disk-devel-01
15:00:57.163 DEBUG o.b.c.c.s.ContributorResponseStatus - Expected contributors: [netarkiv-kbh-disk-devel-01], components that have not answered: []
15:00:57.163 DEBUG o.b.commandline.DeleteFileCmd - Complete: 87153d89: DELETE_FILE for file 1-1-20220125121709634-00000-kb-test-har-003.kb.dk.warc.gz: COMPLETE:
15:00:57.163 INFO o.b.commandline.DeleteFileCmd - Results of the DeleteFile operation for the file '1-1-20220125121709634-00000-kb-test-har-003.kb.dk.warc.gz':
15:00:57.163 DEBUG o.b.commandline.DeleteFileCmd - 87153d89: DELETE_FILE for file 1-1-20220125121709634-00000-kb-test-har-003.kb.dk.warc.gz: COMPLETE:
calling cleanup_hosts
running cleanup_hosts
Removing installation on sb-test-har-001.statsbiblioteket.dk
Removing installation on kb-test-har-003.kb.dk
Removing installation on kb-test-har-004.kb.dk
Removing installation on kb-test-har-005.kb.dk
Removing installation on kb-test-adm-001.kb.dk
Removing installation on kb-test-acs-001.kb.dk
Removing installation on kb-test-way-001.kb.dk
calling cleanup_linuxbarhosts
running cleanup_linuxbarhosts
Removing installation on sb-test-bar-001.statsbiblioteket.dk
Removing bitarchive on sb-test-bar-001.statsbiblioteket.dk
Calling cleanup_windowsbarhosts
running cleanup_windowsbarhosts
Removing C:\bitarkiv\Test6 on kb-test-bar-01.bitarkiv.kb.dk
The system cannot find the file specified.
Removing E:\bitarchive_1\Test6 on kb-test-bar-01.bitarkiv.kb.dk
The system cannot find the file specified.
Removing F:\bitarchive_2\Test6 on kb-test-bar-01.bitarkiv.kb.dk
The system cannot find the file specified.
Removing g:\bitarchive_3\Test6 on kb-test-bar-01.bitarkiv.kb.dk
The system cannot find the file specified.
Removing C:\Users\ba-devel\Test6 on kb-test-bar-01.bitarkiv.kb.dk
The system cannot find the file specified.
TESTX=Test6
PORT=8078
MAILRECEIVERS=csr@kb.dk
VERSION=CSR
DEPLOYCONF=deploy_config_bitmag.xml
H3ZIP=CSR
BITMAGCONF=/home/devel/bitmag_test_config/client_config
Using bundler file /home/devel/nas_versions/bundler/NetarchiveSuite-heritrix3-bundler-CSR.zip
Creating distribution files in /home/devel/prepared_software/Test6.
Will read source files using encoding 'UTF-8' (defaulted)
Branch in settings not found: common.warcRecordService
Branch in settings not found: common.arcrepositoryClient.bitrepository.getFileIDsMaxResults
Branch in settings not found: common.hadoop.kerberos
Branch in settings not found: common.hadoop.username
Branch in settings not found: common.hadoop.mapred.hadoopUberJar
Branch in settings not found: common.hadoop.mapred.inputFilesParentDir
Branch in settings not found: common.hadoop.mapred.cdxJob
Branch in settings not found: common.hadoop.mapred.metadataExtractionJob
Branch in settings not found: common.hadoop.mapred.metadataCDXExtractionJob
Branch in settings not found: common.hadoop.mapred.crawlLogExtractionJob
Branch in settings not found: common.fileResolver.baseUrl
Branch in settings not found: common.fileResolver.keyfile
Branch in settings not found: common.webinterface.metadata_cache_dir
Branch in settings not found: harvester.harvesting.heritrix3.bundle
Branch in settings not found: harvester.harvesting.heritrix3.certificate
Branch in settings not found: harvester.harvesting.heritrix3.bundle
Branch in settings not found: harvester.harvesting.heritrix3.certificate
Branch in settings not found: harvester.harvesting.umbra
Branch in settings not found: harvester.harvesting.heritrix3.bundle
Branch in settings not found: harvester.harvesting.heritrix3.certificate
Branch in settings not found: harvester.harvesting.umbra
Branch in settings not found: harvester.harvesting.heritrix3.bundle
Branch in settings not found: harvester.harvesting.heritrix3.bundle
Machine 'kb-test-adm-001.kb.dk' configured with encoding 'UTF-8' (defaulted)
Machine 'kb-test-har-003.kb.dk' configured with encoding 'UTF-8' (defaulted)
NetarchiveSuite-heritrix3-bundler.zip -> /home/devel/Test6/NetarchiveSuite-heritrix3-bundler.zip
/home/devel/quickstart-nas-5.0-h3/h3server.jks -> /home/devel/Test6/h3server.jks
Machine 'kb-test-har-005.kb.dk' configured with encoding 'UTF-8' (defaulted)
NetarchiveSuite-heritrix3-bundler.zip -> /home/devel/Test6/NetarchiveSuite-heritrix3-bundler.zip
/home/devel/quickstart-nas-5.0-h3/h3server.jks -> /home/devel/Test6/h3server.jks
Machine 'kb-test-har-004.kb.dk' configured with encoding 'UTF-8' (defaulted)
NetarchiveSuite-heritrix3-bundler.zip -> /home/devel/Test6/NetarchiveSuite-heritrix3-bundler.zip
Machine 'kb-test-acs-001.kb.dk' configured with encoding 'UTF-8' (defaulted)
Machine 'kb-test-way-001.kb.dk' configured with encoding 'UTF-8' (defaulted)
Machine 'sb-test-har-001.statsbiblioteket.dk' configured with encoding 'UTF-8' (defaulted)
/home/devel/prepared_software/Test6/NetarchiveSuite-heritrix3-bundler.zip -> /home/netarkdv/Test6/NetarchiveSuite-heritrix3-bundler.zip
15:01:16.093 [main] DEBUG dk.netarkivet.deploy.XmlStructure - Element baseDir is not a branch in the tree [archive, bitpreservation, baseDir]. Null returned
15:01:16.097 [main] DEBUG dk.netarkivet.deploy.XmlStructure - Element is not text. Null returned. Path given to getLeafValue archive
15:01:16.099 [main] DEBUG dk.netarkivet.deploy.XmlStructure - Element baseDir is not a branch in the tree [archive, bitpreservation, baseDir]. Null returned
15:01:16.100 [main] DEBUG dk.netarkivet.deploy.XmlStructure - Element is not text. Null returned. Path given to getLeafValue archive
15:01:16.101 [main] DEBUG dk.netarkivet.deploy.XmlStructure - Element baseDir is not a branch in the tree [archive, bitpreservation, baseDir]. Null returned
15:01:16.101 [main] DEBUG dk.netarkivet.deploy.XmlStructure - Element is not text. Null returned. Path given to getLeafValue archive
15:01:16.101 [main] DEBUG dk.netarkivet.deploy.XmlStructure - Element baseDir is not a branch in the tree [archive, bitpreservation, baseDir]. Null returned
15:01:16.101 [main] DEBUG dk.netarkivet.deploy.XmlStructure - Element is not text. Null returned. Path given to getLeafValue archive
15:01:16.102 [main] DEBUG dk.netarkivet.deploy.XmlStructure - Element baseDir is not a branch in the tree [archive, bitpreservation, baseDir]. Null returned
15:01:16.102 [main] DEBUG dk.netarkivet.deploy.XmlStructure - Element is not text. Null returned. Path given to getLeafValue archive
15:01:16.102 [main] DEBUG dk.netarkivet.deploy.XmlStructure - Element baseDir is not a branch in the tree [archive, bitpreservation, baseDir]. Null returned
15:01:16.102 [main] DEBUG dk.netarkivet.deploy.XmlStructure - Element is not text. Null returned. Path given to getLeafValue archive
15:01:16.162 [main] DEBUG dk.netarkivet.deploy.XmlStructure - Element baseDir is not a branch in the tree [archive, bitpreservation, baseDir]. Null returned
15:01:16.163 [main] DEBUG dk.netarkivet.deploy.XmlStructure - Element is not text. Null returned. Path given to getLeafValue archive
--------------------------------------------
INSTALLING TO MACHINE: devel@kb-test-adm-001.kb.dk
copying NetarchiveSuite-CSR.zip to:kb-test-adm-001.kb.dk
deleting devel@kb-test-adm-001.kb.dk:/home/devel/Test6/lib
unzipping NetarchiveSuite-CSR.zip at:kb-test-adm-001.kb.dk
Creating directories.
preparing for copying of settings and scripts
copying settings and scripts
make scripts executable
make password and access files readonly
--------------------------------------------
INSTALLING TO MACHINE: devel@kb-test-har-003.kb.dk
copying NetarchiveSuite-CSR.zip to:kb-test-har-003.kb.dk
deleting devel@kb-test-har-003.kb.dk:/home/devel/Test6/lib
unzipping NetarchiveSuite-CSR.zip at:kb-test-har-003.kb.dk
Creating directories.
preparing for copying of settings and scripts
copying settings and scripts
make scripts executable
make password and access files readonly
--------------------------------------------
INSTALLING TO MACHINE: devel@kb-test-har-005.kb.dk
copying NetarchiveSuite-CSR.zip to:kb-test-har-005.kb.dk
deleting devel@kb-test-har-005.kb.dk:/home/devel/Test6/lib
unzipping NetarchiveSuite-CSR.zip at:kb-test-har-005.kb.dk
Creating directories.
preparing for copying of settings and scripts
copying settings and scripts
make scripts executable
make password and access files readonly
--------------------------------------------
INSTALLING TO MACHINE: devel@kb-test-har-004.kb.dk
copying NetarchiveSuite-CSR.zip to:kb-test-har-004.kb.dk
deleting devel@kb-test-har-004.kb.dk:/home/devel/Test6/lib
unzipping NetarchiveSuite-CSR.zip at:kb-test-har-004.kb.dk
Creating directories.
preparing for copying of settings and scripts
copying settings and scripts
make scripts executable
make password and access files readonly
--------------------------------------------
INSTALLING TO MACHINE: devel@kb-test-acs-001.kb.dk
copying NetarchiveSuite-CSR.zip to:kb-test-acs-001.kb.dk
deleting devel@kb-test-acs-001.kb.dk:/home/devel/Test6/lib
unzipping NetarchiveSuite-CSR.zip at:kb-test-acs-001.kb.dk
Creating directories.
preparing for copying of settings and scripts
copying settings and scripts
make scripts executable
make password and access files readonly
--------------------------------------------
INSTALLING TO MACHINE: devel@kb-test-way-001.kb.dk
copying NetarchiveSuite-CSR.zip to:kb-test-way-001.kb.dk
deleting devel@kb-test-way-001.kb.dk:/home/devel/Test6/lib
unzipping NetarchiveSuite-CSR.zip at:kb-test-way-001.kb.dk
Creating directories.
preparing for copying of settings and scripts
copying settings and scripts
make scripts executable
make password and access files readonly
--------------------------------------------
--------------------------------------------
INSTALLING TO MACHINE: netarkdv@sb-test-har-001.statsbiblioteket.dk
copying NetarchiveSuite-CSR.zip to:sb-test-har-001.statsbiblioteket.dk
deleting netarkdv@sb-test-har-001.statsbiblioteket.dk:/home/netarkdv/Test6/lib
unzipping NetarchiveSuite-CSR.zip at:sb-test-har-001.statsbiblioteket.dk
Creating directories.
preparing for copying of settings and scripts
copying settings and scripts
make scripts executable
make password and access files readonly
--------------------------------------------
Initializing database for system Test6
+ host=devel@kb-test-adm-001
+ scp -p /home/devel/devel-config/scripts/db/setup_admindb.sh /home/devel/devel-config/scripts/db/setup_harvestdb.sh /home/devel/devel-config/scripts/db/teardown_admindb.sh /home/devel/devel-config/scripts/db/teardown_harvestdb.sh /home/devel/devel-config/scripts/../resources/harvestdb.testdata.sql /home/devel/devel-config/scripts/../resources/default_orderxml.cxml devel@kb-test-adm-001:bin/
+ ssh devel@kb-test-adm-001
Pseudo-terminal will not be allocated because stdin is not a terminal.
Destroying test6_admindb
DEBUG: transaction ID wrap limit is 2257721481, limited by database "test2056_harvestdb"
DROP DATABASE
Creating empty test6_admindb
DEBUG: transaction ID wrap limit is 2257721481, limited by database "test2056_harvestdb"
CREATE DATABASE
Initialising test6_admindb
CREATE TABLE
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "replica_pkey" for table "replica"
NOTICE: CREATE TABLE / UNIQUE will create implicit index "replica_replica_id_key" for table "replica"
NOTICE: CREATE TABLE / UNIQUE will create implicit index "replica_replica_name_key" for table "replica"
CREATE TABLE
CREATE SEQUENCE
ALTER TABLE
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "replicafileinfo_pkey" for table "replicafileinfo"
CREATE TABLE
CREATE SEQUENCE
ALTER TABLE
CREATE INDEX
CREATE INDEX
CREATE INDEX
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "file_pkey" for table "file"
CREATE TABLE
CREATE SEQUENCE
ALTER TABLE
CREATE INDEX
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "segment_pkey" for table "segment"
CREATE TABLE
CREATE SEQUENCE
ALTER TABLE
Destroying test6_harvestdb
DEBUG: transaction ID wrap limit is 2257721481, limited by database "test2056_harvestdb"
DROP DATABASE
DEBUG: transaction ID wrap limit is 2257721481, limited by database "test2056_harvestdb"
CREATE DATABASE
CREATE TABLE
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
GRANT
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "domains_pkey" for table "domains"
NOTICE: CREATE TABLE / UNIQUE will create implicit index "domains_name_key" for table "domains"
CREATE TABLE
CREATE SEQUENCE
ALTER TABLE
CREATE INDEX
CREATE INDEX
GRANT
GRANT
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "configurations_pkey" for table "configurations"
CREATE TABLE
CREATE SEQUENCE
ALTER TABLE
CREATE INDEX
CREATE INDEX
CREATE INDEX
GRANT
GRANT
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "config_passwords_pkey" for table "config_passwords"
CREATE TABLE
GRANT
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "config_seedlists_pkey" for table "config_seedlists"
CREATE TABLE
GRANT
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "seedlists_pkey" for table "seedlists"
CREATE TABLE
CREATE SEQUENCE
ALTER TABLE
CREATE INDEX
CREATE INDEX
GRANT
GRANT
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "passwords_pkey" for table "passwords"
CREATE TABLE
CREATE SEQUENCE
ALTER TABLE
CREATE INDEX
CREATE INDEX
GRANT
GRANT
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "ownerinfo_pkey" for table "ownerinfo"
CREATE TABLE
CREATE SEQUENCE
ALTER TABLE
CREATE INDEX
GRANT
GRANT
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "historyinfo_pkey" for table "historyinfo"
CREATE TABLE
CREATE SEQUENCE
ALTER TABLE
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
GRANT
GRANT
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "harvestdefinitions_pkey" for table "harvestdefinitions"
NOTICE: CREATE TABLE / UNIQUE will create implicit index "harvestdefinitions_name_key" for table "harvestdefinitions"
CREATE TABLE
CREATE INDEX
GRANT
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "fullharvests_pkey" for table "fullharvests"
CREATE TABLE
GRANT
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "partialharvests_pkey" for table "partialharvests"
CREATE TABLE
CREATE INDEX
GRANT
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "harvest_configs_pkey" for table "harvest_configs"
CREATE TABLE
GRANT
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "schedules_pkey" for table "schedules"
NOTICE: CREATE TABLE / UNIQUE will create implicit index "schedules_name_key" for table "schedules"
CREATE TABLE
CREATE SEQUENCE
ALTER TABLE
GRANT
GRANT
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "ordertemplates_pkey" for table "ordertemplates"
NOTICE: CREATE TABLE / UNIQUE will create implicit index "ordertemplates_name_key" for table "ordertemplates"
CREATE TABLE
CREATE SEQUENCE
ALTER TABLE
GRANT
GRANT
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "jobs_pkey" for table "jobs"
CREATE TABLE
CREATE INDEX
CREATE INDEX
GRANT
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "job_configs_pkey" for table "job_configs"
CREATE TABLE
CREATE INDEX
GRANT
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "global_crawler_trap_lists_pkey" for table "global_crawler_trap_lists"
NOTICE: CREATE TABLE / UNIQUE will create implicit index "global_crawler_trap_lists_name_key" for table "global_crawler_trap_lists"
CREATE TABLE
CREATE SEQUENCE
ALTER TABLE
CREATE INDEX
GRANT
GRANT
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "global_crawler_trap_expressions_pkey" for table "global_crawler_trap_expressions"
CREATE TABLE
CREATE SEQUENCE
ALTER TABLE
CREATE INDEX
CREATE INDEX
GRANT
GRANT
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "pkrunningjobshistory" for table "runningjobshistory"
CREATE TABLE
CREATE INDEX
CREATE INDEX
CREATE INDEX
GRANT
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "pkrunningjobsmonitor" for table "runningjobsmonitor"
CREATE TABLE
CREATE INDEX
CREATE INDEX
GRANT
NOTICE: CREATE TABLE / UNIQUE will create implicit index "pkfrontierreportlines" for table "frontierreportmonitor"
CREATE TABLE
GRANT
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "extendedfieldtype_pkey" for table "extendedfieldtype"
CREATE TABLE
CREATE SEQUENCE
ALTER TABLE
GRANT
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "extendedfield_pkey" for table "extendedfield"
CREATE TABLE
CREATE SEQUENCE
ALTER TABLE
GRANT
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "extendedfieldvalue_pkey" for table "extendedfieldvalue"
CREATE TABLE
CREATE SEQUENCE
ALTER TABLE
GRANT
INSERT 0 1
INSERT 0 1
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "harvestchannel_pkey" for table "harvestchannel"
NOTICE: CREATE TABLE / UNIQUE will create implicit index "harvestchannel_name_key" for table "harvestchannel"
CREATE TABLE
CREATE SEQUENCE
ALTER TABLE
CREATE INDEX
GRANT
GRANT
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
CREATE TABLE
NOTICE: ALTER TABLE / ADD PRIMARY KEY will create implicit index "eav_type_attribute_pkey" for table "eav_type_attribute"
ALTER TABLE
CREATE INDEX
CREATE SEQUENCE
CREATE TABLE
NOTICE: ALTER TABLE / ADD PRIMARY KEY will create implicit index "eav_attribute_pkey" for table "eav_attribute"
ALTER TABLE
CREATE INDEX
GRANT
GRANT
GRANT
INSERT 0 1
INSERT 0 1
INSERT 0 1
SET
SET
SET
SET
SET
SET
SET
SET
SET
setval
--------
17
(1 row)
setval
--------
17
(1 row)
setval
--------
17
(1 row)
setval
--------
5
(1 row)
setval
--------
4
(1 row)
setval
--------
17
(1 row)
15:03:28.103 [MLog-Init-Reporter] INFO com.mchange.v2.log.MLog - MLog clients using slf4j logging.
15:03:28.159 [MLog-Init-Reporter] DEBUG com.mchange.v2.log.MLog - Reading VM config for path list /com/mchange/v2/log/default-mchange-log.properties, /mchange-commons.properties, /c3p0.properties, hocon:/reference,/application,/c3p0,/, /mchange-log.properties, /
15:03:28.159 [MLog-Init-Reporter] DEBUG com.mchange.v2.log.MLog - The configuration file for resource identifier '/mchange-commons.properties' could not be found. Skipping.
15:03:28.159 [MLog-Init-Reporter] DEBUG com.mchange.v2.log.MLog - The configuration file for resource identifier '/c3p0.properties' could not be found. Skipping.
15:03:28.159 [MLog-Init-Reporter] DEBUG com.mchange.v2.log.MLog - The configuration file for resource identifier 'hocon:/reference,/application,/c3p0,/' could not be found. Skipping.
15:03:28.159 [MLog-Init-Reporter] DEBUG com.mchange.v2.log.MLog - The configuration file for resource identifier '/mchange-log.properties' could not be found. Skipping.
15:03:28.174 [main] DEBUG com.mchange.v2.cfg.MConfig - The configuration file for resource identifier '/mchange-commons.properties' could not be found. Skipping.
15:03:28.174 [main] DEBUG com.mchange.v2.cfg.MConfig - The configuration file for resource identifier '/mchange-log.properties' could not be found. Skipping.
15:03:28.174 [main] DEBUG com.mchange.v2.cfg.MConfig - The configuration file for resource identifier 'hocon:/reference,/application,/c3p0,/' could not be found. Skipping.
15:03:28.175 [main] DEBUG com.mchange.v2.cfg.MConfig - The configuration file for resource identifier '/c3p0.properties' could not be found. Skipping.
15:03:28.562 [main] INFO com.mchange.v2.c3p0.C3P0Registry - Initializing c3p0-0.9.5.4 [built 23-March-2019 23:00:48 -0700; debug? true; trace: 10]
15:03:28.726 [main] DEBUG com.mchange.v2.c3p0.management.DynamicPooledDataSourceManagerMBean - MBean: com.mchange.v2.c3p0:type=PooledDataSource,identityToken=10bdx52am14duf1g1jth92h|1b37288,name=10bdx52am14duf1g1jth92h|1b37288 registered.
15:03:28.905 [main] DEBUG com.mchange.v2.c3p0.management.DynamicPooledDataSourceManagerMBean - MBean: com.mchange.v2.c3p0:type=PooledDataSource,identityToken=10bdx52am14duf1g1jth92h|1b37288,name=10bdx52am14duf1g1jth92h|1b37288 unregistered, in order to be reregistered after update.
15:03:28.907 [main] DEBUG com.mchange.v2.c3p0.management.DynamicPooledDataSourceManagerMBean - MBean: com.mchange.v2.c3p0:type=PooledDataSource,identityToken=10bdx52am14duf1g1jth92h|1b37288,name=10bdx52am14duf1g1jth92h|1b37288 registered.
15:03:28.933 [main] INFO dk.netarkivet.harvester.datamodel.HarvestDBConnection - Connection pool initialized with the following values:
- minPoolSize=5
- maxPoolSize=20
- acquireIncrement=5
- maxStatements=0
- maxStatementsPerConnection=0
- idleConnTestPeriod=0
- idleConnTestQuery='null'
- idleConnTestOnCheckin=false
15:03:29.071 [main] INFO com.mchange.v2.c3p0.impl.AbstractPoolBackedDataSource - Initializing c3p0 pool... com.mchange.v2.c3p0.ComboPooledDataSource [ acquireIncrement -> 5, acquireRetryAttempts -> 30, acquireRetryDelay -> 1000, autoCommitOnClose -> false, automaticTestTable -> null, breakAfterAcquireFailure -> false, checkoutTimeout -> 0, connectionCustomizerClassName -> null, connectionTesterClassName -> com.mchange.v2.c3p0.impl.DefaultConnectionTester, contextClassLoaderSource -> caller, dataSourceName -> 10bdx52am14duf1g1jth92h|1b37288, debugUnreturnedConnectionStackTraces -> false, description -> null, driverClass -> org.postgresql.Driver, extensions -> {}, factoryClassLocation -> null, forceIgnoreUnresolvedTransactions -> false, forceSynchronousCheckins -> false, forceUseNamedDriverClass -> false, identityToken -> 10bdx52am14duf1g1jth92h|1b37288, idleConnectionTestPeriod -> 0, initialPoolSize -> 3, jdbcUrl -> jdbc:postgresql://localhost:5432/test6_harvestdb, maxAdministrativeTaskTime -> 0, maxConnectionAge -> 3600, maxIdleTime -> 0, maxIdleTimeExcessConnections -> 0, maxPoolSize -> 20, maxStatements -> 0, maxStatementsPerConnection -> 0, minPoolSize -> 5, numHelperThreads -> 3, preferredTestQuery -> null, privilegeSpawnedThreads -> false, properties -> {user=******, password=******}, propertyCycle -> 0, statementCacheNumDeferredCloseThreads -> 0, testConnectionOnCheckin -> false, testConnectionOnCheckout -> false, unreturnedConnectionTimeout -> 0, userOverrides -> {}, usesTraditionalReflectiveProxies -> false ]
15:03:29.135 [main] DEBUG com.mchange.v2.cfg.MConfig - The configuration file for resource identifier '/mchange-commons.properties' could not be found. Skipping.
15:03:29.135 [main] DEBUG com.mchange.v2.cfg.MConfig - The configuration file for resource identifier '/mchange-log.properties' could not be found. Skipping.
15:03:29.135 [main] DEBUG com.mchange.v2.cfg.MConfig - The configuration file for resource identifier '/c3p0.properties' could not be found. Skipping.
15:03:29.135 [main] DEBUG com.mchange.v2.cfg.MConfig - The configuration file for resource identifier 'hocon:/reference,/application,/c3p0,/' could not be found. Skipping.
15:03:29.136 [main] WARN com.mchange.v2.resourcepool.BasicResourcePool - Bad pool size config, start 3 < min 5. Using 5 as start.
15:03:29.152 [main] DEBUG com.mchange.v2.resourcepool.BasicResourcePool - com.mchange.v2.resourcepool.BasicResourcePool@c5cc4c config: [start -> 5; min -> 5; max -> 20; inc -> 5; num_acq_attempts -> 30; acq_attempt_delay -> 1000; check_idle_resources_delay -> 0; max_resource_age -> 3600000; max_idle_time -> 0; excess_max_idle_time -> 0; destroy_unreturned_resc_time -> 0; expiration_enforcement_delay -> 900000; break_on_acquisition_failure -> false; debug_store_checkout_exceptions -> false; force_synchronous_checkins -> false]
15:03:29.152 [main] DEBUG com.mchange.v2.c3p0.impl.C3P0PooledConnectionPoolManager - Created new pool for auth, username (masked): 'de******'.
15:03:29.153 [main] DEBUG com.mchange.v2.resourcepool.BasicResourcePool - acquire test -- pool size: 0; target_pool_size: 5; desired target? 1
15:03:29.153 [main] DEBUG com.mchange.v2.resourcepool.BasicResourcePool - awaitAvailable(): [unknown]
15:03:29.581 [main] DEBUG com.mchange.v2.c3p0.impl.NewProxyPreparedStatement - com.mchange.v2.c3p0.impl.NewProxyPreparedStatement@802ef9 [wrapping: SELECT version FROM schemaversions WHERE tablename = 'ordertemplates'] closed orphaned ResultSet: com.mchange.v2.c3p0.impl.NewProxyResultSet@1010382 [wrapping: null]
15:03:29.600 [main] DEBUG com.mchange.v2.c3p0.impl.NewProxyPreparedStatement - com.mchange.v2.c3p0.impl.NewProxyPreparedStatement@cddd20 [wrapping: SELECT COUNT(*) FROM ordertemplates WHERE name = 'default_orderxml'] closed orphaned ResultSet: com.mchange.v2.c3p0.impl.NewProxyResultSet@16def03 [wrapping: null]
15:03:29.663 [main] DEBUG com.mchange.v2.c3p0.impl.NewProxyPreparedStatement - com.mchange.v2.c3p0.impl.NewProxyPreparedStatement@909414 [wrapping: SELECT COUNT(*) FROM ordertemplates WHERE name = 'default_orderxml'] closed orphaned ResultSet: com.mchange.v2.c3p0.impl.NewProxyResultSet@13d61fb [wrapping: null]
The template 'default_orderxml' has now been updated.
+ set +x
--------------------------------------------
STARTING MACHINE: devel@kb-test-adm-001.kb.dk
Starting all applications on: kb-test-adm-001.kb.dk
Starting linux application: GUIApplication
Starting linux application: ArcRepositoryApplication
Starting linux application: HarvestJobManagerApplication
15:03:32,212 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [/home/devel/Test6/conf/logback_ArcRepositoryApplication.xml] at [file:/home/devel/Test6/conf/logback_ArcRepositoryApplication.xml]
15:03:33,104 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - debug attribute not set
15:03:33,107 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
15:03:33,131 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [STDOUT]
15:03:33,204 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
15:03:33,433 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
15:03:33,468 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [FILE]
15:03:33,667 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@16843650 - No compression will be used
15:03:33,676 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@16843650 - Will use the pattern ./log/ArcRepositoryApplication.%d{yyyy-MM-dd}.%i.log for the active file
15:03:33,694 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@802ef9 - The date pattern is 'yyyy-MM-dd' from file name pattern './log/ArcRepositoryApplication.%d{yyyy-MM-dd}.%i.log'.
15:03:33,694 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@802ef9 - Roll-over at midnight.
15:03:33,729 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@802ef9 - Setting initial period to Tue Jan 25 15:03:33 CET 2022
15:03:33,731 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@802ef9 - SizeAndTimeBasedFNATP is deprecated. Use SizeAndTimeBasedRollingPolicy instead
15:03:33,731 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@802ef9 - For more information see http://logback.qos.ch/manual/appenders.html#SizeAndTimeBasedRollingPolicy
15:03:33,746 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
15:03:33,766 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - Active log file name: ./log/ArcRepositoryApplication.log
15:03:33,766 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - File property is set to [./log/ArcRepositoryApplication.log]
15:03:33,768 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [dk.netarkivet.monitor.logging.CachingSLF4JAppender]
15:03:35,935 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [MONITOR]
15:03:35,950 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to WARN
15:03:35,950 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[ROOT]
15:03:35,960 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [FILE] to Logger[ROOT]
15:03:35,960 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [MONITOR] to Logger[ROOT]
15:03:35,965 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [dk.netarkivet] to DEBUG
15:03:35,966 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
15:03:35,968 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@16faea6 - Registering current configuration as safe fallback point
2022-01-25 15:03:35.991 DEBUG dk.netarkivet.common.utils.Settings.invoke0 - Value found in loaded data: SB,KB
SLF4J: A number (1) of logging calls during the initialization phase have been intercepted and are
SLF4J: now being replayed. These are subject to the filtering rules of the underlying logging system.
SLF4J: See also http://www.slf4j.org/codes.html#replay
Starting dk.netarkivet.archive.arcrepository.ArcRepository
Version: 7.3-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/f0e2fec464f219bbccc5ad1a8c47bb5ab34b5f79)
2022-01-25 15:03:36.045 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Starting dk.netarkivet.archive.arcrepository.ArcRepository
Version: 7.3-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/f0e2fec464f219bbccc5ad1a8c47bb5ab34b5f79)
Java VM: 1.8.0_20
2022-01-25 15:03:36.047 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Java VM: 1.8.0_20
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/devel/Test6/lib/logback-classic-1.2.3.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/devel/Test6/lib/slf4j-log4j12-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
15:03:33,154 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [/home/devel/Test6/conf/logback_HarvestJobManagerApplication.xml] at [file:/home/devel/Test6/conf/logback_HarvestJobManagerApplication.xml]
15:03:33,655 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - debug attribute not set
15:03:33,665 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
15:03:33,725 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [STDOUT]
15:03:33,792 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
15:03:33,989 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
15:03:34,008 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [FILE]
15:03:34,136 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@27445814 - No compression will be used
15:03:34,142 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@27445814 - Will use the pattern ./log/HarvestJobManagerApplication.%d{yyyy-MM-dd}.%i.log for the active file
15:03:34,147 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@44d69d - The date pattern is 'yyyy-MM-dd' from file name pattern './log/HarvestJobManagerApplication.%d{yyyy-MM-dd}.%i.log'.
15:03:34,147 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@44d69d - Roll-over at midnight.
15:03:34,169 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@44d69d - Setting initial period to Tue Jan 25 15:03:34 CET 2022
15:03:34,169 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@44d69d - SizeAndTimeBasedFNATP is deprecated. Use SizeAndTimeBasedRollingPolicy instead
15:03:34,169 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@44d69d - For more information see http://logback.qos.ch/manual/appenders.html#SizeAndTimeBasedRollingPolicy
15:03:34,175 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
15:03:34,182 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - Active log file name: ./log/HarvestJobManagerApplication.log
15:03:34,182 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - File property is set to [./log/HarvestJobManagerApplication.log]
15:03:34,184 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [dk.netarkivet.monitor.logging.CachingSLF4JAppender]
15:03:35,869 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [MONITOR]
15:03:35,887 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to WARN
15:03:35,889 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[ROOT]
15:03:35,892 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [FILE] to Logger[ROOT]
15:03:35,892 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [MONITOR] to Logger[ROOT]
15:03:35,897 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [dk.netarkivet] to DEBUG
15:03:35,897 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
15:03:35,901 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@172ed45 - Registering current configuration as safe fallback point
2022-01-25 15:03:35.935 DEBUG dk.netarkivet.common.utils.Settings.invoke0 - Value found in loaded data: SB,KB
SLF4J: A number (1) of logging calls during the initialization phase have been intercepted and are
SLF4J: now being replayed. These are subject to the filtering rules of the underlying logging system.
SLF4J: See also http://www.slf4j.org/codes.html#replay
2022-01-25 15:03:36.008 INFO d.n.c.distribute.JMSConnectionSunMQ.<init> - Creating instance of dk.netarkivet.common.distribute.JMSConnectionSunMQ
2022-01-25 15:03:36.010 DEBUG d.n.common.distribute.JMSConnection.initConnection - Initializing a JMS connection dk.netarkivet.common.distribute.JMSConnectionSunMQ
2022-01-25 15:03:36.011 INFO d.n.c.distribute.JMSConnectionSunMQ.getConnectionFactory - Establishing SunMQ JMS Connection to 'kb-test-adm-001.kb.dk:7676'
--------------------------------------------
STARTING MACHINE: devel@kb-test-har-003.kb.dk
Starting all applications on: kb-test-har-003.kb.dk
Starting linux application: HarvestControllerApplication_kblow001_h3
15:03:37,202 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [/home/devel/Test6/conf/logback_HarvestControllerApplication_kblow001_h3.xml] at [file:/home/devel/Test6/conf/logback_HarvestControllerApplication_kblow001_h3.xml]
15:03:37,415 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - debug attribute not set
15:03:37,419 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
15:03:37,428 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [STDOUT]
15:03:37,443 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
15:03:37,501 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
15:03:37,506 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [FILE]
15:03:37,529 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@447981768 - No compression will be used
15:03:37,530 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@447981768 - Will use the pattern ./log/HarvestControllerApplication_kblow001_h3.%d{yyyy-MM-dd}.%i.log for the active file
15:03:37,534 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@43195e57 - The date pattern is 'yyyy-MM-dd' from file name pattern './log/HarvestControllerApplication_kblow001_h3.%d{yyyy-MM-dd}.%i.log'.
15:03:37,534 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@43195e57 - Roll-over at midnight.
15:03:37,538 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@43195e57 - Setting initial period to Tue Jan 25 15:03:37 CET 2022
15:03:37,539 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@43195e57 - SizeAndTimeBasedFNATP is deprecated. Use SizeAndTimeBasedRollingPolicy instead
15:03:37,539 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@43195e57 - For more information see http://logback.qos.ch/manual/appenders.html#SizeAndTimeBasedRollingPolicy
15:03:37,541 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
15:03:37,543 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - Active log file name: ./log/HarvestControllerApplication_kblow001_h3.log
15:03:37,543 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - File property is set to [./log/HarvestControllerApplication_kblow001_h3.log]
15:03:37,544 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [dk.netarkivet.monitor.logging.CachingSLF4JAppender]
15:03:38,252 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [MONITOR]
15:03:38,254 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to WARN
15:03:38,254 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[ROOT]
15:03:38,255 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [FILE] to Logger[ROOT]
15:03:38,255 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [MONITOR] to Logger[ROOT]
15:03:38,256 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [dk.netarkivet] to DEBUG
15:03:38,256 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
15:03:38,257 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@333291e3 - Registering current configuration as safe fallback point
2022-01-25 15:03:38.262 DEBUG dk.netarkivet.common.utils.Settings.invoke0 - Value found in loaded data: SB,KB
SLF4J: A number (1) of logging calls during the initialization phase have been intercepted and are
SLF4J: now being replayed. These are subject to the filtering rules of the underlying logging system.
SLF4J: See also http://www.slf4j.org/codes.html#replay
Starting dk.netarkivet.harvester.heritrix3.HarvestControllerServer
Version: 7.3-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/f0e2fec464f219bbccc5ad1a8c47bb5ab34b5f79)
2022-01-25 15:03:38.268 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Starting dk.netarkivet.harvester.heritrix3.HarvestControllerServer
Version: 7.3-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/f0e2fec464f219bbccc5ad1a8c47bb5ab34b5f79)
Java VM: 1.8.0_312
2022-01-25 15:03:38.269 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Java VM: 1.8.0_312
java.home: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.312.b07-1.el7_9.x86_64/jre
2022-01-25 15:03:38.269 INFO d.n.common.utils.ApplicationUtils.logAndPrint - java.home: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.312.b07-1.el7_9.x86_64/jre
Working dir: /home/devel/Test6
2022-01-25 15:03:38.270 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Working dir: /home/devel/Test6
2022-01-25 15:03:38.270 INFO d.n.common.utils.ApplicationUtils.startApp - Using settings files '/home/devel/Test6/conf/settings_HarvestControllerApplication_kblow001_h3.xml'
2022-01-25 15:03:38.442 INFO d.n.m.d.JMSMonitorRegistryClient.register - Registering this client for monitoring every 1 minutes, using hostname 'kb-test-har-003.kb.dk' and JMX/RMI ports 8170/8270
2022-01-25 15:03:38.443 INFO d.n.c.m.MBeanConnectorCreator.exposeJMXMBeanServer - Registered mbean server in registry on port 8170 communicating on port 8270 using password file 'conf/jmxremote.password'.
Service URL is service:jmx:rmi://kb-test-har-003.kb.dk:8270/jndi/rmi://kb-test-har-003.kb.dk:8170/jmxrmi
2022-01-25 15:03:38.454 INFO d.n.c.distribute.JMSConnectionSunMQ.<init> - Creating instance of dk.netarkivet.common.distribute.JMSConnectionSunMQ
2022-01-25 15:03:38.455 DEBUG d.n.common.distribute.JMSConnection.initConnection - Initializing a JMS connection dk.netarkivet.common.distribute.JMSConnectionSunMQ
2022-01-25 15:03:38.455 INFO d.n.c.distribute.JMSConnectionSunMQ.getConnectionFactory - Establishing SunMQ JMS Connection to 'kb-test-adm-001.kb.dk:7676'
dk.netarkivet.harvester.heritrix3.HarvestControllerServer Running
2022-01-25 15:03:38.461 INFO d.n.common.utils.ApplicationUtils.logAndPrint - dk.netarkivet.harvester.heritrix3.HarvestControllerServer Running
2022-01-25 15:03:38.474 INFO d.n.h.h.HarvestControllerServer.<init> - Starting HarvestControllerServer.
2022-01-25 15:03:38.474 INFO d.n.h.h.HarvestControllerServer.<init> - Bound to harvest channel 'LOWPRIORITY'
2022-01-25 15:03:38.474 INFO d.n.h.h.HarvestControllerServer.<init> - Serverdir: 'harvester_low'
2022-01-25 15:03:38.475 INFO d.n.h.h.HarvestControllerServer.<init> - Harvesting requires at least 400000000 bytes free.
2022-01-25 15:03:38.608 DEBUG d.n.c.d.bitrepository.BitmagUtils.initialize - Bitrepository settings will be loaded from /home/devel/extra_configs/bitmag_config.
2022-01-25 15:03:38.612 DEBUG d.n.c.d.bitrepository.BitmagUtils.initialize - Path to Bitrepository settings is /home/devel/extra_configs/bitmag_config.
2022-01-25 15:03:38.618 INFO d.n.c.d.bitrepository.HostNameUtils.getHostName - Local hostname (provided by getCanonicalHostName): kb-test-har-003.kb.dk
2022-01-25 15:03:40.414 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - BitmagArcRepositoryClient will timeout on each get request after 300000 milliseconds.
2022-01-25 15:03:40.415 DEBUG d.n.common.distribute.JMSConnection.setListener - Adding dk.netarkivet.archive.arcrepository.distribute.BitmagArcRepositoryClient@65a4798f as listener to Test6_COMMON_THIS_REPOS_CLIENT_130_226_228_9_HCS_KBLOW001_H3
2022-01-25 15:03:40.423 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - BitmagArcRepositoryClient listens for replies on channel '[Queue 'Test6_COMMON_THIS_REPOS_CLIENT_130_226_228_9_HCS_KBLOW001_H3']'
2022-01-25 15:03:40.423 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - Getting bitmag config from settings.common.arcrepositoryClient.bitrepository.settingsDir=/home/devel/extra_configs/bitmag_config
2022-01-25 15:03:40.423 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - Using 'Test6' as default collectionID
2022-01-25 15:03:40.428 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - Storing tempfiles in folder 'arcrepositoryTemp'
2022-01-25 15:03:40.436 INFO d.n.common.utils.HttpsClientBuilder.<init> - Creating Aggressive retry handler for https queries.
2022-01-25 15:03:40.512 INFO d.n.c.utils.BasicTwoWaySSLProvider.loadPrivateKey - Key file '/home/devel/extra_configs/https_config/https_key.pem' with private key and certificate does not exist!
2022-01-25 15:03:40.551 DEBUG d.n.h.h.HarvestControllerServer.<init> - Obtained JMS connection.
2022-01-25 15:03:40.553 INFO d.n.h.h.HarvestControllerServer.<init> - SEND_READY_DELAY used by HarvestControllerServer is 300
2022-01-25 15:03:40.553 INFO d.n.h.heritrix3.PostProcessing.processOldJobs - Looking for unprocessed crawldata in 'harvester_low'
2022-01-25 15:03:40.554 DEBUG d.n.common.distribute.JMSConnection.setListener - Adding dk.netarkivet.harvester.heritrix3.HarvestControllerServer@9635fa as listener to Test6_COMMON_HCHAN_VAL_RESP
2022-01-25 15:03:40.569 INFO d.n.h.h.HarvestControllerServer.<init> - Requested to check the validity of harvest channel 'LOWPRIORITY'
2022-01-25 15:03:40.666 INFO d.n.h.h.HarvestControllerServer.visit - Received message stating that channel 'LOWPRIORITY' is valid.
2022-01-25 15:03:40.669 INFO d.n.h.h.HarvestControllerServer.beginListeningIfSpaceAvailable - Starts to listen to new jobs on queue '[Queue 'Test6_COMMON_JOB_SNAPSHOT_LOWPRIORITY']'
2022-01-25 15:03:40.669 DEBUG d.n.common.distribute.JMSConnection.setListener - Adding dk.netarkivet.harvester.heritrix3.HarvestControllerServer@9635fa as listener to Test6_COMMON_JOB_SNAPSHOT_LOWPRIORITY
--------------------------------------------
STARTING MACHINE: devel@kb-test-har-005.kb.dk
Starting all applications on: kb-test-har-005.kb.dk
Starting linux application: HarvestControllerApplication_browser_1
Starting linux application: HarvestControllerApplication_browser_2
15:03:42,516 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [/home/devel/Test6/conf/logback_HarvestControllerApplication_browser_1.xml] at [file:/home/devel/Test6/conf/logback_HarvestControllerApplication_browser_1.xml]
15:03:42,869 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - debug attribute not set
15:03:42,873 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
15:03:42,913 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [STDOUT]
15:03:42,940 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
15:03:43,055 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
15:03:43,059 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [FILE]
15:03:43,115 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@447981768 - No compression will be used
15:03:43,116 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@447981768 - Will use the pattern ./log/HarvestControllerApplication_browser_1.%d{yyyy-MM-dd}.%i.log for the active file
15:03:43,119 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@43195e57 - The date pattern is 'yyyy-MM-dd' from file name pattern './log/HarvestControllerApplication_browser_1.%d{yyyy-MM-dd}.%i.log'.
15:03:43,119 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@43195e57 - Roll-over at midnight.
15:03:43,122 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@43195e57 - Setting initial period to Tue Jan 25 15:03:43 CET 2022
15:03:43,122 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@43195e57 - SizeAndTimeBasedFNATP is deprecated. Use SizeAndTimeBasedRollingPolicy instead
15:03:43,122 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@43195e57 - For more information see http://logback.qos.ch/manual/appenders.html#SizeAndTimeBasedRollingPolicy
15:03:43,126 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
15:03:43,128 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - Active log file name: ./log/HarvestControllerApplication_browser_1.log
15:03:43,128 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - File property is set to [./log/HarvestControllerApplication_browser_1.log]
15:03:43,129 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [dk.netarkivet.monitor.logging.CachingSLF4JAppender]
15:03:43,779 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [MONITOR]
15:03:43,780 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to WARN
15:03:43,780 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[ROOT]
15:03:43,781 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [FILE] to Logger[ROOT]
15:03:43,781 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [MONITOR] to Logger[ROOT]
15:03:43,782 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [dk.netarkivet] to DEBUG
15:03:43,782 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
15:03:43,782 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@333291e3 - Registering current configuration as safe fallback point
2022-01-25 15:03:43.792 DEBUG dk.netarkivet.common.utils.Settings.invoke0 - Value found in loaded data: SB,KB
SLF4J: A number (1) of logging calls during the initialization phase have been intercepted and are
SLF4J: now being replayed. These are subject to the filtering rules of the underlying logging system.
SLF4J: See also http://www.slf4j.org/codes.html#replay
Starting dk.netarkivet.harvester.heritrix3.HarvestControllerServer
Version: 7.3-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/f0e2fec464f219bbccc5ad1a8c47bb5ab34b5f79)
2022-01-25 15:03:43.812 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Starting dk.netarkivet.harvester.heritrix3.HarvestControllerServer
Version: 7.3-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/f0e2fec464f219bbccc5ad1a8c47bb5ab34b5f79)
Java VM: 1.8.0_312
2022-01-25 15:03:43.813 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Java VM: 1.8.0_312
java.home: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.312.b07-1.el7_9.x86_64/jre
2022-01-25 15:03:43.813 INFO d.n.common.utils.ApplicationUtils.logAndPrint - java.home: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.312.b07-1.el7_9.x86_64/jre
Working dir: /home/devel/Test6
2022-01-25 15:03:43.813 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Working dir: /home/devel/Test6
2022-01-25 15:03:43.813 INFO d.n.common.utils.ApplicationUtils.startApp - Using settings files '/home/devel/Test6/conf/settings_HarvestControllerApplication_browser_1.xml'
2022-01-25 15:03:44.055 INFO d.n.m.d.JMSMonitorRegistryClient.register - Registering this client for monitoring every 1 minutes, using hostname 'kb-test-har-005.kb.dk' and JMX/RMI ports 8170/8270
2022-01-25 15:03:44.055 INFO d.n.c.m.MBeanConnectorCreator.exposeJMXMBeanServer - Registered mbean server in registry on port 8170 communicating on port 8270 using password file 'conf/jmxremote.password'.
Service URL is service:jmx:rmi://kb-test-har-005.kb.dk:8270/jndi/rmi://kb-test-har-005.kb.dk:8170/jmxrmi
dk.netarkivet.harvester.heritrix3.HarvestControllerServer Running
2022-01-25 15:03:44.060 INFO d.n.common.utils.ApplicationUtils.logAndPrint - dk.netarkivet.harvester.heritrix3.HarvestControllerServer Running
2022-01-25 15:03:44.082 INFO d.n.c.distribute.JMSConnectionSunMQ.<init> - Creating instance of dk.netarkivet.common.distribute.JMSConnectionSunMQ
2022-01-25 15:03:44.082 DEBUG d.n.common.distribute.JMSConnection.initConnection - Initializing a JMS connection dk.netarkivet.common.distribute.JMSConnectionSunMQ
2022-01-25 15:03:44.083 INFO d.n.c.distribute.JMSConnectionSunMQ.getConnectionFactory - Establishing SunMQ JMS Connection to 'kb-test-adm-001.kb.dk:7676'
2022-01-25 15:03:44.131 INFO d.n.h.h.HarvestControllerServer.<init> - Starting HarvestControllerServer.
2022-01-25 15:03:44.131 INFO d.n.h.h.HarvestControllerServer.<init> - Bound to harvest channel 'BROWSER'
2022-01-25 15:03:44.131 INFO d.n.h.h.HarvestControllerServer.<init> - Serverdir: 'harvester_browser_1'
2022-01-25 15:03:44.132 INFO d.n.h.h.HarvestControllerServer.<init> - Harvesting requires at least 400000000 bytes free.
2022-01-25 15:03:44.289 DEBUG d.n.c.d.bitrepository.BitmagUtils.initialize - Bitrepository settings will be loaded from /home/devel/extra_configs/bitmag_config.
2022-01-25 15:03:44.290 DEBUG d.n.c.d.bitrepository.BitmagUtils.initialize - Path to Bitrepository settings is /home/devel/extra_configs/bitmag_config.
2022-01-25 15:03:44.308 INFO d.n.c.d.bitrepository.HostNameUtils.getHostName - Local hostname (provided by getCanonicalHostName): kb-test-har-005.kb.dk
2022-01-25 15:03:46.831 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - BitmagArcRepositoryClient will timeout on each get request after 300000 milliseconds.
2022-01-25 15:03:46.832 DEBUG d.n.common.distribute.JMSConnection.setListener - Adding dk.netarkivet.archive.arcrepository.distribute.BitmagArcRepositoryClient@65a4798f as listener to Test6_COMMON_THIS_REPOS_CLIENT_130_226_228_16_HCS_BROWSER_1
2022-01-25 15:03:46.844 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - BitmagArcRepositoryClient listens for replies on channel '[Queue 'Test6_COMMON_THIS_REPOS_CLIENT_130_226_228_16_HCS_BROWSER_1']'
2022-01-25 15:03:46.845 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - Getting bitmag config from settings.common.arcrepositoryClient.bitrepository.settingsDir=/home/devel/extra_configs/bitmag_config
2022-01-25 15:03:46.845 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - Using 'Test6' as default collectionID
2022-01-25 15:03:46.848 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - Storing tempfiles in folder 'arcrepositoryTemp'
2022-01-25 15:03:46.857 INFO d.n.common.utils.HttpsClientBuilder.<init> - Creating Aggressive retry handler for https queries.
2022-01-25 15:03:46.926 INFO d.n.c.utils.BasicTwoWaySSLProvider.loadPrivateKey - Key file '/home/devel/extra_configs/https_config/https_key.pem' with private key and certificate does not exist!
2022-01-25 15:03:46.953 DEBUG d.n.h.h.HarvestControllerServer.<init> - Obtained JMS connection.
2022-01-25 15:03:46.954 INFO d.n.h.h.HarvestControllerServer.<init> - SEND_READY_DELAY used by HarvestControllerServer is 300
2022-01-25 15:03:46.955 INFO d.n.h.heritrix3.PostProcessing.processOldJobs - Looking for unprocessed crawldata in 'harvester_browser_1'
2022-01-25 15:03:46.955 DEBUG d.n.common.distribute.JMSConnection.setListener - Adding dk.netarkivet.harvester.heritrix3.HarvestControllerServer@9635fa as listener to Test6_COMMON_HCHAN_VAL_RESP
15:03:42,637 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [/home/devel/Test6/conf/logback_HarvestControllerApplication_browser_2.xml] at [file:/home/devel/Test6/conf/logback_HarvestControllerApplication_browser_2.xml]
15:03:43,029 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - debug attribute not set
15:03:43,032 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
15:03:43,054 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [STDOUT]
15:03:43,079 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
15:03:43,184 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
15:03:43,195 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [FILE]
15:03:43,225 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@447981768 - No compression will be used
15:03:43,226 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@447981768 - Will use the pattern ./log/HarvestControllerApplication_browser_2.%d{yyyy-MM-dd}.%i.log for the active file
15:03:43,229 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@43195e57 - The date pattern is 'yyyy-MM-dd' from file name pattern './log/HarvestControllerApplication_browser_2.%d{yyyy-MM-dd}.%i.log'.
15:03:43,229 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@43195e57 - Roll-over at midnight.
15:03:43,238 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@43195e57 - Setting initial period to Tue Jan 25 15:03:43 CET 2022
15:03:43,238 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@43195e57 - SizeAndTimeBasedFNATP is deprecated. Use SizeAndTimeBasedRollingPolicy instead
15:03:43,238 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@43195e57 - For more information see http://logback.qos.ch/manual/appenders.html#SizeAndTimeBasedRollingPolicy
15:03:43,240 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
15:03:43,242 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - Active log file name: ./log/HarvestControllerApplication_browser_2.log
15:03:43,242 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - File property is set to [./log/HarvestControllerApplication_browser_2.log]
15:03:43,243 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [dk.netarkivet.monitor.logging.CachingSLF4JAppender]
15:03:43,973 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [MONITOR]
15:03:43,993 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to WARN
15:03:43,993 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[ROOT]
15:03:43,993 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [FILE] to Logger[ROOT]
15:03:43,993 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [MONITOR] to Logger[ROOT]
15:03:43,994 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [dk.netarkivet] to DEBUG
15:03:43,994 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
15:03:43,995 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@333291e3 - Registering current configuration as safe fallback point
2022-01-25 15:03:43.999 DEBUG dk.netarkivet.common.utils.Settings.invoke0 - Value found in loaded data: SB,KB
SLF4J: A number (1) of logging calls during the initialization phase have been intercepted and are
SLF4J: now being replayed. These are subject to the filtering rules of the underlying logging system.
SLF4J: See also http://www.slf4j.org/codes.html#replay
Starting dk.netarkivet.harvester.heritrix3.HarvestControllerServer
Version: 7.3-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/f0e2fec464f219bbccc5ad1a8c47bb5ab34b5f79)
2022-01-25 15:03:44.004 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Starting dk.netarkivet.harvester.heritrix3.HarvestControllerServer
Version: 7.3-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/f0e2fec464f219bbccc5ad1a8c47bb5ab34b5f79)
Java VM: 1.8.0_312
2022-01-25 15:03:44.005 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Java VM: 1.8.0_312
java.home: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.312.b07-1.el7_9.x86_64/jre
2022-01-25 15:03:44.005 INFO d.n.common.utils.ApplicationUtils.logAndPrint - java.home: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.312.b07-1.el7_9.x86_64/jre
Working dir: /home/devel/Test6
2022-01-25 15:03:44.005 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Working dir: /home/devel/Test6
2022-01-25 15:03:44.006 INFO d.n.common.utils.ApplicationUtils.startApp - Using settings files '/home/devel/Test6/conf/settings_HarvestControllerApplication_browser_2.xml'
2022-01-25 15:03:44.167 INFO d.n.m.d.JMSMonitorRegistryClient.register - Registering this client for monitoring every 1 minutes, using hostname 'kb-test-har-005.kb.dk' and JMX/RMI ports 8175/8275
2022-01-25 15:03:44.168 INFO d.n.c.m.MBeanConnectorCreator.exposeJMXMBeanServer - Registered mbean server in registry on port 8175 communicating on port 8275 using password file 'conf/jmxremote.password'.
Service URL is service:jmx:rmi://kb-test-har-005.kb.dk:8275/jndi/rmi://kb-test-har-005.kb.dk:8175/jmxrmi
dk.netarkivet.harvester.heritrix3.HarvestControllerServer Running
2022-01-25 15:03:44.172 INFO d.n.common.utils.ApplicationUtils.logAndPrint - dk.netarkivet.harvester.heritrix3.HarvestControllerServer Running
2022-01-25 15:03:44.187 INFO d.n.c.distribute.JMSConnectionSunMQ.<init> - Creating instance of dk.netarkivet.common.distribute.JMSConnectionSunMQ
2022-01-25 15:03:44.188 DEBUG d.n.common.distribute.JMSConnection.initConnection - Initializing a JMS connection dk.netarkivet.common.distribute.JMSConnectionSunMQ
2022-01-25 15:03:44.188 INFO d.n.c.distribute.JMSConnectionSunMQ.getConnectionFactory - Establishing SunMQ JMS Connection to 'kb-test-adm-001.kb.dk:7676'
2022-01-25 15:03:44.215 INFO d.n.h.h.HarvestControllerServer.<init> - Starting HarvestControllerServer.
2022-01-25 15:03:44.216 INFO d.n.h.h.HarvestControllerServer.<init> - Bound to harvest channel 'BROWSER'
2022-01-25 15:03:44.216 INFO d.n.h.h.HarvestControllerServer.<init> - Serverdir: 'harvester_browser_2'
2022-01-25 15:03:44.217 INFO d.n.h.h.HarvestControllerServer.<init> - Harvesting requires at least 400000000 bytes free.
2022-01-25 15:03:44.328 DEBUG d.n.c.d.bitrepository.BitmagUtils.initialize - Bitrepository settings will be loaded from /home/devel/extra_configs/bitmag_config.
2022-01-25 15:03:44.329 DEBUG d.n.c.d.bitrepository.BitmagUtils.initialize - Path to Bitrepository settings is /home/devel/extra_configs/bitmag_config.
2022-01-25 15:03:44.333 INFO d.n.c.d.bitrepository.HostNameUtils.getHostName - Local hostname (provided by getCanonicalHostName): kb-test-har-005.kb.dk
2022-01-25 15:03:46.362 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - BitmagArcRepositoryClient will timeout on each get request after 300000 milliseconds.
2022-01-25 15:03:46.363 DEBUG d.n.common.distribute.JMSConnection.setListener - Adding dk.netarkivet.archive.arcrepository.distribute.BitmagArcRepositoryClient@65a4798f as listener to Test6_COMMON_THIS_REPOS_CLIENT_130_226_228_16_HCS_BROWSER_2
2022-01-25 15:03:46.396 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - BitmagArcRepositoryClient listens for replies on channel '[Queue 'Test6_COMMON_THIS_REPOS_CLIENT_130_226_228_16_HCS_BROWSER_2']'
2022-01-25 15:03:46.397 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - Getting bitmag config from settings.common.arcrepositoryClient.bitrepository.settingsDir=/home/devel/extra_configs/bitmag_config
2022-01-25 15:03:46.397 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - Using 'Test6' as default collectionID
2022-01-25 15:03:46.401 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - Storing tempfiles in folder 'arcrepositoryTemp'
2022-01-25 15:03:46.416 INFO d.n.common.utils.HttpsClientBuilder.<init> - Creating Aggressive retry handler for https queries.
2022-01-25 15:03:46.518 INFO d.n.c.utils.BasicTwoWaySSLProvider.loadPrivateKey - Key file '/home/devel/extra_configs/https_config/https_key.pem' with private key and certificate does not exist!
2022-01-25 15:03:46.548 DEBUG d.n.h.h.HarvestControllerServer.<init> - Obtained JMS connection.
2022-01-25 15:03:46.549 INFO d.n.h.h.HarvestControllerServer.<init> - SEND_READY_DELAY used by HarvestControllerServer is 300
2022-01-25 15:03:46.549 INFO d.n.h.heritrix3.PostProcessing.processOldJobs - Looking for unprocessed crawldata in 'harvester_browser_2'
2022-01-25 15:03:46.550 DEBUG d.n.common.distribute.JMSConnection.setListener - Adding dk.netarkivet.harvester.heritrix3.HarvestControllerServer@9635fa as listener to Test6_COMMON_HCHAN_VAL_RESP
2022-01-25 15:03:46.570 INFO d.n.h.h.HarvestControllerServer.<init> - Requested to check the validity of harvest channel 'BROWSER'
2022-01-25 15:03:46.599 INFO d.n.h.h.HarvestControllerServer.visit - Received message stating that channel 'BROWSER' is valid.
2022-01-25 15:03:46.600 INFO d.n.h.h.HarvestControllerServer.beginListeningIfSpaceAvailable - Starts to listen to new jobs on queue '[Queue 'Test6_COMMON_JOB_PARTIAL_BROWSER']'
2022-01-25 15:03:46.600 DEBUG d.n.common.distribute.JMSConnection.setListener - Adding dk.netarkivet.harvester.heritrix3.HarvestControllerServer@9635fa as listener to Test6_COMMON_JOB_PARTIAL_BROWSER
--------------------------------------------
STARTING MACHINE: devel@kb-test-har-004.kb.dk
Starting all applications on: kb-test-har-004.kb.dk
Starting linux application: HarvestControllerApplication_kblow002_h3
Starting linux application: HarvestControllerApplication_kbhigh_h3
15:03:48,061 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [/home/devel/Test6/conf/logback_HarvestControllerApplication_kbhigh_h3.xml] at [file:/home/devel/Test6/conf/logback_HarvestControllerApplication_kbhigh_h3.xml]
15:03:48,496 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - debug attribute not set
15:03:48,505 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
15:03:48,522 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [STDOUT]
15:03:48,551 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
15:03:48,621 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
15:03:48,625 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [FILE]
15:03:48,666 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@1704629915 - No compression will be used
15:03:48,667 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@1704629915 - Will use the pattern ./log/HarvestControllerApplication_kbhigh_h3.%d{yyyy-MM-dd}.%i.log for the active file
15:03:48,673 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@76908cc0 - The date pattern is 'yyyy-MM-dd' from file name pattern './log/HarvestControllerApplication_kbhigh_h3.%d{yyyy-MM-dd}.%i.log'.
15:03:48,673 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@76908cc0 - Roll-over at midnight.
15:03:48,682 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@76908cc0 - Setting initial period to Tue Jan 25 15:03:48 CET 2022
15:03:48,682 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@76908cc0 - SizeAndTimeBasedFNATP is deprecated. Use SizeAndTimeBasedRollingPolicy instead
15:03:48,682 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@76908cc0 - For more information see http://logback.qos.ch/manual/appenders.html#SizeAndTimeBasedRollingPolicy
15:03:48,689 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
15:03:48,695 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - Active log file name: ./log/HarvestControllerApplication_kbhigh_h3.log
15:03:48,695 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - File property is set to [./log/HarvestControllerApplication_kbhigh_h3.log]
15:03:48,696 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [dk.netarkivet.monitor.logging.CachingSLF4JAppender]
15:03:49,280 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [MONITOR]
15:03:49,286 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to WARN
15:03:49,286 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[ROOT]
15:03:49,289 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [FILE] to Logger[ROOT]
15:03:49,289 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [MONITOR] to Logger[ROOT]
15:03:49,289 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [dk.netarkivet] to DEBUG
15:03:49,289 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
15:03:49,294 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@2473d930 - Registering current configuration as safe fallback point
2022-01-25 15:03:49.299 DEBUG dk.netarkivet.common.utils.Settings.invoke0 - Value found in loaded data: SB,KB
SLF4J: A number (1) of logging calls during the initialization phase have been intercepted and are
SLF4J: now being replayed. These are subject to the filtering rules of the underlying logging system.
SLF4J: See also http://www.slf4j.org/codes.html#replay
Starting dk.netarkivet.harvester.heritrix3.HarvestControllerServer
Version: 7.3-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/f0e2fec464f219bbccc5ad1a8c47bb5ab34b5f79)
2022-01-25 15:03:49.320 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Starting dk.netarkivet.harvester.heritrix3.HarvestControllerServer
Version: 7.3-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/f0e2fec464f219bbccc5ad1a8c47bb5ab34b5f79)
Java VM: 1.8.0_25
2022-01-25 15:03:49.321 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Java VM: 1.8.0_25
java.home: /usr/local/java/jdk1.8.0_25_x64/jre
2022-01-25 15:03:49.321 INFO d.n.common.utils.ApplicationUtils.logAndPrint - java.home: /usr/local/java/jdk1.8.0_25_x64/jre
Working dir: /home/devel/Test6
2022-01-25 15:03:49.321 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Working dir: /home/devel/Test6
2022-01-25 15:03:49.321 INFO d.n.common.utils.ApplicationUtils.startApp - Using settings files '/home/devel/Test6/conf/settings_HarvestControllerApplication_kbhigh_h3.xml'
2022-01-25 15:03:49.501 INFO d.n.m.d.JMSMonitorRegistryClient.register - Registering this client for monitoring every 1 minutes, using hostname 'kb-test-har-004.kb.dk' and JMX/RMI ports 8172/8272
2022-01-25 15:03:49.502 INFO d.n.c.m.MBeanConnectorCreator.exposeJMXMBeanServer - Registered mbean server in registry on port 8172 communicating on port 8272 using password file 'conf/jmxremote.password'.
Service URL is service:jmx:rmi://kb-test-har-004.kb.dk:8272/jndi/rmi://kb-test-har-004.kb.dk:8172/jmxrmi
dk.netarkivet.harvester.heritrix3.HarvestControllerServer Running
2022-01-25 15:03:49.511 INFO d.n.common.utils.ApplicationUtils.logAndPrint - dk.netarkivet.harvester.heritrix3.HarvestControllerServer Running
2022-01-25 15:03:49.531 INFO d.n.c.distribute.JMSConnectionSunMQ.<init> - Creating instance of dk.netarkivet.common.distribute.JMSConnectionSunMQ
2022-01-25 15:03:49.533 DEBUG d.n.common.distribute.JMSConnection.initConnection - Initializing a JMS connection dk.netarkivet.common.distribute.JMSConnectionSunMQ
2022-01-25 15:03:49.540 INFO d.n.c.distribute.JMSConnectionSunMQ.getConnectionFactory - Establishing SunMQ JMS Connection to 'kb-test-adm-001.kb.dk:7676'
2022-01-25 15:03:49.558 INFO d.n.h.h.HarvestControllerServer.<init> - Starting HarvestControllerServer.
2022-01-25 15:03:49.559 INFO d.n.h.h.HarvestControllerServer.<init> - Bound to harvest channel 'HIGHPRIORITY'
2022-01-25 15:03:49.559 INFO d.n.h.h.HarvestControllerServer.<init> - Serverdir: 'harvester_high'
2022-01-25 15:03:49.559 INFO d.n.h.h.HarvestControllerServer.<init> - Harvesting requires at least 400000000 bytes free.
2022-01-25 15:03:49.768 DEBUG d.n.c.d.bitrepository.BitmagUtils.initialize - Bitrepository settings will be loaded from /home/devel/extra_configs/bitmag_config.
2022-01-25 15:03:49.786 DEBUG d.n.c.d.bitrepository.BitmagUtils.initialize - Path to Bitrepository settings is /home/devel/extra_configs/bitmag_config.
2022-01-25 15:03:49.801 INFO d.n.c.d.bitrepository.HostNameUtils.getHostName - Local hostname (provided by getCanonicalHostName): kb-test-har-004.kb.dk
15:03:48,054 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [/home/devel/Test6/conf/logback_HarvestControllerApplication_kblow002_h3.xml] at [file:/home/devel/Test6/conf/logback_HarvestControllerApplication_kblow002_h3.xml]
15:03:48,457 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - debug attribute not set
15:03:48,472 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
15:03:48,486 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [STDOUT]
15:03:48,512 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
15:03:48,565 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
15:03:48,569 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [FILE]
15:03:48,613 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@1704629915 - No compression will be used
15:03:48,615 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@1704629915 - Will use the pattern ./log/HarvestControllerApplication_kblow002_h3.%d{yyyy-MM-dd}.%i.log for the active file
15:03:48,618 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@76908cc0 - The date pattern is 'yyyy-MM-dd' from file name pattern './log/HarvestControllerApplication_kblow002_h3.%d{yyyy-MM-dd}.%i.log'.
15:03:48,618 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@76908cc0 - Roll-over at midnight.
15:03:48,624 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@76908cc0 - Setting initial period to Tue Jan 25 15:03:48 CET 2022
15:03:48,624 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@76908cc0 - SizeAndTimeBasedFNATP is deprecated. Use SizeAndTimeBasedRollingPolicy instead
15:03:48,624 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@76908cc0 - For more information see http://logback.qos.ch/manual/appenders.html#SizeAndTimeBasedRollingPolicy
15:03:48,626 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
15:03:48,633 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - Active log file name: ./log/HarvestControllerApplication_kblow002_h3.log
15:03:48,633 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - File property is set to [./log/HarvestControllerApplication_kblow002_h3.log]
15:03:48,634 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [dk.netarkivet.monitor.logging.CachingSLF4JAppender]
15:03:49,271 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [MONITOR]
15:03:49,273 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to WARN
15:03:49,273 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[ROOT]
15:03:49,274 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [FILE] to Logger[ROOT]
15:03:49,274 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [MONITOR] to Logger[ROOT]
15:03:49,274 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [dk.netarkivet] to DEBUG
15:03:49,274 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
15:03:49,280 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@2473d930 - Registering current configuration as safe fallback point
2022-01-25 15:03:49.294 DEBUG dk.netarkivet.common.utils.Settings.invoke0 - Value found in loaded data: SB,KB
SLF4J: A number (1) of logging calls during the initialization phase have been intercepted and are
SLF4J: now being replayed. These are subject to the filtering rules of the underlying logging system.
SLF4J: See also http://www.slf4j.org/codes.html#replay
Starting dk.netarkivet.harvester.heritrix3.HarvestControllerServer
Version: 7.3-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/f0e2fec464f219bbccc5ad1a8c47bb5ab34b5f79)
2022-01-25 15:03:49.311 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Starting dk.netarkivet.harvester.heritrix3.HarvestControllerServer
Version: 7.3-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/f0e2fec464f219bbccc5ad1a8c47bb5ab34b5f79)
Java VM: 1.8.0_25
2022-01-25 15:03:49.311 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Java VM: 1.8.0_25
java.home: /usr/local/java/jdk1.8.0_25_x64/jre
2022-01-25 15:03:49.312 INFO d.n.common.utils.ApplicationUtils.logAndPrint - java.home: /usr/local/java/jdk1.8.0_25_x64/jre
Working dir: /home/devel/Test6
2022-01-25 15:03:49.312 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Working dir: /home/devel/Test6
2022-01-25 15:03:49.312 INFO d.n.common.utils.ApplicationUtils.startApp - Using settings files '/home/devel/Test6/conf/settings_HarvestControllerApplication_kblow002_h3.xml'
2022-01-25 15:03:49.476 INFO d.n.m.d.JMSMonitorRegistryClient.register - Registering this client for monitoring every 1 minutes, using hostname 'kb-test-har-004.kb.dk' and JMX/RMI ports 8170/8270
2022-01-25 15:03:49.477 INFO d.n.c.m.MBeanConnectorCreator.exposeJMXMBeanServer - Registered mbean server in registry on port 8170 communicating on port 8270 using password file 'conf/jmxremote.password'.
Service URL is service:jmx:rmi://kb-test-har-004.kb.dk:8270/jndi/rmi://kb-test-har-004.kb.dk:8170/jmxrmi
dk.netarkivet.harvester.heritrix3.HarvestControllerServer Running
2022-01-25 15:03:49.486 INFO d.n.common.utils.ApplicationUtils.logAndPrint - dk.netarkivet.harvester.heritrix3.HarvestControllerServer Running
2022-01-25 15:03:49.507 INFO d.n.c.distribute.JMSConnectionSunMQ.<init> - Creating instance of dk.netarkivet.common.distribute.JMSConnectionSunMQ
2022-01-25 15:03:49.507 DEBUG d.n.common.distribute.JMSConnection.initConnection - Initializing a JMS connection dk.netarkivet.common.distribute.JMSConnectionSunMQ
2022-01-25 15:03:49.507 INFO d.n.c.distribute.JMSConnectionSunMQ.getConnectionFactory - Establishing SunMQ JMS Connection to 'kb-test-adm-001.kb.dk:7676'
2022-01-25 15:03:49.521 INFO d.n.h.h.HarvestControllerServer.<init> - Starting HarvestControllerServer.
2022-01-25 15:03:49.521 INFO d.n.h.h.HarvestControllerServer.<init> - Bound to harvest channel 'LOWPRIORITY'
2022-01-25 15:03:49.521 INFO d.n.h.h.HarvestControllerServer.<init> - Serverdir: 'harvester_low'
2022-01-25 15:03:49.521 INFO d.n.h.h.HarvestControllerServer.<init> - Harvesting requires at least 400000000 bytes free.
2022-01-25 15:03:49.690 DEBUG d.n.c.d.bitrepository.BitmagUtils.initialize - Bitrepository settings will be loaded from /home/devel/extra_configs/bitmag_config.
2022-01-25 15:03:49.696 DEBUG d.n.c.d.bitrepository.BitmagUtils.initialize - Path to Bitrepository settings is /home/devel/extra_configs/bitmag_config.
2022-01-25 15:03:49.702 INFO d.n.c.d.bitrepository.HostNameUtils.getHostName - Local hostname (provided by getCanonicalHostName): kb-test-har-004.kb.dk
--------------------------------------------
STARTING MACHINE: devel@kb-test-acs-001.kb.dk
Starting all applications on: kb-test-acs-001.kb.dk
Starting linux application: IndexServerApplication
Starting linux application: ViewerProxyApplication
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/devel/Test6/lib/logback-classic-1.2.3.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/devel/Test6/lib/slf4j-log4j12-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
15:03:53,933 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [/home/devel/Test6/conf/logback_IndexServerApplication.xml] at [file:/home/devel/Test6/conf/logback_IndexServerApplication.xml]
15:03:54,230 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - debug attribute not set
15:03:54,234 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
15:03:54,242 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [STDOUT]
15:03:54,262 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
15:03:54,318 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
15:03:54,333 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [FILE]
15:03:54,374 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@1954406292 - No compression will be used
15:03:54,376 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@1954406292 - Will use the pattern ./log/IndexServerApplication.%d{yyyy-MM-dd}.%i.log for the active file
15:03:54,378 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@35e2d654 - The date pattern is 'yyyy-MM-dd' from file name pattern './log/IndexServerApplication.%d{yyyy-MM-dd}.%i.log'.
15:03:54,378 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@35e2d654 - Roll-over at midnight.
15:03:54,382 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@35e2d654 - Setting initial period to Tue Jan 25 15:03:54 CET 2022
15:03:54,383 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@35e2d654 - SizeAndTimeBasedFNATP is deprecated. Use SizeAndTimeBasedRollingPolicy instead
15:03:54,383 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@35e2d654 - For more information see http://logback.qos.ch/manual/appenders.html#SizeAndTimeBasedRollingPolicy
15:03:54,385 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
15:03:54,387 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - Active log file name: ./log/IndexServerApplication.log
15:03:54,387 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - File property is set to [./log/IndexServerApplication.log]
15:03:54,390 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [dk.netarkivet.monitor.logging.CachingSLF4JAppender]
15:03:55,038 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [MONITOR]
15:03:55,047 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to WARN
15:03:55,047 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[ROOT]
15:03:55,048 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [FILE] to Logger[ROOT]
15:03:55,048 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [MONITOR] to Logger[ROOT]
15:03:55,048 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [dk.netarkivet] to DEBUG
15:03:55,048 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
15:03:55,049 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@1bd4fdd - Registering current configuration as safe fallback point
SLF4J: Actual binding is of type [ch.qos.logback.classic.util.ContextSelectorStaticBinder]
2022-01-25 15:03:55.054 DEBUG dk.netarkivet.common.utils.Settings.invoke0 - Value found in loaded data: SB,KB
SLF4J: A number (1) of logging calls during the initialization phase have been intercepted and are
SLF4J: now being replayed. These are subject to the filtering rules of the underlying logging system.
SLF4J: See also http://www.slf4j.org/codes.html#replay
Starting dk.netarkivet.harvester.indexserver.IndexServer
Version: 7.3-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/f0e2fec464f219bbccc5ad1a8c47bb5ab34b5f79)
2022-01-25 15:03:55.070 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Starting dk.netarkivet.harvester.indexserver.IndexServer
Version: 7.3-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/f0e2fec464f219bbccc5ad1a8c47bb5ab34b5f79)
Java VM: 1.8.0_312
2022-01-25 15:03:55.071 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Java VM: 1.8.0_312
java.home: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.312.b07-1.el7_9.x86_64/jre
2022-01-25 15:03:55.071 INFO d.n.common.utils.ApplicationUtils.logAndPrint - java.home: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.312.b07-1.el7_9.x86_64/jre
Working dir: /home/devel/Test6
2022-01-25 15:03:55.071 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Working dir: /home/devel/Test6
2022-01-25 15:03:55.071 INFO d.n.common.utils.ApplicationUtils.startApp - Using settings files '/home/devel/Test6/conf/settings_IndexServerApplication.xml'
2022-01-25 15:03:55.290 INFO d.n.m.d.JMSMonitorRegistryClient.register - Registering this client for monitoring every 1 minutes, using hostname 'kb-test-acs-001.kb.dk' and JMX/RMI ports 8371/8471
2022-01-25 15:03:55.291 INFO d.n.c.m.MBeanConnectorCreator.exposeJMXMBeanServer - Registered mbean server in registry on port 8371 communicating on port 8471 using password file 'conf/jmxremote.password'.
Service URL is service:jmx:rmi://kb-test-acs-001.kb.dk:8471/jndi/rmi://kb-test-acs-001.kb.dk:8371/jmxrmi
dk.netarkivet.harvester.indexserver.IndexServer Running
2022-01-25 15:03:55.295 INFO d.n.common.utils.ApplicationUtils.logAndPrint - dk.netarkivet.harvester.indexserver.IndexServer Running
2022-01-25 15:03:55.298 INFO d.n.c.distribute.JMSConnectionSunMQ.<init> - Creating instance of dk.netarkivet.common.distribute.JMSConnectionSunMQ
2022-01-25 15:03:55.299 DEBUG d.n.common.distribute.JMSConnection.initConnection - Initializing a JMS connection dk.netarkivet.common.distribute.JMSConnectionSunMQ
2022-01-25 15:03:55.300 INFO d.n.h.indexserver.FileBasedCache.<init> - Metadata cache for 'cdxdata' uses directory '/home/devel/Test6/cache/cdxdata'
2022-01-25 15:03:55.308 INFO d.n.c.distribute.JMSConnectionSunMQ.getConnectionFactory - Establishing SunMQ JMS Connection to 'kb-test-adm-001.kb.dk:7676'
2022-01-25 15:03:55.308 DEBUG d.n.c.d.bitrepository.BitmagUtils.initialize - Bitrepository settings will be loaded from /home/devel/extra_configs/bitmag_config.
2022-01-25 15:03:55.342 DEBUG d.n.c.d.bitrepository.BitmagUtils.initialize - Path to Bitrepository settings is /home/devel/extra_configs/bitmag_config.
2022-01-25 15:03:55.354 INFO d.n.c.d.bitrepository.HostNameUtils.getHostName - Local hostname (provided by getCanonicalHostName): kb-test-acs-001.kb.dk
2022-01-25 15:03:57.433 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - BitmagArcRepositoryClient will timeout on each get request after 300000 milliseconds.
2022-01-25 15:03:57.434 DEBUG d.n.common.distribute.JMSConnection.setListener - Adding dk.netarkivet.archive.arcrepository.distribute.BitmagArcRepositoryClient@107ed6fc as listener to Test6_COMMON_THIS_REPOS_CLIENT_130_226_228_5_IS
2022-01-25 15:03:57.446 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - BitmagArcRepositoryClient listens for replies on channel '[Queue 'Test6_COMMON_THIS_REPOS_CLIENT_130_226_228_5_IS']'
2022-01-25 15:03:57.446 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - Getting bitmag config from settings.common.arcrepositoryClient.bitrepository.settingsDir=/home/devel/extra_configs/bitmag_config
2022-01-25 15:03:57.447 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - Using 'Test6' as default collectionID
2022-01-25 15:03:57.451 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - Storing tempfiles in folder 'arcrepositoryTemp'
2022-01-25 15:03:57.459 INFO d.n.common.utils.HttpsClientBuilder.<init> - Creating Aggressive retry handler for https queries.
2022-01-25 15:03:57.562 DEBUG d.n.c.utils.BasicTwoWaySSLProvider.loadPrivateKey - X509CertificateHolder found
2022-01-25 15:03:57.563 DEBUG d.n.c.utils.BasicTwoWaySSLProvider.loadPrivateKey - PrivateKeyInfo found
2022-01-25 15:03:57.605 INFO d.n.h.indexserver.RawMetadataCache.<init> - Metadata cache for 'cdxdata' is fetching metadata with urls matching 'metadata://[^/]*/crawl/index/cdx.*' and mimetype matching 'application/x-cdx'. Migration of duplicate records is disabled
2022-01-25 15:03:57.606 INFO d.n.h.indexserver.FileBasedCache.<init> - Metadata cache for 'cdxindex' uses directory '/home/devel/Test6/cache/cdxindex'
2022-01-25 15:03:57.608 INFO d.n.h.indexserver.FileBasedCache.<init> - Metadata cache for 'crawllog' uses directory '/home/devel/Test6/cache/crawllog'
2022-01-25 15:03:57.609 INFO d.n.h.indexserver.RawMetadataCache.<init> - Metadata cache for 'crawllog' is fetching metadata with urls matching 'metadata://[^/]*/crawl/logs/crawl\.log.*' and mimetype matching 'text/plain'. Migration of duplicate records is disabled
2022-01-25 15:03:57.609 INFO d.n.h.indexserver.FileBasedCache.<init> - Metadata cache for 'dedupcrawllogindex' uses directory '/home/devel/Test6/cache/dedupcrawllogindex'
2022-01-25 15:03:57.610 INFO d.n.h.indexserver.FileBasedCache.<init> - Metadata cache for 'cdxdata' uses directory '/home/devel/Test6/cache/cdxdata'
2022-01-25 15:03:57.610 INFO d.n.h.indexserver.RawMetadataCache.<init> - Metadata cache for 'cdxdata' is fetching metadata with urls matching 'metadata://[^/]*/crawl/index/cdx.*' and mimetype matching 'application/x-cdx'. Migration of duplicate records is disabled
2022-01-25 15:03:57.611 INFO d.n.h.indexserver.FileBasedCache.<init> - Metadata cache for 'crawllog' uses directory '/home/devel/Test6/cache/crawllog'
2022-01-25 15:03:57.612 INFO d.n.h.indexserver.RawMetadataCache.<init> - Metadata cache for 'crawllog' is fetching metadata with urls matching 'metadata://[^/]*/crawl/logs/crawl\.log.*' and mimetype matching 'text/plain'. Migration of duplicate records is disabled
2022-01-25 15:03:57.612 INFO d.n.h.indexserver.FileBasedCache.<init> - Metadata cache for 'fullcrawllogindex' uses directory '/home/devel/Test6/cache/fullcrawllogindex'
2022-01-25 15:03:57.612 INFO d.n.h.indexserver.FileBasedCache.<init> - Metadata cache for 'cdxdata' uses directory '/home/devel/Test6/cache/cdxdata'
2022-01-25 15:03:57.613 INFO d.n.h.indexserver.RawMetadataCache.<init> - Metadata cache for 'cdxdata' is fetching metadata with urls matching 'metadata://[^/]*/crawl/index/cdx.*' and mimetype matching 'application/x-cdx'. Migration of duplicate records is disabled
2022-01-25 15:03:57.613 DEBUG d.n.h.indexserver.FileBasedCache.cache - Waiting to enter synchronization on /home/devel/Test6/cache/cdxindex/empty-cache.working
2022-01-25 15:03:57.614 DEBUG d.n.h.indexserver.FileBasedCache.cache - locking filechannel for file '/home/devel/Test6/cache/cdxindex/empty-cache.working' (thread = main)
2022-01-25 15:03:57.624 DEBUG d.n.common.utils.ProcessUtils.runProcess - Running external program: sort /home/devel/Test6/cache/cdxindex/empty-cache -o /home/devel/Test6/cache/cdxindex/empty-cache.unsorted with environment LANG=C
2022-01-25 15:03:57.646 DEBUG d.n.h.indexserver.FileBasedCache.cache - release lock on filechannel sun.nio.ch.FileChannelImpl@b672aa8
2022-01-25 15:03:57.646 INFO d.n.h.indexserver.FileBasedCache.getIndex - Generated index '/home/devel/Test6/cache/cdxindex/empty-cache' of id '[]', request was for '[]'
2022-01-25 15:03:57.647 DEBUG d.n.h.indexserver.FileBasedCache.cache - Waiting to enter synchronization on /home/devel/Test6/cache/dedupcrawllogindex/empty-cache.working
2022-01-25 15:03:57.647 DEBUG d.n.h.indexserver.FileBasedCache.cache - locking filechannel for file '/home/devel/Test6/cache/dedupcrawllogindex/empty-cache.working' (thread = main)
2022-01-25 15:03:57.648 INFO d.n.h.indexserver.CrawlLogIndexCache.prepareCombine - Starting to generate dedupcrawllogindex for the 0 jobs: []
2022-01-25 15:03:57.648 INFO d.n.h.indexserver.CrawlLogIndexCache.combine - Starting combine task #1. This combines a dataset with 0 crawl logs (thread = main)
2022-01-25 15:03:57.780 INFO d.n.h.indexserver.CrawlLogIndexCache.combine - Adding the final 0 subindices to main index. Forcing index to contain max 42 files (related to combine task #1)
2022-01-25 15:03:57.789 INFO d.n.h.indexserver.CrawlLogIndexCache.combine - Adding operation completed (combine task #1)!
2022-01-25 15:03:57.791 INFO d.n.h.indexserver.CrawlLogIndexCache.combine - Closed index (related to combine task #1
2022-01-25 15:03:57.791 INFO d.n.h.indexserver.CrawlLogIndexCache.combine - Gzip-compressing the individual 2 index files of combine task # 1
2022-01-25 15:03:57.797 INFO d.n.h.indexserver.CrawlLogIndexCache.combine - Completed combine task #1 that combined a dataset with 0 crawl logs (entries in combined index: 0) - compressed index has size 72 bytes
2022-01-25 15:03:57.798 DEBUG d.n.h.indexserver.FileBasedCache.cache - release lock on filechannel sun.nio.ch.FileChannelImpl@7728643a
2022-01-25 15:03:57.799 INFO d.n.h.indexserver.FileBasedCache.getIndex - Generated index '/home/devel/Test6/cache/dedupcrawllogindex/empty-cache' of id '[]', request was for '[]'
2022-01-25 15:03:57.799 DEBUG d.n.h.indexserver.FileBasedCache.cache - Waiting to enter synchronization on /home/devel/Test6/cache/fullcrawllogindex/empty-cache.working
2022-01-25 15:03:57.799 DEBUG d.n.h.indexserver.FileBasedCache.cache - locking filechannel for file '/home/devel/Test6/cache/fullcrawllogindex/empty-cache.working' (thread = main)
2022-01-25 15:03:57.800 INFO d.n.h.indexserver.CrawlLogIndexCache.prepareCombine - Starting to generate fullcrawllogindex for the 0 jobs: []
2022-01-25 15:03:57.800 INFO d.n.h.indexserver.CrawlLogIndexCache.combine - Starting combine task #1. This combines a dataset with 0 crawl logs (thread = main)
2022-01-25 15:03:57.803 INFO d.n.h.indexserver.CrawlLogIndexCache.combine - Adding the final 0 subindices to main index. Forcing index to contain max 42 files (related to combine task #1)
2022-01-25 15:03:57.806 INFO d.n.h.indexserver.CrawlLogIndexCache.combine - Adding operation completed (combine task #1)!
2022-01-25 15:03:57.807 INFO d.n.h.indexserver.CrawlLogIndexCache.combine - Closed index (related to combine task #1
2022-01-25 15:03:57.807 INFO d.n.h.indexserver.CrawlLogIndexCache.combine - Gzip-compressing the individual 2 index files of combine task # 1
2022-01-25 15:03:57.809 INFO d.n.h.indexserver.CrawlLogIndexCache.combine - Completed combine task #1 that combined a dataset with 0 crawl logs (entries in combined index: 0) - compressed index has size 72 bytes
2022-01-25 15:03:57.810 DEBUG d.n.h.indexserver.FileBasedCache.cache - release lock on filechannel sun.nio.ch.FileChannelImpl@5167268
2022-01-25 15:03:57.810 INFO d.n.h.indexserver.FileBasedCache.getIndex - Generated index '/home/devel/Test6/cache/fullcrawllogindex/empty-cache' of id '[]', request was for '[]'
15:03:53,849 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [/home/devel/Test6/conf/logback_ViewerProxyApplication.xml] at [file:/home/devel/Test6/conf/logback_ViewerProxyApplication.xml]
15:03:54,200 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - debug attribute not set
15:03:54,207 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
15:03:54,214 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [STDOUT]
15:03:54,239 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
15:03:54,314 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
15:03:54,332 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [FILE]
15:03:54,369 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@1285524499 - No compression will be used
15:03:54,370 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@1285524499 - Will use the pattern ./log/ViewerProxyApplication.%d{yyyy-MM-dd}.%i.log for the active file
15:03:54,373 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@5ae50ce6 - The date pattern is 'yyyy-MM-dd' from file name pattern './log/ViewerProxyApplication.%d{yyyy-MM-dd}.%i.log'.
15:03:54,373 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@5ae50ce6 - Roll-over at midnight.
15:03:54,377 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@5ae50ce6 - Setting initial period to Tue Jan 25 15:03:54 CET 2022
15:03:54,377 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@5ae50ce6 - SizeAndTimeBasedFNATP is deprecated. Use SizeAndTimeBasedRollingPolicy instead
15:03:54,377 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@5ae50ce6 - For more information see http://logback.qos.ch/manual/appenders.html#SizeAndTimeBasedRollingPolicy
15:03:54,379 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
15:03:54,383 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - Active log file name: ./log/ViewerProxyApplication.log
15:03:54,383 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - File property is set to [./log/ViewerProxyApplication.log]
15:03:54,384 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [dk.netarkivet.monitor.logging.CachingSLF4JAppender]
15:03:55,084 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [MONITOR]
15:03:55,086 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to WARN
15:03:55,086 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[ROOT]
15:03:55,086 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [FILE] to Logger[ROOT]
15:03:55,087 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [MONITOR] to Logger[ROOT]
15:03:55,087 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [dk.netarkivet] to DEBUG
15:03:55,087 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
15:03:55,095 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@6f96c77 - Registering current configuration as safe fallback point
2022-01-25 15:03:55.100 DEBUG dk.netarkivet.common.utils.Settings.invoke0 - Value found in loaded data: SB,KB
SLF4J: A number (1) of logging calls during the initialization phase have been intercepted and are
SLF4J: now being replayed. These are subject to the filtering rules of the underlying logging system.
SLF4J: See also http://www.slf4j.org/codes.html#replay
Starting dk.netarkivet.viewerproxy.ViewerProxy
Version: 7.3-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/f0e2fec464f219bbccc5ad1a8c47bb5ab34b5f79)
2022-01-25 15:03:55.107 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Starting dk.netarkivet.viewerproxy.ViewerProxy
Version: 7.3-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/f0e2fec464f219bbccc5ad1a8c47bb5ab34b5f79)
Java VM: 1.8.0_312
2022-01-25 15:03:55.108 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Java VM: 1.8.0_312
java.home: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.312.b07-1.el7_9.x86_64/jre
2022-01-25 15:03:55.108 INFO d.n.common.utils.ApplicationUtils.logAndPrint - java.home: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.312.b07-1.el7_9.x86_64/jre
Working dir: /home/devel/Test6
2022-01-25 15:03:55.108 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Working dir: /home/devel/Test6
2022-01-25 15:03:55.109 INFO d.n.common.utils.ApplicationUtils.startApp - Using settings files '/home/devel/Test6/conf/settings_ViewerProxyApplication.xml'
2022-01-25 15:03:55.296 INFO d.n.m.d.JMSMonitorRegistryClient.register - Registering this client for monitoring every 1 minutes, using hostname 'kb-test-acs-001.kb.dk' and JMX/RMI ports 8370/8470
2022-01-25 15:03:55.297 INFO d.n.c.m.MBeanConnectorCreator.exposeJMXMBeanServer - Registered mbean server in registry on port 8370 communicating on port 8470 using password file 'conf/jmxremote.password'.
Service URL is service:jmx:rmi://kb-test-acs-001.kb.dk:8470/jndi/rmi://kb-test-acs-001.kb.dk:8370/jmxrmi
dk.netarkivet.viewerproxy.ViewerProxy Running
2022-01-25 15:03:55.299 INFO d.n.common.utils.ApplicationUtils.logAndPrint - dk.netarkivet.viewerproxy.ViewerProxy Running
2022-01-25 15:03:55.308 INFO d.n.c.distribute.JMSConnectionSunMQ.<init> - Creating instance of dk.netarkivet.common.distribute.JMSConnectionSunMQ
2022-01-25 15:03:55.314 DEBUG d.n.c.d.bitrepository.BitmagUtils.initialize - Bitrepository settings will be loaded from /home/devel/extra_configs/bitmag_config.
2022-01-25 15:03:55.314 DEBUG d.n.common.distribute.JMSConnection.initConnection - Initializing a JMS connection dk.netarkivet.common.distribute.JMSConnectionSunMQ
2022-01-25 15:03:55.315 INFO d.n.c.distribute.JMSConnectionSunMQ.getConnectionFactory - Establishing SunMQ JMS Connection to 'kb-test-adm-001.kb.dk:7676'
2022-01-25 15:03:55.328 DEBUG d.n.c.d.bitrepository.BitmagUtils.initialize - Path to Bitrepository settings is /home/devel/extra_configs/bitmag_config.
2022-01-25 15:03:55.353 INFO d.n.c.d.bitrepository.HostNameUtils.getHostName - Local hostname (provided by getCanonicalHostName): kb-test-acs-001.kb.dk
2022-01-25 15:03:57.426 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - BitmagArcRepositoryClient will timeout on each get request after 300000 milliseconds.
2022-01-25 15:03:57.426 DEBUG d.n.common.distribute.JMSConnection.setListener - Adding dk.netarkivet.archive.arcrepository.distribute.BitmagArcRepositoryClient@56bca85b as listener to Test6_COMMON_THIS_REPOS_CLIENT_130_226_228_5_VP
2022-01-25 15:03:57.440 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - BitmagArcRepositoryClient listens for replies on channel '[Queue 'Test6_COMMON_THIS_REPOS_CLIENT_130_226_228_5_VP']'
2022-01-25 15:03:57.441 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - Getting bitmag config from settings.common.arcrepositoryClient.bitrepository.settingsDir=/home/devel/extra_configs/bitmag_config
2022-01-25 15:03:57.441 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - Using 'Test6' as default collectionID
2022-01-25 15:03:57.447 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - Storing tempfiles in folder 'arcrepositoryTemp'
2022-01-25 15:03:57.454 INFO d.n.common.utils.HttpsClientBuilder.<init> - Creating Aggressive retry handler for https queries.
2022-01-25 15:03:57.560 DEBUG d.n.c.utils.BasicTwoWaySSLProvider.loadPrivateKey - X509CertificateHolder found
2022-01-25 15:03:57.562 DEBUG d.n.c.utils.BasicTwoWaySSLProvider.loadPrivateKey - PrivateKeyInfo found
2022-01-25 15:03:57.595 INFO d.n.h.indexserver.FileBasedCache.<init> - Metadata cache for 'FULL_CRAWL_LOG' uses directory '/home/devel/Test6/cache/FULL_CRAWL_LOG'
2022-01-25 15:03:57.605 INFO d.n.viewerproxy.ARCArchiveAccess.<init> - Constructed instance of ARCArchiveAccess with TryToLookupUriAsFtp: false
2022-01-25 15:03:57.630 INFO dk.netarkivet.viewerproxy.WebProxy.<init> - Starting viewerproxy jetty on port 8078
--------------------------------------------
STARTING MACHINE: devel@kb-test-way-001.kb.dk
Starting all applications on: kb-test-way-001.kb.dk
Starting linux application: WaybackIndexerApplication
Starting linux application: AggregatorApplication
15:03:59,689 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [/home/devel/Test6/conf/logback_AggregatorApplication.xml] at [file:/home/devel/Test6/conf/logback_AggregatorApplication.xml]
15:03:59,975 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - debug attribute not set
15:03:59,982 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
15:04:00,014 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [STDOUT]
15:04:00,073 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
15:04:00,241 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
15:04:00,261 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [FILE]
15:04:00,334 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@477289012 - No compression will be used
15:04:00,335 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@477289012 - Will use the pattern ./log/AggregatorApplication.%d{yyyy-MM-dd}.%i.log for the active file
15:04:00,347 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@6b0c2d26 - The date pattern is 'yyyy-MM-dd' from file name pattern './log/AggregatorApplication.%d{yyyy-MM-dd}.%i.log'.
15:04:00,347 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@6b0c2d26 - Roll-over at midnight.
15:04:00,372 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@6b0c2d26 - Setting initial period to Tue Jan 25 15:04:00 CET 2022
15:04:00,373 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@6b0c2d26 - SizeAndTimeBasedFNATP is deprecated. Use SizeAndTimeBasedRollingPolicy instead
15:04:00,373 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@6b0c2d26 - For more information see http://logback.qos.ch/manual/appenders.html#SizeAndTimeBasedRollingPolicy
15:04:00,376 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
15:04:00,384 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - Active log file name: ./log/AggregatorApplication.log
15:04:00,384 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - File property is set to [./log/AggregatorApplication.log]
15:04:00,385 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [dk.netarkivet.monitor.logging.CachingSLF4JAppender]
15:04:01,681 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [MONITOR]
15:04:01,685 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to WARN
15:04:01,690 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[ROOT]
15:04:01,691 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [FILE] to Logger[ROOT]
15:04:01,691 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [MONITOR] to Logger[ROOT]
15:04:01,691 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [dk.netarkivet] to DEBUG
15:04:01,691 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
15:04:01,702 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@3d3fcdb0 - Registering current configuration as safe fallback point
2022-01-25 15:04:01.729 DEBUG dk.netarkivet.common.utils.Settings.invoke0 - Value found in loaded data: SB,KB
SLF4J: A number (1) of logging calls during the initialization phase have been intercepted and are
SLF4J: now being replayed. These are subject to the filtering rules of the underlying logging system.
SLF4J: See also http://www.slf4j.org/codes.html#replay
Starting dk.netarkivet.wayback.aggregator.AggregationWorker
Version: 7.3-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/f0e2fec464f219bbccc5ad1a8c47bb5ab34b5f79)
2022-01-25 15:04:01.754 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Starting dk.netarkivet.wayback.aggregator.AggregationWorker
Version: 7.3-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/f0e2fec464f219bbccc5ad1a8c47bb5ab34b5f79)
Java VM: 1.8.0_25
2022-01-25 15:04:01.759 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Java VM: 1.8.0_25
java.home: /usr/local/java.old/jdk1.8.0_25_x64/jre
2022-01-25 15:04:01.761 INFO d.n.common.utils.ApplicationUtils.logAndPrint - java.home: /usr/local/java.old/jdk1.8.0_25_x64/jre
Working dir: /home/devel/Test6
2022-01-25 15:04:01.762 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Working dir: /home/devel/Test6
2022-01-25 15:04:01.763 INFO d.n.common.utils.ApplicationUtils.startApp - Using settings files '/home/devel/Test6/conf/settings_AggregatorApplication.xml'
2022-01-25 15:04:02.136 INFO d.n.m.d.JMSMonitorRegistryClient.register - Registering this client for monitoring every 1 minutes, using hostname 'kb-test-way-001.kb.dk' and JMX/RMI ports 9170/9270
2022-01-25 15:04:02.137 INFO d.n.c.m.MBeanConnectorCreator.exposeJMXMBeanServer - Registered mbean server in registry on port 9170 communicating on port 9270 using password file 'conf/jmxremote.password'.
Service URL is service:jmx:rmi://kb-test-way-001.kb.dk:9270/jndi/rmi://kb-test-way-001.kb.dk:9170/jmxrmi
dk.netarkivet.wayback.aggregator.AggregationWorker Running
2022-01-25 15:04:02.151 INFO d.n.common.utils.ApplicationUtils.logAndPrint - dk.netarkivet.wayback.aggregator.AggregationWorker Running
2022-01-25 15:04:02.180 INFO d.n.c.distribute.JMSConnectionSunMQ.<init> - Creating instance of dk.netarkivet.common.distribute.JMSConnectionSunMQ
2022-01-25 15:04:02.180 DEBUG d.n.common.distribute.JMSConnection.initConnection - Initializing a JMS connection dk.netarkivet.common.distribute.JMSConnectionSunMQ
2022-01-25 15:04:02.181 INFO d.n.c.distribute.JMSConnectionSunMQ.getConnectionFactory - Establishing SunMQ JMS Connection to 'kb-test-adm-001.kb.dk:7676'
2022-01-25 15:04:02.198 WARN d.n.w.aggregator.AggregationWorker.runAggregation - Input directory '/home/devel/Test6/batchOutputDir' was not found: skipping this aggregation
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/devel/Test6/lib/logback-classic-1.2.3.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/devel/Test6/lib/slf4j-log4j12-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
15:03:59,986 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [/home/devel/Test6/conf/logback_WaybackIndexerApplication.xml] at [file:/home/devel/Test6/conf/logback_WaybackIndexerApplication.xml]
15:04:00,575 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - debug attribute not set
15:04:00,589 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
15:04:00,608 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [STDOUT]
15:04:00,649 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
15:04:00,792 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
15:04:00,800 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [FILE]
15:04:00,905 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@1201484275 - No compression will be used
15:04:00,913 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@1201484275 - Will use the pattern ./log/WaybackIndexerApplication.%d{yyyy-MM-dd}.%i.log for the active file
15:04:00,923 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@40ef3420 - The date pattern is 'yyyy-MM-dd' from file name pattern './log/WaybackIndexerApplication.%d{yyyy-MM-dd}.%i.log'.
15:04:00,923 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@40ef3420 - Roll-over at midnight.
15:04:00,944 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@40ef3420 - Setting initial period to Tue Jan 25 15:04:00 CET 2022
15:04:00,944 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@40ef3420 - SizeAndTimeBasedFNATP is deprecated. Use SizeAndTimeBasedRollingPolicy instead
15:04:00,944 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@40ef3420 - For more information see http://logback.qos.ch/manual/appenders.html#SizeAndTimeBasedRollingPolicy
15:04:00,950 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
15:04:00,965 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - Active log file name: ./log/WaybackIndexerApplication.log
15:04:00,965 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - File property is set to [./log/WaybackIndexerApplication.log]
15:04:00,968 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [dk.netarkivet.monitor.logging.CachingSLF4JAppender]
15:04:02,393 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [MONITOR]
15:04:02,398 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to WARN
15:04:02,398 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[ROOT]
15:04:02,403 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [FILE] to Logger[ROOT]
15:04:02,404 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [MONITOR] to Logger[ROOT]
15:04:02,404 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [dk.netarkivet] to DEBUG
15:04:02,404 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
15:04:02,405 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@498d318c - Registering current configuration as safe fallback point
SLF4J: Actual binding is of type [ch.qos.logback.classic.util.ContextSelectorStaticBinder]
2022-01-25 15:04:02.418 DEBUG dk.netarkivet.common.utils.Settings.invoke0 - Value found in loaded data: SB,KB
SLF4J: A number (1) of logging calls during the initialization phase have been intercepted and are
SLF4J: now being replayed. These are subject to the filtering rules of the underlying logging system.
SLF4J: See also http://www.slf4j.org/codes.html#replay
Starting dk.netarkivet.wayback.indexer.WaybackIndexer
Version: 7.3-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/f0e2fec464f219bbccc5ad1a8c47bb5ab34b5f79)
2022-01-25 15:04:02.438 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Starting dk.netarkivet.wayback.indexer.WaybackIndexer
Version: 7.3-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/f0e2fec464f219bbccc5ad1a8c47bb5ab34b5f79)
Java VM: 1.8.0_25
2022-01-25 15:04:02.438 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Java VM: 1.8.0_25
java.home: /usr/local/java.old/jdk1.8.0_25_x64/jre
2022-01-25 15:04:02.441 INFO d.n.common.utils.ApplicationUtils.logAndPrint - java.home: /usr/local/java.old/jdk1.8.0_25_x64/jre
Working dir: /home/devel/Test6
2022-01-25 15:04:02.444 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Working dir: /home/devel/Test6
2022-01-25 15:04:02.445 INFO d.n.common.utils.ApplicationUtils.startApp - Using settings files '/home/devel/Test6/conf/settings_WaybackIndexerApplication.xml'
2022-01-25 15:04:02.634 INFO d.n.m.d.JMSMonitorRegistryClient.register - Registering this client for monitoring every 1 minutes, using hostname 'kb-test-way-001.kb.dk' and JMX/RMI ports 8171/8271
2022-01-25 15:04:02.635 INFO d.n.c.m.MBeanConnectorCreator.exposeJMXMBeanServer - Registered mbean server in registry on port 8171 communicating on port 8271 using password file 'conf/jmxremote.password'.
Service URL is service:jmx:rmi://kb-test-way-001.kb.dk:8271/jndi/rmi://kb-test-way-001.kb.dk:8171/jmxrmi
dk.netarkivet.wayback.indexer.WaybackIndexer Running
2022-01-25 15:04:02.645 INFO d.n.common.utils.ApplicationUtils.logAndPrint - dk.netarkivet.wayback.indexer.WaybackIndexer Running
2022-01-25 15:04:02.658 INFO d.n.c.distribute.JMSConnectionSunMQ.<init> - Creating instance of dk.netarkivet.common.distribute.JMSConnectionSunMQ
2022-01-25 15:04:02.664 DEBUG d.n.common.distribute.JMSConnection.initConnection - Initializing a JMS connection dk.netarkivet.common.distribute.JMSConnectionSunMQ
2022-01-25 15:04:02.667 DEBUG d.n.c.d.bitrepository.BitmagUtils.initialize - Bitrepository settings will be loaded from /home/devel/extra_configs/bitmag_config.
2022-01-25 15:04:02.667 INFO d.n.c.distribute.JMSConnectionSunMQ.getConnectionFactory - Establishing SunMQ JMS Connection to 'kb-test-adm-001.kb.dk:7676'
2022-01-25 15:04:02.678 DEBUG d.n.c.d.bitrepository.BitmagUtils.initialize - Path to Bitrepository settings is /home/devel/extra_configs/bitmag_config.
2022-01-25 15:04:02.693 INFO d.n.c.d.bitrepository.HostNameUtils.getHostName - Local hostname (provided by getCanonicalHostName): kb-test-way-001.kb.dk
--------------------------------------------
--------------------------------------------
STARTING MACHINE: netarkdv@sb-test-har-001.statsbiblioteket.dk
Starting all applications on: sb-test-har-001.statsbiblioteket.dk
Starting linux application: HarvestControllerApplication_sbhigh_h3
15:04:04,930 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [/home/netarkdv/Test6/conf/logback_HarvestControllerApplication_sbhigh_h3.xml] at [file:/home/netarkdv/Test6/conf/logback_HarvestControllerApplication_sbhigh_h3.xml]
15:04:05,071 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - debug attribute not set
15:04:05,073 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
15:04:05,079 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [STDOUT]
15:04:05,087 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
15:04:05,119 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
15:04:05,121 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [FILE]
15:04:05,135 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@251210093 - No compression will be used
15:04:05,135 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@251210093 - Will use the pattern ./log/HarvestControllerApplication_sbhigh_h3.%d{yyyy-MM-dd}.%i.log for the active file
15:04:05,137 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@36c88a32 - The date pattern is 'yyyy-MM-dd' from file name pattern './log/HarvestControllerApplication_sbhigh_h3.%d{yyyy-MM-dd}.%i.log'.
15:04:05,137 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@36c88a32 - Roll-over at midnight.
15:04:05,142 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@36c88a32 - Setting initial period to Tue Jan 25 15:04:05 CET 2022
15:04:05,143 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@36c88a32 - SizeAndTimeBasedFNATP is deprecated. Use SizeAndTimeBasedRollingPolicy instead
15:04:05,143 |-WARN in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@36c88a32 - For more information see http://logback.qos.ch/manual/appenders.html#SizeAndTimeBasedRollingPolicy
15:04:05,146 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
15:04:05,150 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - Active log file name: ./log/HarvestControllerApplication_sbhigh_h3.log
15:04:05,150 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - File property is set to [./log/HarvestControllerApplication_sbhigh_h3.log]
15:04:05,152 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [dk.netarkivet.monitor.logging.CachingSLF4JAppender]
15:04:05,448 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [MONITOR]
15:04:05,449 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to WARN
15:04:05,449 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[ROOT]
15:04:05,450 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [FILE] to Logger[ROOT]
15:04:05,450 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [MONITOR] to Logger[ROOT]
15:04:05,450 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [dk.netarkivet] to DEBUG
15:04:05,450 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
15:04:05,451 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@7880cdf3 - Registering current configuration as safe fallback point
2022-01-25 15:04:05.454 DEBUG dk.netarkivet.common.utils.Settings.invoke0 - Value found in loaded data: SB,KB
SLF4J: A number (1) of logging calls during the initialization phase have been intercepted and are
SLF4J: now being replayed. These are subject to the filtering rules of the underlying logging system.
SLF4J: See also http://www.slf4j.org/codes.html#replay
Starting dk.netarkivet.harvester.heritrix3.HarvestControllerServer
Version: 7.3-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/f0e2fec464f219bbccc5ad1a8c47bb5ab34b5f79)
2022-01-25 15:04:05.460 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Starting dk.netarkivet.harvester.heritrix3.HarvestControllerServer
Version: 7.3-SNAPSHOT (https://github.com/netarchivesuite/netarchivesuite/commit/f0e2fec464f219bbccc5ad1a8c47bb5ab34b5f79)
Java VM: 1.8.0_232
2022-01-25 15:04:05.461 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Java VM: 1.8.0_232
java.home: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.232.b09-0.el7_7.x86_64/jre
2022-01-25 15:04:05.461 INFO d.n.common.utils.ApplicationUtils.logAndPrint - java.home: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.232.b09-0.el7_7.x86_64/jre
Working dir: /home/netarkdv/Test6
2022-01-25 15:04:05.461 INFO d.n.common.utils.ApplicationUtils.logAndPrint - Working dir: /home/netarkdv/Test6
2022-01-25 15:04:05.461 INFO d.n.common.utils.ApplicationUtils.startApp - Using settings files '/home/netarkdv/Test6/conf/settings_HarvestControllerApplication_sbhigh_h3.xml'
2022-01-25 15:04:05.614 INFO d.n.m.d.JMSMonitorRegistryClient.register - Registering this client for monitoring every 1 minutes, using hostname 'sb-test-har-001.statsbiblioteket.dk' and JMX/RMI ports 8170/8270
2022-01-25 15:04:05.614 INFO d.n.c.m.MBeanConnectorCreator.exposeJMXMBeanServer - Registered mbean server in registry on port 8170 communicating on port 8270 using password file 'conf/jmxremote.password'.
Service URL is service:jmx:rmi://sb-test-har-001.statsbiblioteket.dk:8270/jndi/rmi://sb-test-har-001.statsbiblioteket.dk:8170/jmxrmi
dk.netarkivet.harvester.heritrix3.HarvestControllerServer Running
2022-01-25 15:04:05.622 INFO d.n.common.utils.ApplicationUtils.logAndPrint - dk.netarkivet.harvester.heritrix3.HarvestControllerServer Running
2022-01-25 15:04:05.624 INFO d.n.c.distribute.JMSConnectionSunMQ.<init> - Creating instance of dk.netarkivet.common.distribute.JMSConnectionSunMQ
2022-01-25 15:04:05.624 DEBUG d.n.common.distribute.JMSConnection.initConnection - Initializing a JMS connection dk.netarkivet.common.distribute.JMSConnectionSunMQ
2022-01-25 15:04:05.624 INFO d.n.c.distribute.JMSConnectionSunMQ.getConnectionFactory - Establishing SunMQ JMS Connection to 'kb-test-adm-001.kb.dk:7676'
2022-01-25 15:04:05.637 INFO d.n.h.h.HarvestControllerServer.<init> - Starting HarvestControllerServer.
2022-01-25 15:04:05.640 INFO d.n.h.h.HarvestControllerServer.<init> - Bound to harvest channel 'HIGHPRIORITY'
2022-01-25 15:04:05.645 INFO d.n.h.h.HarvestControllerServer.<init> - Serverdir: 'harvester_high'
2022-01-25 15:04:05.646 INFO d.n.h.h.HarvestControllerServer.<init> - Harvesting requires at least 400000000 bytes free.
2022-01-25 15:04:05.778 DEBUG d.n.c.d.bitrepository.BitmagUtils.initialize - Bitrepository settings will be loaded from /home/netarkdv/extra_configs/bitmag_config.
2022-01-25 15:04:05.779 DEBUG d.n.c.d.bitrepository.BitmagUtils.initialize - Path to Bitrepository settings is /home/netarkdv/extra_configs/bitmag_config.
2022-01-25 15:04:05.783 INFO d.n.c.d.bitrepository.HostNameUtils.getHostName - Local hostname (provided by getCanonicalHostName): sb-test-har-001.statsbiblioteket.dk
2022-01-25 15:04:06.864 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - BitmagArcRepositoryClient will timeout on each get request after 300000 milliseconds.
2022-01-25 15:04:06.869 DEBUG d.n.common.distribute.JMSConnection.setListener - Adding dk.netarkivet.archive.arcrepository.distribute.BitmagArcRepositoryClient@5b04476e as listener to Test6_COMMON_THIS_REPOS_CLIENT_130_225_27_140_HCS_SBHIGH_H3
2022-01-25 15:04:06.898 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - BitmagArcRepositoryClient listens for replies on channel '[Queue 'Test6_COMMON_THIS_REPOS_CLIENT_130_225_27_140_HCS_SBHIGH_H3']'
2022-01-25 15:04:06.901 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - Getting bitmag config from settings.common.arcrepositoryClient.bitrepository.settingsDir=/home/netarkdv/extra_configs/bitmag_config
2022-01-25 15:04:06.904 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - Using 'Test6' as default collectionID
2022-01-25 15:04:06.916 INFO d.n.a.a.d.BitmagArcRepositoryClient.<init> - Storing tempfiles in folder 'arcrepositoryTemp'
2022-01-25 15:04:06.922 INFO d.n.common.utils.HttpsClientBuilder.<init> - Creating Aggressive retry handler for https queries.
2022-01-25 15:04:06.957 INFO d.n.c.utils.BasicTwoWaySSLProvider.loadPrivateKey - Key file '/home/devel/extra_configs/https_config/https_key.pem' with private key and certificate does not exist!
2022-01-25 15:04:06.979 DEBUG d.n.h.h.HarvestControllerServer.<init> - Obtained JMS connection.
2022-01-25 15:04:06.983 INFO d.n.h.h.HarvestControllerServer.<init> - SEND_READY_DELAY used by HarvestControllerServer is 300
2022-01-25 15:04:06.985 INFO d.n.h.heritrix3.PostProcessing.processOldJobs - Looking for unprocessed crawldata in 'harvester_high'
2022-01-25 15:04:06.987 DEBUG d.n.common.distribute.JMSConnection.setListener - Adding dk.netarkivet.harvester.heritrix3.HarvestControllerServer@305f031 as listener to Test6_COMMON_HCHAN_VAL_RESP
2022-01-25 15:04:07.021 INFO d.n.h.h.HarvestControllerServer.<init> - Requested to check the validity of harvest channel 'HIGHPRIORITY'
2022-01-25 15:04:07.053 INFO d.n.h.h.HarvestControllerServer.visit - Received message stating that channel 'HIGHPRIORITY' is valid.
2022-01-25 15:04:07.056 INFO d.n.h.h.HarvestControllerServer.beginListeningIfSpaceAvailable - Starts to listen to new jobs on queue '[Queue 'Test6_COMMON_JOB_PARTIAL_HIGHPRIORITY']'
2022-01-25 15:04:07.056 DEBUG d.n.common.distribute.JMSConnection.setListener - Adding dk.netarkivet.harvester.heritrix3.HarvestControllerServer@305f031 as listener to Test6_COMMON_JOB_PARTIAL_HIGHPRIORITY
--------------------------------------------
Completed the script all_test.sh
Finished: SUCCESS
Console Output
Help us localize this page