[INFO] [INFO] ------------------------------------------------------------------------ [INFO] Building NetarchiveSuite - heritrix1 extensions 5.3-SNAPSHOT [INFO] ------------------------------------------------------------------------ [INFO] [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ heritrix1-extensions --- [INFO] Deleting /home/cibuild02/workspace/NetarchiveSuite-sonar/harvester/heritrix1/heritrix1-extensions/target [INFO] [INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ heritrix1-extensions --- [INFO] Using 'UTF-8' encoding to copy filtered resources. [INFO] skip non existing resourceDirectory /home/cibuild02/workspace/NetarchiveSuite-sonar/harvester/heritrix1/heritrix1-extensions/src/main/resources [INFO] [INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ heritrix1-extensions --- [INFO] Changes detected - recompiling the module! [INFO] Compiling 10 source files to /home/cibuild02/workspace/NetarchiveSuite-sonar/harvester/heritrix1/heritrix1-extensions/target/classes [WARNING] /home/cibuild02/workspace/NetarchiveSuite-sonar/harvester/heritrix1/heritrix1-extensions/src/main/java/dk/netarkivet/harvester/tools/TwitterDecidingScope.java: Some input files use unchecked or unsafe operations. [WARNING] /home/cibuild02/workspace/NetarchiveSuite-sonar/harvester/heritrix1/heritrix1-extensions/src/main/java/dk/netarkivet/harvester/tools/TwitterDecidingScope.java: Recompile with -Xlint:unchecked for details. [INFO] [INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ heritrix1-extensions --- [INFO] Using 'UTF-8' encoding to copy filtered resources. [INFO] Copying 3 resources [INFO] [INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @ heritrix1-extensions --- [INFO] Changes detected - recompiling the module! [INFO] Compiling 7 source files to /home/cibuild02/workspace/NetarchiveSuite-sonar/harvester/heritrix1/heritrix1-extensions/target/test-classes [INFO] [INFO] --- maven-surefire-plugin:2.17:test (default-test) @ heritrix1-extensions --- [INFO] Surefire report directory: /home/cibuild02/workspace/NetarchiveSuite-sonar/harvester/heritrix1/heritrix1-extensions/target/surefire-reports ------------------------------------------------------- T E S T S ------------------------------------------------------- Running dk.netarkivet.harvester.harvesting.WARCWriterProcessorTester Feb 07, 2017 12:06:17 AM dk.netarkivet.harvester.harvesting.WARCWriterProcessor getFirstrecordBody SEVERE: Error missing metadata Feb 07, 2017 12:06:18 AM dk.netarkivet.harvester.harvesting.WARCWriterProcessor getFirstrecordBody SEVERE: Error missing metadata Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.988 sec - in dk.netarkivet.harvester.harvesting.WARCWriterProcessorTester Running dk.netarkivet.harvester.harvesting.extractor.ExtractorOAITest Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.113 sec - in dk.netarkivet.harvester.harvesting.extractor.ExtractorOAITest Running dk.netarkivet.harvester.harvesting.DomainnameQueueAssignmentPolicyTester 00:06:18.212 [main] DEBUG d.n.h.h.DomainnameQueueAssignmentPolicy - Heritrix broke getting class key candidate for 00:06:18.216 [main] INFO dk.netarkivet.common.utils.TLD - Did not found external public suffix list at '/home/cibuild02/workspace/NetarchiveSuite-sonar/harvester/heritrix1/heritrix1-extensions/conf/public_suffix_list.dat'! Reading instead the public suffixes list from embedded file 'dk/netarkivet/common/utils/public_suffix_list.dat' in common-core.jar-VERSION.jar. 00:06:18.241 [main] WARN dk.netarkivet.common.utils.TLD - Invalid tld '*.bd', ignoring 00:06:18.247 [main] WARN dk.netarkivet.common.utils.TLD - Invalid tld '*.bn', ignoring 00:06:18.252 [main] WARN dk.netarkivet.common.utils.TLD - Invalid tld '*.nom.br', ignoring 00:06:18.259 [main] WARN dk.netarkivet.common.utils.TLD - Invalid tld '*.ck', ignoring 00:06:18.259 [main] WARN dk.netarkivet.common.utils.TLD - Invalid tld '!www.ck', ignoring 00:06:18.274 [main] WARN dk.netarkivet.common.utils.TLD - Invalid tld '*.er', ignoring 00:06:18.276 [main] WARN dk.netarkivet.common.utils.TLD - Invalid tld '*.fj', ignoring 00:06:18.276 [main] WARN dk.netarkivet.common.utils.TLD - Invalid tld '*.fk', ignoring 00:06:18.283 [main] WARN dk.netarkivet.common.utils.TLD - Invalid tld '*.gu', ignoring 00:06:18.388 [main] WARN dk.netarkivet.common.utils.TLD - Invalid tld '*.jm', ignoring 00:06:18.392 [main] WARN dk.netarkivet.common.utils.TLD - Invalid tld '*.kawasaki.jp', ignoring 00:06:18.392 [main] WARN dk.netarkivet.common.utils.TLD - Invalid tld '*.kitakyushu.jp', ignoring 00:06:18.392 [main] WARN dk.netarkivet.common.utils.TLD - Invalid tld '*.kobe.jp', ignoring 00:06:18.392 [main] WARN dk.netarkivet.common.utils.TLD - Invalid tld '*.nagoya.jp', ignoring 00:06:18.393 [main] WARN dk.netarkivet.common.utils.TLD - Invalid tld '*.sapporo.jp', ignoring 00:06:18.393 [main] WARN dk.netarkivet.common.utils.TLD - Invalid tld '*.sendai.jp', ignoring 00:06:18.394 [main] WARN dk.netarkivet.common.utils.TLD - Invalid tld '*.yokohama.jp', ignoring 00:06:18.394 [main] WARN dk.netarkivet.common.utils.TLD - Invalid tld '!city.kawasaki.jp', ignoring 00:06:18.394 [main] WARN dk.netarkivet.common.utils.TLD - Invalid tld '!city.kitakyushu.jp', ignoring 00:06:18.394 [main] WARN dk.netarkivet.common.utils.TLD - Invalid tld '!city.kobe.jp', ignoring 00:06:18.395 [main] WARN dk.netarkivet.common.utils.TLD - Invalid tld '!city.nagoya.jp', ignoring 00:06:18.395 [main] WARN dk.netarkivet.common.utils.TLD - Invalid tld '!city.sapporo.jp', ignoring 00:06:18.396 [main] WARN dk.netarkivet.common.utils.TLD - Invalid tld '!city.sendai.jp', ignoring 00:06:18.396 [main] WARN dk.netarkivet.common.utils.TLD - Invalid tld '!city.yokohama.jp', ignoring 00:06:18.442 [main] WARN dk.netarkivet.common.utils.TLD - Invalid tld '*.ke', ignoring 00:06:18.443 [main] WARN dk.netarkivet.common.utils.TLD - Invalid tld '*.kh', ignoring 00:06:18.444 [main] WARN dk.netarkivet.common.utils.TLD - Invalid tld '*.kw', ignoring 00:06:18.447 [main] WARN dk.netarkivet.common.utils.TLD - Invalid tld '*.mm', ignoring 00:06:18.463 [main] WARN dk.netarkivet.common.utils.TLD - Invalid tld '*.mz', ignoring 00:06:18.463 [main] WARN dk.netarkivet.common.utils.TLD - Invalid tld '!teledata.mz', ignoring 00:06:18.482 [main] WARN dk.netarkivet.common.utils.TLD - Invalid tld '*.np', ignoring 00:06:18.483 [main] WARN dk.netarkivet.common.utils.TLD - Invalid tld '*.pg', ignoring 00:06:18.502 [main] WARN dk.netarkivet.common.utils.TLD - Invalid tld '*.sch.uk', ignoring 00:06:18.511 [main] WARN dk.netarkivet.common.utils.TLD - Invalid tld '*.ye', ignoring 00:06:18.512 [main] WARN dk.netarkivet.common.utils.TLD - Invalid tld '*.zw', ignoring 00:06:18.538 [main] WARN dk.netarkivet.common.utils.TLD - Invalid tld '*.compute.estate', ignoring 00:06:18.539 [main] WARN dk.netarkivet.common.utils.TLD - Invalid tld '*.alces.network', ignoring 00:06:18.562 [main] WARN dk.netarkivet.common.utils.TLD - Invalid tld '*.platform.sh', ignoring 00:06:18.563 [main] WARN dk.netarkivet.common.utils.TLD - Invalid tld '*.cryptonomic.net', ignoring 00:06:18.579 [main] WARN dk.netarkivet.common.utils.TLD - Invalid tld '*.api.githubcloud.com', ignoring 00:06:18.579 [main] WARN dk.netarkivet.common.utils.TLD - Invalid tld '*.ext.githubcloud.com', ignoring 00:06:18.580 [main] WARN dk.netarkivet.common.utils.TLD - Invalid tld '*.githubcloudusercontent.com', ignoring 00:06:18.580 [main] WARN dk.netarkivet.common.utils.TLD - Invalid tld '*.0emm.com', ignoring 00:06:18.581 [main] WARN dk.netarkivet.common.utils.TLD - Invalid tld '*.magentosite.cloud', ignoring The list of loaded data settings is empty. Is this OK?00:06:18.598 [main] DEBUG dk.netarkivet.common.utils.TLD - No tlds found in settingsfiles 00:06:18.599 [main] INFO dk.netarkivet.common.utils.TLD - Did not found external public suffix list at '/home/cibuild02/workspace/NetarchiveSuite-sonar/harvester/heritrix1/heritrix1-extensions/conf/public_suffix_list.dat'! Reading instead the public suffixes list from embedded file 'dk/netarkivet/common/utils/public_suffix_list.dat' in common-core.jar-VERSION.jar. 00:06:18.608 [main] WARN dk.netarkivet.common.utils.TLD - Invalid tld '*.bd', ignoring 00:06:18.608 [main] WARN dk.netarkivet.common.utils.TLD - Invalid tld '*.bn', ignoring 00:06:18.609 [main] WARN dk.netarkivet.common.utils.TLD - Invalid tld '*.nom.br', ignoring 00:06:18.610 [main] WARN dk.netarkivet.common.utils.TLD - Invalid tld '*.ck', ignoring 00:06:18.610 [main] WARN dk.netarkivet.common.utils.TLD - Invalid tld '!www.ck', ignoring 00:06:18.612 [main] WARN dk.netarkivet.common.utils.TLD - Invalid tld '*.er', ignoring 00:06:18.614 [main] WARN dk.netarkivet.common.utils.TLD - Invalid tld '*.fj', ignoring 00:06:18.615 [main] WARN dk.netarkivet.common.utils.TLD - Invalid tld '*.fk', ignoring 00:06:18.615 [main] WARN dk.netarkivet.common.utils.TLD - Invalid tld '*.gu', ignoring 00:06:18.621 [main] WARN dk.netarkivet.common.utils.TLD - Invalid tld '*.jm', ignoring 00:06:18.621 [main] WARN dk.netarkivet.common.utils.TLD - Invalid tld '*.kawasaki.jp', ignoring 00:06:18.622 [main] WARN dk.netarkivet.common.utils.TLD - Invalid tld '*.kitakyushu.jp', ignoring 00:06:18.622 [main] WARN dk.netarkivet.common.utils.TLD - Invalid tld '*.kobe.jp', ignoring 00:06:18.622 [main] WARN dk.netarkivet.common.utils.TLD - Invalid tld '*.nagoya.jp', ignoring 00:06:18.622 [main] WARN dk.netarkivet.common.utils.TLD - Invalid tld '*.sapporo.jp', ignoring 00:06:18.623 [main] WARN dk.netarkivet.common.utils.TLD - Invalid tld '*.sendai.jp', ignoring 00:06:18.624 [main] WARN dk.netarkivet.common.utils.TLD - Invalid tld '*.yokohama.jp', ignoring 00:06:18.624 [main] WARN dk.netarkivet.common.utils.TLD - Invalid tld '!city.kawasaki.jp', ignoring 00:06:18.624 [main] WARN dk.netarkivet.common.utils.TLD - Invalid tld '!city.kitakyushu.jp', ignoring 00:06:18.625 [main] WARN dk.netarkivet.common.utils.TLD - Invalid tld '!city.kobe.jp', ignoring 00:06:18.625 [main] WARN dk.netarkivet.common.utils.TLD - Invalid tld '!city.nagoya.jp', ignoring 00:06:18.625 [main] WARN dk.netarkivet.common.utils.TLD - Invalid tld '!city.sapporo.jp', ignoring 00:06:18.626 [main] WARN dk.netarkivet.common.utils.TLD - Invalid tld '!city.sendai.jp', ignoring 00:06:18.626 [main] WARN dk.netarkivet.common.utils.TLD - Invalid tld '!city.yokohama.jp', ignoring 00:06:18.640 [main] WARN dk.netarkivet.common.utils.TLD - Invalid tld '*.ke', ignoring 00:06:18.640 [main] WARN dk.netarkivet.common.utils.TLD - Invalid tld '*.kh', ignoring 00:06:18.641 [main] WARN dk.netarkivet.common.utils.TLD - Invalid tld '*.kw', ignoring 00:06:18.642 [main] WARN dk.netarkivet.common.utils.TLD - Invalid tld '*.mm', ignoring 00:06:18.648 [main] WARN dk.netarkivet.common.utils.TLD - Invalid tld '*.mz', ignoring 00:06:18.648 [main] WARN dk.netarkivet.common.utils.TLD - Invalid tld '!teledata.mz', ignoring 00:06:18.653 [main] WARN dk.netarkivet.common.utils.TLD - Invalid tld '*.np', ignoring 00:06:18.655 [main] WARN dk.netarkivet.common.utils.TLD - Invalid tld '*.pg', ignoring 00:06:18.660 [main] WARN dk.netarkivet.common.utils.TLD - Invalid tld '*.sch.uk', ignoring 00:06:18.663 [main] WARN dk.netarkivet.common.utils.TLD - Invalid tld '*.ye', ignoring 00:06:18.663 [main] WARN dk.netarkivet.common.utils.TLD - Invalid tld '*.zw', ignoring 00:06:18.673 [main] WARN dk.netarkivet.common.utils.TLD - Invalid tld '*.compute.estate', ignoring 00:06:18.674 [main] WARN dk.netarkivet.common.utils.TLD - Invalid tld '*.alces.network', ignoring 00:06:18.675 [main] WARN dk.netarkivet.common.utils.TLD - Invalid tld '*.platform.sh', ignoring 00:06:18.676 [main] WARN dk.netarkivet.common.utils.TLD - Invalid tld '*.cryptonomic.net', ignoring 00:06:18.679 [main] WARN dk.netarkivet.common.utils.TLD - Invalid tld '*.api.githubcloud.com', ignoring 00:06:18.679 [main] WARN dk.netarkivet.common.utils.TLD - Invalid tld '*.ext.githubcloud.com', ignoring 00:06:18.679 [main] WARN dk.netarkivet.common.utils.TLD - Invalid tld '*.githubcloudusercontent.com', ignoring 00:06:18.681 [main] WARN dk.netarkivet.common.utils.TLD - Invalid tld '*.0emm.com', ignoring 00:06:18.682 [main] WARN dk.netarkivet.common.utils.TLD - Invalid tld '*.magentosite.cloud', ignoring The list of loaded data settings is empty. Is this OK?00:06:18.688 [main] DEBUG dk.netarkivet.common.utils.TLD - No tlds found in settingsfiles 00:06:18.717 [main] DEBUG d.n.h.h.DomainnameQueueAssignmentPolicy - Illegal class key candidate 'default...' for '' 00:06:18.717 [main] DEBUG d.n.h.h.DomainnameQueueAssignmentPolicy - Heritrix broke getting class key candidate for 00:06:18.718 [main] DEBUG d.n.h.h.DomainnameQueueAssignmentPolicy - Illegal class key candidate 'default...' for '' 00:06:18.724 [main] DEBUG d.n.h.h.DomainnameQueueAssignmentPolicy - Heritrix broke getting class key candidate for null 00:06:18.725 [main] DEBUG d.n.h.h.DomainnameQueueAssignmentPolicy - Illegal class key candidate 'default...' for 'null' 00:06:18.726 [main] DEBUG d.n.h.h.DomainnameQueueAssignmentPolicy - Heritrix broke getting class key candidate for 00:06:18.726 [main] DEBUG d.n.h.h.DomainnameQueueAssignmentPolicy - Illegal class key candidate 'default...' for '' 00:06:18.734 [main] DEBUG d.n.h.h.DomainnameQueueAssignmentPolicy - Illegal class key candidate 'x.fnord.barbar' for 'http://x.fnord.barbar/' 00:06:18.735 [main] DEBUG d.n.h.h.DomainnameQueueAssignmentPolicy - Illegal class key candidate 'default...' for 'about:blank' Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.53 sec - in dk.netarkivet.harvester.harvesting.DomainnameQueueAssignmentPolicyTester Running dk.netarkivet.harvester.harvesting.PersistentJobDataTester harvestInfo.xml harvestInfo-0.4.xml 00:06:18.980 [main] DEBUG d.n.h.datamodel.H1HeritrixTemplate - WARC format selected to be used by Heritrix 00:06:19.100 [main] WARN d.n.h.datamodel.H1HeritrixTemplate - No attribute insertion is done for H1 templates 00:06:19.101 [main] DEBUG d.n.h.datamodel.H1HeritrixTemplate - In H1 templates we don't enable/disable deduplication. 00:06:19.111 [main] WARN d.netarkivet.harvester.datamodel.Job - HarvestnamePrefix not yet set for job 1. Set it by using the naming scheme. This should only happen for old jobs being read 00:06:19.113 [main] DEBUG d.netarkivet.harvester.datamodel.Job - Applying the default ArchiveFileNaming class 'dk.netarkivet.harvester.harvesting.LegacyNamingConvention'. 00:06:19.113 [main] DEBUG d.netarkivet.harvester.datamodel.Job - The harvestPrefix of this job is: 1-1 Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.388 sec - in dk.netarkivet.harvester.harvesting.PersistentJobDataTester Running dk.netarkivet.harvester.harvesting.OnNSDomainsDecideRuleTester Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.012 sec - in dk.netarkivet.harvester.harvesting.OnNSDomainsDecideRuleTester Results : Tests run: 16, Failures: 0, Errors: 0, Skipped: 0 [JENKINS] Recording test results [INFO] [INFO] --- maven-jgit-buildnumber-plugin:1.2.9:extract-buildnumber (git-buildnumber) @ heritrix1-extensions --- [INFO] [INFO] --- maven-jar-plugin:2.4:jar (default-jar) @ heritrix1-extensions --- [INFO] Building jar: /home/cibuild02/workspace/NetarchiveSuite-sonar/harvester/heritrix1/heritrix1-extensions/target/heritrix1-extensions-5.3-SNAPSHOT.jar [INFO] [INFO] >>> maven-source-plugin:2.2.1:jar (attach-sources) @ heritrix1-extensions >>> [WARNING] Failed to getClass for org.apache.maven.plugin.source.SourceJarMojo [INFO] [INFO] <<< maven-source-plugin:2.2.1:jar (attach-sources) @ heritrix1-extensions <<< [INFO] [INFO] --- maven-source-plugin:2.2.1:jar (attach-sources) @ heritrix1-extensions --- [INFO] Building jar: /home/cibuild02/workspace/NetarchiveSuite-sonar/harvester/heritrix1/heritrix1-extensions/target/heritrix1-extensions-5.3-SNAPSHOT-sources.jar [INFO] [INFO] --- maven-javadoc-plugin:2.10.3:jar (attach-javadocs) @ heritrix1-extensions --- [INFO] Building jar: /home/cibuild02/workspace/NetarchiveSuite-sonar/harvester/heritrix1/heritrix1-extensions/target/heritrix1-extensions-5.3-SNAPSHOT-javadoc.jar [INFO] [INFO] --- maven-install-plugin:2.4:install (default-install) @ heritrix1-extensions --- [INFO] Installing /home/cibuild02/workspace/NetarchiveSuite-sonar/harvester/heritrix1/heritrix1-extensions/target/heritrix1-extensions-5.3-SNAPSHOT.jar to /home/cibuild02/.m2/repository/org/netarchivesuite/heritrix1-extensions/5.3-SNAPSHOT/heritrix1-extensions-5.3-SNAPSHOT.jar [INFO] Installing /home/cibuild02/workspace/NetarchiveSuite-sonar/harvester/heritrix1/heritrix1-extensions/pom.xml to /home/cibuild02/.m2/repository/org/netarchivesuite/heritrix1-extensions/5.3-SNAPSHOT/heritrix1-extensions-5.3-SNAPSHOT.pom [INFO] Installing /home/cibuild02/workspace/NetarchiveSuite-sonar/harvester/heritrix1/heritrix1-extensions/target/heritrix1-extensions-5.3-SNAPSHOT-sources.jar to /home/cibuild02/.m2/repository/org/netarchivesuite/heritrix1-extensions/5.3-SNAPSHOT/heritrix1-extensions-5.3-SNAPSHOT-sources.jar [INFO] Installing /home/cibuild02/workspace/NetarchiveSuite-sonar/harvester/heritrix1/heritrix1-extensions/target/heritrix1-extensions-5.3-SNAPSHOT-javadoc.jar to /home/cibuild02/.m2/repository/org/netarchivesuite/heritrix1-extensions/5.3-SNAPSHOT/heritrix1-extensions-5.3-SNAPSHOT-javadoc.jar [JENKINS] Archiving disabled