Uploaded image for project: 'NetarchiveSuite'
  1. NetarchiveSuite
  2. NAS-2847

Upload Failure to Bitmagasin is treated too lightly

    XMLWordPrintable

Details

    • Bug
    • Resolution: Unresolved
    • Major
    • None
    • None
    • Bitpreservation

    Description

      Despite all this, the job is still marked as completed normally in NAS.

      14:54:35.004 INFO  d.n.h.heritrix3.PostProcessing.uploadFiles - File '3-1-20181219133113616-00000-sb-test-har-001.statsbiblioteket.dk.warc.gz' uploaded successfully to the arcrepository.
      14:54:35.005 INFO  d.n.h.heritrix3.PostProcessing.storeFiles - Beginning upload of the 1 metadafile(s) 
      14:54:35.009 INFO  d.n.h.heritrix3.PostProcessing.uploadFiles - Uploading file #1 - '3-metadata-1.warc.gz' to arcrepository.
      14:54:35.122 DEBUG d.n.c.d.a.b.Bitrepository.debug - Received event: 537147c6: GET_FILE_IDS for file 3-metadata-1.warc.gz: IDENTIFY_REQUEST_SENT: , Identifying contributers for get fileIDs
      14:54:35.916 DEBUG d.n.c.d.a.b.Bitrepository.debug - Received event: 537147c6: GET_FILE_IDS for file 3-metadata-1.warc.gz: COMPONENT_FAILED: ContributorID netarkiv-checksum1 ResponseCode: FILE_NOT_FOUND_FAILURE, File not found.
      14:54:36.019 DEBUG d.n.c.d.a.b.Bitrepository.debug - Received event: 537147c6: GET_FILE_IDS for file 3-metadata-1.warc.gz: COMPONENT_FAILED: ContributorID netarkiv-online1 ResponseCode: FILE_NOT_FOUND_FAILURE, File not found.
      14:54:38.056 WARN  d.n.c.d.a.b.Bitrepository.putTheFile - The putFile Operation was not a complete success (Putting the file 'harvester_high/3_1545226255421/metadata/3-metadata-1.warc.gz' with the file id '3-metadata-1.warc.gz' from Netarchivesuite). Checksum whether we accept anyway.
      org.bitrepository.common.exceptions.OperationFailedException: Failed operation. Cause(s):
      [85de09ea: PUT_FILE for file 3-metadata-1.warc.gz: COMPONENT_FAILED: ContributorID netarkiv-online1 ResponseCode: FAILURE, java.lang.IllegalStateException: Could not move the file '/home/netarkdv/bitmag/bitrepository-quickstart/var/file1pillar/files/TEST0/tmpDir/3-metadata-1.warc.gz' to the location '/home/netarkdv/bitmag/bitrepository-quickstart/var/file1pillar/files/TEST0/fileDir/3-metadata-1.warc.gz']
              at org.bitrepository.modify.putfile.BlockingPutFileClient.putFile(BlockingPutFileClient.java:73)
              at dk.netarkivet.common.distribute.arcrepository.bitrepository.Bitrepository.putTheFile(Bitrepository.java:239)
              at dk.netarkivet.common.distribute.arcrepository.bitrepository.Bitrepository.uploadFile(Bitrepository.java:199)
              at dk.netarkivet.archive.arcrepository.distribute.JMSBitmagArcRepositoryClient.store(JMSBitmagArcRepositoryClient.java:273)
              at dk.netarkivet.harvester.heritrix3.PostProcessing.uploadFiles(PostProcessing.java:315)
              at dk.netarkivet.harvester.heritrix3.PostProcessing.storeFiles(PostProcessing.java:287)
              at dk.netarkivet.harvester.heritrix3.PostProcessing.doPostProcessing(PostProcessing.java:161)
              at dk.netarkivet.harvester.heritrix3.HarvestControllerServer$HarvesterThread.run(HarvestControllerServer.java:490)
      14:54:38.113 INFO  d.n.c.d.a.b.Bitrepository.uploadFile - File '/home/netarkdv/TEST0/harvester_high/3_1545226255421/metadata/3-metadata-1.warc.gz' uploaded successfully. 
      14:54:38.115 INFO  d.n.a.a.d.JMSBitmagArcRepositoryClient.store - Upload to collection 'TEST0' of file '3-metadata-1.warc.gz' was successful
      14:54:38.115 INFO  d.n.h.heritrix3.PostProcessing.uploadFiles - File '3-metadata-1.warc.gz' uploaded successfully to the arcrepository.
      14:54:38.126 INFO  d.n.h.h.r.HarvestReportGenerator.preProcess - Starting pre-processing of harvest report for job 3
      14:54:38.129 INFO  d.n.h.h.r.HarvestReportGenerator.parseCrawlLog - DISREGARD_SEEDURL_INFORMATION_IN_CRAWLLOG: false
      

      Attachments

        Activity

          People

            Unassigned Unassigned
            csr Colin Rosenthal
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated: