[NAS-2479] MetadataFileWriterWarc writes faulty digest headers Created: 19/Jan/16  Updated: 08/Feb/16  Resolved: 20/Jan/16

Status: Resolved
Project: NetarchiveSuite
Component/s: Common
Affects Version/s: None
Fix Version/s: 5.1

Type: Bug Priority: Minor
Reporter: Nicholas Clarke (Inactive) Assignee: Nicholas Clarke (Inactive)
Resolution: Fixed  
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Verification:
  • make a standard harvest of netarkivet.dk
  • get hold of the metadata warc file created with the harvest
  • Verify that the warc-file is valid using jwattols

 Description   

Bug introduced after converting the ARC/WARC metadata writers to JWAT from IA writers.



 Comments   
Comment by Søren Vejrup Carlsen (Inactive) [ 08/Feb/16 ]

Verified with

bash jwattools.sh test -e 1-metadata-1.warc

gave this output:

[devel@kb-prod-udv-001 jwat-tools-0.6.2]$ bash jwattools.sh test -e ../verify-nas-2479/1-metadata-1.warc 
Showing errors: true
Validate digest: true
Using 1 thread(s).
ThreadPool started.
Output Thread started.
Queued 1 file(s).
ThreadPool shut down.                                               
Output Thread stopped.
#
# Job summary
#
GZip files: 0
  +  Arc: 0
  + Warc: 0
 Arc files: 0
Warc files: 1
    Errors: 0
  Warnings: 0
RuntimeErr: 0
   Skipped: 0
      Time: 00:00:01 (1314 ms.)
TotalBytes: 1.9 mb
  AvgBytes: 1.9 mb/s

and compared with test of 9-metadata-1.warc generated with a earlier version of 5.1 giving this output:

[devel@kb-prod-udv-001 jwat-tools-0.6.2]$ bash jwattools.sh test -e ../verify-nas-2479/9-metadata-1.warc 
Showing errors: true
Validate digest: true
Using 1 thread(s).
ThreadPool started.
Output Thread started.
Queued 1 file(s).
ThreadPool shut down.                                                 
Output Thread stopped.
#
# Job summary
#
GZip files: 0
  +  Arc: 0
  + Warc: 0
 Arc files: 0
Warc files: 1
    Errors: 25
  Warnings: 0
RuntimeErr: 0
   Skipped: 0
      Time: 00:00:01 (1214 ms.)
TotalBytes: 384.5 kb
  AvgBytes: 384.5 kb/s
INVALID_EXPECTED: 25
'WARC-Block-Digest' value: 25
Comment by Søren Vejrup Carlsen (Inactive) [ 20/Jan/16 ]

Have been reviewed

Generated at Sat Apr 27 06:46:03 CEST 2024 using Jira 9.4.15#940015-sha1:bdaa9cbecfb6791ea579749728cab771f0dfe90b.