Details
-
Improvement
-
Status: Resolved
-
Minor
-
Resolution: Fixed
-
None
-
None
-
BNF
Description
Add a prefix to (W)ARC metadata files
We have added the parameter settings.harvester.harvesting.metadata.metadataFileNameFormat in HarvesterSettings.java and modify code so that if this parameter is "prefix", the metadata file name will be either
collectionName + "-" + jobID + "-" + harvestID + "-metadata-" + 1 + ."arc" + possibleGzSuffix
or
collectionName + "-" + jobID + "-" + harvestID + "-metadata-" + 1 + ."warc" + possibleGzSuffix
We also modify wherever it was necessary to handle this case (in particular in test classes).
Attachments
Issue Links
- related to
-
NAS-2713 Add a prefix to (W)ARC files
-
- Resolved
-
-
NAS-2712 Indexserver includes too much in the specifiedPattern used by RawMetadataCache#cacheData(Long id)
-
- Ready for release test
-
-
NAS-2715 Make writing of outlinks in WARC metadata records optional
-
- Resolved
-
-
NAS-2716 Make overriding of writeRequests, writeMetadata, writeMetadataOutlinks, writeRevisitForIdenticalDigests and writeRevisitForNotModified optional
-
- Resolved
-