public abstract class MetadataFileWriter extends Object
Modifier and Type | Field and Description |
---|---|
protected static String |
CDX_URI_SCHEME
Constants used in constructing URI for CDX content.
|
static int |
MDF_ARC
Constant representing the ARC format.
|
static int |
MDF_WARC
Constant representing the WARC format.
|
protected static int |
metadataFormat
Constant representing the metadata Format.
|
Constructor and Description |
---|
MetadataFileWriter() |
Modifier and Type | Method and Description |
---|---|
abstract void |
close()
Close the metadatafile Writer.
|
static MetadataFileWriter |
createWriter(File metadataArchiveFile)
Create a writer that writes data to the given archive file.
|
static URI |
getAlternateCDXURI(long jobID,
String filename)
Generates a URI identifying CDX info for one harvested ARC file.
|
static URI |
getCDXURI(String harvestID,
String jobID,
String filename)
Generates a URI identifying CDX info for one harvested (W)ARC file.
|
abstract File |
getFile() |
static String |
getMetadataArchiveFileName(String jobID)
Generates a name for an archive(ARC/WARC) file containing metadata regarding a given job.
|
protected static void |
initializeMetadataFormat()
Initialize the used metadata format from settings.
|
void |
insertFiles(File parentDir,
FilenameFilter filter,
String mimetype,
long harvestId,
long jobId)
Append the files contained in the directory to the metadata archive file, but only if the filename matches the
supplied filter.
|
static void |
resetMetadataFormat()
Reset the metadata format.
|
abstract void |
write(String uri,
String contentType,
String hostIP,
long fetchBeginTimeStamp,
byte[] payload)
Write a record to the archive file.
|
abstract void |
writeFileTo(File file,
String uri,
String mime)
Write the given file to the metadata file.
|
abstract boolean |
writeTo(File fileToArchive,
String URL,
String mimetype)
Writes a File to an ARCWriter, if available, otherwise logs the failure to the class-logger.
|
public static final int MDF_ARC
public static final int MDF_WARC
protected static int metadataFormat
protected static final String CDX_URI_SCHEME
public MetadataFileWriter()
protected static void initializeMetadataFormat()
public static String getMetadataArchiveFileName(String jobID) throws ArgumentNotValid
jobID
- The number of the job that generated the archive file.ArgumentNotValid
- if any parameter was null.public static MetadataFileWriter createWriter(File metadataArchiveFile)
metadataArchiveFile
- The archive file to write to.public abstract void close()
public abstract void writeFileTo(File file, String uri, String mime)
file
- A given file with metadata to write to the metadata archive file.uri
- The uri associated with the piece of metadatamime
- The mimetype associated with the piece of metadatapublic abstract boolean writeTo(File fileToArchive, String URL, String mimetype)
fileToArchive
- the File to archiveURL
- the URL with which it is stored in the arcfilemimetype
- The mimetype of the File-contentspublic abstract void write(String uri, String contentType, String hostIP, long fetchBeginTimeStamp, byte[] payload) throws IOException
uri
- record URIcontentType
- content-type of recordhostIP
- resource ip-addressfetchBeginTimeStamp
- record datetimepayload
- A byte array containing the payloadIOException
ARCWriter.write(String uri, String contentType, String hostIP, long fetchBeginTimeStamp,
long recordLength, InputStream in)
public void insertFiles(File parentDir, FilenameFilter filter, String mimetype, long harvestId, long jobId)
parentDir
- directory containing the files to append to metadatafilter
- filter describing which files to accept and which to ignoremimetype
- The content-type to write along with the files in the metadata outputharvestId
- The harvestId of the harvestjobId
- The jobId of the harvestpublic static void resetMetadataFormat()
public static URI getCDXURI(String harvestID, String jobID, String filename) throws ArgumentNotValid, UnknownID
harvestID
- The number of the harvest that generated the (W)ARC file.jobID
- The number of the job that generated the (W)ARC file.filename
- The name of the ARC or WARC file behind the cdx-dataArgumentNotValid
- if any parameter is null.UnknownID
- if something goes terribly wrong in our URI construction.public static URI getAlternateCDXURI(long jobID, String filename) throws ArgumentNotValid, UnknownID
jobID
- The number of the job that generated the ARC file.filename
- the filename.ArgumentNotValid
- if any parameter is null.UnknownID
- if something goes terribly wrong in our URI construction.Copyright © 2005–2015 The Royal Danish Library, the Danish State and University Library, the National Library of France and the Austrian National Library.. All rights reserved.