Package dk.netarkivet.harvester.harvesting.metadata

Class MetadataFileWriterWarc

java.lang.Object

dk.netarkivet.harvester.harvesting.metadata.MetadataFileWriter

dk.netarkivet.harvester.harvesting.metadata.MetadataFileWriterWarc

public class MetadataFileWriterWarc
extends MetadataFileWriter

MetadataFileWriter that writes to WARC files.

Field Summary

Fields inherited from class dk.netarkivet.harvester.harvesting.metadata.MetadataFileWriter

CDX_URI_SCHEME, MDF_ARC, MDF_WARC, metadataFormat

Constructor Summary

Constructors

Constructor	Description
MetadataFileWriterWarc()

Method Summary

Modifier and Type	Method	Description
void	close()	Close the metadatafile Writer.
static MetadataFileWriter	createWriter(java.io.File metadataWarcFile)	Create a MetadataFileWriter for WARC output.
java.io.File	getFile()
void	insertInfoRecord(org.jwat.common.ANVLRecord payloadToInfoRecord)	Insert a warcInfoRecord in the WARC-file, if it doesn't already exists.
protected void	open()
void	write(java.lang.String uri, java.lang.String contentType, java.lang.String hostIP, long fetchBeginTimeStamp, byte[] payload)	Write a record to the archive file.
void	writeFileTo(java.io.File file, java.lang.String uri, java.lang.String mime)	Write the given file to the metadata file.
boolean	writeTo(java.io.File fileToArchive, java.lang.String URL, java.lang.String mimetype)	Writes a File to an ARCWriter, if available, otherwise logs the failure to the class-logger.

Methods inherited from class dk.netarkivet.harvester.harvesting.metadata.MetadataFileWriter

compressRecords, getAlternateCDXURI, getCDXURI, getMetadataArchiveFileName, initializeMetadataFormat, insertFiles, resetMetadataFormat

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

MetadataFileWriterWarc

public MetadataFileWriterWarc()

Method Detail

createWriter

public static MetadataFileWriter createWriter(java.io.File metadataWarcFile)

Create a MetadataFileWriter for WARC output.

Parameters:

metadataWarcFile - The WARC output file

Returns:

MetadataFileWriter for writing metadata files in WARC

open

protected void open()

close

public void close()

Description copied from class: MetadataFileWriter

Close the metadatafile Writer.

Specified by:

close in class MetadataFileWriter

getFile

public java.io.File getFile()

Specified by:

getFile in class MetadataFileWriter

Returns:

the finished metadataFile

insertInfoRecord

public void insertInfoRecord(org.jwat.common.ANVLRecord payloadToInfoRecord)

Insert a warcInfoRecord in the WARC-file, if it doesn't already exists. saves the recordID of the written info-record for future reference to be used for later in the

Parameters:

payloadToInfoRecord - the given payload for this record.

writeFileTo

public void writeFileTo(java.io.File file,
                        java.lang.String uri,
                        java.lang.String mime)

Description copied from class: MetadataFileWriter

Write the given file to the metadata file.

Specified by:

writeFileTo in class MetadataFileWriter

Parameters:

file - A given file with metadata to write to the metadata archive file.

uri - The uri associated with the piece of metadata

mime - The mimetype associated with the piece of metadata

writeTo

public boolean writeTo(java.io.File fileToArchive,
                       java.lang.String URL,
                       java.lang.String mimetype)

Description copied from class: MetadataFileWriter

Writes a File to an ARCWriter, if available, otherwise logs the failure to the class-logger.

Specified by:

writeTo in class MetadataFileWriter

Parameters:

fileToArchive - the File to archive

URL - the URL with which it is stored in the arcfile

mimetype - The mimetype of the File-contents

Returns:

true, if file exists, and is written to the arcfile.

write

public void write(java.lang.String uri,
                  java.lang.String contentType,
                  java.lang.String hostIP,
                  long fetchBeginTimeStamp,
                  byte[] payload)
           throws java.io.IOException

Description copied from class: MetadataFileWriter

Write a record to the archive file.

Specified by:

write in class MetadataFileWriter

Parameters:

uri - record URI

contentType - content-type of record

hostIP - resource ip-address

fetchBeginTimeStamp - record datetime

payload - A byte array containing the payload

Throws:

java.io.IOException

See Also:

ARCWriter.write(String uri, String contentType, String hostIP, long fetchBeginTimeStamp, long recordLength, InputStream in)