Serialized Form


Package dk.netarkivet.archive.arcrepository.bitpreservation

Class dk.netarkivet.archive.arcrepository.bitpreservation.AdminDataMessage extends ArchiveMessage implements Serializable

Serialized Fields

fileName

java.lang.String fileName
The filename to be updated in AdminData.


bitarchiveName

java.lang.String bitarchiveName
The name of the bitarchive, where the file resides.


newvalue

BitArchiveStoreState newvalue
the new storestate for the filename. Used only when changestorestate is true.


checksum

java.lang.String checksum
the new checksum for the filename. Used only when changechecksum is true.


changestorestate

boolean changestorestate
change storestate flag. default = false.


changechecksum

boolean changechecksum
change checksum flag. default = false.

Class dk.netarkivet.archive.arcrepository.bitpreservation.ChecksumJob extends FileBatchJob implements Serializable

Serialization Methods

readObject

private void readObject(java.io.ObjectInputStream s)
Invoke default method for deserializing object, and reinitialise the logger.


writeObject

private void writeObject(java.io.ObjectOutputStream s)
Invoke default method for serializing object.

Class dk.netarkivet.archive.arcrepository.bitpreservation.FileListJob extends FileBatchJob implements Serializable

Serialization Methods

readObject

private void readObject(java.io.ObjectInputStream s)
Invoke default method for deserializing object, and reinitialise the logger.


Package dk.netarkivet.archive.arcrepository.distribute

Class dk.netarkivet.archive.arcrepository.distribute.StoreMessage extends ArchiveMessage implements Serializable

Serialized Fields

theRemoteFile

RemoteFile theRemoteFile
The actual data.


Package dk.netarkivet.archive.bitarchive.distribute

Class dk.netarkivet.archive.bitarchive.distribute.BatchEndedMessage extends ArchiveMessage implements Serializable

Serialized Fields

BA_ApplicationId

java.lang.String BA_ApplicationId
the identifier for BA application, that performed the batch-job.


originatingBatchMsgId

java.lang.String originatingBatchMsgId
The identifier for the message, that initiated the batch-job.


noOfFilesProcessed

int noOfFilesProcessed
Number of files processed by the batch-job.


filesFailed

java.util.Collection<E> filesFailed
Collection of files that the batch-job could not process.


rf

RemoteFile rf
The resultfile of the batchJob.

Class dk.netarkivet.archive.bitarchive.distribute.BatchMessage extends ArchiveMessage implements Serializable

Serialized Fields

job

FileBatchJob job
The batch job, this message is sent to initiate.


locationName

java.lang.String locationName
The name of this location.

Class dk.netarkivet.archive.bitarchive.distribute.BatchReplyMessage extends ArchiveMessage implements Serializable

Serialized Fields

noOfFilesProcessed

int noOfFilesProcessed
Number of files processed by the BatchJob.


filesFailed

java.util.HashSet<E> filesFailed
Set of files that the BatchJob could not process.


resultFile

RemoteFile resultFile
The result of the BatchJob.

Class dk.netarkivet.archive.bitarchive.distribute.GetFileMessage extends ArchiveMessage implements Serializable

Serialized Fields

arcfileName

java.lang.String arcfileName
the file to retrieve.


remoteFile

RemoteFile remoteFile
The actual data.


locationName

java.lang.String locationName
This location.

Class dk.netarkivet.archive.bitarchive.distribute.GetMessage extends ArchiveMessage implements Serializable

Serialized Fields

arcfile

java.lang.String arcfile
the arcfile to retrieve an record from.


index

long index
offset of the record to retrieve.


record

BitarchiveRecord record
the retrieved record.

Class dk.netarkivet.archive.bitarchive.distribute.HeartBeatMessage extends ArchiveMessage implements Serializable

Serialized Fields

timestamp

long timestamp
time when heartbeat occurred. Note that timestamps cannot be compared between processes.


applicationId

java.lang.String applicationId
id of the application sending the heartbeat.

Class dk.netarkivet.archive.bitarchive.distribute.RemoveAndGetFileMessage extends ArchiveMessage implements Serializable

Serialized Fields

arcfileName

java.lang.String arcfileName
The file to retrieve.


remoteFile

RemoteFile remoteFile
The actual data.


locationName

java.lang.String locationName
This location.


checksum

java.lang.String checksum
The checksum of the file to remove.


credentials

java.lang.String credentials
The bitarchive credentials.

Class dk.netarkivet.archive.bitarchive.distribute.UploadMessage extends ArchiveMessage implements Serializable

Serialized Fields

arcfileName

java.lang.String arcfileName
the name of the file to upload.


theRemoteFile

RemoteFile theRemoteFile
The actual data.


Package dk.netarkivet.archive.distribute

Class dk.netarkivet.archive.distribute.ArchiveMessage extends NetarkivetMessage implements Serializable


Package dk.netarkivet.archive.indexserver.distribute

Class dk.netarkivet.archive.indexserver.distribute.IndexRequestMessage extends ArchiveMessage implements Serializable

Serialization Methods

readObject

private void readObject(java.io.ObjectInputStream s)
Invoke default method for deserializing object, and reinitialise the logger.


writeObject

private void writeObject(java.io.ObjectOutputStream s)
Invoke default method for serializing object.

Serialized Fields

requestedJobs

java.util.Set<E> requestedJobs
List of jobs for which an index is requested. Should always be set.


requestType

RequestType requestType
Type of index is requested. Should always be set.


foundJobs

java.util.Set<E> foundJobs
List of jobs for which an index _can_ be generated. Should only be set on reply. Should always be a subset of requestedJobs. If This set is equal to the requested set, resultFile should also be set.


resultFiles

java.util.List<E> resultFiles
The list of files that make up the generated index. Should only be set on reply, and only if index was generated for all files if indexIsStoredInDirectory is false, this list must contain exactly one file (or not have been set yet).


indexIsStoredInDirectory

boolean indexIsStoredInDirectory
If true, the underlying cache uses a directory to store its files (which may be zero or more files), otherwise just a single file is used.


Package dk.netarkivet.common.distribute

Class dk.netarkivet.common.distribute.AbstractRemoteFile extends java.lang.Object implements Serializable

Serialized Fields

file

java.io.File file
The file this is remote file for


useChecksums

boolean useChecksums
If true, communication is checksummed.


fileDeletable

boolean fileDeletable
If true, the file may be deleted after all transfers are done.


multipleDownloads

boolean multipleDownloads
If true, the file may be downloaded multple times. Otherwise, the remote file is invalidated after first transfer.


filesize

long filesize
The size of the file.

Class dk.netarkivet.common.distribute.ChannelID extends java.lang.Object implements Serializable

Serialization Methods

readObject

private void readObject(java.io.ObjectInputStream s)
Method used by Java serialization. Our coding guidelines prescribes that this method should always be implemented, even if it only calls the default method: http://asterix.kb.dk/twiki/bin/view/Netarkiv/ImplementeringOgTestAfSerializable See also "Effective Java", pages 219 and 224.

Throws:
IOFailure - if Java could not deserialize the object.

writeObject

private void writeObject(java.io.ObjectOutputStream s)
Method used by Java serialization. Our coding guidelines prescribes that this method should always be implemented, even if it only calls the default method: http://asterix.kb.dk/twiki/bin/view/Netarkiv/ImplementeringOgTestAfSerializable See also "Effective Java", pages 219 and 224.

Throws:
IOFailure - if Java could not serialize the object.
Serialized Fields

name

java.lang.String name
A ChannelID is identified by its name. It has one bit of state information: is it a queue or a topic?


isTopic

boolean isTopic

Class dk.netarkivet.common.distribute.FTPRemoteFile extends AbstractRemoteFile implements Serializable

Serialized Fields

ftpServerName

java.lang.String ftpServerName
Ftp-connection information. Read ftp-related settings from settings.xml. Notice that these settings get transferred to the receiver, which is necessary to allow the receiver to get data from different servers.


ftpServerPort

int ftpServerPort

ftpUserName

java.lang.String ftpUserName

ftpUserPassword

java.lang.String ftpUserPassword

ftpFileName

java.lang.String ftpFileName
The name that we use for the file on the FTP server. This is only for internal use.


checksum

java.lang.String checksum
If useChecksums is true, contains the file checksum.

Class dk.netarkivet.common.distribute.HTTPRemoteFile extends AbstractRemoteFile implements Serializable

Serialized Fields

hostname

java.lang.String hostname
The host name this file orignated on.


url

java.net.URL url
The url that exposes this remote file.


checksum

java.lang.String checksum
If useChecksums is true, contains the file checksum.

Class dk.netarkivet.common.distribute.NetarkivetMessage extends java.lang.Object implements Serializable

Serialization Methods

readObject

private void readObject(java.io.ObjectInputStream s)
Invoke default method for deserializing object.


writeObject

private void writeObject(java.io.ObjectOutputStream s)
Invoke default method for serializing object.

Serialized Fields

errMsg

java.lang.String errMsg

isOk

boolean isOk

id

java.lang.String id

to

ChannelID to

replyTo

ChannelID replyTo

replyOfId

java.lang.String replyOfId

Class dk.netarkivet.common.distribute.NullRemoteFile extends java.lang.Object implements Serializable


Package dk.netarkivet.common.distribute.arcrepository

Class dk.netarkivet.common.distribute.arcrepository.BitarchiveRecord extends java.lang.Object implements Serializable

Serialized Fields

fileName

java.lang.String fileName
The file the data were retrieved from.


objectBuffer

byte[] objectBuffer
The actual data.


offset

long offset
The offset of the ARCRecord contained.


length

long length
The length of the ARCRecord contained.


objectAsRemoteFile

RemoteFile objectAsRemoteFile
The actual data as a remote file.


isStoredAsRemoteFile

boolean isStoredAsRemoteFile
Is the data stored in a RemoteFile.


hasRemoteFileBeenDeleted

boolean hasRemoteFileBeenDeleted
Set after deleting RemoteFile.


LIMIT_FOR_SAVING_DATA_IN_OBJECT_BUFFER

long LIMIT_FOR_SAVING_DATA_IN_OBJECT_BUFFER
How large the ARCRecord can before saving as RemoteFile.


Package dk.netarkivet.common.exceptions

Class dk.netarkivet.common.exceptions.ArgumentNotValid extends NetarkivetException implements Serializable

Class dk.netarkivet.common.exceptions.ForwardedToErrorPage extends NetarkivetException implements Serializable

Class dk.netarkivet.common.exceptions.IllegalState extends NetarkivetException implements Serializable

Class dk.netarkivet.common.exceptions.IOFailure extends NetarkivetException implements Serializable

Class dk.netarkivet.common.exceptions.NetarkivetException extends java.lang.RuntimeException implements Serializable

Class dk.netarkivet.common.exceptions.NotImplementedException extends NetarkivetException implements Serializable

Class dk.netarkivet.common.exceptions.PermissionDenied extends NetarkivetException implements Serializable

Class dk.netarkivet.common.exceptions.UnknownID extends NetarkivetException implements Serializable


Package dk.netarkivet.common.utils.arc

Class dk.netarkivet.common.utils.arc.ARCBatchJob extends FileBatchJob implements Serializable

Serialized Fields

exceptionList

java.util.ArrayList<E> exceptionList
List for collecting caught Exceptions.


noOfRecordsProcessed

int noOfRecordsProcessed
The total number of records processed.

Class dk.netarkivet.common.utils.arc.BatchFilter extends java.lang.Object implements Serializable

Class dk.netarkivet.common.utils.arc.FileBatchJob extends java.lang.Object implements Serializable

Serialized Fields

filesToProcess

java.util.regex.Pattern filesToProcess
Regular expression for the files to process with this job. By default, all files are processed. This pattern must match the entire filename, but not the path (e.g. .*foo.* for any file with foo in it).


noOfFilesProcessed

int noOfFilesProcessed

filesFailed

java.util.Set<E> filesFailed

Package dk.netarkivet.common.utils.cdx

Class dk.netarkivet.common.utils.cdx.ARCFilenameCDXRecordFilter extends SimpleCDXRecordFilter implements Serializable

Serialized Fields

arcfilenamepattern

java.lang.String arcfilenamepattern

p

java.util.regex.Pattern p

Class dk.netarkivet.common.utils.cdx.ExtractCDXJob extends ARCBatchJob implements Serializable

Serialized Fields

fields

java.lang.String[] fields
The fields to be included in CDX output.


includeChecksum

boolean includeChecksum
True if we put an MD5 in each CDX line as well.


log

org.apache.commons.logging.Log log
Logger for this class.

Class dk.netarkivet.common.utils.cdx.SimpleCDXRecordFilter extends java.lang.Object implements Serializable

Serialized Fields

filtername

java.lang.String filtername
Variable holding the filtername.


Package dk.netarkivet.harvester.datamodel

Class dk.netarkivet.harvester.datamodel.Job extends java.lang.Object implements Serializable

Serialization Methods

readObject

private void readObject(java.io.ObjectInputStream s)
Invoke default method for deserializing object, and reinitialise the logger.


writeObject

private void writeObject(java.io.ObjectOutputStream s)
Invoke default method for serializing object.

Serialized Fields

jobID

java.lang.Long jobID

origHarvestDefinitionID

java.lang.Long origHarvestDefinitionID

status

JobStatus status

priority

JobPriority priority
The priority of this job.


forceMaxObjectsPerDomain

long forceMaxObjectsPerDomain
Overrides the individual configurations maximum setting for objects retrieved from a domain when set to a positive value.


forceMaxBytesPerDomain

long forceMaxBytesPerDomain
Overrides the invidual configurations maximum setting for bytes retrieved from a domain when set to other than -1.


orderXMLname

java.lang.String orderXMLname

orderXMLdoc

org.dom4j.Document orderXMLdoc

settingsXMLfiles

java.io.File[] settingsXMLfiles

settingsXMLdocs

org.dom4j.Document[] settingsXMLdocs

seedList

java.lang.String seedList
A newline-separated ('\n') list of seeds. Note: During construction this variable may be set to null, in which case it has been transferred to the seedListSet-variable. In this case it should be regenerated is seen in the getSeedList()-method - the preferred way of reading this variable is therefore the method even from inside the class.


harvestNum

int harvestNum
Which run of the harvest definition this is.


harvestErrors

java.lang.String harvestErrors

harvestErrorDetails

java.lang.String harvestErrorDetails

uploadErrors

java.lang.String uploadErrors

uploadErrorDetails

java.lang.String uploadErrorDetails

actualStart

java.util.Date actualStart

actualStop

java.util.Date actualStop

edition

long edition
Edition is used by the DAO to keep track of changes.


domainConfigurationMap

java.util.Map<K,V> domainConfigurationMap
A map (domainName, domainConfigurationName), must be accessible in order to update job information (see Ass. 2.4.3)


configsChanged

boolean configsChanged
A hint to the DAO that configurations have changed. Since configurations are large, the DAO can use that this is false to avoid updating the config list. The DAO can set it to false after saving configurations.


configurationSetsLimit

boolean configurationSetsLimit
Whether the maxBytes field was defined by the harvest definition or the configuration limit. This is deciding for whether we accept smaller configurations or not when building jobs. True means the limit is defined by the configuration, false means by the harvest definition.


minCountObjects

long minCountObjects
The lowest number of objects expected by a configuration.


maxCountObjects

long maxCountObjects
The highest number of objects expected by a configuration.


totalCountObjects

long totalCountObjects
The total number of objects expected by all added configurations.


seedListSet

java.util.Set<E> seedListSet
A set of seeds involved in this job. This is not accessible from the outside, but will be used to form the seedList variable when that is requested the first time.


underConstruction

boolean underConstruction
If true, this job object is still undergoing changes due to having more configurations added. When set to false, the object is no longer considered immutable except for updating status. Jobs loaded from the DAO are never under construction anymore.


LIM_MAX_REL_SIZE

long LIM_MAX_REL_SIZE
Job limits read from settings during construction.


LIM_MIN_ABS_SIZE

long LIM_MIN_ABS_SIZE

LIM_MAX_TOTAL_SIZE

long LIM_MAX_TOTAL_SIZE

Class dk.netarkivet.harvester.datamodel.RepeatingSchedule extends Schedule implements Serializable

Serialized Fields

repeats

int repeats
How many times this schedule should be repeated.

Class dk.netarkivet.harvester.datamodel.Schedule extends java.lang.Object implements Serializable

Serialized Fields

name

java.lang.String name
Human readable name for the schedule.


comments

java.lang.String comments
Any comments added by the user.


startDate

java.util.Date startDate
first run of job: date, time (hour:min:sec). May be null, meaning at any time


frequency

Frequency frequency
Frequency of runs, possibly with a time it should happen at.


edition

long edition
Edition is used by the DAO to keep track of changes.


id

java.lang.Long id
ID autogenerated by DB, ignored otherwise.

Class dk.netarkivet.harvester.datamodel.SeedList extends java.lang.Object implements Serializable

Serialized Fields

name

java.lang.String name
The name of the seedlist. Used for sorting.


seeds

java.util.List<E> seeds
The List of Seeds; Each String in the List holds one seed.


comments

java.lang.String comments
Any comments associated with this seedlist.


id

java.lang.Long id
ID autogenerated by DB, ignored otherwise.

Class dk.netarkivet.harvester.datamodel.TimedSchedule extends Schedule implements Serializable

Serialized Fields

endDate

java.util.Date endDate
The day this schedule should end.


Package dk.netarkivet.harvester.distribute

Class dk.netarkivet.harvester.distribute.HarvesterMessage extends NetarkivetMessage implements Serializable


Package dk.netarkivet.harvester.harvesting

Class dk.netarkivet.harvester.harvesting.ContentSizeAnnotationPostProcessor extends org.archive.crawler.framework.Processor implements Serializable

Class dk.netarkivet.harvester.harvesting.FixedUURI extends org.archive.net.UURI implements Serializable

Class dk.netarkivet.harvester.harvesting.HeritrixDomainHarvestReport extends DomainHarvestReport implements Serializable

Serialized Fields

defaultStopReason

StopReason defaultStopReason
The default reason why we stopped harvesting this domain. This value is set by looking for a CRAWL ENDED in the crawl.log.


Package dk.netarkivet.harvester.harvesting.distribute

Class dk.netarkivet.harvester.harvesting.distribute.CrawlStatusMessage extends HarvesterMessage implements Serializable

Serialized Fields

jobID

long jobID
the id for the crawlJob, for which this message reports.


statusCode

JobStatus statusCode
The current state of the crawl-job.


domainHarvestReport

DomainHarvestReport domainHarvestReport
A domainHarvestReport created at the end of the crawl.


harvestErrors

java.lang.String harvestErrors
harvest errors encountered.


harvestErrorDetails

java.lang.String harvestErrorDetails
harvest errrors encountered with details.


uploadErrors

java.lang.String uploadErrors
upload errors encountered.


uploadErrorDetails

java.lang.String uploadErrorDetails
upload errors encountered with details.

Class dk.netarkivet.harvester.harvesting.distribute.DomainHarvestReport extends java.lang.Object implements Serializable

Serialized Fields

domainstats

java.util.Map<K,V> domainstats
Datastructure holding the domain-information contained in one harvest.

Class dk.netarkivet.harvester.harvesting.distribute.DomainStats extends java.lang.Object implements Serializable

Serialized Fields

objectCount

long objectCount
Count of how many objects have been harvested from this domain.


byteCount

long byteCount
Count of how many bytes have been harvested from this domain .


stopReason

StopReason stopReason
The reason why we 'only' harvested byteCount bytes or objectCount objects.

Class dk.netarkivet.harvester.harvesting.distribute.DoOneCrawlMessage extends HarvesterMessage implements Serializable

Serialization Methods

readObject

private void readObject(java.io.ObjectInputStream s)
                 throws java.lang.ClassNotFoundException,
                        java.io.IOException
Method needed to de-serializable an object of this class.

Throws:
java.lang.ClassNotFoundException
java.io.IOException

writeObject

private void writeObject(java.io.ObjectOutputStream s)
                  throws java.lang.ClassNotFoundException,
                         java.io.IOException
Method needed to serializable an object of this class.

Throws:
java.lang.ClassNotFoundException
java.io.IOException
Serialized Fields

submittedJob

Job submittedJob
the Job to crawl.


metadata

java.util.List<E> metadata
Extra metadata associated with the crawl-job.

Class dk.netarkivet.harvester.harvesting.distribute.MetadataEntry extends java.lang.Object implements Serializable

Serialization Methods

readObject

private void readObject(java.io.ObjectInputStream s)
                 throws java.lang.ClassNotFoundException,
                        java.io.IOException
Method needed to de-serializable an object of this class.

Throws:
java.lang.ClassNotFoundException
java.io.IOException

writeObject

private void writeObject(java.io.ObjectOutputStream s)
                  throws java.lang.ClassNotFoundException,
                         java.io.IOException
Method needed to serializable an object of this class.

Throws:
java.lang.ClassNotFoundException
java.io.IOException
Serialized Fields

url

java.lang.String url
The URL for this metadataEntry: Used as the unique identifier for this bit of metadata in the Netarchive.


mimeType

java.lang.String mimeType
The mimetype for this metadataEntry: Identifies which type of document this bit of metadata is.


data

byte[] data
the metadata itself as byte array.