dk.netarkivet.archive.arcrepository.distribute
Class JMSArcRepositoryClient

java.lang.Object
  dk.netarkivet.common.distribute.Synchronizer
      dk.netarkivet.archive.arcrepository.distribute.JMSArcRepositoryClient

All Implemented Interfaces:

ArcRepositoryClient, HarvesterArcRepositoryClient, PreservationArcRepositoryClient, ViewerArcRepositoryClient, javax.jms.MessageListener

public class JMSArcRepositoryClient
extends Synchronizer
implements ArcRepositoryClient

Client side usage of an arc repository. All requests are forwarded to the ArcRepositoryServer over the network. get and store messages are retried a number of time before giving up, and will timeout after a specified time.

Field Summary

static java.lang.String	ARCREPOSITORY_GET_TIMEOUT settings.common.arcrepositoryClient.getTimeout: The setting for how many milliseconds we will wait before giving up on a lookup request to the Arcrepository.
static java.lang.String	ARCREPOSITORY_STORE_RETRIES settings.common.arcrepositoryClient.storeRetries: The setting for the number of times to try sending a store message before failing, including the first attempt.
static java.lang.String	ARCREPOSITORY_STORE_TIMEOUT settings.common.arcrepositoryClient.storeTimeout: the setting for the timeout in milliseconds before retrying when calling ArcRepositoryClient.store(File).
protected org.apache.commons.logging.Log	log Logging output place.

Constructor Summary

protected

JMSArcRepositoryClient() Adds this Synchronizer as listener on a jms connection.

Method Summary

BatchStatus	batch(FileBatchJob job, java.lang.String replicaId, java.lang.String... args) Runs a batch batch job on each file in the ArcRepository.
BatchStatus	batch(FileBatchJob job, java.lang.String replicaId, java.lang.String batchId, java.lang.String... args) Runs a batch job on each file in the ArcRepository.
void	close() Removes this object as a JMS listener.
java.io.File	correct(java.lang.String replicaId, java.lang.String checksum, java.io.File file, java.lang.String credentials) Method for correcting an entry in a replica.
BitarchiveRecord	get(java.lang.String arcfile, long index) Sends a GetMessage on the "TheArcrepos" queue and waits for a reply.
java.io.File	getAllChecksums(java.lang.String replicaId) Retrieves all the checksum from the replica through a GetAllChecksumMessage.
java.io.File	getAllFilenames(java.lang.String replicaId) Retrieves the names of all the files in the replica through a GetAllFilenamesMessage.
java.lang.String	getChecksum(java.lang.String replicaId, java.lang.String filename) Retrieves the checksum of a specific file.
void	getFile(java.lang.String arcfilename, Replica replica, java.io.File toFile) Synchronously retrieves a file from a bitarchive and places it in a local file.
static JMSArcRepositoryClient	getInstance() Get an JMSArcRepositoryClient instance.
java.io.File	removeAndGetFile(java.lang.String fileName, java.lang.String bitarchiveId, java.lang.String checksum, java.lang.String credentials) Removes a file from the bitarchives, if given credentials and checksum are correct.
void	store(java.io.File file) Sends a StoreMessage via the synchronized JMS connection method sendAndWaitForOneReply().
void	updateAdminChecksum(java.lang.String filename, java.lang.String checksum) Request update of admin data to specific checksum.
void	updateAdminData(java.lang.String fileName, java.lang.String replicaId, ReplicaStoreState newval) Request update of admin data to specific state.

Methods inherited from class dk.netarkivet.common.distribute.Synchronizer

onMessage, sendAndWaitForOneReply

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

log

protected final org.apache.commons.logging.Log log

Logging output place.

ARCREPOSITORY_GET_TIMEOUT

public static final java.lang.String ARCREPOSITORY_GET_TIMEOUT

settings.common.arcrepositoryClient.getTimeout:
The setting for how many milliseconds we will wait before giving up on a lookup request to the Arcrepository.

See Also:

Constant Field Values

ARCREPOSITORY_STORE_RETRIES

public static final java.lang.String ARCREPOSITORY_STORE_RETRIES

settings.common.arcrepositoryClient.storeRetries:
The setting for the number of times to try sending a store message before failing, including the first attempt.

See Also:

Constant Field Values

ARCREPOSITORY_STORE_TIMEOUT

public static final java.lang.String ARCREPOSITORY_STORE_TIMEOUT

settings.common.arcrepositoryClient.storeTimeout:
the setting for the timeout in milliseconds before retrying when calling ArcRepositoryClient.store(File).

See Also:

Constant Field Values

Constructor Detail

JMSArcRepositoryClient

protected JMSArcRepositoryClient()

Adds this Synchronizer as listener on a jms connection.

Method Detail

getInstance

public static JMSArcRepositoryClient getInstance()

Get an JMSArcRepositoryClient instance. This is guaranteed to be a singleton.

Returns:

an JMSArcRepositoryClient instance.

close

public void close()

Removes this object as a JMS listener.

Specified by:

close in interface ArcRepositoryClient

Specified by:

close in interface HarvesterArcRepositoryClient

Specified by:

close in interface PreservationArcRepositoryClient

Specified by:

close in interface ViewerArcRepositoryClient

get

public BitarchiveRecord get(java.lang.String arcfile,
                            long index)
                     throws ArgumentNotValid,
                            IOFailure

Sends a GetMessage on the "TheArcrepos" queue and waits for a reply. This is a blocking call. Returns null if no message is returned within Settings.ARCREPOSITORY_GET_TIMEOUT

Specified by:

get in interface ArcRepositoryClient

Specified by:

get in interface PreservationArcRepositoryClient

Specified by:

get in interface ViewerArcRepositoryClient

Parameters:

arcfile - The name of a file.

index - The offset of the wanted record in the file

Returns:

a BitarchiveRecord-object or null if request times out or object is not found.

Throws:

ArgumentNotValid - If the given arcfile is null or empty, or the given index is negative.

IOFailure - If a wrong message is returned or the get operation failed.

getFile

public void getFile(java.lang.String arcfilename,
                    Replica replica,
                    java.io.File toFile)
             throws ArgumentNotValid,
                    IOFailure

Synchronously retrieves a file from a bitarchive and places it in a local file. This is the interface for sending GetFileMessage on the "TheArcrepos" queue. This is a blocking call.

Specified by:

getFile in interface ArcRepositoryClient

Specified by:

getFile in interface PreservationArcRepositoryClient

Specified by:

getFile in interface ViewerArcRepositoryClient

Parameters:

arcfilename - Name of the arcfile to retrieve.

replica - The bitarchive to retrieve the data from.

toFile - Filename of a place where the file fetched can be put.

Throws:

ArgumentNotValid - If the arcfilename are null or empty, or if either replica or toFile is null.

IOFailure - if there are problems getting a reply or the file could not be found.

store

public void store(java.io.File file)
           throws IOFailure,
                  ArgumentNotValid

Sends a StoreMessage via the synchronized JMS connection method sendAndWaitForOneReply(). After a successful storage operation, both the local copy of the file and the copy on the ftp server are deleted.

Specified by:

store in interface ArcRepositoryClient

Specified by:

store in interface HarvesterArcRepositoryClient

Specified by:

store in interface PreservationArcRepositoryClient

Parameters:

file - A file to be stored. Must exist.

Throws:

IOFailure - thrown if store is unsuccessful, or failed to clean up files locally or on the ftp server after the store operation.

ArgumentNotValid - if file parameter is null or file is not an existing file.

batch

public BatchStatus batch(FileBatchJob job,
                         java.lang.String replicaId,
                         java.lang.String... args)

Runs a batch batch job on each file in the ArcRepository. Note: The id for the batchjob is the empty string, which removes the possibility of terminating the batchjob remotely while it is running.

Specified by:

batch in interface ArcRepositoryClient

Specified by:

batch in interface PreservationArcRepositoryClient

Specified by:

batch in interface ViewerArcRepositoryClient

Parameters:

job - An object that implements the FileBatchJob interface. The initialize() method will be called before processing and the finish() method will be called afterwards. The process() method will be called with each File entry. An optional function postProcess() allows handling the combined results of the batchjob, e.g. summing the results, sorting, etc.

replicaId - The archive to execute the job on.

args - The arguments for the batchjob.

Returns:

The status of the batch job after it ended.

batch

public BatchStatus batch(FileBatchJob job,
                         java.lang.String replicaId,
                         java.lang.String batchId,
                         java.lang.String... args)
                  throws IOFailure,
                         ArgumentNotValid

Runs a batch job on each file in the ArcRepository.

Parameters:

job - An object that implements the FileBatchJob interface. The initialize() method will be called before processing and the finish() method will be called afterwards. The process() method will be called with each File entry. An optional function postProcess() allows handling the combined results of the batchjob, e.g. summing the results, sorting, etc.

replicaId - The archive to execute the job on.

args - The arguments for the batchjob. This is allowed to be null.

batchId - The id for the batch process.

Returns:

The status of the batch job after it ended.

Throws:

ArgumentNotValid - If the job is null or the replicaId is either null or the empty string.

IOFailure - If no result file is returned.

updateAdminData

public void updateAdminData(java.lang.String fileName,
                            java.lang.String replicaId,
                            ReplicaStoreState newval)
                     throws ArgumentNotValid,
                            IOFailure

Request update of admin data to specific state.

Specified by:

updateAdminData in interface ArcRepositoryClient

Specified by:

updateAdminData in interface PreservationArcRepositoryClient

Parameters:

fileName - The file for which admin data should be updated.

replicaId - The id if the replica that the administrative data for fileName is wrong for.

newval - The new value in admin data.

Throws:

ArgumentNotValid - If one of the arguments are invalid (null or empty string).

IOFailure - If the reply to the request update timed out.

updateAdminChecksum

public void updateAdminChecksum(java.lang.String filename,
                                java.lang.String checksum)

Request update of admin data to specific checksum.

Specified by:

updateAdminChecksum in interface ArcRepositoryClient

Specified by:

updateAdminChecksum in interface PreservationArcRepositoryClient

Parameters:

filename - The file for which admin data should be updated.

checksum - The new checksum for the file

removeAndGetFile

public java.io.File removeAndGetFile(java.lang.String fileName,
                                     java.lang.String bitarchiveId,
                                     java.lang.String checksum,
                                     java.lang.String credentials)
                              throws IOFailure,
                                     ArgumentNotValid

Removes a file from the bitarchives, if given credentials and checksum are correct.

Specified by:

removeAndGetFile in interface ArcRepositoryClient

Specified by:

removeAndGetFile in interface PreservationArcRepositoryClient

Parameters:

fileName - The name of the file to delete

bitarchiveId - The id of the bitarchive to delete the file in

checksum - The checksum of the deleted file

credentials - The credentials used to delete the file

Returns:

The file that was removed

Throws:

ArgumentNotValid - if arguments are null or equal to the empty string

IOFailure - if we could not delete the remote file, or there was no response to our RemoveAndGetFileMessage within the allotted time defined by the setting ARCREPOSITORY_STORE_TIMEOUT.

getAllChecksums

public java.io.File getAllChecksums(java.lang.String replicaId)
                             throws IOFailure,
                                    ArgumentNotValid

Retrieves all the checksum from the replica through a GetAllChecksumMessage. This is the checksum archive alternative to running a ChecksumBatchJob.

Specified by:

getAllChecksums in interface PreservationArcRepositoryClient

Parameters:

replicaId - The id of the replica from which the checksums should be retrieved.

Returns:

A file containing filename and checksum of all the files in an archive in the same format as a ChecksumJob.

Throws:

IOFailure - If the reply is not of type GetAllChecksumsMessage or if the file could not properly be retrieved from the reply message or if the message timed out.

ArgumentNotValid - If the replicaId is null or empty.

See Also:

GetAllChecksumsMessage

getAllFilenames

public java.io.File getAllFilenames(java.lang.String replicaId)
                             throws ArgumentNotValid,
                                    IOFailure

Retrieves the names of all the files in the replica through a GetAllFilenamesMessage. This is the checksum archive alternative to running a FilelistBatchJob.

Specified by:

getAllFilenames in interface PreservationArcRepositoryClient

Parameters:

replicaId - The id of the replica from which the list of filenames should be retrieved.

Returns:

A file with all the filenames within the archive of the given replica. A null is returned if the message timeout.

Throws:

IOFailure - If the reply is not of type GetAllFilenamesMessage or if the file could not properly be retrieved from the reply message

ArgumentNotValid - If the replicaId is null or empty.

See Also:

GetAllFilenamesMessage

getChecksum

public java.lang.String getChecksum(java.lang.String replicaId,
                                    java.lang.String filename)
                             throws ArgumentNotValid,
                                    IOFailure

Retrieves the checksum of a specific file. This is the checksum archive alternative to running a ChecksumJob limited to a specific file.

Specified by:

getChecksum in interface PreservationArcRepositoryClient

Parameters:

replicaId - The ID of the replica to send the message.

filename - The name of the file for whom the checksum should be retrieved.

Returns:

The checksum of the file in the replica.

Throws:

IOFailure - If the reply is not of type GetChecksumMessage. Or if the message timed out.

ArgumentNotValid - If either the replicaId of the filename is null or empty.

correct

public java.io.File correct(java.lang.String replicaId,
                            java.lang.String checksum,
                            java.io.File file,
                            java.lang.String credentials)
                     throws IOFailure,
                            ArgumentNotValid

Method for correcting an entry in a replica. This is done by sending a correct message to the replica. The file which is removed from the replica is put into the tempDir.

Specified by:

correct in interface PreservationArcRepositoryClient

Parameters:

replicaId - The id of the replica to send the message.

checksum - The checksum of the corrupt entry in the archive. It is important to validate that the checksum actually is wrong before correcting the entry.

file - The file to correct the entry in the archive of the replica.

credentials - A string with the password for allowing changes inside an archive. If it does not correspond to the credentials of the archive, the correction will not be allowed.

Returns:

The corrupted file from the archive.

Throws:

IOFailure - If the message is not handled properly.

ArgumentNotValid - If the replicaId, the checksum or the credentials are either null or empty, or if file is null.