Package dk.netarkivet.archive.arcrepository.distribute

Class BitmagArcRepositoryClient

java.lang.Object

dk.netarkivet.common.distribute.Synchronizer

dk.netarkivet.archive.arcrepository.distribute.BitmagArcRepositoryClient

All Implemented Interfaces:

ArcRepositoryClient, HarvesterArcRepositoryClient, PreservationArcRepositoryClient, ViewerArcRepositoryClient, java.lang.AutoCloseable, javax.jms.MessageListener

public class BitmagArcRepositoryClient
extends Synchronizer
implements ArcRepositoryClient, java.lang.AutoCloseable

Client side usage of an arc repository. All non-writing requests are forwarded to the ArcRepositoryServer over the network. Store requests are sent directly to the bitrepository messagebus. Get and store messages are retried a number of time before giving up, and will timeout after a specified time. So called mixed-mode client, where the store is done to a Bitmagrepository, and the rest (access and batch-processing) is done using the distributed netarchivesuite archive.

Field Summary

Fields

Modifier and Type	Field	Description
static java.lang.String	ARCREPOSITORY_GET_TIMEOUT	settings.common.arcrepositoryClient.getTimeout: The setting for how many milliseconds we will wait before giving up on a lookup request to the Arcrepository.
protected static org.slf4j.Logger	log	Logging output place.

Method Summary

Modifier and Type	Method	Description
BatchStatus	batch(FileBatchJob job, java.lang.String replicaId, java.lang.String... args)	Runs a batch batch job on each file in the ArcRepository.
BatchStatus	batch(FileBatchJob job, java.lang.String replicaId, java.lang.String batchId, java.lang.String... args)	Runs a batch job on each file in the ArcRepository.
void	close()	Removes this object as a JMS listener.
java.io.File	correct(java.lang.String replicaId, java.lang.String checksum, java.io.File file, java.lang.String credentials)	Deprecated.
protected void	error(java.lang.String errMsg)	Handle an error situation.
BitarchiveRecord	get(java.lang.String arcfile, long index)	Initializes a WarcRecordClient to interact with the warc records service and uses it to request a record with the given name and offset/index.
java.io.File	getAllChecksums(java.lang.String replicaId)	Deprecated.
java.io.File	getAllFilenames(java.lang.String replicaId)	Deprecated.
java.lang.String	getChecksum(java.lang.String replicaId, java.lang.String filename)	Deprecated.
void	getFile(java.lang.String arcfilename, Replica replica, java.io.File toFile)	Synchronously retrieves a file from a bitarchive and places it in a local file.
static BitmagArcRepositoryClient	getInstance()	Get an instance of this class.
java.io.File	removeAndGetFile(java.lang.String fileName, java.lang.String bitarchiveId, java.lang.String checksum, java.lang.String credentials)	Deprecated.
void	store(java.io.File file)	Store the file in the bitrepository.
void	updateAdminChecksum(java.lang.String filename, java.lang.String checksum)	Not implemented.
void	updateAdminData(java.lang.String fileName, java.lang.String replicaId, ReplicaStoreState newval)	Not implemented.
boolean	uploadFile(java.io.File file, java.lang.String fileId)	Attempts to upload a given file.

Methods inherited from class dk.netarkivet.common.distribute.Synchronizer

onMessage, sendAndWaitForOneReply

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

log

protected static final org.slf4j.Logger log

Logging output place.

ARCREPOSITORY_GET_TIMEOUT

public static final java.lang.String ARCREPOSITORY_GET_TIMEOUT

settings.common.arcrepositoryClient.getTimeout:
The setting for how many milliseconds we will wait before giving up on a lookup request to the Arcrepository.

See Also:

Constant Field Values

Method Detail

getInstance

public static BitmagArcRepositoryClient getInstance()

Get an instance of this class. This is guaranteed to be a singleton.

Returns:

an JMSArcRepositoryClient instance.

close

public void close()

Removes this object as a JMS listener.

Specified by:

close in interface ArcRepositoryClient

Specified by:

close in interface java.lang.AutoCloseable

Specified by:

close in interface HarvesterArcRepositoryClient

Specified by:

close in interface PreservationArcRepositoryClient

Specified by:

close in interface ViewerArcRepositoryClient

get

public BitarchiveRecord get(java.lang.String arcfile,
                            long index)
                     throws ArgumentNotValid,
                            IOFailure

Initializes a WarcRecordClient to interact with the warc records service and uses it to request a record with the given name and offset/index.

Specified by:

get in interface ArcRepositoryClient

Specified by:

get in interface PreservationArcRepositoryClient

Specified by:

get in interface ViewerArcRepositoryClient

Parameters:

arcfile - The name of a file.

index - The offset of the wanted record in the file

Returns:

a BitarchiveRecord-object.

Throws:

ArgumentNotValid - If the given arcfile is null or empty, or the given index is negative.

IOFailure - If an invalid URL is provided for the warc record service or the operation failed.

getFile

public void getFile(java.lang.String arcfilename,
                    Replica replica,
                    java.io.File toFile)
             throws ArgumentNotValid,
                    IOFailure

Synchronously retrieves a file from a bitarchive and places it in a local file. This implementation retrieves the file using bitrepository.org software.

Specified by:

getFile in interface ArcRepositoryClient

Specified by:

getFile in interface PreservationArcRepositoryClient

Specified by:

getFile in interface ViewerArcRepositoryClient

Parameters:

arcfilename - Name of the arcfile to retrieve.

replica - This parameter is ignored in this implementation. The file is retrieved from the fastest pillar.

toFile - Filename of a place where the file fetched can be put. If this file already exists it must be empty otherwise this method-call will fail.

Throws:

ArgumentNotValid - If the arcfilename are null or empty, or if either replica or toFile is null.

IOFailure - if there are problems getting a reply or the file could not be found.

store

public void store(java.io.File file)
           throws IOFailure,
                  ArgumentNotValid

Store the file in the bitrepository. After a successful storage operation, both the local copy of the file and the copy on the ftp server are deleted.

Specified by:

store in interface ArcRepositoryClient

Specified by:

store in interface HarvesterArcRepositoryClient

Specified by:

store in interface PreservationArcRepositoryClient

Parameters:

file - A file to be stored. Must exist.

Throws:

IOFailure - thrown if store is unsuccessful, or failed to clean up files locally or on the ftp server after the store operation.

ArgumentNotValid - if file parameter is null or file is not an existing file.

error

protected void error(java.lang.String errMsg)

Handle an error situation. Sends a notification, and throws an error.

Parameters:

errMsg - The message for the error.

batch

public BatchStatus batch(FileBatchJob job,
                         java.lang.String replicaId,
                         java.lang.String... args)

Runs a batch batch job on each file in the ArcRepository.

Note: The id for the batchjob is the empty string, which removes the possibility of terminating the batchjob remotely while it is running.

Specified by:

batch in interface ArcRepositoryClient

Specified by:

batch in interface PreservationArcRepositoryClient

Specified by:

batch in interface ViewerArcRepositoryClient

Parameters:

job - An object that implements the FileBatchJob interface. The initialize() method will be called before processing and the finish() method will be called afterwards. The process() method will be called with each File entry. An optional function postProcess() allows handling the combined results of the batchjob, e.g. summing the results, sorting, etc.

replicaId - The archive to execute the job on.

args - The arguments for the batchjob.

Returns:

The status of the batch job after it ended.

batch

public BatchStatus batch(FileBatchJob job,
                         java.lang.String replicaId,
                         java.lang.String batchId,
                         java.lang.String... args)
                  throws IOFailure,
                         ArgumentNotValid

Runs a batch job on each file in the ArcRepository.

Parameters:

job - An object that implements the FileBatchJob interface. The initialize() method will be called before processing and the finish() method will be called afterwards. The process() method will be called with each File entry. An optional function postProcess() allows handling the combined results of the batchjob, e.g. summing the results, sorting, etc.

replicaId - The archive to execute the job on.

args - The arguments for the batchjob. This is allowed to be null.

batchId - The id for the batch process.

Returns:

The status of the batch job after it ended.

Throws:

ArgumentNotValid - If the job is null or the replicaId is either null or the empty string.

IOFailure - If no result file is returned.

updateAdminData

public void updateAdminData(java.lang.String fileName,
                            java.lang.String replicaId,
                            ReplicaStoreState newval)
                     throws ArgumentNotValid,
                            IOFailure

Not implemented. This functionality is delegated to bitrepository software.

Specified by:

updateAdminData in interface ArcRepositoryClient

Specified by:

updateAdminData in interface PreservationArcRepositoryClient

Parameters:

fileName - The name of a file stored in the ArcRepository.

replicaId - The id of the replica that the administrative data for fileName is wrong for.

newval - What the administrative data will be updated to.

Throws:

ArgumentNotValid

IOFailure

updateAdminChecksum

public void updateAdminChecksum(java.lang.String filename,
                                java.lang.String checksum)

Not implemented. This functionality is delegated to bitrepository software.

Specified by:

updateAdminChecksum in interface ArcRepositoryClient

Specified by:

updateAdminChecksum in interface PreservationArcRepositoryClient

Parameters:

filename - The name of a file stored in the ArcRepository.

checksum - The new checksum.

removeAndGetFile

@Deprecated
public java.io.File removeAndGetFile(java.lang.String fileName,
                                     java.lang.String bitarchiveId,
                                     java.lang.String checksum,
                                     java.lang.String credentials)
                              throws IOFailure,
                                     ArgumentNotValid

Deprecated.

Not implemented. This functionality is delegated to bitrepository software.

Specified by:

removeAndGetFile in interface ArcRepositoryClient

Specified by:

removeAndGetFile in interface PreservationArcRepositoryClient

Parameters:

fileName - The name of the file to remove.

bitarchiveId - The id of the replica from which to remove the file.

checksum - The checksum of the file to be removed.

credentials - A string that shows that the user is allowed to perform this operation.

Returns:

A local copy of the file removed.

Throws:

IOFailure

ArgumentNotValid

getAllChecksums

@Deprecated
public java.io.File getAllChecksums(java.lang.String replicaId)
                             throws IOFailure,
                                    ArgumentNotValid

Deprecated.

Not implemented. This functionality is delegated to bitrepository software.

Specified by:

getAllChecksums in interface PreservationArcRepositoryClient

Parameters:

replicaId - The id of the replica from which the checksums should be retrieved.

Returns:

A list of ChecksumEntries which is the results of the GetAllChecksumMessage.

Throws:

IOFailure

ArgumentNotValid

See Also:

GetAllChecksumsMessage

getAllFilenames

@Deprecated
public java.io.File getAllFilenames(java.lang.String replicaId)
                             throws ArgumentNotValid,
                                    IOFailure

Deprecated.

Not implemented. This functionality is delegated to bitrepository software.

Specified by:

getAllFilenames in interface PreservationArcRepositoryClient

Parameters:

replicaId - The id of the replica from which the list of filenames should be retrieved.

Returns:

A list of all the filenames within the archive of the given replica.

Throws:

ArgumentNotValid

IOFailure

See Also:

GetAllFilenamesMessage

getChecksum

@Deprecated
public java.lang.String getChecksum(java.lang.String replicaId,
                                    java.lang.String filename)
                             throws ArgumentNotValid,
                                    IOFailure

Deprecated.

Not implemented. This functionality is delegated to bitrepository software.

Specified by:

getChecksum in interface PreservationArcRepositoryClient

Parameters:

replicaId - The name of the replica to send the message.

filename - The name of the file for whom the checksum should be retrieved.

Returns:

The checksum of the file in the replica. Or null if an error occurred.

Throws:

ArgumentNotValid

IOFailure

correct

@Deprecated
public java.io.File correct(java.lang.String replicaId,
                            java.lang.String checksum,
                            java.io.File file,
                            java.lang.String credentials)
                     throws IOFailure,
                            ArgumentNotValid

Deprecated.

Not implemented. This functionality is delegated to bitrepository software.

Specified by:

correct in interface PreservationArcRepositoryClient

Parameters:

replicaId - The identification of the replica.

checksum - The checksum of the corrupt entry in the archive. It is important to validate that the checksum actually is wrong before correcting the entry.

file - The new file to replace the old one.

credentials - The password for allowing to remove a file entry in the archive.

Returns:

The corrupted file from the archive.

Throws:

IOFailure

ArgumentNotValid

uploadFile

public boolean uploadFile(java.io.File file,
                          java.lang.String fileId)

Attempts to upload a given file.

Parameters:

file - The file to upload. Should exist. The packageId is the name of the file

fileId - The Id of the file to upload

Returns:

true if the upload succeeded, false otherwise.