Class BitmagArcRepositoryClient
- java.lang.Object
-
- dk.netarkivet.common.distribute.Synchronizer
-
- dk.netarkivet.archive.arcrepository.distribute.BitmagArcRepositoryClient
-
- All Implemented Interfaces:
ArcRepositoryClient
,HarvesterArcRepositoryClient
,PreservationArcRepositoryClient
,ViewerArcRepositoryClient
,AutoCloseable
,javax.jms.MessageListener
public class BitmagArcRepositoryClient extends Synchronizer implements ArcRepositoryClient, AutoCloseable
Client side usage of an arc repository. All non-writing requests are forwarded to the ArcRepositoryServer over the network. Store requests are sent directly to the bitrepository messagebus. Get and store messages are retried a number of time before giving up, and will timeout after a specified time. So called mixed-mode client, where the store is done to a Bitmagrepository, and the rest (access and batch-processing) is done using the distributed netarchivesuite archive.
-
-
Field Summary
Fields Modifier and Type Field Description static String
ARCREPOSITORY_GET_TIMEOUT
settings.common.arcrepositoryClient.getTimeout:
The setting for how many milliseconds we will wait before giving up on a lookup request to the Arcrepository.protected static org.slf4j.Logger
log
Logging output place.
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Deprecated Methods Modifier and Type Method Description BatchStatus
batch(FileBatchJob job, String replicaId, String... args)
Runs a batch batch job on each file in the ArcRepository.BatchStatus
batch(FileBatchJob job, String replicaId, String batchId, String... args)
Runs a batch job on each file in the ArcRepository.void
close()
Removes this object as a JMS listener.File
correct(String replicaId, String checksum, File file, String credentials)
Deprecated.protected void
error(String errMsg)
Handle an error situation.BitarchiveRecord
get(String arcfile, long index)
Initializes a WarcRecordClient to interact with the warc records service and uses it to request a record with the given name and offset/index.File
getAllChecksums(String replicaId)
Deprecated.File
getAllFilenames(String replicaId)
Deprecated.String
getChecksum(String replicaId, String filename)
Deprecated.void
getFile(String arcfilename, Replica replica, File toFile)
Synchronously retrieves a file from a bitarchive and places it in a local file.static BitmagArcRepositoryClient
getInstance()
Get an instance of this class.File
removeAndGetFile(String fileName, String bitarchiveId, String checksum, String credentials)
Deprecated.void
store(File file)
Store the file in the bitrepository.void
updateAdminChecksum(String filename, String checksum)
Not implemented.void
updateAdminData(String fileName, String replicaId, ReplicaStoreState newval)
Not implemented.boolean
uploadFile(File file, String fileId)
Attempts to upload a given file.-
Methods inherited from class dk.netarkivet.common.distribute.Synchronizer
onMessage, sendAndWaitForOneReply
-
-
-
-
Field Detail
-
log
protected static final org.slf4j.Logger log
Logging output place.
-
ARCREPOSITORY_GET_TIMEOUT
public static final String ARCREPOSITORY_GET_TIMEOUT
settings.common.arcrepositoryClient.getTimeout:
The setting for how many milliseconds we will wait before giving up on a lookup request to the Arcrepository.- See Also:
- Constant Field Values
-
-
Method Detail
-
getInstance
public static BitmagArcRepositoryClient getInstance()
Get an instance of this class. This is guaranteed to be a singleton.- Returns:
- an JMSArcRepositoryClient instance.
-
close
public void close()
Removes this object as a JMS listener.- Specified by:
close
in interfaceArcRepositoryClient
- Specified by:
close
in interfaceAutoCloseable
- Specified by:
close
in interfaceHarvesterArcRepositoryClient
- Specified by:
close
in interfacePreservationArcRepositoryClient
- Specified by:
close
in interfaceViewerArcRepositoryClient
-
get
public BitarchiveRecord get(String arcfile, long index) throws ArgumentNotValid, IOFailure
Initializes a WarcRecordClient to interact with the warc records service and uses it to request a record with the given name and offset/index.- Specified by:
get
in interfaceArcRepositoryClient
- Specified by:
get
in interfacePreservationArcRepositoryClient
- Specified by:
get
in interfaceViewerArcRepositoryClient
- Parameters:
arcfile
- The name of a file.index
- The offset of the wanted record in the file- Returns:
- a BitarchiveRecord-object.
- Throws:
ArgumentNotValid
- If the given arcfile is null or empty, or the given index is negative.IOFailure
- If an invalid URL is provided for the warc record service or the operation failed.
-
getFile
public void getFile(String arcfilename, Replica replica, File toFile) throws ArgumentNotValid, IOFailure
Synchronously retrieves a file from a bitarchive and places it in a local file. This implementation retrieves the file using bitrepository.org software.- Specified by:
getFile
in interfaceArcRepositoryClient
- Specified by:
getFile
in interfacePreservationArcRepositoryClient
- Specified by:
getFile
in interfaceViewerArcRepositoryClient
- Parameters:
arcfilename
- Name of the arcfile to retrieve.replica
- This parameter is ignored in this implementation. The file is retrieved from the fastest pillar.toFile
- Filename of a place where the file fetched can be put. If this file already exists it must be empty otherwise this method-call will fail.- Throws:
ArgumentNotValid
- If the arcfilename are null or empty, or if either replica or toFile is null.IOFailure
- if there are problems getting a reply or the file could not be found.
-
store
public void store(File file) throws IOFailure, ArgumentNotValid
Store the file in the bitrepository.- Specified by:
store
in interfaceArcRepositoryClient
- Specified by:
store
in interfaceHarvesterArcRepositoryClient
- Specified by:
store
in interfacePreservationArcRepositoryClient
- Parameters:
file
- A file to be stored. Must exist.- Throws:
IOFailure
- thrown if store is unsuccessful, or failed to clean up files locally or on the ftp server after the store operation.ArgumentNotValid
- if file parameter is null or file is not an existing file.
-
error
protected void error(String errMsg)
Handle an error situation. Sends a notification, and throws an error.- Parameters:
errMsg
- The message for the error.
-
batch
public BatchStatus batch(FileBatchJob job, String replicaId, String... args)
Runs a batch batch job on each file in the ArcRepository.Note: The id for the batchjob is the empty string, which removes the possibility of terminating the batchjob remotely while it is running.
- Specified by:
batch
in interfaceArcRepositoryClient
- Specified by:
batch
in interfacePreservationArcRepositoryClient
- Specified by:
batch
in interfaceViewerArcRepositoryClient
- Parameters:
job
- An object that implements the FileBatchJob interface. The initialize() method will be called before processing and the finish() method will be called afterwards. The process() method will be called with each File entry. An optional function postProcess() allows handling the combined results of the batchjob, e.g. summing the results, sorting, etc.replicaId
- The archive to execute the job on.args
- The arguments for the batchjob.- Returns:
- The status of the batch job after it ended.
-
batch
public BatchStatus batch(FileBatchJob job, String replicaId, String batchId, String... args) throws IOFailure, ArgumentNotValid
Runs a batch job on each file in the ArcRepository.- Parameters:
job
- An object that implements the FileBatchJob interface. The initialize() method will be called before processing and the finish() method will be called afterwards. The process() method will be called with each File entry. An optional function postProcess() allows handling the combined results of the batchjob, e.g. summing the results, sorting, etc.replicaId
- The archive to execute the job on.args
- The arguments for the batchjob. This is allowed to be null.batchId
- The id for the batch process.- Returns:
- The status of the batch job after it ended.
- Throws:
ArgumentNotValid
- If the job is null or the replicaId is either null or the empty string.IOFailure
- If no result file is returned.
-
updateAdminData
public void updateAdminData(String fileName, String replicaId, ReplicaStoreState newval) throws ArgumentNotValid, IOFailure
Not implemented. This functionality is delegated to bitrepository software.- Specified by:
updateAdminData
in interfaceArcRepositoryClient
- Specified by:
updateAdminData
in interfacePreservationArcRepositoryClient
- Parameters:
fileName
- The name of a file stored in the ArcRepository.replicaId
- The id of the replica that the administrative data for fileName is wrong for.newval
- What the administrative data will be updated to.- Throws:
ArgumentNotValid
IOFailure
-
updateAdminChecksum
public void updateAdminChecksum(String filename, String checksum)
Not implemented. This functionality is delegated to bitrepository software.- Specified by:
updateAdminChecksum
in interfaceArcRepositoryClient
- Specified by:
updateAdminChecksum
in interfacePreservationArcRepositoryClient
- Parameters:
filename
- The name of a file stored in the ArcRepository.checksum
- The new checksum.
-
removeAndGetFile
@Deprecated public File removeAndGetFile(String fileName, String bitarchiveId, String checksum, String credentials) throws IOFailure, ArgumentNotValid
Deprecated.Not implemented. This functionality is delegated to bitrepository software.- Specified by:
removeAndGetFile
in interfaceArcRepositoryClient
- Specified by:
removeAndGetFile
in interfacePreservationArcRepositoryClient
- Parameters:
fileName
- The name of the file to remove.bitarchiveId
- The id of the replica from which to remove the file.checksum
- The checksum of the file to be removed.credentials
- A string that shows that the user is allowed to perform this operation.- Returns:
- A local copy of the file removed.
- Throws:
IOFailure
ArgumentNotValid
-
getAllChecksums
@Deprecated public File getAllChecksums(String replicaId) throws IOFailure, ArgumentNotValid
Deprecated.Not implemented. This functionality is delegated to bitrepository software.- Specified by:
getAllChecksums
in interfacePreservationArcRepositoryClient
- Parameters:
replicaId
- The id of the replica from which the checksums should be retrieved.- Returns:
- A list of ChecksumEntries which is the results of the GetAllChecksumMessage.
- Throws:
IOFailure
ArgumentNotValid
- See Also:
GetAllChecksumsMessage
-
getAllFilenames
@Deprecated public File getAllFilenames(String replicaId) throws ArgumentNotValid, IOFailure
Deprecated.Not implemented. This functionality is delegated to bitrepository software.- Specified by:
getAllFilenames
in interfacePreservationArcRepositoryClient
- Parameters:
replicaId
- The id of the replica from which the list of filenames should be retrieved.- Returns:
- A list of all the filenames within the archive of the given replica.
- Throws:
ArgumentNotValid
IOFailure
- See Also:
GetAllFilenamesMessage
-
getChecksum
@Deprecated public String getChecksum(String replicaId, String filename) throws ArgumentNotValid, IOFailure
Deprecated.Not implemented. This functionality is delegated to bitrepository software.- Specified by:
getChecksum
in interfacePreservationArcRepositoryClient
- Parameters:
replicaId
- The name of the replica to send the message.filename
- The name of the file for whom the checksum should be retrieved.- Returns:
- The checksum of the file in the replica. Or null if an error occurred.
- Throws:
ArgumentNotValid
IOFailure
-
correct
@Deprecated public File correct(String replicaId, String checksum, File file, String credentials) throws IOFailure, ArgumentNotValid
Deprecated.Not implemented. This functionality is delegated to bitrepository software.- Specified by:
correct
in interfacePreservationArcRepositoryClient
- Parameters:
replicaId
- The identification of the replica.checksum
- The checksum of the corrupt entry in the archive. It is important to validate that the checksum actually is wrong before correcting the entry.file
- The new file to replace the old one.credentials
- The password for allowing to remove a file entry in the archive.- Returns:
- The corrupted file from the archive.
- Throws:
IOFailure
ArgumentNotValid
-
-