Package dk.netarkivet.archive.arcrepositoryadmin

Class ReplicaCacheHelpers

    • Field Detail

      • log

        protected static org.slf4j.Logger log
        The log.

    • Method Detail

      • existsReplicaFileInfoInDB

        protected static boolean existsReplicaFileInfoInDB​(long fileid,
                                                   String replicaID,
                                                   Connection con)
                                            throws IllegalState
        Method for checking whether a replicafileinfo is in the database.
        Parameters:
        fileid - The id of the file.
        replicaID - The id of the replica.
        con - An open connection to the archive database
        Returns:
        Whether the replicafileinfo was there or not.
        Throws:
        IllegalState - If more than one copy of the replicafileinfo is placed in the database.

      • insertReplicaIntoDB

        protected static void insertReplicaIntoDB​(Replica rep,
                                          Connection con)
                                   throws IOFailure
        Method for inserting a Replica into the replica table. The replica_guid is automatically given by the database, and the values in the fields replica_id, replica_name and replica_type is created from the replica argument.
        Parameters:
        rep - The Replica to insert into the replica table.
        con - An open connection to the archive database
        Throws:
        IOFailure - If a SQLException is caught.

      • insertFileIntoDB

        protected static long insertFileIntoDB​(String filename,
                                       Connection connection)
                                throws IllegalState
        Method to create a new entry in the file table in the database. The file_id is automatically created by the database, and the argument is used for the filename for this new entry to the table. This will also create a replicafileinfo entry for each replica.
        Parameters:
        filename - The filename for the new entry in the file table.
        connection - An open connection to the archive database
        Returns:
        created file_id for the new entry.
        Throws:
        IllegalState - If the file cannot be inserted into the database.

      • createReplicaFileInfoEntriesInDB

        protected static void createReplicaFileInfoEntriesInDB​(long fileId,
                                                       Connection con)
                                                throws IllegalState
        When a new file is inserted into the database, each replica gets a new entry in the replicafileinfo table for this file. The fields for this new entry are set to the following: - file_id = argument. - replica_id = The id of the current replica. - filelist_status = NO_FILELIST_STATUS. - checksum_status = UNKNOWN. - upload_status = NO_UPLOAD_STATUS.

        The replicafileinfo_guid is automatically created by the database, and the dates are set to null.

        Parameters:
        fileId - The id for the file.
        con - An open connection to the archive database
        Throws:
        IllegalState - If the file could not be entered into the database.

      • retrieveIdsFromReplicaTable

        protected static List<String> retrieveIdsFromReplicaTable​(Connection con)
        Method for retrieving the replica IDs within the database.
        Parameters:
        con - An open connection to the archive database
        Returns:
        The list of replicaIds from the replica table in the database.

      • retrieveIdsFromFileTable

        protected static List<String> retrieveIdsFromFileTable​(Connection con)
        Method for retrieving all the file IDs within the database.
        Parameters:
        con - An open connection to the archive database
        Returns:
        The list of fileIds from the file table in the database.

      • retrieveIdsFromReplicaFileInfoTable

        protected static List<String> retrieveIdsFromReplicaFileInfoTable​(Connection con)
        Method for retrieving all the ReplicaFileInfo GUIDs within the database.
        Parameters:
        con - An open connection to the archive database
        Returns:
        A list of the replicafileinfo_guid for all entries in the replicafileinfo table.

      • retrieveIdForFile

        protected static long retrieveIdForFile​(String filename,
                                        Connection con)
        Retrieves the file_id for the corresponding filename. An error is thrown if no such file_id is found in the file table, and if more than one instance with the given name is found, then a warning is issued. If more than one is found, then it is logged and only the first is returned.
        Parameters:
        filename - The entry in the filename list where the corresponding file_id should be found.
        con - An open connection to the archive database
        Returns:
        The file_id for the file, or -1 if the file was not found.

      • retrieveReplicaFileInfoGuid

        protected static long retrieveReplicaFileInfoGuid​(long fileId,
                                                  String replicaId,
                                                  Connection con)
        Method for retrieving the replicafileinfo_guid for a specific instance defined from the fileId and the replicaId. If more than one is found, then it is logged and only the first is returned.
        Parameters:
        fileId - The identifier for the file.
        replicaId - The identifier for the replica.
        con - An open connection to the archive database
        Returns:
        The identifier for the replicafileinfo, or -1 if not found.

      • retrieveReplicaFileInfoGuidsForReplica

        protected static Set<Long> retrieveReplicaFileInfoGuidsForReplica​(String replicaId,
                                                                  Connection con)
        Method for retrieving the list of all the replicafileinfo_guids for a specific replica.
        Parameters:
        replicaId - The id for the replica to contain the files.
        con - An open connection to the archiveDatabase.
        Returns:
        The list of all the replicafileinfo_guid.

      • retrieveReplicaType

        protected static ReplicaType retrieveReplicaType​(String replicaId,
                                                 Connection con)
        Method for retrieving the replica type for a specific replica.
        Parameters:
        replicaId - The id of the replica.
        con - An open connection to the archiveDatabase.
        Returns:
        The type of the replica.

      • retrieveReplicaIdsWithOKChecksumStatus

        protected static List<String> retrieveReplicaIdsWithOKChecksumStatus​(String filename,
                                                                     Connection con)
        Method for retrieving the list of replica, where the file with the given name has the checksum_status 'OK'.
        Parameters:
        filename - The name of the file.
        con - An open connection to the archive database
        Returns:
        The list of replicas where the status for the checksum of the file is OK.

      • retrieveFilenameForFileId

        protected static String retrieveFilenameForFileId​(long fileId,
                                                  Connection con)
        Method for retrieving the filename from the entry in the file table which has the fileId as file_id.
        Parameters:
        fileId - The file_id of the entry in the file table for which to retrieve the filename.
        con - An open connection to the archive database
        Returns:
        The filename corresponding to the fileId in the file table.

      • retrieveFileListStatusFromReplicaFileInfo

        protected static int retrieveFileListStatusFromReplicaFileInfo​(String filename,
                                                               String replicaId,
                                                               Connection con)
        Method for retrieving the filelist_status for the entry in the replicafileinfo table associated with the given filename for the replica identified with a given id.
        Parameters:
        filename - the filename of the file for which you want a status.
        replicaId - The identifier of the replica
        con - An open connection to the archive database
        Returns:
        The above mentioned filelist_status of the file

      • updateReplicaFileInfoChecksum

        protected static void updateReplicaFileInfoChecksum​(long replicafileinfoId,
                                                    String checksum,
                                                    Connection con)
        This is used for updating a replicafileinfo instance based on the results of a checksumjob. Updates the following fields for the entry in the replicafileinfo:
        - checksum = checksum argument.
        - upload_status = completed.
        - filelist_status = ok.
        - checksum_status = UNKNOWN.
        - checksum_checkdatetime = now.
        - filelist_checkdatetime = now.
        Parameters:
        replicafileinfoId - The unique id for the replicafileinfo.
        checksum - The new checksum for the entry.
        con - An open connection to the archive database

      • updateReplicaFileInfoFilelist

        protected static void updateReplicaFileInfoFilelist​(long replicafileinfoId,
                                                    Connection con)
        Method for updating the filelist of a replicafileinfo instance. Updates the following fields for the entry in the replicafileinfo:
        filelist_status = OK.
        filelist_checkdatetime = current time.
        Parameters:
        replicafileinfoId - The id of the replicafileinfo.
        con - An open connection to the archive database

      • updateReplicaFileInfoMissingFromFilelist

        protected static void updateReplicaFileInfoMissingFromFilelist​(long replicafileinfoId,
                                                               Connection con)
        Method for updating the filelist of a replicafileinfo instance. Updates the following fields for the entry in the replicafileinfo:
        filelist_status = missing.
        filelist_checkdatetime = current time.

        The replicafileinfo is in the filelist.

        Parameters:
        replicafileinfoId - The id of the replicafileinfo.
        con - An open connection to the archive database

      • updateReplicaFileInfoChecksumCorrupt

        protected static void updateReplicaFileInfoChecksumCorrupt​(long replicafileinfoId,
                                                           Connection con)
        Method for updating the checksum status of a replicafileinfo instance. Updates the following fields for the entry in the replicafileinfo:
        checksum_status = CORRUPT.
        checksum_checkdatetime = current time.

        The replicafileinfo is in the filelist.

        Parameters:
        replicafileinfoId - The id of the replicafileinfo.
        con - An open connection to the archive database

      • retrieveGuidForFilenameOnReplica

        protected static long retrieveGuidForFilenameOnReplica​(String filename,
                                                       String replicaId,
                                                       Connection con)
        Retrieve the guid stored for a filename on a given replica.
        Parameters:
        filename - a given filename
        replicaId - An identifier for a replica.
        con - An open connection to the archive database
        Returns:
        the abovementioned guid.

      • updateReplicaFileInfoChecksumUnknown

        protected static void updateReplicaFileInfoChecksumUnknown​(long replicafileinfoId,
                                                           Connection con)
        Method for updating the checksum status of a replicafileinfo instance. Updates the following fields for the entry in the replicafileinfo:
        checksum_status = UNKNOWN.
        checksum_checkdatetime = current time.

        The replicafileinfo is in the filelist.

        Parameters:
        replicafileinfoId - The id of the replicafileinfo.
        con - An open connection to the archive database

      • updateReplicaFileInfoChecksumOk

        protected static void updateReplicaFileInfoChecksumOk​(long replicafileinfoId,
                                                      Connection con)
        Method for updating the checksum status of a replicafileinfo instance. Updates the following fields for the entry in the replicafileinfo:
        checksum_status = OK.
        upload_status = UPLOAD_COMLPETE.
        checksum_checkdatetime = current time.

        The file is required to exist in the replica.
        Parameters:
        replicafileinfoId - The id of the replicafileinfo.
        con - An open connection to the archive database

      • updateChecksumDateForReplica

        protected static void updateChecksumDateForReplica​(Replica rep,
                                                   Connection con)
        Method for updating the checksum_updated field for a given replica in the replica table. This is called when a checksum_job has been handled.

        The following fields for the entry in the replica table:
        checksum_updated = now.

        Parameters:
        rep - The replica which has just been updated.
        con - An open connection to the archive database

      • updateFilelistDateForReplica

        protected static void updateFilelistDateForReplica​(Replica rep,
                                                   Connection connection)
        Method for updating the filelist_updated field for a given replica in the replica table. This is called when a filelist_job or a checksum_job has been handled.

        The following fields for the entry in the replica table:
        filelist_updated = now.

        Parameters:
        rep - The replica which has just been updated.
        connection - An open connection to the archive database

      • setFilelistDateForReplica

        protected static void setFilelistDateForReplica​(Replica rep,
                                                Date date,
                                                Connection con)
        Method for setting the filelist_updated field for a given replica in the replica table to a specified value. This is only called when the admin.data is converted.

        The following fields for the entry in the replica table:
        filelist_updated = date.

        Parameters:
        rep - The replica which has just been updated.
        date - The date for the last filelist update.
        con - An open connection to the archive database

      • setChecksumlistDateForReplica

        protected static void setChecksumlistDateForReplica​(Replica rep,
                                                    Date date,
                                                    Connection con)
        Method for setting the checksum_updated field for a given replica in the replica table to a specified value. This is only called when the admin.data is converted.

        The following fields for the entry in the replica table:
        checksum_updated = date.

        Parameters:
        rep - The replica which has just been updated.
        date - The date for the last checksum update.
        con - An open connection to the archive database

      • existsReplicaInDB

        protected static boolean existsReplicaInDB​(Replica rep,
                                           Connection con)
        Method for testing whether a replica already is within the database.
        Parameters:
        rep - The replica to find in the database.
        con - An open connection to the archive database
        Returns:
        Whether the replica is found in the database.

      • getReplicaFileInfo

        protected static ReplicaFileInfo getReplicaFileInfo​(long replicaFileInfoGuid,
                                                    Connection con)
        Method for retrieving ReplicaFileInfo entry in the database.
        Parameters:
        replicaFileInfoGuid - The guid for the specific replicafileinfo.
        con - An open connection to the archive database
        Returns:
        The replicafileinfo.

      • retrieveReplicaFileInfosWithChecksum

        protected static List<ReplicaFileInfo> retrieveReplicaFileInfosWithChecksum​(List<Long> rfiGuids,
                                                                            Connection con)
        Method for retrieving the data for the wanted entries in the replicafileinfo table. All the replicafileinfo entries with no checksum defined is ignored.
        Parameters:
        rfiGuids - The list of guids for the entries in the replicafileinfo table which is wanted.
        con - An open connection to the archive database
        Returns:
        The complete data for these entries in the replicafileinfo table.

      • updateReplicaFileInfo

        protected static void updateReplicaFileInfo​(long replicafileinfoGuid,
                                            String checksum,
                                            ReplicaStoreState state,
                                            Connection con)
                                     throws IOFailure
        Method for updating an entry in the replicafileinfo table. This method does not update the 'checksum_checkdatetime' and 'filelist_checkdatetime'.
        Parameters:
        replicafileinfoGuid - The guid to update.
        checksum - The new checksum for the entry.
        state - The state for the upload.
        con - An open connection to the archive database
        Throws:
        IOFailure - If an error occurs in the database connection.

      • updateReplicaFileInfo

        protected static void updateReplicaFileInfo​(long replicafileinfoGuid,
                                            String checksum,
                                            Date date,
                                            ReplicaStoreState state,
                                            Connection con)
                                     throws IOFailure
        Method for updating an entry in the replicafileinfo table. This method updates the 'checksum_checkdatetime' and 'filelist_checkdatetime' with the given date argument.
        Parameters:
        replicafileinfoGuid - The guid to update.
        checksum - The new checksum for the entry.
        date - The date for the update.
        state - The status for the upload.
        con - An open connection to the archive database
        Throws:
        IOFailure - If an error occurs in the connection to the database.

      • retrieveUploadStatus

        protected static ReplicaStoreState retrieveUploadStatus​(long fileGuid,
                                                        String repId,
                                                        Connection con)
        Retrieves the UploadStatus for a specific entry in the replicafileinfo table identified by the file guid and the replica id.
        Parameters:
        fileGuid - The id of the file.
        repId - The id of the replica.
        con - An open connection to the archive database
        Returns:
        The upload status of the corresponding replicafileinfo entry.

      • retrieveChecksumForReplicaFileInfoEntry

        protected static String retrieveChecksumForReplicaFileInfoEntry​(long fileGuid,
                                                                String repId,
                                                                Connection con)
        Retrieves the checksum for a specific entry in the replicafileinfo table identified by the file guid and the replica id.
        Parameters:
        fileGuid - The guid of the file in the file table.
        repId - The id of the replica.
        con - An open connection to the archive database
        Returns:
        The checksum of the corresponding replicafileinfo entry.

      • retrieveChecksumStatusForReplicaFileInfoEntry

        protected static ChecksumStatus retrieveChecksumStatusForReplicaFileInfoEntry​(long fileGuid,
                                                                              String repId,
                                                                              Connection con)
        Retrieves the checksum status for a specific entry in the replicafileinfo table identified by the file guid and the replica id.
        Parameters:
        fileGuid - The guid of the file in the file table.
        repId - The id of the replica.
        con - An open connection to the archive database
        Returns:
        The checksum status of the corresponding replicafileinfo entry.

      • vote

        protected static String vote​(List<String> checksums)
        Method for finding the checksum which is present most times in the list.
        Parameters:
        checksums - The list of checksum to vote about.
        Returns:
        The most common checksum, or null if several exists.

      • fileChecksumVote

        protected static void fileChecksumVote​(long fileId,
                                       Connection con)
        The method for voting about the checksum of a file.
        Each entry in the replicafileinfo table containing the file is retrieved. All the unique checksums are retrieved, e.g. if a checksum is found more than one, then it is ignored.
        If only one unique checksum is found, then if must be the correct one, and all the replicas with this file will have their checksum_status set to 'OK'.
        If more than one checksum is found, then a vote for the correct checksum is performed. This is done by counting the amount of time each of the unique checksum is found among the replicafileinfo entries for the current file. The checksum with most votes is chosen as the correct one, and the checksum_status for all the replicafileinfo entries with this checksum is set to 'OK', whereas the replicafileinfo entries with a different checksum is set to 'CORRUPT'.
        If no winner is found then a warning and a notification is issued, and the checksum_status for all the replicafileinfo entries with for the current file is set to 'UNKNOWN'.
        Parameters:
        fileId - The id for the file to vote about.
        con - An open connection to the archive database

      • addFileInformation

        protected static long addFileInformation​(String file,
                                         Replica replica,
                                         Connection con)
        Add information about one file in a given replica.
        Parameters:
        file - The name of a file
        replica - A replica
        con - An open connection to the ArchiveDatabase
        Returns:
        the ReplicaFileInfo ID for the given filename and replica in the database

      • processChecksumline

        public static long processChecksumline​(String filename,
                                       String checksum,
                                       Replica replica,
                                       Connection con)
        Process checksum information about one file in a given replica. and update the database accordingly.
        Parameters:
        filename - The name of a file
        checksum - The checksum of that file.
        replica - A replica
        con - An open connection to the ArchiveDatabase
        Returns:
        the ReplicaFileInfo ID for the given filename and replica in the database