Package dk.netarkivet.harvester.indexserver

Class CDXIndexCache

  • All Implemented Interfaces:
    JobIndexCache
    public class CDXIndexCache
extends CombiningMultiFileBasedCache<Long>
implements JobIndexCache
    A cache that serves CDX index files for job IDs.

    Notice that since data for some IDs may not be available, the actual cached file might not correspond in its content to what was asked for. For instance, if asking for data for IDs 2, 3, and 4, and 3 fails, a cached file for IDs 2 and 4 will be returned. There is currently no way to tell if you got everything you asked for.

    This cache uses the Unix sort(1) command as an external process call, as that one is optimized for handling large, disk-based sorts.

    • Constructor Detail

      • CDXIndexCache

        public CDXIndexCache()
        Creates a new cache for CDX index files.

    • Method Detail

      • combine

        protected void combine​(Map<Long,​File> filesFound)
        Combine parts of an index into one big index.
        Specified by:
        combine in class CombiningMultiFileBasedCache<Long>
        Parameters:
        filesFound - A map of IDs and the files caching their content.

      • requestIndex

        public void requestIndex​(Set<Long> jobSet,
                         Long harvestId)
        Description copied from interface: JobIndexCache
        Request an index from the indexserver. Prepare the index but don't give it to me.
        Specified by:
        requestIndex in interface JobIndexCache
        Parameters:
        jobSet - Set of job IDs to generate index for.
        harvestId - Harvestdefinition associated with this set of jobs