dk.netarkivet.wayback.aggregator
Class AggregationWorker

java.lang.Object
  dk.netarkivet.wayback.aggregator.AggregationWorker

All Implemented Interfaces:

CleanupIF

public class AggregationWorker
extends java.lang.Object
implements CleanupIF

The AggregationWorker singleton contains the schedule and file bookkeeping functionality needed in the aggregation of indexes. The AggregationWorker has the responsibility of ensuring each index in the raw index files ends up appearing exactly once in the index files used by Wayback. If this isn't possible the fallback is to allow duplicate occurrences of index lines ensuring index lines appears at least once.

Field Summary

static java.io.File	FINAL_INDEX_FILE The final Wayback index file currently used to intermediate indexes into.
(package private) static java.io.File	indexOutputDir See WaybackSettings.WAYBACK_AGGREGATOR_OUTPUT_DIR.
static java.io.File	INTERMEDIATE_INDEX_FILE The intermediate Wayback index file currently used to merge new indexes into.
static java.io.File	TEMP_FILE_INDEX The Files to store sorted indexes until they have been merge into a intermediate index files.
(package private) static java.io.File	tempFinalIndexFile The file to use for creating temporary final index file, which subsequent are merge into the working final index file.
(package private) static java.io.File	tempIntermediateIndexFile The file to use for creating temporary intermediate index file, which subsequent are merge into the final intermediate index file.

Method Summary

void	cleanup() Used to clean up a class from within a shutdown hook.
static AggregationWorker	getInstance() Factory method which creates a singleton aggregator and sets it running.
protected void	initialize() Creates the needed working directories.
protected void	runAggregation() Runs the actual aggregation.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

indexOutputDir

static java.io.File indexOutputDir

See WaybackSettings.WAYBACK_AGGREGATOR_OUTPUT_DIR.

tempIntermediateIndexFile

static java.io.File tempIntermediateIndexFile

The file to use for creating temporary intermediate index file, which subsequent are merge into the final intermediate index file.

tempFinalIndexFile

static java.io.File tempFinalIndexFile

The file to use for creating temporary final index file, which subsequent are merge into the working final index file.

TEMP_FILE_INDEX

public static final java.io.File TEMP_FILE_INDEX

The Files to store sorted indexes until they have been merge into a intermediate index files.

INTERMEDIATE_INDEX_FILE

public static final java.io.File INTERMEDIATE_INDEX_FILE

The intermediate Wayback index file currently used to merge new indexes into. If the intermediate files size exceeds the WaybackSettings#WAYBACK_AGGREGATOR_INTERMEDIATE_INDEX_FILE_SIZE_LIMIT

FINAL_INDEX_FILE

public static final java.io.File FINAL_INDEX_FILE

The final Wayback index file currently used to intermediate indexes into. A new working file is created and used, when the current file size + new indexes would exceed WaybackSettings#WAYBACK_AGGREGATOR_FINAL_INDEX_FILE_SIZE_LIMIT

Method Detail

getInstance

public static AggregationWorker getInstance()

Factory method which creates a singleton aggregator and sets it running. It has the side effect of creating the output directories for the indexer if these do not already exist. A temp directory is create if it doesn't exist. The aggregator won't run if a temp directory is already present, as this might indicate an instance of the aggregator already running.

Returns:

the indexer.

runAggregation

protected void runAggregation()

Runs the actual aggregation. See package description for details. Is synchronized so several subsequent scheduled runs of the method will have to run one at a time.

cleanup

public void cleanup()

Description copied from interface: CleanupIF

Used to clean up a class from within a shutdown hook. Must not do any logging. Program defensively, please.

Specified by:

cleanup in interface CleanupIF

initialize

protected void initialize()

Creates the needed working directories. Also checks whether a temp directory exists, which might be an indication of a unclean shutdown.