Package dk.netarkivet.wayback.aggregator

Class AggregationWorker

java.lang.Object

dk.netarkivet.wayback.aggregator.AggregationWorker

All Implemented Interfaces:

CleanupIF

public class AggregationWorker
extends Object
implements CleanupIF

The AggregationWorker singleton contains the schedule and file bookkeeping functionality needed in the aggregation of indexes.

The AggregationWorker has the responsibility of ensuring each index in the raw index files ends up appearing exactly once in the index files used by Wayback. If this isn't possible the fallback is to allow duplicate occurrences of index lines ensuring index lines appears at least once.

Field Summary

Fields

Modifier and Type	Field	Description
static File	FINAL_INDEX_FILE	The final Wayback index file currently used to intermediate indexes into.
static File	INTERMEDIATE_INDEX_FILE	The intermediate Wayback index file currently used to merge new indexes into.
static File	TEMP_FILE_INDEX	The Files to store sorted indexes until they have been merge into a intermediate index files.

Method Summary

Modifier and Type	Method	Description
void	cleanup()	Used to clean up a class from within a shutdown hook.
static AggregationWorker	getInstance()	Factory method which creates a singleton aggregator and sets it running.
protected void	initialize()	Creates the needed working directories.
protected void	runAggregation()	Runs the actual aggregation.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

TEMP_FILE_INDEX

public static final File TEMP_FILE_INDEX

The Files to store sorted indexes until they have been merge into a intermediate index files.

INTERMEDIATE_INDEX_FILE

public static final File INTERMEDIATE_INDEX_FILE

The intermediate Wayback index file currently used to merge new indexes into. If the intermediate files size exceeds the WaybackSettings#WAYBACK_AGGREGATOR_INTERMEDIATE_INDEX_FILE_SIZE_LIMIT

FINAL_INDEX_FILE

public static final File FINAL_INDEX_FILE

The final Wayback index file currently used to intermediate indexes into. A new working file is created and used, when the current file size + new indexes would exceed WaybackSettings#WAYBACK_AGGREGATOR_FINAL_INDEX_FILE_SIZE_LIMIT

Method Detail

getInstance

public static AggregationWorker getInstance()

Factory method which creates a singleton aggregator and sets it running. It has the side effect of creating the output directories for the indexer if these do not already exist.

A temp directory is create if it doesn't exist. The aggregator won't run if a temp directory is already present, as this might indicate an instance of the aggregator already running.

Returns:

the indexer.

runAggregation

protected void runAggregation()

Runs the actual aggregation. See package description for details.

Is synchronized so several subsequent scheduled runs of the method will have to run one at a time.

cleanup

public void cleanup()

Description copied from interface: CleanupIF

Used to clean up a class from within a shutdown hook. Must not do any logging. Program defensively, please.

Specified by:

cleanup in interface CleanupIF

initialize

protected void initialize()

Creates the needed working directories. Also checks whether a temp directory exists, which might be an indication of a unclean shutdown.