dk.netarkivet.harvester.harvesting.controller
Class JMXHeritrixController

java.lang.Object
  dk.netarkivet.harvester.harvesting.controller.AbstractJMXHeritrixController
      dk.netarkivet.harvester.harvesting.controller.JMXHeritrixController

All Implemented Interfaces:

HeritrixController

Deprecated. Use the BnfHeritrixController instead

public class JMXHeritrixController
extends AbstractJMXHeritrixController

This implementation of the HeritrixController interface starts Heritrix as a separate process and uses JMX to communicate with it. Each instance executes exactly one process that runs exactly one crawl job.

Constructor Summary

JMXHeritrixController(HeritrixFiles files)
Deprecated. Create a JMXHeritrixController object.

Method Summary

boolean	atFinish() Deprecated. Query whether Heritrix is in a state where it can finish crawling.
void	beginCrawlStop() Deprecated. Tell Heritrix to stop crawling.
void	cleanup() Deprecated. Cleanup after an Heritrix process.
boolean	crawlIsEnded() Deprecated. Check if the crawl has ended, either because Heritrix finished of its own, or because we terminated it.
int	getActiveToeCount() Deprecated. Get the number of currently active ToeThreads (crawler threads).
int	getCurrentProcessedKBPerSec() Deprecated. Get an estimate of the rate, in kb, at which documents are currently being processed by the crawler.
java.lang.String	getHarvestInformation() Deprecated. Return the URL for monitoring this instance.
java.lang.String	getProgressStats() Deprecated. Get a human-readable set of statistics on the progress of the crawl.
long	getQueuedUriCount() Deprecated. Get the number of URIs currently on the queue to be processed.
void	initialize() Deprecated. Initialize a new CrawlController for executing a Heritrix crawl.
boolean	isPaused() Deprecated. Returns true if the crawler has been paused, and thus not supposed to fetch anything.
void	requestCrawlStart() Deprecated. Request that Heritrix start crawling.
void	requestCrawlStop(java.lang.String reason) Deprecated. Request that crawling stops.

Methods inherited from class dk.netarkivet.harvester.harvesting.controller.AbstractJMXHeritrixController

getGuiPort, getHeritrixFiles, getHostName, getJmxPort, getJobDescription, processHasExited, toString, waitForHeritrixProcessExit

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

JMXHeritrixController

public JMXHeritrixController(HeritrixFiles files)

Deprecated.

Create a JMXHeritrixController object.

Parameters:

files - Files that are used to set up Heritrix.

Method Detail

initialize

public void initialize()

Deprecated.

Description copied from interface: HeritrixController

Initialize a new CrawlController for executing a Heritrix crawl. This does not start the crawl.

Throws:

IOFailure - If Heritrix dies before initialization, or we encounter any problems during the initialization.

See Also:

HeritrixController.initialize()

requestCrawlStart

public void requestCrawlStart()

Deprecated.

Description copied from interface: HeritrixController

Request that Heritrix start crawling. When this method returns, either Heritrix has failed in the early stages, or the crawljob has been successfully created. Actual crawling will commence at some point hereafter.

Throws:

IOFailure - if unable to communicate with Heritrix

See Also:

HeritrixController.requestCrawlStart()

atFinish

public boolean atFinish()

Deprecated.

Description copied from interface: HeritrixController

Query whether Heritrix is in a state where it can finish crawling. Returns true if no uris remain to be harvested, or it has met either the maxbytes limit, the document limit, or the time-limit for the current harvest.

Returns:

True if Heritrix thinks it is time to stop crawling.

See Also:

HeritrixController.atFinish()

beginCrawlStop

public void beginCrawlStop()

Deprecated.

Description copied from interface: HeritrixController

Tell Heritrix to stop crawling. Heritrix may take a while to actually stop, so you cannot assume that crawling is stopped when this method returns.

Throws:

IOFailure - if unable to communicate with Heritrix

See Also:

HeritrixController.beginCrawlStop()

getActiveToeCount

public int getActiveToeCount()

Deprecated.

Description copied from interface: HeritrixController

Get the number of currently active ToeThreads (crawler threads).

Returns:

Number of ToeThreads currently active within Heritrix.

See Also:

HeritrixController.getActiveToeCount()

requestCrawlStop

public void requestCrawlStop(java.lang.String reason)

Deprecated.

Description copied from interface: HeritrixController

Request that crawling stops. This makes a call to beginCrawlStop(), unless the crawler is already shutting down. In that case it does nothing.

Parameters:

reason - A human-readable reason the crawl is being stopped.

See Also:

HeritrixController.requestCrawlStop(String)

getQueuedUriCount

public long getQueuedUriCount()

Deprecated.

Description copied from interface: HeritrixController

Get the number of URIs currently on the queue to be processed. This number may not be exact and should only be used in informal texts.

Returns:

How many URIs Heritrix have lined up for processing.

See Also:

HeritrixController.getQueuedUriCount()

getCurrentProcessedKBPerSec

public int getCurrentProcessedKBPerSec()

Deprecated.

Description copied from interface: HeritrixController

Get an estimate of the rate, in kb, at which documents are currently being processed by the crawler.

Returns:

Number of KB data downloaded by Heritrix over an undefined interval up to now.

See Also:

HeritrixController.getCurrentProcessedKBPerSec()

getProgressStats

public java.lang.String getProgressStats()

Deprecated.

Description copied from interface: HeritrixController

Get a human-readable set of statistics on the progress of the crawl. The statistics is discovered uris, queued uris, downloaded uris, doc/s(avg), KB/s(avg), dl-failures, busy-thread, mem-use-KB, heap-size-KB, congestion, max-depth, avg-depth. If no statistics are available, the string "No statistics available" is returned. Note: this method may disappear in the future.

Returns:

Some ascii-formatted statistics on the progress of the crawl.

See Also:

HeritrixController.getProgressStats()

isPaused

public boolean isPaused()

Deprecated.

Description copied from interface: HeritrixController

Returns true if the crawler has been paused, and thus not supposed to fetch anything. Heritrix may still be fetching stuff, as it takes some time for it to go into full pause mode. This method can be used as an indicator that we should not be worried if Heritrix appears to be idle.

Returns:

True if the crawler has been paused, e.g. by using the Heritrix GUI.

See Also:

HeritrixController.isPaused()

crawlIsEnded

public boolean crawlIsEnded()

Deprecated.

Check if the crawl has ended, either because Heritrix finished of its own, or because we terminated it.

Returns:

True if the crawl has ended, either because Heritrix finished or because we terminated it. Otherwise we return false.

See Also:

HeritrixController.crawlIsEnded()

cleanup

public void cleanup()

Deprecated.

Cleanup after an Heritrix process. This entails sending the shutdown command to the Heritrix process, and killing it forcefully, if it is still alive after waiting the period of time specified by the CommonSettings.PROCESS_TIMEOUT setting.

See Also:

HeritrixController.cleanup()

getHarvestInformation

public java.lang.String getHarvestInformation()

Deprecated.

Return the URL for monitoring this instance.

Returns:

the URL for monitoring this instance.