Package | Description |
---|---|
dk.netarkivet.harvester.datamodel | |
dk.netarkivet.harvester.harvesting |
This module handles defining, scheduling, and execution of harvests.
|
dk.netarkivet.harvester.harvesting.distribute | |
dk.netarkivet.harvester.harvesting.report | |
dk.netarkivet.harvester.heritrix3 |
This module handles defining, scheduling, and execution of harvests.
|
dk.netarkivet.harvester.scheduler | |
dk.netarkivet.harvester.scheduler.jobgen | |
dk.netarkivet.harvester.webinterface.servlet |
Modifier and Type | Method and Description |
---|---|
Job |
JobDBDAO.read(long jobID)
Read a single job from the job database.
|
abstract Job |
JobDAO.read(long jobID)
Reads a job from persistent storage.
|
Modifier and Type | Method and Description |
---|---|
Iterator<Job> |
JobDBDAO.getAll()
Return a list of all jobs.
|
abstract Iterator<Job> |
JobDAO.getAll()
Return a list of all jobs .
|
Iterator<Job> |
JobDBDAO.getAll(JobStatus status)
Return a list of all jobs with the given status, ordered by id.
|
abstract Iterator<Job> |
JobDAO.getAll(JobStatus status)
Return a list of all jobs with the given status.
|
Iterator<Job> |
JobDAO.iterator()
Gets an iterator of all jobs.
|
Modifier and Type | Method and Description |
---|---|
void |
JobDBDAO.create(Job job)
Creates an instance in persistent storage of the given job.
|
abstract void |
JobDAO.create(Job job)
Creates an instance in persistent storage of the given job.
|
abstract HarvestInfo |
DomainDAO.getDomainJobInfo(Job job,
String domainName,
String configName)
Get the HarvestInfo object for a certain job and DomainConfiguration defined by domainName and configName.
|
HarvestInfo |
DomainDBDAO.getDomainJobInfo(Job j,
String domainName,
String configName) |
List<AliasInfo> |
JobDBDAO.getJobAliasInfo(Job job)
Get a list of AliasInfo objects for all the domains included in the job.
|
abstract List<AliasInfo> |
JobDAO.getJobAliasInfo(Job job)
Get a list of AliasInfo objects for all the domains included in the job.
|
void |
H1HeritrixTemplate.insertWarcInfoMetadata(Job ajob,
String origHarvestdefinitionName,
String scheduleName,
String performer) |
void |
H3HeritrixTemplate.insertWarcInfoMetadata(Job ajob,
String origHarvestdefinitionName,
String scheduleName,
String performer) |
abstract void |
HeritrixTemplate.insertWarcInfoMetadata(Job ajob,
String origHarvestdefinitionName,
String scheduleName,
String performer)
Method to add settings to the WARCWriterProcesser, so that it can generate a proper WARCINFO record.
|
void |
JobDBDAO.update(Job job)
Update a Job in persistent storage.
|
abstract void |
JobDAO.update(Job job)
Update a Job in persistent storage.
|
Modifier and Type | Method and Description |
---|---|
String |
ArchiveFileNaming.getPrefix(Job job)
Make a prefix to be used by Heritrix.
|
String |
CollectionPrefixNamingConvention.getPrefix(Job theJob) |
String |
LegacyNamingConvention.getPrefix(Job theJob) |
void |
PersistentJobData.write(Job harvestJob,
HarvestDefinitionInfo hdi)
Write information about given Job to XML-structure.
|
HeritrixFiles |
HarvestController.writeHarvestFiles(File crawldir,
Job job,
HarvestDefinitionInfo hdi,
List<MetadataEntry> metadataEntries)
Writes the files involved with a harvests.
|
Modifier and Type | Method and Description |
---|---|
Job |
DoOneCrawlMessage.getJob() |
Constructor and Description |
---|
DoOneCrawlMessage(Job submittedJob,
ChannelID to,
HarvestDefinitionInfo harvestInfo,
List<MetadataEntry> metadata)
A NetarkivetMessage that contains a Job for Heritrix.
|
Modifier and Type | Method and Description |
---|---|
void |
BnfHarvestReport.postProcess(Job job)
Post-processing happens on the scheduler side when ARC files have been uploaded.
|
void |
HarvestReport.postProcess(Job job)
Post-processing happens on the scheduler side when ARC files have been uploaded.
|
void |
LegacyHarvestReport.postProcess(Job job)
Post-processing happens on the scheduler side when ARC files have been uploaded.
|
Modifier and Type | Method and Description |
---|---|
static Heritrix3Files |
Heritrix3Files.getH3HeritrixFiles(File crawldir,
Job job) |
void |
HarvestJob.init(Job job,
HarvestDefinitionInfo origHarvestInfo,
List<MetadataEntry> metadataEntries)
Initialization of the harvestJob.
|
Heritrix3Files |
HarvestJob.writeHarvestFiles(File crawldir,
Job job,
HarvestDefinitionInfo hdi,
List<MetadataEntry> metadataEntries)
Writes the files needed to start a harvest..
|
Modifier and Type | Method and Description |
---|---|
void |
JobDispatcher.doOneCrawl(Job job,
String origHarvestName,
String origHarvestDesc,
String origHarvestSchedule,
HarvestChannel channel,
String origHarvestAudience,
List<MetadataEntry> metadata)
Submit an doOneCrawl request to a HarvestControllerServer.
|
Modifier and Type | Method and Description |
---|---|
boolean |
JobGenerator.canAccept(Job job,
DomainConfiguration cfg,
DomainConfiguration previousCfg)
Tests if a configuration fits into this Job.
|
protected boolean |
FixedDomainConfigurationCountJobGenerator.checkSpecificAcceptConditions(Job job,
DomainConfiguration cfg) |
protected boolean |
DefaultJobGenerator.checkSpecificAcceptConditions(Job job,
DomainConfiguration cfg) |
Modifier and Type | Field and Description |
---|---|
Job |
Heritrix3JobMonitor.job |
Copyright © 2005–2016 The Royal Danish Library, the Danish State and University Library, the National Library of France and the Austrian National Library.. All rights reserved.