dk.netarkivet.harvester.datamodel
Class DBSpecifics

java.lang.Object
  dk.netarkivet.common.utils.SettingsFactory<DBSpecifics>
      dk.netarkivet.harvester.datamodel.DBSpecifics

Direct Known Subclasses:

DerbySpecifics, MySQLSpecifics, PostgreSQLSpecifics

public abstract class DBSpecifics
extends SettingsFactory<DBSpecifics>

Defines database specific implementations used by the Harvester. The actual actual implementation which is loaded is defined by the CommonSettings.DB_SPECIFICS_CLASS setting. See the sub class list for available implementations

Field Summary

protected static java.lang.String	CONFIGPASSWORDS_TABLE
protected static int	CONFIGPASSWORDS_TABLE_REQUIRED_VERSION
protected static java.lang.String	CONFIGSEEDLISTS_TABLE
protected static int	CONFIGSEEDLISTS_TABLE_REQUIRED_VERSION
protected static java.lang.String	CONFIGURATIONS_TABLE
protected static int	CONFIGURATIONS_TABLE_REQUIRED_VERSION
protected static java.lang.String	DOMAINS_TABLE
protected static int	DOMAINS_TABLE_REQUIRED_VERSION
protected static java.lang.String	EXTENDEDFIELD_TABLE
protected static int	EXTENDEDFIELD_TABLE_REQUIRED_VERSION
protected static java.lang.String	EXTENDEDFIELDTYPE_TABLE
protected static int	EXTENDEDFIELDTYPE_TABLE_REQUIRED_VERSION
protected static java.lang.String	EXTENDEDFIELDVALUE_TABLE
protected static int	EXTENDEDFIELDVALUE_TABLE_REQUIRED_VERSION
protected static java.lang.String	FRONTIERREPORTMONITOR_TABLE
protected static int	FRONTIERREPORTMONITOR_TABLE_REQUIRED_VERSION
protected static java.lang.String	FULLHARVESTS_TABLE
protected static int	FULLHARVESTS_TABLE_REQUIRED_VERSION
protected static java.lang.String	GLOBALCRAWLERTRAPEXPRESSIONS_TABLE
protected static int	GLOBALCRAWLERTRAPEXPRESSIONS_TABLE_REQUIRED_VERSION
protected static java.lang.String	GLOBALCRAWLERTRAPLISTS_TABLE
protected static int	GLOBALCRAWLERTRAPLISTS_TABLE_REQUIRED_VERSION
protected static java.lang.String	HARVESTCONFIGS_TABLE
protected static int	HARVESTCONFIGS_TABLE_REQUIRED_VERSION
protected static java.lang.String	HARVESTDEFINITIONS_TABLE
protected static int	HARVESTDEFINITIONS_TABLE_REQUIRED_VERSION
protected static java.lang.String	HISTORYINFO_TABLE
protected static int	HISTORYINFO_TABLE_REQUIRED_VERSION
protected static java.lang.String	JOBCONFIGS_TABLE
protected static int	JOBCONFIGS_TABLE_REQUIRED_VERSION
protected static java.lang.String	JOBS_TABLE
protected static int	JOBS_TABLE_REQUIRED_VERSION
(package private) org.apache.commons.logging.Log	log
protected static java.lang.String	ORDERTEMPLATES_TABLE
protected static int	ORDERTEMPLATES_TABLE_REQUIRED_VERSION
protected static java.lang.String	OWNERINFO_TABLE
protected static int	OWNERINFO_TABLE_REQUIRED_VERSION
protected static java.lang.String	PARTIALHARVESTS_TABLE
protected static int	PARTIALHARVESTS_TABLE_REQUIRED_VERSION
protected static java.lang.String	PASSWORDS_TABLE
protected static int	PASSWORDS_TABLE_REQUIRED_VERSION
protected static java.lang.String	RUNNINGJOBSHISTORY_TABLE
protected static int	RUNNINGJOBSHISTORY_TABLE_REQUIRED_VERSION
protected static java.lang.String	RUNNINGJOBSMONITOR_TABLE
protected static int	RUNNINGJOBSMONITOR_TABLE_REQUIRED_VERSION
protected static java.lang.String	SCHEDULES_TABLE
protected static int	SCHEDULES_TABLE_REQUIRED_VERSION
protected static java.lang.String	SEEDLISTS_TABLE
protected static int	SEEDLISTS_TABLE_REQUIRED_VERSION

Constructor Summary

DBSpecifics()

Method Summary

protected abstract void	createExtendedFieldTable() Create the extendedfield table in the database.
protected abstract void	createExtendedFieldTypeTable() Create the extendedfieldtype table in the database.
protected abstract void	createExtendedFieldValueTable() Create the extendedfieldvalue table in the database.
abstract void	createFrontierReportMonitorTable() Create the frontierReportMonitor table in the database.
protected abstract void	createGlobalCrawlerTrapExpressions() Creates the initial (version 1) of table 'global_crawler_trap_expressions'.
protected abstract void	createGlobalCrawlerTrapLists() Creates the initial (version 1) of table 'global_crawler_trap_lists'.
abstract void	createRunningJobsHistoryTable() Create the frontierReportMonitor table in the database.
abstract void	createRunningJobsMonitorTable() Create the frontierReportMonitor table in the database.
abstract void	dropJobConfigsTmpTable(java.sql.Connection c, java.lang.String tableName) Dispose of a temporary table gotten with getTemporaryTable.
abstract java.lang.String	getDriverClassName() Get the name of the JDBC driver class that handles interfacing to this server.
static DBSpecifics	getInstance() Get the singleton instance of the DBSpecifics implementation class.
abstract java.lang.String	getJobConfigsTmpTable(java.sql.Connection c) Get a temporary table for short-time use.
abstract java.lang.String	getOrderByLimitAndOffsetSubClause(long limit, long offset) Formats the LIMIT sub-clause of an SQL order clause.
protected abstract void	migrateConfigurationsv3ov4() Migrates the 'configurations' table from version 3 to version 4.
protected abstract void	migrateConfigurationsv4tov5() Migrates the 'configurations' table from version 4 to version 5.
protected abstract void	migrateFullharvestsv2tov3() Migrates the 'fullharvests' table from version 2 to version 3.
protected abstract void	migrateFullharvestsv3tov4() Migrates the 'fullharvests' table from version 3 to version 4.
protected abstract void	migrateFullharvestsv4tov5() Migrates the 'fullharvests' table from version 4 to version 5.
protected abstract void	migrateJobsv3tov4() Migrates the 'jobs' table from version 3 to version 4 consisting of a change of the field forcemaxbytes from int to bigint and setting its default to -1.
protected abstract void	migrateJobsv4tov5() Migrates the 'jobs' table from version 4 to version 5 consisting of adding new fields 'resubmitted_as_job' and 'submittedDate'.
protected abstract void	migrateJobsv5tov6() Migrates the 'jobs' table from version 5 to version 6.
protected abstract void	migrateRunningJobsHistoryTableV1ToV2() Migrates the 'runningjobshistory' table from version 1 to version 2.
protected abstract void	migrateRunningJobsMonitorTableV1ToV2() Migrates the 'runningjobsmonitor' table from version 1 to version 2.
abstract boolean	supportsClob() Returns true if the target RDBMS supports CLOBs.
void	updateTable(java.lang.String tableName, int toVersion) Update a table to a newer version, if necessary.

Methods inherited from class dk.netarkivet.common.utils.SettingsFactory

getInstance

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

log

org.apache.commons.logging.Log log

DOMAINS_TABLE

protected static final java.lang.String DOMAINS_TABLE

See Also:

Constant Field Values

DOMAINS_TABLE_REQUIRED_VERSION

protected static final int DOMAINS_TABLE_REQUIRED_VERSION

See Also:

Constant Field Values

CONFIGURATIONS_TABLE

protected static final java.lang.String CONFIGURATIONS_TABLE

See Also:

Constant Field Values

CONFIGURATIONS_TABLE_REQUIRED_VERSION

protected static final int CONFIGURATIONS_TABLE_REQUIRED_VERSION

See Also:

Constant Field Values

SEEDLISTS_TABLE

protected static final java.lang.String SEEDLISTS_TABLE

See Also:

Constant Field Values

SEEDLISTS_TABLE_REQUIRED_VERSION

protected static final int SEEDLISTS_TABLE_REQUIRED_VERSION

See Also:

Constant Field Values

PASSWORDS_TABLE

protected static final java.lang.String PASSWORDS_TABLE

See Also:

Constant Field Values

PASSWORDS_TABLE_REQUIRED_VERSION

protected static final int PASSWORDS_TABLE_REQUIRED_VERSION

See Also:

Constant Field Values

OWNERINFO_TABLE

protected static final java.lang.String OWNERINFO_TABLE

See Also:

Constant Field Values

OWNERINFO_TABLE_REQUIRED_VERSION

protected static final int OWNERINFO_TABLE_REQUIRED_VERSION

See Also:

Constant Field Values

HISTORYINFO_TABLE

protected static final java.lang.String HISTORYINFO_TABLE

See Also:

Constant Field Values

HISTORYINFO_TABLE_REQUIRED_VERSION

protected static final int HISTORYINFO_TABLE_REQUIRED_VERSION

See Also:

Constant Field Values

CONFIGPASSWORDS_TABLE

protected static final java.lang.String CONFIGPASSWORDS_TABLE

See Also:

Constant Field Values

CONFIGPASSWORDS_TABLE_REQUIRED_VERSION

protected static final int CONFIGPASSWORDS_TABLE_REQUIRED_VERSION

See Also:

Constant Field Values

CONFIGSEEDLISTS_TABLE

protected static final java.lang.String CONFIGSEEDLISTS_TABLE

See Also:

Constant Field Values

CONFIGSEEDLISTS_TABLE_REQUIRED_VERSION

protected static final int CONFIGSEEDLISTS_TABLE_REQUIRED_VERSION

See Also:

Constant Field Values

HARVESTDEFINITIONS_TABLE

protected static final java.lang.String HARVESTDEFINITIONS_TABLE

See Also:

Constant Field Values

HARVESTDEFINITIONS_TABLE_REQUIRED_VERSION

protected static final int HARVESTDEFINITIONS_TABLE_REQUIRED_VERSION

See Also:

Constant Field Values

PARTIALHARVESTS_TABLE

protected static final java.lang.String PARTIALHARVESTS_TABLE

See Also:

Constant Field Values

PARTIALHARVESTS_TABLE_REQUIRED_VERSION

protected static final int PARTIALHARVESTS_TABLE_REQUIRED_VERSION

See Also:

Constant Field Values

FULLHARVESTS_TABLE

protected static final java.lang.String FULLHARVESTS_TABLE

See Also:

Constant Field Values

FULLHARVESTS_TABLE_REQUIRED_VERSION

protected static final int FULLHARVESTS_TABLE_REQUIRED_VERSION

See Also:

Constant Field Values

HARVESTCONFIGS_TABLE

protected static final java.lang.String HARVESTCONFIGS_TABLE

See Also:

Constant Field Values

HARVESTCONFIGS_TABLE_REQUIRED_VERSION

protected static final int HARVESTCONFIGS_TABLE_REQUIRED_VERSION

See Also:

Constant Field Values

SCHEDULES_TABLE

protected static final java.lang.String SCHEDULES_TABLE

See Also:

Constant Field Values

SCHEDULES_TABLE_REQUIRED_VERSION

protected static final int SCHEDULES_TABLE_REQUIRED_VERSION

See Also:

Constant Field Values

ORDERTEMPLATES_TABLE

protected static final java.lang.String ORDERTEMPLATES_TABLE

See Also:

Constant Field Values

ORDERTEMPLATES_TABLE_REQUIRED_VERSION

protected static final int ORDERTEMPLATES_TABLE_REQUIRED_VERSION

See Also:

Constant Field Values

JOBS_TABLE

protected static final java.lang.String JOBS_TABLE

See Also:

Constant Field Values

JOBS_TABLE_REQUIRED_VERSION

protected static final int JOBS_TABLE_REQUIRED_VERSION

See Also:

Constant Field Values

JOBCONFIGS_TABLE

protected static final java.lang.String JOBCONFIGS_TABLE

See Also:

Constant Field Values

JOBCONFIGS_TABLE_REQUIRED_VERSION

protected static final int JOBCONFIGS_TABLE_REQUIRED_VERSION

See Also:

Constant Field Values

GLOBALCRAWLERTRAPLISTS_TABLE

protected static final java.lang.String GLOBALCRAWLERTRAPLISTS_TABLE

See Also:

Constant Field Values

GLOBALCRAWLERTRAPLISTS_TABLE_REQUIRED_VERSION

protected static final int GLOBALCRAWLERTRAPLISTS_TABLE_REQUIRED_VERSION

See Also:

Constant Field Values

GLOBALCRAWLERTRAPEXPRESSIONS_TABLE

protected static final java.lang.String GLOBALCRAWLERTRAPEXPRESSIONS_TABLE

See Also:

Constant Field Values

GLOBALCRAWLERTRAPEXPRESSIONS_TABLE_REQUIRED_VERSION

protected static final int GLOBALCRAWLERTRAPEXPRESSIONS_TABLE_REQUIRED_VERSION

See Also:

Constant Field Values

RUNNINGJOBSHISTORY_TABLE

protected static final java.lang.String RUNNINGJOBSHISTORY_TABLE

See Also:

Constant Field Values

RUNNINGJOBSHISTORY_TABLE_REQUIRED_VERSION

protected static final int RUNNINGJOBSHISTORY_TABLE_REQUIRED_VERSION

See Also:

Constant Field Values

RUNNINGJOBSMONITOR_TABLE

protected static final java.lang.String RUNNINGJOBSMONITOR_TABLE

See Also:

Constant Field Values

RUNNINGJOBSMONITOR_TABLE_REQUIRED_VERSION

protected static final int RUNNINGJOBSMONITOR_TABLE_REQUIRED_VERSION

See Also:

Constant Field Values

FRONTIERREPORTMONITOR_TABLE

protected static final java.lang.String FRONTIERREPORTMONITOR_TABLE

See Also:

Constant Field Values

FRONTIERREPORTMONITOR_TABLE_REQUIRED_VERSION

protected static final int FRONTIERREPORTMONITOR_TABLE_REQUIRED_VERSION

See Also:

Constant Field Values

EXTENDEDFIELD_TABLE

protected static final java.lang.String EXTENDEDFIELD_TABLE

See Also:

Constant Field Values

EXTENDEDFIELD_TABLE_REQUIRED_VERSION

protected static final int EXTENDEDFIELD_TABLE_REQUIRED_VERSION

See Also:

Constant Field Values

EXTENDEDFIELDVALUE_TABLE

protected static final java.lang.String EXTENDEDFIELDVALUE_TABLE

See Also:

Constant Field Values

EXTENDEDFIELDVALUE_TABLE_REQUIRED_VERSION

protected static final int EXTENDEDFIELDVALUE_TABLE_REQUIRED_VERSION

See Also:

Constant Field Values

EXTENDEDFIELDTYPE_TABLE

protected static final java.lang.String EXTENDEDFIELDTYPE_TABLE

See Also:

Constant Field Values

EXTENDEDFIELDTYPE_TABLE_REQUIRED_VERSION

protected static final int EXTENDEDFIELDTYPE_TABLE_REQUIRED_VERSION

See Also:

Constant Field Values

Constructor Detail

DBSpecifics

public DBSpecifics()

Method Detail

getInstance

public static DBSpecifics getInstance()

Get the singleton instance of the DBSpecifics implementation class.

Returns:

An instance of DBSpecifics with implementations for a given DB.

getJobConfigsTmpTable

public abstract java.lang.String getJobConfigsTmpTable(java.sql.Connection c)
                                                throws java.sql.SQLException

Get a temporary table for short-time use. The table should be disposed of with dropTemporaryTable. The table has two columns domain_name varchar(Constants.MAX_NAME_SIZE) + config_name varchar(Constants.MAX_NAME_SIZE) All rows in the table must be deleted at commit or rollback.

Parameters:

c - The DB connection to use.

Returns:

The name of the created table

Throws:

java.sql.SQLException - if there is a problem getting the table.

dropJobConfigsTmpTable

public abstract void dropJobConfigsTmpTable(java.sql.Connection c,
                                            java.lang.String tableName)

Dispose of a temporary table gotten with getTemporaryTable. This can be expected to be called from within a finally clause, so it mustn't throw exceptions.

Parameters:

c - The DB connection to use.

tableName - The name of the temporarily created table.

getDriverClassName

public abstract java.lang.String getDriverClassName()

Get the name of the JDBC driver class that handles interfacing to this server.

Returns:

The name of a JDBC driver class

updateTable

public void updateTable(java.lang.String tableName,
                        int toVersion)

Update a table to a newer version, if necessary. This will check the schemaversions table to see the current version and perform a table-specific update if required.

Parameters:

tableName - The table to update

toVersion - The version to update the table to.

Throws:

IllegalState - If the table is an unsupported version, and the toVersion is less than the current version of the table

NotImplementedException - If no method exists for migration from current version of the table to the toVersion of the table.

IOFailure - in case of problems in interacting with the database

migrateJobsv3tov4

protected abstract void migrateJobsv3tov4()

Migrates the 'jobs' table from version 3 to version 4 consisting of a change of the field forcemaxbytes from int to bigint and setting its default to -1. Furthermore the default value for field num_configs is set to 0.

Throws:

IOFailure - in case of problems in interacting with the database

migrateJobsv4tov5

protected abstract void migrateJobsv4tov5()

Migrates the 'jobs' table from version 4 to version 5 consisting of adding new fields 'resubmitted_as_job' and 'submittedDate'.

Throws:

IOFailure - in case of problems in interacting with the database

migrateConfigurationsv3ov4

protected abstract void migrateConfigurationsv3ov4()

Migrates the 'configurations' table from version 3 to version 4. This consists of altering the default value of field 'maxbytes' to -1.

migrateFullharvestsv2tov3

protected abstract void migrateFullharvestsv2tov3()

Migrates the 'fullharvests' table from version 2 to version 3. This consists of altering the default value of field 'maxbytes' to -1.

migrateRunningJobsHistoryTableV1ToV2

protected abstract void migrateRunningJobsHistoryTableV1ToV2()

Migrates the 'runningjobshistory' table from version 1 to version 2. This consists of adding the new column 'retiredQueuesCount'.

migrateRunningJobsMonitorTableV1ToV2

protected abstract void migrateRunningJobsMonitorTableV1ToV2()

Migrates the 'runningjobsmonitor' table from version 1 to version 2. This consists of adding the new column 'retiredQueuesCount'.

createGlobalCrawlerTrapLists

protected abstract void createGlobalCrawlerTrapLists()

Creates the initial (version 1) of table 'global_crawler_trap_lists'.

createGlobalCrawlerTrapExpressions

protected abstract void createGlobalCrawlerTrapExpressions()

Creates the initial (version 1) of table 'global_crawler_trap_expressions'.

getOrderByLimitAndOffsetSubClause

public abstract java.lang.String getOrderByLimitAndOffsetSubClause(long limit,
                                                                   long offset)

Formats the LIMIT sub-clause of an SQL order clause. This sub-clause allows to paginate query results and its syntax might be dependant on the target RDBMS

Parameters:

limit - the maximum number of rows to fetch.

offset - the starting offset in the full query results.

Returns:

the proper sub-clause.

supportsClob

public abstract boolean supportsClob()

Returns true if the target RDBMS supports CLOBs. If possible seedlists will be stored as CLOBs.

Returns:

true if CLOBs are supported, false otherwise.

createFrontierReportMonitorTable

public abstract void createFrontierReportMonitorTable()

Create the frontierReportMonitor table in the database.

createRunningJobsHistoryTable

public abstract void createRunningJobsHistoryTable()

Create the frontierReportMonitor table in the database.

createRunningJobsMonitorTable

public abstract void createRunningJobsMonitorTable()

Create the frontierReportMonitor table in the database.

migrateJobsv5tov6

protected abstract void migrateJobsv5tov6()

Migrates the 'jobs' table from version 5 to version 6. Adds the field 'forcemaxrunningtime'.

Throws:

IOFailure - in case of problems in interacting with the database

migrateConfigurationsv4tov5

protected abstract void migrateConfigurationsv4tov5()

Migrates the 'configurations' table from version 4 to version 5. This consists of altering the field 'maxobjects' from being an int to a bigint.

migrateFullharvestsv3tov4

protected abstract void migrateFullharvestsv3tov4()

Migrates the 'fullharvests' table from version 3 to version 4. This consists of adding the field 'maxjobrunningtime'.

migrateFullharvestsv4tov5

protected abstract void migrateFullharvestsv4tov5()

Migrates the 'fullharvests' table from version 4 to version 5. This consists of adding the field 'isindexready'.

createExtendedFieldTypeTable

protected abstract void createExtendedFieldTypeTable()

Create the extendedfieldtype table in the database.

createExtendedFieldTable

protected abstract void createExtendedFieldTable()

Create the extendedfield table in the database.

createExtendedFieldValueTable

protected abstract void createExtendedFieldValueTable()

Create the extendedfieldvalue table in the database.