Package dk.netarkivet.common

Class CommonSettings

  • public class CommonSettings
extends java.lang.Object
    Settings common to the entire NetarchiveSuite.

    • Field Summary

      Fields 
      Modifier and Type Field Description
      static java.lang.String ACCESS_TRUSTSTORE_PATH
      Path to the default truststore used for safe communication over HTTPS with FileResolver and WarcRecordService
      static java.lang.String APPLICATION_INSTANCE_ID
      settings.common.applicationInstanceId:
      The identifier of the instance of the application.
      static java.lang.String APPLICATION_NAME
      settings.common.applicationName:
      The name of the application, e.g.
      static java.lang.String ARC_REPOSITORY_CLIENT
      settings.common.arcrepositoryClient.class:
      The class that implements the ArcRepositoryClient.
      static java.lang.String BATCH_DEFAULT_TIMEOUT
      settings.common.batch.defaultBatchTimeout:
      The default timeout for batchjobs.
      static java.lang.String BATCH_LOGGING_INTERVAL
      settings.common.batch.loggingInterval:
      The time between logging the status of a batch job.
      static java.lang.String BATCHJOBS_BASEDIR
      settings.common.batch.baseDir:
      The directory where the resulting files will be placed when running a batchjob through the GUI interface.
      static java.lang.String BATCHJOBS_CLASS
      settings.common.batch.batchjobs.batchjob.class:
      The list of batchjobs to be runnable from the GUI.
      static java.lang.String BATCHJOBS_JARFILE
      settings.common.batch.batchjobs.batchjob.jarfile:
      The list of the corresponding jar-files containing the batchjob.
      static java.lang.String BITARCHIVE_LIMIT_FOR_RECORD_DATATRANSFER_IN_FILE
      settings.common.repository.limitForRecordDatatransferInFile:
      When the length record exceeds this number, the contents of the record will be transferred using a RemoteFile.
      static java.lang.String CACHE_DIR
      settings.common.cacheDir:
      The name of the directory where cache data global to the entire machine can be stored.
      static java.lang.String DB_BASE_URL
      settings.common.database.baseUrl:
      The URL to use to connect to the database specified in the DB_SPECIFICS_CLASS setting.
      static java.lang.String DB_CONN_VALID_CHECK_TIMEOUT
      settings.common.database.validityCheckTimeout:
      Timeout in seconds to check for the validity of a JDBC connection on the server.
      static java.lang.String DB_DIR
      settings.common.database.dir:
      Used for the external harvest definition database.
      static java.lang.String DB_IS_DERBY_IF_CONTAINS
      If DB_SPECIFICS_CLASS contains this string then a Derby database is in use.
      static java.lang.String DB_MACHINE
      settings.common.database.machine:
      Used for the external harvest definition database.
      static java.lang.String DB_PASSWORD
      The password for login to the harvest database.
      static java.lang.String DB_POOL_ACQ_INC
      settings.common.database.pool.acquireInc:
      Configure the increment size DB connection pool.
      static java.lang.String DB_POOL_IDLE_CONN_TEST_ON_CHECKIN
      settings.common.database.pool.idleConnTestOnCheckin:
      Configure if a connection validity should be checked when returned to the pool.
      static java.lang.String DB_POOL_IDLE_CONN_TEST_PERIOD
      settings.common.database.pool.idleConnTestPeriod:
      Configure idle connection testing period in seconds.
      static java.lang.String DB_POOL_IDLE_CONN_TEST_QUERY
      settings.common.database.pool.idleConnTestQuery:
      The SQL query to be used when testing an idle connection.
      static java.lang.String DB_POOL_MAX_CONNECTION_AGE
      settings.common.database.pool.maxConnectionAge:
      The maxConnectionAge in seconds for each connection in the pool.
      static java.lang.String DB_POOL_MAX_SIZE
      settings.common.database.pool.maxSize:
      Configure the maximum size of the DB connection pool.
      static java.lang.String DB_POOL_MAX_STM
      settings.common.database.pool.maxStm:
      Configure statement pooling, by setting the global maximum number of pooled prepared statements for a data source.
      static java.lang.String DB_POOL_MAX_STM_PER_CONN
      settings.common.database.pool.maxStmPerConn:
      Configure statement pooling, by setting the global maximum number of pooled prepared statements for a data source.
      static java.lang.String DB_POOL_MIN_SIZE
      settings.common.database.pool.minSize:
      Configure the minimum size of the DB connection pool.
      static java.lang.String DB_PORT
      settings.common.database.port:
      Used for the external harvest definition database.
      static java.lang.String DB_SPECIFICS_CLASS
      settings.common.database.class:
      The class that defines DB-specific methods.
      static java.lang.String DB_USERNAME
      The username for login to the harvest database.
      static java.lang.String DIR_COMMONTEMPDIR
      settings.common.tempDir:
      Common temporary directory for all applications.
      static java.lang.String ENVIRONMENT_NAME
      settings.common.environmentName:
      The name of the environment in which this code is running.
      static java.lang.String FILE_RESOLVER_BASE_URL
      The base url for the REST implementation of FileResolver.
      static java.lang.String FILE_RESOLVER_CLASS
      The implementation of FileResolver to use.
      static java.lang.String FILE_RESOLVER_KEYFILE
      The path to the keyfile to use when communicating with the file resolver service over HTTPS.
      static java.lang.String FREESPACE_PROVIDER_CLASS
      settings.common.freespaceprovider.class:
      The implementation class for free space provider, e.g.
      static java.lang.String FTP_DATATIMEOUT_SETTINGS
      settings.common.remoteFile.datatimeout:
      The setting for the FTP data timeout in seconds.
      static java.lang.String FTP_RETRIES_SETTINGS
      settings.common.remoteFile.retries:
      The setting for the number of times FTPRemoteFile should try before giving up a copyTo operation or logOn operation.
      static java.lang.String FTP_SERVER_NAME
      settings.common.remoteFile.serverName:
      The setting for the FTP-server used.
      static java.lang.String FTP_SERVER_PORT
      settings.common.remoteFile.serverPort:
      The setting for the FTP-server port used.
      static java.lang.String FTP_USER_NAME
      settings.common.remoteFile.userName:
      The setting for the FTP username.
      static java.lang.String FTP_USER_PASSWORD
      settings.common.remoteFile.userPassword:
      The setting for the FTP password.
      static java.lang.String HADOOP_DEFAULT_FS
      Deprecated. 
      static java.lang.String HADOOP_HDFS_URI
      Deprecated. 
      static java.lang.String HADOOP_KERBEROS_CONF
      The path to the kerberos configuration to use when running hadoop jobs.
      static java.lang.String HADOOP_KERBEROS_KEYTAB
      The path to the kerberos keytab file to use when running hadoop jobs.
      static java.lang.String HADOOP_KERBEROS_PRINCIPAL
      The kerberos principal to use when running hadoop jobs.
      static java.lang.String HADOOP_MAPRED_CDXJOB_INPUT_DIR
      The setting for the input path to use in cdx-indexing map-/reduce jobs.
      static java.lang.String HADOOP_MAPRED_CDXJOB_OUTPUT_DIR
      The setting for the output path to use in cdx-indexing map-/reduce jobs.
      static java.lang.String HADOOP_MAPRED_CRAWLLOG_EXTRACTIONJOB_INPUT_DIR
      The directory/path for Hadoop to use as input path in crawl log extraction map-/reduce jobs (Hadoop equivalent of the CrawlLogLinesMatchingRegexp batch job).
      static java.lang.String HADOOP_MAPRED_CRAWLLOG_EXTRACTIONJOB_OUTPUT_DIR
      The directory/path for Hadoop to use as output path in crawl log extraction map-/reduce jobs (Hadoop equivalent of the CrawlLogLinesMatchingRegexp batch job).
      static java.lang.String HADOOP_MAPRED_FRAMEWORK
      Deprecated. 
      static java.lang.String HADOOP_MAPRED_INPUT_FILES_PARENT_DIR
      Specifies which directory to find the individual input files for a job in.
      static java.lang.String HADOOP_MAPRED_METADATA_CDX_EXTRACTIONJOB_INPUT_DIR
      The directory/path for Hadoop to use as input path in metadata CDX-indexing map-/reduce jobs (Hadoop equivalent of ArchiveExtractCDXJob).
      static java.lang.String HADOOP_MAPRED_METADATA_CDX_EXTRACTIONJOB_OUTPUT_DIR
      The directory/path for Hadoop to use as output path in metadata CDX-indexing map-/reduce jobs (Hadoop equivalent of ArchiveExtractCDXJob).
      static java.lang.String HADOOP_MAPRED_METADATA_EXTRACTIONJOB_INPUT_DIR
      The directory/path for Hadoop to use as input path in metadata extraction map-/reduce jobs.
      static java.lang.String HADOOP_MAPRED_METADATA_EXTRACTIONJOB_OUTPUT_DIR
      The directory/path for Hadoop to use as output path in metadata extraction map-/reduce jobs.
      static java.lang.String HADOOP_MAPRED_UBER_JAR
      Path on the client machine where the uber-jar file containing the map-/reduce job and dependencies for the harvester and indexer module is to be found.
      static java.lang.String HADOOP_RESOURCEMANAGER_ADDRESS
      Deprecated. 
      static java.lang.String HADOOP_USER_NAME
      Deprecated. 
      static java.lang.String HARVEST_STATUS_DFT_PAGE_SIZE
      settings.common.webinterface.harvestStatus.defaultPageSize:
      The default number of jobs to show in the harvest status section, on one result page.
      static java.lang.String HIDE_INACTIVE_TEMPLATES
      settings.common.webinterface.hideInactiveTemplates;
      Boolean valued String.
      static java.lang.String HTTP_PORT_NUMBER
      settings.common.http.port:
      The *unique* (per host) port number that may or may not be used to serve http.
      static java.lang.String INDEXSERVER_CLIENT
      settings.common.indexClient.class:
      The class instantiated to give access to indices.
      static java.lang.String JMS_BROKER_CLASS
      settings.common.jms.class:
      Selects the broker class to be used.
      static java.lang.String JMS_BROKER_RETRIES
      settings.common.jms.retries:
      Selects the number of times the JMS connection tries to reconnect to the broker.
      static java.lang.String JMX_ACCESS_FILE
      settings.common.jmx.accessFile:
      The access file, containing information about who have which JMX roles have which access privileges.
      static java.lang.String JMX_PASSWORD_FILE
      settings.common.jmx.passwordFile:
      The password file, containing information about who may connect to the beans.
      static java.lang.String JMX_PORT
      settings.common.jmx.port:
      The port to use for JMX.
      static java.lang.String JMX_RMI_PORT
      settings.common.jmx.rmiPort:
      The JMX's RMI port to use for internal communication with beans.
      static java.lang.String JMX_TIMEOUT
      settings.common.jmx.timeout:
      How many seconds we will wait before giving up on a JMX connection.
      static java.lang.String MAIL_SERVER
      settings.common.mail.server:
      The mail server to use when sending mails.
      static java.lang.String MAX_CONNECTIONS_PER_ROUTE
      settings.common.webinterface.warc.maxConnectionsPerRoute
      Increase default max connections per route, e.g.
      static java.lang.String MAX_NUM_BATCH_EXCEPTIONS
      settings.common.batch.maxExceptions:
      The maximum number of exceptions to be stored for a batch job.
      static java.lang.String MAX_TOTAL_CONNECTIONS
      settings.common.webinterface.warc.maxTotalConnections:
      Increase max total connections to e.g.
      static java.lang.String METADATAFILE_REGEX_SUFFIX
      Specifies the suffix of a regex which can identify valid metadata files by job number.
      static java.lang.String MONITOR_REGISTRY_CLIENT
      settings.common.monitorregistryClient.class:
      Which class to use for monitor registry.
      static java.lang.String MONITOR_REGISTRY_CLIENT_REREGISTERDELAY
      settings.common.monitorregistryClient.reregisterdelay:
      Delay between every reregistering in minutes, e.g.
      static java.lang.String NOTIFICATIONS_CLASS
      settings.common.notifications.class:
      The implementation class for notifications, e.g.
      static java.lang.String ORGANIZATION
      settings.common.organization:
      The name of the organization responsible.
      static java.lang.String PROCESS_TIMEOUT
      settings.common.processTimeout:
      The number of milliseconds we wait for processes to react to shutdown requests.
      static java.lang.String REMOTE_FILE_CLASS
      settings.common.remoteFile.class:
      The class to use for RemoteFile objects (for transferring files around).
      static java.lang.String REPLICA_IDS
      settings.common.replicas.replica.replicaId:
      The identifiers of all replicas in the environment.
      static java.lang.String REPLICA_TAG
      settings.common.replicas.replica:
      The path to settings belonging to an individual replica, placed under the replicas setting.
      static java.lang.String REPLICAID_TAG
      settings.common.replicas.replica.replicaId:
      The tags for identifier of the replica, placed under the replica tag.
      static java.lang.String REPLICANAME_TAG
      settings.common.replicas.replica.replicaName:
      The tags for name of the replica, placed under the replica tag.
      static java.lang.String REPLICAS_SETTINGS
      settings.common.replicas:
      The entire settings for all replicas in the environment.
      static java.lang.String REPLICATYPE_TAG
      settings.common.replicas.replica.replicaType:
      The tags for type of the replica, placed under the replica tag.
      static java.lang.String SETTINGS
      The path in the XML-tree for the settings branch.
      static java.lang.String SITESECTION_CLASS
      settings.common.webinterface.siteSection.class:
      The subclass of SiteSection that defines a part of the web interface.
      static java.lang.String SITESECTION_WEBAPPLICATION
      settings.common.webinterface.siteSection.webapplication:
      The directory or war-file containing the web application for a site section.
      static java.lang.String THIS_PHYSICAL_LOCATION
      settings.common.thisPhysicalLocation:
      Physical location of where the application is running.
      static java.lang.String TLD_SILENTLY_IGNORE_STARRED_TLDS
      settings.common.topLevelDomains.silentlyIgnoreStarredTlds:
      If set to true, silently ignores starred TLDS and excluded tlds which are currently not supported by NetarchiveSuite (NAS-2689).
      static java.lang.String TLDS
      settings.common.topLevelDomains.tld:
      Extra valid top level domain, like .co.uk, .dk, .org., not part of current embedded public_suffix_list.dat file in common/common-core/src/main/resources/dk/netarkivet/common/utils/public_suffix_list.dat downloaded from https://www.publicsuffix.org/list/public_suffix_list.dat
      static java.lang.String TRUSTSTORE_PASSWORD
      Password for the default truststore.
      static java.lang.String UNIX_SORT_USE_COMMON_TEMP_DIR
      settings.common.unixSort.useCommonTempDir:
      When using the Unix sort command, by default it stores temporary files in the system temp dir, but if this setting is set to true, then it will be configured to use the common temp dir defined by common.settings.tempDir.
      static java.lang.String USE_BITMAG_HADOOP_BACKEND
      Boolean setting specifying if the Bitrepository project is used as backend storage together with Hadoop for mass processing jobs.
      static java.lang.String USE_REPLICA_ID
      settings.common.useReplicaId:
      Default bitarchive to use for e.g.
      static java.lang.String WEBINTERFACE_LANGUAGE
      settings.common.webinterface.language:
      The entire language setting structure under the webinterface setting.
      static java.lang.String WEBINTERFACE_LANGUAGE_LOCALE
      settings.common.webinterface.language.locale:
      The locale the GUI is available as under specific language setting.
      static java.lang.String WEBINTERFACE_LANGUAGE_NAME
      settings.common.webinterface.language.name:
      The native name of the language for the locale under specific language setting.
      static java.lang.String WEBINTERFACE_SETTINGS
      settings.common.webinterface:
      The entire webinterface setting structure.
      static java.lang.String WRS_BASE_URL
      The base url for the Warc Record Service.
      static java.lang.String WRS_KEYFILE
      The path to the keyfile to use when communicating with the Warc Record Service over HTTPS.

    • Constructor Summary

      Constructors 
      Constructor Description
      CommonSettings()  

    • Method Summary

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    • Field Detail

      • SETTINGS

        public static java.lang.String SETTINGS
        The path in the XML-tree for the settings branch. (Used by deploy).

      • ENVIRONMENT_NAME

        public static java.lang.String ENVIRONMENT_NAME
        settings.common.environmentName:
        The name of the environment in which this code is running. E.g. PROD, RELEASETEST. It is used as a Common prefix to all JMS channels created in a NetarchiveSuite installation.

      • DIR_COMMONTEMPDIR

        public static java.lang.String DIR_COMMONTEMPDIR
        settings.common.tempDir:
        Common temporary directory for all applications. Some subdirs of this directory must be set to have AllPermision in the conf/security.conf file, or the web pages won't work.

      • REMOTE_FILE_CLASS

        public static java.lang.String REMOTE_FILE_CLASS
        settings.common.remoteFile.class:
        The class to use for RemoteFile objects (for transferring files around). This class must implement the dk.netarkivet.common.distribute.RemoteFile interface.

      • JMS_BROKER_CLASS

        public static java.lang.String JMS_BROKER_CLASS
        settings.common.jms.class:
        Selects the broker class to be used. Must be subclass of dk.netarkivet.common.distribute.JMSConnection.

      • JMS_BROKER_RETRIES

        public static java.lang.String JMS_BROKER_RETRIES
        settings.common.jms.retries:
        Selects the number of times the JMS connection tries to reconnect to the broker.

      • HTTP_PORT_NUMBER

        public static java.lang.String HTTP_PORT_NUMBER
        settings.common.http.port:
        The *unique* (per host) port number that may or may not be used to serve http.

      • ARC_REPOSITORY_CLIENT

        public static java.lang.String ARC_REPOSITORY_CLIENT
        settings.common.arcrepositoryClient.class:
        The class that implements the ArcRepositoryClient. The class must implement the interface dk.netarkivet.common.distribute.arcrepository.ArcRepositoryClient This class will be instantiated by the ArcRepositoryClientFactory.

      • MAX_NUM_BATCH_EXCEPTIONS

        public static java.lang.String MAX_NUM_BATCH_EXCEPTIONS
        settings.common.batch.maxExceptions:
        The maximum number of exceptions to be stored for a batch job.

      • BATCH_LOGGING_INTERVAL

        public static java.lang.String BATCH_LOGGING_INTERVAL
        settings.common.batch.loggingInterval:
        The time between logging the status of a batch job.

      • BATCH_DEFAULT_TIMEOUT

        public static java.lang.String BATCH_DEFAULT_TIMEOUT
        settings.common.batch.defaultBatchTimeout:
        The default timeout for batchjobs. This will be used for batchjobs which has the batchJobTimeout set to -1 (inherited value from FileBatchJob).

      • MONITOR_REGISTRY_CLIENT

        public static java.lang.String MONITOR_REGISTRY_CLIENT
        settings.common.monitorregistryClient.class:
        Which class to use for monitor registry. Must implement the interface dk.netarkivet.common.distribute.monitorregistry.MonitorRegistryClient.

      • MONITOR_REGISTRY_CLIENT_REREGISTERDELAY

        public static java.lang.String MONITOR_REGISTRY_CLIENT_REREGISTERDELAY
        settings.common.monitorregistryClient.reregisterdelay:
        Delay between every reregistering in minutes, e.g. 1 for one minute.

      • INDEXSERVER_CLIENT

        public static java.lang.String INDEXSERVER_CLIENT
        settings.common.indexClient.class:
        The class instantiated to give access to indices. The class must implement the interface dk.netarkivet.common.distribute.indexserver.JobIndexCache The class instantiations are manufactored by IndexClientFactory.

      • UNIX_SORT_USE_COMMON_TEMP_DIR

        public static java.lang.String UNIX_SORT_USE_COMMON_TEMP_DIR
        settings.common.unixSort.useCommonTempDir:
        When using the Unix sort command, by default it stores temporary files in the system temp dir, but if this setting is set to true, then it will be configured to use the common temp dir defined by common.settings.tempDir. By default the value is false (use system temp), which is the legacy behavior. This setting is part of the fix for Jira issue NAS-1995.
        See Also:
        DIR_COMMONTEMPDIR

      • CACHE_DIR

        public static java.lang.String CACHE_DIR
        settings.common.cacheDir:
        The name of the directory where cache data global to the entire machine can be stored. Various kinds of caches should be stored in subdirectories of this.

      • PROCESS_TIMEOUT

        public static java.lang.String PROCESS_TIMEOUT
        settings.common.processTimeout:
        The number of milliseconds we wait for processes to react to shutdown requests.

      • NOTIFICATIONS_CLASS

        public static java.lang.String NOTIFICATIONS_CLASS
        settings.common.notifications.class:
        The implementation class for notifications, e.g. error notification. The class must extend dk.netarkivet.common.utils.Notifications

      • MAIL_SERVER

        public static java.lang.String MAIL_SERVER
        settings.common.mail.server:
        The mail server to use when sending mails.

      • JMX_PORT

        public static java.lang.String JMX_PORT
        settings.common.jmx.port:
        The port to use for JMX.

      • JMX_RMI_PORT

        public static java.lang.String JMX_RMI_PORT
        settings.common.jmx.rmiPort:
        The JMX's RMI port to use for internal communication with beans.

      • JMX_PASSWORD_FILE

        public static java.lang.String JMX_PASSWORD_FILE
        settings.common.jmx.passwordFile:
        The password file, containing information about who may connect to the beans. The file has a format defined by the JMX standard,
        See Also:

      • JMX_ACCESS_FILE

        public static java.lang.String JMX_ACCESS_FILE
        settings.common.jmx.accessFile:
        The access file, containing information about who have which JMX roles have which access privileges. The file has a format defined by the JMX standard,
        See Also:

      • JMX_TIMEOUT

        public static java.lang.String JMX_TIMEOUT
        settings.common.jmx.timeout:
        How many seconds we will wait before giving up on a JMX connection.

      • WEBINTERFACE_SETTINGS

        public static java.lang.String WEBINTERFACE_SETTINGS
        settings.common.webinterface:
        The entire webinterface setting structure.

      • HIDE_INACTIVE_TEMPLATES

        public static java.lang.String HIDE_INACTIVE_TEMPLATES
        settings.common.webinterface.hideInactiveTemplates;
        Boolean valued String. If true, inactive harvest templates are entirely hidden in the GUI - it will be impossible to see them or change their state to "active". If false, the GUI will include functionality to change the status of template from inactive to active and vice-versa.

      • WEBINTERFACE_LANGUAGE

        public static java.lang.String WEBINTERFACE_LANGUAGE
        settings.common.webinterface.language:
        The entire language setting structure under the webinterface setting. Is repeated for each language

      • WEBINTERFACE_LANGUAGE_LOCALE

        public static java.lang.String WEBINTERFACE_LANGUAGE_LOCALE
        settings.common.webinterface.language.locale:
        The locale the GUI is available as under specific language setting.

      • WEBINTERFACE_LANGUAGE_NAME

        public static java.lang.String WEBINTERFACE_LANGUAGE_NAME
        settings.common.webinterface.language.name:
        The native name of the language for the locale under specific language setting.

      • SITESECTION_CLASS

        public static java.lang.String SITESECTION_CLASS
        settings.common.webinterface.siteSection.class:
        The subclass of SiteSection that defines a part of the web interface. Is part of repeated siteSection settings for each part.

      • SITESECTION_WEBAPPLICATION

        public static java.lang.String SITESECTION_WEBAPPLICATION
        settings.common.webinterface.siteSection.webapplication:
        The directory or war-file containing the web application for a site section. Is part of repeated siteSection settings for each part.

      • HARVEST_STATUS_DFT_PAGE_SIZE

        public static java.lang.String HARVEST_STATUS_DFT_PAGE_SIZE
        settings.common.webinterface.harvestStatus.defaultPageSize:
        The default number of jobs to show in the harvest status section, on one result page.

      • TLDS

        public static java.lang.String TLDS
        settings.common.topLevelDomains.tld:
        Extra valid top level domain, like .co.uk, .dk, .org., not part of current embedded public_suffix_list.dat file in common/common-core/src/main/resources/dk/netarkivet/common/utils/public_suffix_list.dat downloaded from https://www.publicsuffix.org/list/public_suffix_list.dat

      • TLD_SILENTLY_IGNORE_STARRED_TLDS

        public static java.lang.String TLD_SILENTLY_IGNORE_STARRED_TLDS
        settings.common.topLevelDomains.silentlyIgnoreStarredTlds:
        If set to true, silently ignores starred TLDS and excluded tlds which are currently not supported by NetarchiveSuite (NAS-2689). default: true

      • DB_SPECIFICS_CLASS

        public static java.lang.String DB_SPECIFICS_CLASS
        settings.common.database.class:
        The class that defines DB-specific methods. This class must extend the DBSpecifics class

      • DB_IS_DERBY_IF_CONTAINS

        public static java.lang.String DB_IS_DERBY_IF_CONTAINS
        If DB_SPECIFICS_CLASS contains this string then a Derby database is in use.

      • DB_BASE_URL

        public static java.lang.String DB_BASE_URL
        settings.common.database.baseUrl:
        The URL to use to connect to the database specified in the DB_SPECIFICS_CLASS setting.

      • DB_MACHINE

        public static java.lang.String DB_MACHINE
        settings.common.database.machine:
        Used for the external harvest definition database. The machine where the harvest definition database is located.

      • DB_PORT

        public static java.lang.String DB_PORT
        settings.common.database.port:
        Used for the external harvest definition database. The port where the external harvest definition database is attached.

      • DB_DIR

        public static java.lang.String DB_DIR
        settings.common.database.dir:
        Used for the external harvest definition database. The directory where the external harvest definition database is located.

      • DB_CONN_VALID_CHECK_TIMEOUT

        public static java.lang.String DB_CONN_VALID_CHECK_TIMEOUT
        settings.common.database.validityCheckTimeout:
        Timeout in seconds to check for the validity of a JDBC connection on the server. This is the time in seconds to wait for the database operation used to validate the connection to complete. If the timeout period expires before the operation completes, this method returns false. A value of 0 indicates a timeout is not applied to the database operation.

        Connection.isValid(int)

      • DB_POOL_MIN_SIZE

        public static java.lang.String DB_POOL_MIN_SIZE
        settings.common.database.pool.minSize:
        Configure the minimum size of the DB connection pool. Default value is 5.

      • DB_POOL_MAX_SIZE

        public static java.lang.String DB_POOL_MAX_SIZE
        settings.common.database.pool.maxSize:
        Configure the maximum size of the DB connection pool. Default value is 10.

      • DB_POOL_ACQ_INC

        public static java.lang.String DB_POOL_ACQ_INC
        settings.common.database.pool.acquireInc:
        Configure the increment size DB connection pool. Default value is 5 (half the max size).

      • DB_POOL_MAX_STM

        public static java.lang.String DB_POOL_MAX_STM
        settings.common.database.pool.maxStm:
        Configure statement pooling, by setting the global maximum number of pooled prepared statements for a data source. Default value is 0. Note that if both DB_POOL_MAX_STM and DB_POOL_MAX_STM_PER_CONN are set to zero, statement pooling is fully deactivated.
        See Also:
        c3p0 documentation

      • DB_POOL_MAX_STM_PER_CONN

        public static java.lang.String DB_POOL_MAX_STM_PER_CONN
        settings.common.database.pool.maxStmPerConn:
        Configure statement pooling, by setting the global maximum number of pooled prepared statements for a data source. Default value is 0. Note that if both DB_POOL_MAX_STM and DB_POOL_MAX_STM_PER_CONN are set to zero, statement pooling is fully deactivated.
        See Also:
        c3p0 documentation

      • DB_POOL_IDLE_CONN_TEST_PERIOD

        public static java.lang.String DB_POOL_IDLE_CONN_TEST_PERIOD
        settings.common.database.pool.idleConnTestPeriod:
        Configure idle connection testing period in seconds. Default is 0, which means no idle connection testing
        See Also:

      • DB_POOL_IDLE_CONN_TEST_ON_CHECKIN

        public static java.lang.String DB_POOL_IDLE_CONN_TEST_ON_CHECKIN
        settings.common.database.pool.idleConnTestOnCheckin:
        Configure if a connection validity should be checked when returned to the pool. Default is false.
        See Also:

      • DB_POOL_IDLE_CONN_TEST_QUERY

        public static java.lang.String DB_POOL_IDLE_CONN_TEST_QUERY
        settings.common.database.pool.idleConnTestQuery:
        The SQL query to be used when testing an idle connection. Default is empty, which means using c3p0 defaults.
        See Also:

      • DB_POOL_MAX_CONNECTION_AGE

        public static java.lang.String DB_POOL_MAX_CONNECTION_AGE
        settings.common.database.pool.maxConnectionAge:
        The maxConnectionAge in seconds for each connection in the pool. The default is one hour (3600 seconds)
        See Also:

      • DB_USERNAME

        public static java.lang.String DB_USERNAME
        The username for login to the harvest database.

      • DB_PASSWORD

        public static java.lang.String DB_PASSWORD
        The password for login to the harvest database.

      • BITARCHIVE_LIMIT_FOR_RECORD_DATATRANSFER_IN_FILE

        public static java.lang.String BITARCHIVE_LIMIT_FOR_RECORD_DATATRANSFER_IN_FILE
        settings.common.repository.limitForRecordDatatransferInFile:
        When the length record exceeds this number, the contents of the record will be transferred using a RemoteFile.

      • REPLICAS_SETTINGS

        public static java.lang.String REPLICAS_SETTINGS
        settings.common.replicas:
        The entire settings for all replicas in the environment. NOTE: settings for checksum replicas are not use yet

      • REPLICA_TAG

        public static java.lang.String REPLICA_TAG
        settings.common.replicas.replica:
        The path to settings belonging to an individual replica, placed under the replicas setting.

      • REPLICAID_TAG

        public static java.lang.String REPLICAID_TAG
        settings.common.replicas.replica.replicaId:
        The tags for identifier of the replica, placed under the replica tag. The replica id is used internally in e.g. naming of channels.

      • REPLICANAME_TAG

        public static java.lang.String REPLICANAME_TAG
        settings.common.replicas.replica.replicaName:
        The tags for name of the replica, placed under the replica tag. The replica name is used in interfaces like the GUI or command-line batch-programs. The name can be the same value as the id.

      • REPLICATYPE_TAG

        public static java.lang.String REPLICATYPE_TAG
        settings.common.replicas.replica.replicaType:
        The tags for type of the replica, placed under the replica tag. The type is used to identify whether it is a bitarchive or a checksum replica. NOTE: checksum replicas are not implemented yet Possible values are defined in ReplicaType

      • REPLICA_IDS

        public static java.lang.String REPLICA_IDS
        settings.common.replicas.replica.replicaId:
        The identifiers of all replicas in the environment.

      • USE_REPLICA_ID

        public static java.lang.String USE_REPLICA_ID
        settings.common.useReplicaId:
        Default bitarchive to use for e.g. batch jobs (if none is specified).

      • THIS_PHYSICAL_LOCATION

        public static java.lang.String THIS_PHYSICAL_LOCATION
        settings.common.thisPhysicalLocation:
        Physical location of where the application is running. Only use for System state GUI and deploy

      • APPLICATION_NAME

        public static java.lang.String APPLICATION_NAME
        settings.common.applicationName:
        The name of the application, e.g. "BitarchiveServerApplication". The monitor puts this with each log message.

      • APPLICATION_INSTANCE_ID

        public static java.lang.String APPLICATION_INSTANCE_ID
        settings.common.applicationInstanceId:
        The identifier of the instance of the application. This is used when there are more than one of the same application running on the same machine, e.g. when more harvesters are running on the same machine or more bitarchive applications are running on the same machine.

      • FREESPACE_PROVIDER_CLASS

        public static java.lang.String FREESPACE_PROVIDER_CLASS
        settings.common.freespaceprovider.class:
        The implementation class for free space provider, e.g. dk.netarkivet.common.utils.DefaultFreeSpaceProvider. The class must implement FreeSpaceProvider-Interface.

      • BATCHJOBS_CLASS

        public static java.lang.String BATCHJOBS_CLASS
        settings.common.batch.batchjobs.batchjob.class:
        The list of batchjobs to be runnable from the GUI. Must be the complete path to the batchjob classes (e.g. dk.netarkivet.archive.arcrepository.bitpreservation.ChecksumJob). Must inherit FileBatchJob.

      • BATCHJOBS_JARFILE

        public static java.lang.String BATCHJOBS_JARFILE
        settings.common.batch.batchjobs.batchjob.jarfile:
        The list of the corresponding jar-files containing the batchjob. This will be used for LoadableJarBatchJobs. If no file is specified, it is assumed, that the batchjob exists with the default classpath of the involved applications (BitarchiveMonitor, ArcRepository, GUIWebServer and BitArchive).

      • BATCHJOBS_BASEDIR

        public static java.lang.String BATCHJOBS_BASEDIR
        settings.common.batch.baseDir:
        The directory where the resulting files will be placed when running a batchjob through the GUI interface.

      • FTP_SERVER_NAME

        public static java.lang.String FTP_SERVER_NAME
        settings.common.remoteFile.serverName:
        The setting for the FTP-server used.

      • FTP_SERVER_PORT

        public static java.lang.String FTP_SERVER_PORT
        settings.common.remoteFile.serverPort:
        The setting for the FTP-server port used.

      • FTP_USER_NAME

        public static java.lang.String FTP_USER_NAME
        settings.common.remoteFile.userName:
        The setting for the FTP username.

      • FTP_USER_PASSWORD

        public static java.lang.String FTP_USER_PASSWORD
        settings.common.remoteFile.userPassword:
        The setting for the FTP password. *

      • FTP_RETRIES_SETTINGS

        public static java.lang.String FTP_RETRIES_SETTINGS
        settings.common.remoteFile.retries:
        The setting for the number of times FTPRemoteFile should try before giving up a copyTo operation or logOn operation.

      • FTP_DATATIMEOUT_SETTINGS

        public static java.lang.String FTP_DATATIMEOUT_SETTINGS
        settings.common.remoteFile.datatimeout:
        The setting for the FTP data timeout in seconds. The default value is 600 (10 minutes).

      • ORGANIZATION

        public static java.lang.String ORGANIZATION
        settings.common.organization:
        The name of the organization responsible. The default value is netarkivet.dk

      • METADATAFILE_REGEX_SUFFIX

        public static java.lang.String METADATAFILE_REGEX_SUFFIX
        Specifies the suffix of a regex which can identify valid metadata files by job number. Thus preceding the value of this setting with .* will find all metadata files.

      • HADOOP_DEFAULT_FS

        public static java.lang.String HADOOP_DEFAULT_FS
        Deprecated.
        Specifies the name of the default file system for Hadoop to use (URI consisting of scheme and authority). If not set, Hadoop defaults to 'file:///'. If using hdfs Hadoop paths are auto-resolved to 'hdfs://hostname/user/current_user/specified_path In production we now load the entire hadoop configuration from the classpath so this is

      • HADOOP_HDFS_URI

        public static java.lang.String HADOOP_HDFS_URI
        Deprecated.
        The setting specifying the URI on which the used HDFS is located. In production we now load the entire hadoop configuration from the classpath so this is

      • HADOOP_RESOURCEMANAGER_ADDRESS

        public static java.lang.String HADOOP_RESOURCEMANAGER_ADDRESS
        Deprecated.
        Specifies the address on where to locate the ResourceManager (e.g. YARN). In production we now load the entire hadoop configuration from the classpath so this is

      • HADOOP_MAPRED_FRAMEWORK

        public static java.lang.String HADOOP_MAPRED_FRAMEWORK
        Deprecated.
        Specifies which framework is used for running mapreduce (e.g. YARN). In production we now load the entire hadoop configuration from the classpath so this is

      • HADOOP_MAPRED_INPUT_FILES_PARENT_DIR

        public static java.lang.String HADOOP_MAPRED_INPUT_FILES_PARENT_DIR
        Specifies which directory to find the individual input files for a job in.

      • HADOOP_MAPRED_CDXJOB_INPUT_DIR

        public static java.lang.String HADOOP_MAPRED_CDXJOB_INPUT_DIR
        The setting for the input path to use in cdx-indexing map-/reduce jobs.

      • HADOOP_MAPRED_CDXJOB_OUTPUT_DIR

        public static java.lang.String HADOOP_MAPRED_CDXJOB_OUTPUT_DIR
        The setting for the output path to use in cdx-indexing map-/reduce jobs.

      • HADOOP_MAPRED_METADATA_EXTRACTIONJOB_INPUT_DIR

        public static java.lang.String HADOOP_MAPRED_METADATA_EXTRACTIONJOB_INPUT_DIR
        The directory/path for Hadoop to use as input path in metadata extraction map-/reduce jobs. Used when Hadoop metadata extraction jobs are started by the IndexRequestServer after receiving requests.

      • HADOOP_MAPRED_METADATA_EXTRACTIONJOB_OUTPUT_DIR

        public static java.lang.String HADOOP_MAPRED_METADATA_EXTRACTIONJOB_OUTPUT_DIR
        The directory/path for Hadoop to use as output path in metadata extraction map-/reduce jobs. Used when Hadoop metadata extraction jobs are started by the IndexRequestServer after receiving requests.

      • HADOOP_MAPRED_METADATA_CDX_EXTRACTIONJOB_INPUT_DIR

        public static java.lang.String HADOOP_MAPRED_METADATA_CDX_EXTRACTIONJOB_INPUT_DIR
        The directory/path for Hadoop to use as input path in metadata CDX-indexing map-/reduce jobs (Hadoop equivalent of ArchiveExtractCDXJob).

      • HADOOP_MAPRED_METADATA_CDX_EXTRACTIONJOB_OUTPUT_DIR

        public static java.lang.String HADOOP_MAPRED_METADATA_CDX_EXTRACTIONJOB_OUTPUT_DIR
        The directory/path for Hadoop to use as output path in metadata CDX-indexing map-/reduce jobs (Hadoop equivalent of ArchiveExtractCDXJob).

      • HADOOP_MAPRED_CRAWLLOG_EXTRACTIONJOB_INPUT_DIR

        public static java.lang.String HADOOP_MAPRED_CRAWLLOG_EXTRACTIONJOB_INPUT_DIR
        The directory/path for Hadoop to use as input path in crawl log extraction map-/reduce jobs (Hadoop equivalent of the CrawlLogLinesMatchingRegexp batch job).

      • HADOOP_MAPRED_CRAWLLOG_EXTRACTIONJOB_OUTPUT_DIR

        public static java.lang.String HADOOP_MAPRED_CRAWLLOG_EXTRACTIONJOB_OUTPUT_DIR
        The directory/path for Hadoop to use as output path in crawl log extraction map-/reduce jobs (Hadoop equivalent of the CrawlLogLinesMatchingRegexp batch job).

      • HADOOP_MAPRED_UBER_JAR

        public static java.lang.String HADOOP_MAPRED_UBER_JAR
        Path on the client machine where the uber-jar file containing the map-/reduce job and dependencies for the harvester and indexer module is to be found.

      • HADOOP_USER_NAME

        public static java.lang.String HADOOP_USER_NAME
        Deprecated.
        Username to start map-/reduce jobs under on the Hadoop cluster. In production we now use kerberos so this is

      • USE_BITMAG_HADOOP_BACKEND

        public static java.lang.String USE_BITMAG_HADOOP_BACKEND
        Boolean setting specifying if the Bitrepository project is used as backend storage together with Hadoop for mass processing jobs.

      • MAX_TOTAL_CONNECTIONS

        public static java.lang.String MAX_TOTAL_CONNECTIONS
        settings.common.webinterface.warc.maxTotalConnections:
        Increase max total connections to e.g. 200.

      • MAX_CONNECTIONS_PER_ROUTE

        public static java.lang.String MAX_CONNECTIONS_PER_ROUTE
        settings.common.webinterface.warc.maxConnectionsPerRoute
        Increase default max connections per route, e.g. to 20.

      • WRS_BASE_URL

        public static java.lang.String WRS_BASE_URL
        The base url for the Warc Record Service.

      • WRS_KEYFILE

        public static java.lang.String WRS_KEYFILE
        The path to the keyfile to use when communicating with the Warc Record Service over HTTPS.

      • FILE_RESOLVER_CLASS

        public static java.lang.String FILE_RESOLVER_CLASS
        The implementation of FileResolver to use.

      • FILE_RESOLVER_BASE_URL

        public static java.lang.String FILE_RESOLVER_BASE_URL
        The base url for the REST implementation of FileResolver.

      • FILE_RESOLVER_KEYFILE

        public static java.lang.String FILE_RESOLVER_KEYFILE
        The path to the keyfile to use when communicating with the file resolver service over HTTPS.

      • ACCESS_TRUSTSTORE_PATH

        public static java.lang.String ACCESS_TRUSTSTORE_PATH
        Path to the default truststore used for safe communication over HTTPS with FileResolver and WarcRecordService

      • TRUSTSTORE_PASSWORD

        public static java.lang.String TRUSTSTORE_PASSWORD
        Password for the default truststore.

      • HADOOP_KERBEROS_PRINCIPAL

        public static java.lang.String HADOOP_KERBEROS_PRINCIPAL
        The kerberos principal to use when running hadoop jobs.

      • HADOOP_KERBEROS_KEYTAB

        public static java.lang.String HADOOP_KERBEROS_KEYTAB
        The path to the kerberos keytab file to use when running hadoop jobs.

      • HADOOP_KERBEROS_CONF

        public static java.lang.String HADOOP_KERBEROS_CONF
        The path to the kerberos configuration to use when running hadoop jobs. The default is /etc/krb5.conf .