dk.netarkivet.deploy
Class LinuxMachine

java.lang.Object
  dk.netarkivet.deploy.Machine
      dk.netarkivet.deploy.LinuxMachine

public class LinuxMachine
extends Machine

A LinuxMachine is the instance of the abstract machine class, which runs the operating system Linux or another Unix dependent operation system. This class only contains the operating system specific functions.

Field Summary

Fields inherited from class dk.netarkivet.deploy.Machine

applications, arcDatabaseFile, databaseFile, inheritedLogPropFile, inheritedSecurityPolicyFile, jarFolder, log, machineDirectory, machineParameters, machineRoot, name, netarchiveSuiteFileName, operatingSystem, resetTempDir, scriptExtension, settings

Constructor Summary

LinuxMachine(org.dom4j.Element subTreeRoot, XmlStructure parentSettings, Parameters param, java.lang.String netarchiveSuiteSource, java.io.File logProp, java.io.File securityPolicy, java.io.File dbFile, java.io.File arcdbFile, boolean resetDir, java.io.File externalJarFolder)
The constructor.

Method Summary

protected java.lang.String	callKillArchiveDatabase() Method for generating the command for running the external_database_kill script.
protected java.lang.String	callKillHarvestDatabase() Method for generating the command for running the external_database_kill script.
protected java.lang.String	callStartArchiveDatabase() Method for generating the command for running the external_admin_database_start script.
protected java.lang.String	callStartHarvestDatabase() Method for generating the command for running the external_harvest_database_start script.
protected java.lang.String	changeFileDirPathForSecurity(java.lang.String path) Changes the file directory path to the format used in the security policy.
protected void	createApplicationKillScripts(java.io.File directory) Creates the kill scripts for all the applications.
protected void	createApplicationStartScripts(java.io.File directory) Creates the start scripts for all the applications.
protected void	createArchiveDatabaseKillScript(java.io.File dir) Creates a script for killing the archive database on a given machine.
protected void	createArchiveDatabaseStartScript(java.io.File dir) Creates a script for starting the archive database on a given machine.
protected void	createHarvestDatabaseKillScript(java.io.File dir) Creates a script for killing the harvest database on a given machine.
protected void	createHarvestDatabaseStartScript(java.io.File dir) Creates a script for starting the harvest database on a given machine.
protected void	createInstallDirScript(java.io.File dir) Dummy function on linux machine.
protected void	createOSLocalKillAllScript(java.io.File directory) This function creates the script to kill all applications on this machine.
protected void	createOSLocalStartAllScript(java.io.File directory) This function creates the script to start all applications on this machine.
protected java.lang.String	createPathToDir(java.lang.String dir) Function for creating the directories along the path until the end directory.
protected void	createRestartScript(java.io.File dir) Creates script for restarting all the applications on a machine.
protected java.lang.String	getAppDirectories() Creates the script for creating the application specified directories.
protected java.lang.String	getConfDirPath() The operation system specific path to the conf directory.
protected java.lang.String	getInstallDirPath() The operation system specific path to the installation directory.
protected java.lang.String	getJMXremoteFilesCommand() This method does the following: Retrieves the path to the jmxremote.access and jmxremote.password files.
protected java.lang.String	osGetClassPath(Application app) Makes all the class paths into the operation system specific syntax, and puts them into a string where they are separated by the operation system specific separator (':' for linux, ';' for windows).
protected java.lang.String	osInstallArchiveDatabase() Checks if a specific directory for the archive database is given in the settings, and thus if the archive database should be installed on this machine.
protected java.lang.String	osInstallDatabase() Checks if a specific directory for the database is given in the settings, and thus if the database should be installed on this machine.
protected java.lang.String	osInstallExternalJarFiles() This function makes the part of the install script for installing the external jar files from within the jarFolder.
protected java.lang.String	osInstallScript() Creates the operation system specific installation script for this machine.
protected java.lang.String	osInstallScriptCreateDir() Creates the specified directories in the deploy-configuration file.
protected java.lang.String	osKillScript() Creates the operation system specific killing script for this machine.
protected java.lang.String	osStartScript() Creates the operation system specific starting script for this machine.
protected java.lang.String	scriptCreateDir(java.lang.String dir, boolean clean) This functions makes the script for creating the new directories.

Methods inherited from class dk.netarkivet.deploy.Machine

createJmxRemoteAccessFile, createJmxRemotePasswordFile, createLogPropertyFiles, createSecurityPolicyFile, getEnvironmentName, getHeritrixLogin, getHeritrixUsername, getMonitorLogin, getMonitorUsername, machineUserLogin, write, writeToGlobalInstallScript, writeToGlobalKillScript, writeToGlobalStartScript

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

LinuxMachine

public LinuxMachine(org.dom4j.Element subTreeRoot,
                    XmlStructure parentSettings,
                    Parameters param,
                    java.lang.String netarchiveSuiteSource,
                    java.io.File logProp,
                    java.io.File securityPolicy,
                    java.io.File dbFile,
                    java.io.File arcdbFile,
                    boolean resetDir,
                    java.io.File externalJarFolder)

The constructor. Starts by initialising the parent abstract class, then sets the operating system dependent variables.

Parameters:

subTreeRoot - The XML root element.

parentSettings - The Settings to be inherited from the PhysicalLocation, where this machine is placed.

param - The machine parameters to be inherited from the PhysicalLocation.

netarchiveSuiteSource - The name of the NetarchiveSuite package file. Must end with '.zip'.

logProp - The logging property file, to be copied into machine directory.

securityPolicy - The security policy file, to be copied into machine directory.

dbFile - The name of the database file.

arcdbFile - The name of the archive file.

resetDir - Whether the temporary directory should be reset.

externalJarFolder - The folder containing the external jar library files.

Method Detail

osInstallScript

protected java.lang.String osInstallScript()

Description copied from class: Machine

Creates the operation system specific installation script for this machine.

Specified by:

osInstallScript in class Machine

Returns:

Operation system specific part of the installscript.

osKillScript

protected java.lang.String osKillScript()

Description copied from class: Machine

Creates the operation system specific killing script for this machine.

Specified by:

osKillScript in class Machine

Returns:

Operation system specific part of the killscript.

osStartScript

protected java.lang.String osStartScript()

Creates the operation system specific starting script for this machine. pseudo code: - ssh maclogin ". /etc/profile; conf/startall.sh; sleep 5; cat install/*.log" where: maclogin = login for machine (username@machine). conf = path to /conf directory. install = path to install directory.

Specified by:

osStartScript in class Machine

Returns:

Operation system specific part of the startscript.

getInstallDirPath

protected java.lang.String getInstallDirPath()

Description copied from class: Machine

The operation system specific path to the installation directory.

Specified by:

getInstallDirPath in class Machine

Returns:

Install path.

getConfDirPath

protected java.lang.String getConfDirPath()

Description copied from class: Machine

The operation system specific path to the conf directory.

Specified by:

getConfDirPath in class Machine

Returns:

Conf path.

createOSLocalKillAllScript

protected void createOSLocalKillAllScript(java.io.File directory)
                                   throws IOFailure

This function creates the script to kill all applications on this machine. The scripts calls all the kill script for each application. It also runs the script for killing any external database. pseudo code: - echo Killing all applications at machine: mac - if [ -e ./kill_app.sh ] - ./kill_app.sh - fi - ... where: mac = machine name. app = application name. ... = the same for other applications.

Specified by:

createOSLocalKillAllScript in class Machine

Parameters:

directory - The directory for this machine (use global variable?).

Throws:

IOFailure - If an error occurred during the creation of the local killall script.

createOSLocalStartAllScript

protected void createOSLocalStartAllScript(java.io.File directory)
                                    throws IOFailure

This function creates the script to start all applications on this machine. The scripts calls all the start script for each application. It also runs the script for starting any external database. pseudo code: - echo Starting all applications at machine: mac - if [ -e ./start_app.sh ] - ./start_app.sh - fi - ... where: mac = machine name. app = application name. ... = the same for other applications.

Specified by:

createOSLocalStartAllScript in class Machine

Parameters:

directory - The directory for this machine (use global variable?).

Throws:

IOFailure - If an error occurred during the creation of the local startall script.

createApplicationKillScripts

protected void createApplicationKillScripts(java.io.File directory)
                                     throws IOFailure

Creates the kill scripts for all the applications. The script starts by finding all running processes of the application. If it finds any processes, it kills them. The kill_app.sh should have the following structure: - echo Killing linux application. - #!/bin/bash - PIDS = $(ps -wwfe | grep fullapp | grep -v grep | grep path\settings_app.xml | awk "{print \\$2}") - if [ -n "$PIDS" ]; then - kill -9 $PIDS; - fi Also, if a heritrix process is started, the following is added: - PIDS = $(ps -wwfe | grep heritrix | grep -v grep | grep path\settings_app.xml | awk "{print \\$2}") - if [ -n "$PIDS" ]; then - kill -9 $PIDS; - fi where: path = the path to the ./conf directory. fullapp = the full application name with class path. app = the id of the application (name + instanceId). heritrix = the heritrix class path.

Specified by:

createApplicationKillScripts in class Machine

Parameters:

directory - The directory for this machine (use global variable?).

Throws:

IOFailure - If an error occured during the creation of the kill application script file.

createApplicationStartScripts

protected void createApplicationStartScripts(java.io.File directory)
                                      throws IOFailure

Creates the start scripts for all the applications. The application should only be started, if it is not running already. The script starts by finding all running processes of the application. If any processes are found, a new application should not be started. Otherwise start the application. The start_app.sh should have the following structure: - echo Starting linux application: app - cd path - #!/bin/bash - PIDS = $(ps -wwfe | grep fullapp | grep -v grep | grep path\settings_app.xml | awk "{print \\$2}") - if [ -n "$PIDS" ]; then - echo Application already running. - else - export CLASSPATH = cp:$CLASSPATH; - JAVA - fi where: path = the path to the install directory. fullapp = the full name application with java path. app = the name of the application. cp = the classpaths for the application. JAVA = the command to run the java application.

Specified by:

createApplicationStartScripts in class Machine

Parameters:

directory - The directory for this machine (use global variable?).

Throws:

IOFailure - If an error occurred during the creation of the start application script file.

osGetClassPath

protected java.lang.String osGetClassPath(Application app)

Description copied from class: Machine

Makes all the class paths into the operation system specific syntax, and puts them into a string where they are separated by the operation system specific separator (':' for linux, ';' for windows).

Specified by:

osGetClassPath in class Machine

Parameters:

app - The application which has the class paths.

Returns:

The class paths in operation system specific syntax.

osInstallDatabase

protected java.lang.String osInstallDatabase()

Description copied from class: Machine

Checks if a specific directory for the database is given in the settings, and thus if the database should be installed on this machine. If no specific database is given as deploy argument (databaseFileName = null) then use the standard database extracted from NetarchiveSuite.zip. Else send the given new database to the standard database location. Extract the database in the standard database location to the specified database directory.

Specified by:

osInstallDatabase in class Machine

Returns:

The script for installing the database (if needed).

osInstallExternalJarFiles

protected java.lang.String osInstallExternalJarFiles()

Description copied from class: Machine

This function makes the part of the install script for installing the external jar files from within the jarFolder. If the jarFolder is null, then no action will be performed.

Specified by:

osInstallExternalJarFiles in class Machine

Returns:

The script for installing the external jar files (if needed).

osInstallArchiveDatabase

protected java.lang.String osInstallArchiveDatabase()

Description copied from class: Machine

Checks if a specific directory for the archive database is given in the settings, and thus if the archive database should be installed on this machine. If not specific database is given (adminDatabaseFileName = null) then use the default in the NetarchiveSuite.zip package. Else send the new archive database to the standard database location, and extract it to the given location.

Specified by:

osInstallArchiveDatabase in class Machine

Returns:

The script for installing the archive database (if needed).

osInstallScriptCreateDir

protected java.lang.String osInstallScriptCreateDir()

Creates the specified directories in the deploy-configuration file. Structure - ssh login cd path; DIRS; CLEANDIR; exit; where: login = username@machine. path = path to install directory. DIRS = the way to create directories. CLEANDIR = the command to clean the tempDir (if chosen as optional) The install creation of DIR has the following structure for directory dir: if [ ! -d dir ]; then mkdir dir; fi;

Specified by:

osInstallScriptCreateDir in class Machine

Returns:

The script for creating the directories.

scriptCreateDir

protected java.lang.String scriptCreateDir(java.lang.String dir,
                                           boolean clean)

Description copied from class: Machine

This functions makes the script for creating the new directories. Linux creates directories directly through ssh. Windows creates an install a script file for installing the directories, which has to be sent to the machine, then executed and finally deleted.

Specified by:

scriptCreateDir in class Machine

Parameters:

dir - The name of the directory to create.

clean - Whether the directory should be cleaned\reset.

Returns:

The lines of code for creating the directories.

See Also:

Machine.createInstallDirScript(File)

createPathToDir

protected java.lang.String createPathToDir(java.lang.String dir)

Function for creating the directories along the path until the end directory. Does not create the end directory.

Parameters:

dir - The path to the directory.

Returns:

The script for creating the directory.

getAppDirectories

protected java.lang.String getAppDirectories()

Description copied from class: Machine

Creates the script for creating the application specified directories. Also creates the directories along the path to the directories.

Specified by:

getAppDirectories in class Machine

Returns:

The script for creating the application specified directories.

createInstallDirScript

protected void createInstallDirScript(java.io.File dir)

Dummy function on linux machine. This is only used for windows machines!

Specified by:

createInstallDirScript in class Machine

Parameters:

dir - The directory to put the file.

changeFileDirPathForSecurity

protected java.lang.String changeFileDirPathForSecurity(java.lang.String path)

Description copied from class: Machine

Changes the file directory path to the format used in the security policy.

Specified by:

changeFileDirPathForSecurity in class Machine

Parameters:

path - The current path.

Returns:

The formatted path.

getJMXremoteFilesCommand

protected java.lang.String getJMXremoteFilesCommand()

Description copied from class: Machine

This method does the following: Retrieves the path to the jmxremote.access and jmxremote.password files. Moves these files, if they are different from standard. Makes the jmxremote.access and jmxremote.password files readonly.

Specified by:

getJMXremoteFilesCommand in class Machine

Returns:

The commands for handling the jmxremote files.

createRestartScript

protected void createRestartScript(java.io.File dir)
                            throws IOFailure

Creates script for restarting all the applications on a machine. This script should start by killing all the existing processes, and then starting them again. First the killall scripts is called, then wait for 5 seconds for the applications to be fully terminated, and finally the startall script is called.

Specified by:

createRestartScript in class Machine

Parameters:

dir - The directory where the script file will be placed.

Throws:

IOFailure - If the restart script cannot be created.

createArchiveDatabaseStartScript

protected void createArchiveDatabaseStartScript(java.io.File dir)
                                         throws IOFailure

Creates a script for starting the archive database on a given machine. This is only created if the <globalArchiveDatabaseDir> parameter is defined on the machine level.
> #!/bin/bash
> cd InstallDir
> java -cp 'DB-CLASSPATH' org.apache.derby.drda.NetworkServerControl start < /dev/null > start_external_database.log 2>&1 &

Specified by:

createArchiveDatabaseStartScript in class Machine

Parameters:

dir - The directory where the script will be placed.

Throws:

IOFailure - If the script cannot be written.

callStartArchiveDatabase

protected java.lang.String callStartArchiveDatabase()

Method for generating the command for running the external_admin_database_start script. This should be called before the application on the machines have been started.
> echo Starting external database
> if [ -e ./start_external_admin_database.sh ]; then
> ./start_external_admin_database.sh &
> sleep 5
> fi

Returns:

The command for running external_admin_database_start script.

createArchiveDatabaseKillScript

protected void createArchiveDatabaseKillScript(java.io.File dir)
                                        throws IOFailure

Creates a script for killing the archive database on a given machine. This is only created if the <globalArchiveDatabaseDir> parameter is defined on the machine level. The output is appended to the log, thus the '>>' instead of the standard '>' when redirecting the output.
> #!/bin/bash
> cd InstallDir
> java -cp 'DB-CLASSPATH' org.apache.derby.drda.NetworkServerControl shutdown < /dev/null >> start_external_database.log 2>&1 &
where 'PORT' is in the setting: settings.archive.admin.database.port

Specified by:

createArchiveDatabaseKillScript in class Machine

Parameters:

dir - The directory where the script will be placed.

Throws:

IOFailure - If the script cannot be created.

callKillArchiveDatabase

protected java.lang.String callKillArchiveDatabase()

Method for generating the command for running the external_database_kill script. This should be called when the application on the machines have been killed.
> echo Killing external database
> if [ -e ./kill_external_database.sh ]; then
> ./kill_external_database.sh
> fi

Returns:

The command for running external_database_kill script.

createHarvestDatabaseStartScript

protected void createHarvestDatabaseStartScript(java.io.File dir)
                                         throws IOFailure

Creates a script for starting the harvest database on a given machine. This is only created if the <deployHarvestDatabaseDir> parameter is defined on the machine level.
> #!/bin/bash
> cd InstallDir
> java -cp 'DB-CLASSPATH' org.apache.derby.drda.NetworkServerControl start < /dev/null > start_external_harvest_database.log 2>&1 &

Specified by:

createHarvestDatabaseStartScript in class Machine

Parameters:

dir - The directory where the script will be placed.

Throws:

IOFailure - If the script cannot be written.

callStartHarvestDatabase

protected java.lang.String callStartHarvestDatabase()

Method for generating the command for running the external_harvest_database_start script. This should be called before the application on the machines have been started.
> echo Starting external harvest database
> if [ -e ./start_external_harvest_database.sh ]; then
> ./start_external_harvest_database.sh &
> sleep 5
> fi

Returns:

The command for running external_harvest_database_start script.

createHarvestDatabaseKillScript

protected void createHarvestDatabaseKillScript(java.io.File dir)
                                        throws IOFailure

Creates a script for killing the harvest database on a given machine. This is only created if the <globalHarvestDatabaseDir> parameter is defined on the machine level. The output is appended to the log, thus the '>>' instead of the standard '>' when redirecting the output.
> #!/bin/bash
> cd InstallDir
> java -cp 'DB-CLASSPATH' org.apache.derby.drda.NetworkServerControl shutdown < /dev/null >> start_external_harvest_database.log 2>&1 &
where 'PORT' is in the setting: settings.common.database.port

Specified by:

createHarvestDatabaseKillScript in class Machine

Parameters:

dir - The directory where the script will be placed.

Throws:

IOFailure - If the script cannot be created.

callKillHarvestDatabase

protected java.lang.String callKillHarvestDatabase()

Method for generating the command for running the external_database_kill script. This should be called when the application on the machines have been killed.
> echo Killing external harvest database
> if [ -e ./kill_external_harvest_database.sh ]; then
> ./kill_external_harvest_database.sh
> fi

Returns:

The command for running external_harvest_database_kill script.