We have prepared a bash shell script that starts all the necessary components on one machine. We will use this script throughout this quickstart manual to allow you to get a feel for what the system can do and how it works without having to deal with issues of distributing to other servers.
|Table of Contents|
Base system required
For the quick startup, NetarchiveSuite requires:
- A Linux system with a minimum of 2GB free diskspace. The QuickStart guide was tested with Ubuntu.
- Sun/Oracle Java 6 8 SE runtime (or higher). Other Java versions such as OpenJDK and Oracle Java 7 are not tested or recommended.
A An additional user is named "test". The commands to install NetarchiveSuite are run from your own login. The commands install and run the NetarchiveSuite software under user "test". This simulates the more realistic productions situation where the software runs under various logins on one or more machines in a distributed network. For convenience, it is a good idea to configure the test-user to have password-free ssh access - i.e. you should be able to execute "ssh test@localhost" in a shell without entering the test-user's password.
NetarchiveSuite uses Java Messaging Service (JMS) for communication between the different components. (The implementation used is OpenMQ 5.1)
Download the openmq installation script:
chmod +x mq.sh
Install the openmq broker:
sh ./mq.sh install
This will download openmq, install and start it.
Note: OpenMQ will as defaut be installed to ~/openmq4MessageQueue5.5. A alternative installdir can be defined the installdir variable prior to using the mq script, eg.
export installdir="netarchive/openmq" sh mq.sh start
Binary releases are available from https://sbforge.org/downloads/netarchivesuite/releases/.
Create a working directory and navigate to it
mkdir netarchive; cd netarchive
EITHER download Download the latest release (as of 2014-04-22 this is 4.2)
wget -N -O NetarchiveSuite.zip https://sbforge.org/downloads/netarchivesuite/releases/stable/45.2/NetarchiveSuite-45.2.0.zip
OR - if you need the latest and untested code - download the latest successful build:together with the latest bundled harvester
wget -N -O NetarchiveSuite-heritrix3-bundler.zip https://sbforge.org/jenkinsdownloads/jobnetarchivesuite/NetarchiveSuite-unittestreleases/lastSuccessfulBuildstable/artifact5.2/NetarchiveSuite.zip
wget -N https://sbforgeraw.orggithubusercontent.com/svnnetarchivesuite/netarchivesuite/trunk/master/deploy/distribution/src/main/resources/examples/deploy_standalone_example.xml wget -N https://sbforgeraw.orggithubusercontent.com/svnnetarchivesuite/netarchivesuite/trunk/master/deploy/deploy-core/scripts/RunNetarchiveSuite.sh chmod +x RunNetarchiveSuite.sh
The first script is a simple script for doing all the steps during deployment. It takes a NetarchiveSuite package ('.zip'), a configuration file (the second file), and a temporary installation directory, and the heritrix3 bundler zipfile as arguments (in the given order). The different ports used by the application for communication are included in the deploy_standalone_example.xml file.
In the configuration file all the applications are placed on one machine, the current machine (localhost).
When the installation script is run it will unpack the installation files into the netarchive/deploy directory as the current user and then - as the user "test" - install NetarchiveSuite into the /home/test/QUICKSTART directory (using ssh).
Important: As of 2014-04-22 an Oracle Java 6 JVM is required as the default Java for the user "test"! OpenJDK or Oracle Java 7 will not workRemember that Sun/Oracle Java 8 is required for the Quickstart procedure.
If you already have a Quickstart installation, the existing bitarchive, database and admin.data files will be untouched. You must explicitly remove any previous installation, if you want a clean empty installation.
Now To do the deployment:
./RunNetarchiveSuite.sh NetarchiveSuite-5,2.zip deploy_standalone_example.xml deploy/ NetarchiveSuite-heritrix3-bundler-5.2.zip
Note that if you have not setup your automatic ssh test user login (using key based login), you need to login some multiple times before the installation finishes successfully. You must also have permission to ssh and scp to test@localhost (try e.g ssh test@localhost)
The script creates a deployment folder named "QUICKSTART" in e.g. /home/test/QUICKSTARTunder the test users home directory, which contains methods for starting and stopping NetarchiveSuite, and starts the whole NetarchiveSuite. The files to run the installation will be placed in the ~/netarchive/deploy directory.
deploy under the directory where you ran the
Now configure your browser:
- Start a web browser.
Setup the browser to proxy on port 8070 on the host executing the netarchive, and exclude localhost and the hostname (used by the Heritrix GUI). In Firefox it is done as follows:
Anchor proxy proxy
Choose in the Firefox toolbar: Edit->Preferences->Advanced->Network->Settings Checkmark: Manual Proxy Configuration and add: Http proxy: name-of-host Port: 8070 No Proxy for: localhost, name-of-host
- Write the following url in the started browser http://localhost:8074/HarvestDefinition (if running on the local machine, otherwise go to port 8074 on the host running NetarchiveSuite).
You should see the Netarchive harvest definition page:
If you see a stacktrace instead of the Netarchive pages, this is most likely because the browser tries to go through the ViewerProxy. Revisit the browser configuration and try again.
- You can now see the webinterface in the browser. You can now create jobs, run them, and browse according to the following or the User Manualoutput following the basic instructions in the rest of this manual, or the more complete description in the User Manual.
- You can stop and start the entire NAS system with:
ssh test@localhost cd QUICKSTART ./conf/killall.sh ./conf/startall.sh
- If you want to try other deploy examples, then go to "Examples of deploy configuration files" in the Installation Manual.