Package dk.netarkivet.common.utils.hadoop

Class HadoopJobTool

java.lang.Object

org.apache.hadoop.conf.Configured

dk.netarkivet.common.utils.hadoop.HadoopJobTool

All Implemented Interfaces:

org.apache.hadoop.conf.Configurable, org.apache.hadoop.util.Tool

public class HadoopJobTool
extends org.apache.hadoop.conf.Configured
implements org.apache.hadoop.util.Tool

A simple generic Hadoop map-only tool that runs a given mapper on the passed input file containing new-line separated file paths and outputs the job's resulting files in the passed output path

Constructor Summary

Constructors

Constructor	Description
HadoopJobTool(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.mapreduce.Mapper<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,org.apache.hadoop.io.NullWritable,org.apache.hadoop.io.Text> mapper)

Method Summary

Modifier and Type	Method	Description
int	run(String[] args)	Method for running the tool/job.

Methods inherited from class org.apache.hadoop.conf.Configured

getConf, setConf

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.hadoop.conf.Configurable

getConf, setConf

Constructor Detail

HadoopJobTool

public HadoopJobTool(org.apache.hadoop.conf.Configuration conf,
                     org.apache.hadoop.mapreduce.Mapper<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,org.apache.hadoop.io.NullWritable,org.apache.hadoop.io.Text> mapper)

Method Detail

run

public int run(String[] args)
        throws InterruptedException,
               IOException,
               ClassNotFoundException

Method for running the tool/job.

Specified by:

run in interface org.apache.hadoop.util.Tool

Parameters:

args - Expects two strings representing the job's in- and output paths (Tool interface dictates String[])

Returns:

An exitcode to report back if the job succeeded.

Throws:

InterruptedException

IOException

ClassNotFoundException