Package dk.netarkivet.wayback.hadoop

Class CDXMapper

  • public class CDXMapper
extends org.apache.hadoop.mapreduce.Mapper<org.apache.hadoop.io.LongWritable,​org.apache.hadoop.io.Text,​org.apache.hadoop.io.NullWritable,​org.apache.hadoop.io.Text>
    Hadoop Mapper for creating the CDX indexes. The input is a key (not used) and a Text line, which we assume is the path to an archive file. The output is an exit code (not used), and the generated CDX lines.

    • Nested Class Summary

      • Nested classes/interfaces inherited from class org.apache.hadoop.mapreduce.Mapper

        org.apache.hadoop.mapreduce.Mapper.Context

    • Constructor Summary

      Constructors 
      Constructor Description
      CDXMapper()  

    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      protected void map​(org.apache.hadoop.io.LongWritable linenumber, org.apache.hadoop.io.Text archiveFilePath, org.apache.hadoop.mapreduce.Mapper.Context context)
      Mapping method.

      • Methods inherited from class org.apache.hadoop.mapreduce.Mapper

        cleanup, run, setup

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    • Method Detail

      • map

        protected void map​(org.apache.hadoop.io.LongWritable linenumber,
                   org.apache.hadoop.io.Text archiveFilePath,
                   org.apache.hadoop.mapreduce.Mapper.Context context)
            throws java.io.IOException,
                   java.lang.InterruptedException
        Mapping method.
        Overrides:
        map in class org.apache.hadoop.mapreduce.Mapper<org.apache.hadoop.io.LongWritable,​org.apache.hadoop.io.Text,​org.apache.hadoop.io.NullWritable,​org.apache.hadoop.io.Text>
        Parameters:
        linenumber - The linenumber. Is ignored.
        archiveFilePath - The path to the archive file.
        context - Context used for writing output.
        Throws:
        java.io.IOException - If it fails to generate the CDX indexes.
        java.lang.InterruptedException