org.apache.hadoop.mapred
Class IFile.Reader<K,V>

java.lang.Object
  org.apache.hadoop.mapred.IFile.Reader<K,V>

Direct Known Subclasses:

InMemoryReader

Enclosing class:

IFile

@InterfaceAudience.Private
@InterfaceStability.Unstable
public static class IFile.Reader<K,V>
extends Object

IFile.Reader to read intermediate map-outputs.

Field Summary

protected byte[]	buffer
protected int	bufferSize
long	bytesRead
protected int	currentKeyLength
protected int	currentValueLength
protected DataInputStream	dataIn
protected boolean	eof
protected long	fileLength
protected int	recNo

Constructor Summary

IFile.Reader(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path file, org.apache.hadoop.io.compress.CompressionCodec codec, Counters.Counter readsCounter)
Construct an IFile Reader.

IFile.Reader(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.FSDataInputStream in, long length, org.apache.hadoop.io.compress.CompressionCodec codec, Counters.Counter readsCounter)
Construct an IFile Reader.

Method Summary

void	close()
void	disableChecksumValidation()
long	getLength()
long	getPosition()
boolean	nextRawKey(org.apache.hadoop.io.DataInputBuffer key)
void	nextRawValue(org.apache.hadoop.io.DataInputBuffer value)
protected boolean	positionToNextRecord(DataInput dIn)
void	reset(int offset)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

bytesRead

public long bytesRead

fileLength

protected final long fileLength

eof

protected boolean eof

buffer

protected byte[] buffer

bufferSize

protected int bufferSize

dataIn

protected DataInputStream dataIn

recNo

protected int recNo

currentKeyLength

protected int currentKeyLength

currentValueLength

protected int currentValueLength

Constructor Detail

IFile.Reader

public IFile.Reader(org.apache.hadoop.conf.Configuration conf,
                    org.apache.hadoop.fs.FileSystem fs,
                    org.apache.hadoop.fs.Path file,
                    org.apache.hadoop.io.compress.CompressionCodec codec,
                    Counters.Counter readsCounter)
             throws IOException

Construct an IFile Reader.

Parameters:

conf - Configuration File

fs - FileSystem

file - Path of the file to be opened. This file should have checksum bytes for the data at the end of the file.

codec - codec

readsCounter - Counter for records read from disk

Throws:

IOException

IFile.Reader

public IFile.Reader(org.apache.hadoop.conf.Configuration conf,
                    org.apache.hadoop.fs.FSDataInputStream in,
                    long length,
                    org.apache.hadoop.io.compress.CompressionCodec codec,
                    Counters.Counter readsCounter)
             throws IOException

Construct an IFile Reader.

Parameters:

conf - Configuration File

in - The input stream

length - Length of the data in the stream, including the checksum bytes.

codec - codec

readsCounter - Counter for records read from disk

Throws:

IOException

Method Detail

getLength

public long getLength()

getPosition

public long getPosition()
                 throws IOException

Throws:

IOException

positionToNextRecord

protected boolean positionToNextRecord(DataInput dIn)
                                throws IOException

Throws:

IOException

nextRawKey

public boolean nextRawKey(org.apache.hadoop.io.DataInputBuffer key)
                   throws IOException

Throws:

IOException

nextRawValue

public void nextRawValue(org.apache.hadoop.io.DataInputBuffer value)
                  throws IOException

Throws:

IOException

close

public void close()
           throws IOException

Throws:

IOException

reset

public void reset(int offset)

disableChecksumValidation

public void disableChecksumValidation()