org.apache.hadoop.mapreduce

Class CryptoUtils

java.lang.Object

org.apache.hadoop.mapreduce.CryptoUtils

@InterfaceAudience.Private
@InterfaceStability.Unstable
public class CryptoUtils
extends Object

This class provides utilities to make it easier to work with Cryptographic Streams. Specifically for dealing with encrypting intermediate data such MapReduce spill files.

Constructor Summary

Constructors

Constructor and Description
CryptoUtils()

Method Summary

Methods

Modifier and Type	Method and Description
static byte[]	createIV(org.apache.hadoop.conf.Configuration conf) This method creates and initializes an IV (Initialization Vector)
static int	cryptoPadding(org.apache.hadoop.conf.Configuration conf)
static boolean	isShuffleEncrypted(org.apache.hadoop.conf.Configuration conf)
static org.apache.hadoop.fs.FSDataInputStream	wrapIfNecessary(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.FSDataInputStream in) Wraps a given FSDataInputStream with a CryptoInputStream.
static org.apache.hadoop.fs.FSDataOutputStream	wrapIfNecessary(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.FSDataOutputStream out) Wraps a given FSDataOutputStream with a CryptoOutputStream.
static InputStream	wrapIfNecessary(org.apache.hadoop.conf.Configuration conf, InputStream in, long length) Wraps a given InputStream with a CryptoInputStream.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

CryptoUtils

public CryptoUtils()

Method Detail

isShuffleEncrypted

public static boolean isShuffleEncrypted(org.apache.hadoop.conf.Configuration conf)

createIV

public static byte[] createIV(org.apache.hadoop.conf.Configuration conf)
                       throws IOException

This method creates and initializes an IV (Initialization Vector)

Parameters:

conf -

Returns:

byte[]

Throws:

IOException

cryptoPadding

public static int cryptoPadding(org.apache.hadoop.conf.Configuration conf)

wrapIfNecessary

public static org.apache.hadoop.fs.FSDataOutputStream wrapIfNecessary(org.apache.hadoop.conf.Configuration conf,
                                                      org.apache.hadoop.fs.FSDataOutputStream out)
                                                               throws IOException

Wraps a given FSDataOutputStream with a CryptoOutputStream. The size of the data buffer required for the stream is specified by the "mapreduce.job.encrypted-intermediate-data.buffer.kb" Job configuration variable.

Parameters:

conf -

out -

Returns:

FSDataOutputStream

Throws:

IOException

wrapIfNecessary

public static InputStream wrapIfNecessary(org.apache.hadoop.conf.Configuration conf,
                          InputStream in,
                          long length)
                                   throws IOException

Wraps a given InputStream with a CryptoInputStream. The size of the data buffer required for the stream is specified by the "mapreduce.job.encrypted-intermediate-data.buffer.kb" Job configuration variable. If the value of 'length' is > -1, The InputStream is additionally wrapped in a LimitInputStream. CryptoStreams are late buffering in nature. This means they will always try to read ahead if they can. The LimitInputStream will ensure that the CryptoStream does not read past the provided length from the given Input Stream.

Parameters:

conf -

in -

length -

Returns:

InputStream

Throws:

IOException

wrapIfNecessary

public static org.apache.hadoop.fs.FSDataInputStream wrapIfNecessary(org.apache.hadoop.conf.Configuration conf,
                                                     org.apache.hadoop.fs.FSDataInputStream in)
                                                              throws IOException

Wraps a given FSDataInputStream with a CryptoInputStream. The size of the data buffer required for the stream is specified by the "mapreduce.job.encrypted-intermediate-data.buffer.kb" Job configuration variable.

Parameters:

conf -

in -

Returns:

FSDataInputStream

Throws:

IOException