ReaderParameters

These are properties for customizing mainframe binary data reader.

isEbcdic: If true the input data file encoding is EBCDIC, otherwise it is ASCII
ebcdicCodePage: Specifies what code page to use for EBCDIC to ASCII/Unicode conversions
ebcdicCodePageClass: An optional custom code page conversion class provided by a user
asciiCharset: A charset for ASCII data
floatingPointFormat: A format of floating-point numbers
variableSizeOccurs: If true, OCCURS DEPENDING ON data size will depend on the number of elements
lengthFieldName: A name of a field that contains record length. Optional. If not set the copybook record length will be used.
isRecordSequence: Does input files have 4 byte record length headers
isRdwBigEndian: Is RDW big endian? It may depend on flavor of mainframe and/or mainframe to PC transfer method
isRdwPartRecLength: Does RDW count itself as part of record length itself
rdwAdjustment: Controls a mismatch between RDW and record length
isIndexGenerationNeeded: Is indexing input file before processing is requested
inputSplitRecords: The number of records to include in each partition. Notice mainframe records may have variable size, inputSplitMB is the recommended option
inputSplitSizeMB: A partition size to target. In certain circumstances this size may not be exactly that, but the library will do the best effort to target that size
hdfsDefaultBlockSize: Default HDFS block size for the HDFS filesystem used. This value is used as the default split size if inputSplitSizeMB is not specified
startOffset: An offset to the start of the record in each binary data block.
endOffset: An offset from the end of the record to the end of the binary data block.
fileStartOffset: A number of bytes to skip at the beginning of each file
fileEndOffset: A number of bytes to skip at the end of each file
generateRecordId: If true, a record id field will be prepended to each record.
schemaPolicy: Specifies a policy to transform the input schema. The default policy is to keep the schema exactly as it is in the copybook.
stringTrimmingPolicy: Specifies if and how strings should be trimmed when parsed.
multisegment: Parameters specific to reading multisegment files
commentPolicy: A comment truncation policy
dropGroupFillers: If true the parser will drop all FILLER fields, even GROUP FILLERS that have non-FILLER nested fields
nonTerminals: A list of non-terminals (GROUPS) to combine and parse as primitive fields
recordHeaderParser: A parser used to parse data field record headers
rhpAdditionalInfo: An optional additional option string passed to a custom record header parser
inputFileNameColumn: A column name to add to the dataframe. The column will contain input file name for each record similar to 'input_file_name()' function

Linear Supertypes

Serializable, Serializable, Product, Equals, AnyRef, Any

Instance Constructors

new ReaderParameters(isEbcdic: Boolean = true, ebcdicCodePage: String = "common", ebcdicCodePageClass: Option[String] = None, asciiCharset: String = "", floatingPointFormat: FloatingPointFormat = FloatingPointFormat.IBM, variableSizeOccurs: Boolean = false, lengthFieldName: Option[String] = None, isRecordSequence: Boolean = false, isRdwBigEndian: Boolean = false, isRdwPartRecLength: Boolean = false, rdwAdjustment: Int = 0, isIndexGenerationNeeded: Boolean = false, inputSplitRecords: Option[Int] = None, inputSplitSizeMB: Option[Int] = None, hdfsDefaultBlockSize: Option[Int] = None, startOffset: Int = 0, endOffset: Int = 0, fileStartOffset: Int = 0, fileEndOffset: Int = 0, generateRecordId: Boolean = false, schemaPolicy: SchemaRetentionPolicy = SchemaRetentionPolicy.KeepOriginal, stringTrimmingPolicy: StringTrimmingPolicy = StringTrimmingPolicy.TrimBoth, multisegment: Option[MultisegmentParameters] = None, commentPolicy: CommentPolicy = CommentPolicy(), dropGroupFillers: Boolean = false, nonTerminals: Seq[String] = Nil, recordHeaderParser: Option[String] = None, rhpAdditionalInfo: Option[String] = None, inputFileNameColumn: String = "")

isEbcdic
If true the input data file encoding is EBCDIC, otherwise it is ASCII
ebcdicCodePage
Specifies what code page to use for EBCDIC to ASCII/Unicode conversions
ebcdicCodePageClass
An optional custom code page conversion class provided by a user
asciiCharset
A charset for ASCII data
floatingPointFormat
A format of floating-point numbers
variableSizeOccurs
If true, OCCURS DEPENDING ON data size will depend on the number of elements
lengthFieldName
A name of a field that contains record length. Optional. If not set the copybook record length will be used.
isRecordSequence
Does input files have 4 byte record length headers
isRdwBigEndian
Is RDW big endian? It may depend on flavor of mainframe and/or mainframe to PC transfer method
isRdwPartRecLength
Does RDW count itself as part of record length itself
rdwAdjustment
Controls a mismatch between RDW and record length
isIndexGenerationNeeded
Is indexing input file before processing is requested
inputSplitRecords
The number of records to include in each partition. Notice mainframe records may have variable size, inputSplitMB is the recommended option
inputSplitSizeMB
A partition size to target. In certain circumstances this size may not be exactly that, but the library will do the best effort to target that size
hdfsDefaultBlockSize
Default HDFS block size for the HDFS filesystem used. This value is used as the default split size if inputSplitSizeMB is not specified
startOffset
An offset to the start of the record in each binary data block.
endOffset
An offset from the end of the record to the end of the binary data block.
fileStartOffset
A number of bytes to skip at the beginning of each file
fileEndOffset
A number of bytes to skip at the end of each file
generateRecordId
If true, a record id field will be prepended to each record.
schemaPolicy
Specifies a policy to transform the input schema. The default policy is to keep the schema exactly as it is in the copybook.
stringTrimmingPolicy
Specifies if and how strings should be trimmed when parsed.
multisegment
Parameters specific to reading multisegment files
commentPolicy
A comment truncation policy
dropGroupFillers
If true the parser will drop all FILLER fields, even GROUP FILLERS that have non-FILLER nested fields
nonTerminals
A list of non-terminals (GROUPS) to combine and parse as primitive fields
recordHeaderParser
A parser used to parse data field record headers
rhpAdditionalInfo
An optional additional option string passed to a custom record header parser
inputFileNameColumn
A column name to add to the dataframe. The column will contain input file name for each record similar to 'input_file_name()' function

Value Members

final def !=(arg0: Any): Boolean

Definition Classes
AnyRef → Any
final def ##(): Int

Definition Classes
AnyRef → Any
final def ==(arg0: Any): Boolean

Definition Classes
AnyRef → Any
final def asInstanceOf[T0]: T0

Definition Classes
Any
val asciiCharset: String

A charset for ASCII data
def clone(): AnyRef

Attributes
protected[java.lang]
Definition Classes
AnyRef
Annotations
@throws( ... )
val commentPolicy: CommentPolicy

A comment truncation policy
val dropGroupFillers: Boolean

If true the parser will drop all FILLER fields, even GROUP FILLERS that have non-FILLER nested fields
val ebcdicCodePage: String

Specifies what code page to use for EBCDIC to ASCII/Unicode conversions
val ebcdicCodePageClass: Option[String]

An optional custom code page conversion class provided by a user
val endOffset: Int

An offset from the end of the record to the end of the binary data block.
final def eq(arg0: AnyRef): Boolean

Definition Classes
AnyRef
val fileEndOffset: Int

A number of bytes to skip at the end of each file
val fileStartOffset: Int

A number of bytes to skip at the beginning of each file
def finalize(): Unit

Attributes
protected[java.lang]
Definition Classes
AnyRef
Annotations
@throws( classOf[java.lang.Throwable] )
val floatingPointFormat: FloatingPointFormat

A format of floating-point numbers
val generateRecordId: Boolean

If true, a record id field will be prepended to each record.
final def getClass(): Class[_]

Definition Classes
AnyRef → Any
val hdfsDefaultBlockSize: Option[Int]

Default HDFS block size for the HDFS filesystem used.
Default HDFS block size for the HDFS filesystem used. This value is used as the default split size if inputSplitSizeMB is not specified
val inputFileNameColumn: String

A column name to add to the dataframe.
A column name to add to the dataframe. The column will contain input file name for each record similar to 'input_file_name()' function
val inputSplitRecords: Option[Int]

The number of records to include in each partition.
The number of records to include in each partition. Notice mainframe records may have variable size, inputSplitMB is the recommended option
val inputSplitSizeMB: Option[Int]

A partition size to target.
A partition size to target. In certain circumstances this size may not be exactly that, but the library will do the best effort to target that size
val isEbcdic: Boolean

If true the input data file encoding is EBCDIC, otherwise it is ASCII
val isIndexGenerationNeeded: Boolean

Is indexing input file before processing is requested
final def isInstanceOf[T0]: Boolean

Definition Classes
Any
val isRdwBigEndian: Boolean

Is RDW big endian? It may depend on flavor of mainframe and/or mainframe to PC transfer method
val isRdwPartRecLength: Boolean

Does RDW count itself as part of record length itself
val isRecordSequence: Boolean

Does input files have 4 byte record length headers
val lengthFieldName: Option[String]

A name of a field that contains record length.
A name of a field that contains record length. Optional. If not set the copybook record length will be used.
val multisegment: Option[MultisegmentParameters]

Parameters specific to reading multisegment files
final def ne(arg0: AnyRef): Boolean

Definition Classes
AnyRef
val nonTerminals: Seq[String]

A list of non-terminals (GROUPS) to combine and parse as primitive fields
final def notify(): Unit

Definition Classes
AnyRef
final def notifyAll(): Unit

Definition Classes
AnyRef
val rdwAdjustment: Int

Controls a mismatch between RDW and record length
val recordHeaderParser: Option[String]

A parser used to parse data field record headers
val rhpAdditionalInfo: Option[String]

An optional additional option string passed to a custom record header parser
val schemaPolicy: SchemaRetentionPolicy

Specifies a policy to transform the input schema.
Specifies a policy to transform the input schema. The default policy is to keep the schema exactly as it is in the copybook.
val startOffset: Int

An offset to the start of the record in each binary data block.
val stringTrimmingPolicy: StringTrimmingPolicy

Specifies if and how strings should be trimmed when parsed.
final def synchronized[T0](arg0: ⇒ T0): T0

Definition Classes
AnyRef
val variableSizeOccurs: Boolean

If true, OCCURS DEPENDING ON data size will depend on the number of elements
final def wait(): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )
final def wait(arg0: Long, arg1: Int): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )
final def wait(arg0: Long): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )

Related Doc: package parameters

Instance Constructors

Value Members

final def !=(arg0: Any): Boolean

final def ##(): Int

final def ==(arg0: Any): Boolean

final def asInstanceOf[T0]: T0

val asciiCharset: String

def clone(): AnyRef

val commentPolicy: CommentPolicy

val dropGroupFillers: Boolean

val ebcdicCodePage: String

val ebcdicCodePageClass: Option[String]

val endOffset: Int

final def eq(arg0: AnyRef): Boolean

val fileEndOffset: Int

val fileStartOffset: Int

def finalize(): Unit

val floatingPointFormat: FloatingPointFormat

val generateRecordId: Boolean

final def getClass(): Class[_]

val hdfsDefaultBlockSize: Option[Int]

val inputFileNameColumn: String

val inputSplitRecords: Option[Int]

val inputSplitSizeMB: Option[Int]

val isEbcdic: Boolean

val isIndexGenerationNeeded: Boolean

final def isInstanceOf[T0]: Boolean

val isRdwBigEndian: Boolean

val isRdwPartRecLength: Boolean

val isRecordSequence: Boolean

val lengthFieldName: Option[String]

val multisegment: Option[MultisegmentParameters]

final def ne(arg0: AnyRef): Boolean

val nonTerminals: Seq[String]

final def notify(): Unit

final def notifyAll(): Unit

val rdwAdjustment: Int

val recordHeaderParser: Option[String]

val rhpAdditionalInfo: Option[String]

val schemaPolicy: SchemaRetentionPolicy

val startOffset: Int

val stringTrimmingPolicy: StringTrimmingPolicy

final def synchronized[T0](arg0: ⇒ T0): T0

val variableSizeOccurs: Boolean

final def wait(): Unit

final def wait(arg0: Long, arg1: Int): Unit

final def wait(arg0: Long): Unit

Inherited from Serializable

Inherited from Serializable

Inherited from Product

Inherited from Equals

Inherited from AnyRef

Inherited from Any

Ungrouped