| Package | Description |
|---|---|
| org.datavec.api.transform |
| Modifier and Type | Method and Description |
|---|---|
TransformProcess.Builder |
TransformProcess.Builder.addConstantColumn(String newColumnName,
ColumnType newColumnType,
Writable fixedValue)
Add a new column, where all values in the column are identical and as specified.
|
TransformProcess.Builder |
TransformProcess.Builder.addConstantDoubleColumn(String newColumnName,
double value)
Add a new double column, where the value for that column (for all records) are identical
|
TransformProcess.Builder |
TransformProcess.Builder.addConstantIntegerColumn(String newColumnName,
int value)
Add a new integer column, where th
e value for that column (for all records) are identical
|
TransformProcess.Builder |
TransformProcess.Builder.addConstantLongColumn(String newColumnName,
long value)
Add a new integer column, where the value for that column (for all records) are identical
|
TransformProcess.Builder |
TransformProcess.Builder.appendStringColumnTransform(String column,
String toAppend)
Append a String to a specified column
|
TransformProcess.Builder |
TransformProcess.Builder.calculateSortedRank(String newColumnName,
String sortOnColumn,
WritableComparator comparator)
CalculateSortedRank: calculate the rank of each example, after sorting example.
|
TransformProcess.Builder |
TransformProcess.Builder.calculateSortedRank(String newColumnName,
String sortOnColumn,
WritableComparator comparator,
boolean ascending)
CalculateSortedRank: calculate the rank of each example, after sorting example.
|
TransformProcess.Builder |
TransformProcess.Builder.categoricalToInteger(String... columnNames)
Convert the specified column(s) from a categorical representation to an integer representation.
|
TransformProcess.Builder |
TransformProcess.Builder.categoricalToOneHot(String... columnNames)
Convert the specified column(s) from a categorical representation to a one-hot representation.
|
TransformProcess.Builder |
TransformProcess.Builder.conditionalCopyValueTransform(String columnToReplace,
String sourceColumn,
Condition condition)
Replace the value in a specified column with a new value taken from another column, if a condition is satisfied/true.
Note that the condition can be any generic condition, including on other column(s), different to the column that will be modified if the condition is satisfied/true. |
TransformProcess.Builder |
TransformProcess.Builder.conditionalReplaceValueTransform(String column,
Writable newValue,
Condition condition)
Replace the values in a specified column with a specified new value, if some condition holds.
|
TransformProcess.Builder |
TransformProcess.Builder.conditionalReplaceValueTransformWithDefault(String column,
Writable yesVal,
Writable noVal,
Condition condition)
Replace the values in a specified column with a specified "yes" value, if some condition holds.
|
TransformProcess.Builder |
TransformProcess.Builder.convertFromSequence()
Convert a sequence to a set of individual values (by treating each value in each sequence as a separate example)
|
TransformProcess.Builder |
TransformProcess.Builder.convertToDouble(String inputColumn)
Convert the specified column to a double.
|
TransformProcess.Builder |
TransformProcess.Builder.convertToInteger(String inputColumn)
Convert the specified column to an integer.
|
TransformProcess.Builder |
TransformProcess.Builder.convertToSequence()
Convert a set of independent records/examples into a sequence; each example is simply treated as a sequence
of length 1, without any join/group operations.
|
TransformProcess.Builder |
TransformProcess.Builder.convertToSequence(List<String> keyColumns,
SequenceComparator comparator)
Convert a set of independent records/examples into a sequence, where each sequence is grouped according to
one or more key values (i.e., the values in one or more columns)
Within each sequence, values are ordered using the provided
SequenceComparator |
TransformProcess.Builder |
TransformProcess.Builder.convertToSequence(String keyColumn,
SequenceComparator comparator)
Convert a set of independent records/examples into a sequence, according to some key.
|
TransformProcess.Builder |
TransformProcess.Builder.convertToString(String inputColumn)
Convert the specified column to a string.
|
TransformProcess.Builder |
TransformProcess.Builder.doubleColumnsMathOp(String newColumnName,
MathOp mathOp,
String... columnNames)
Calculate and add a new double column by performing a mathematical operation on a number of existing columns.
|
TransformProcess.Builder |
TransformProcess.Builder.doubleMathFunction(String columnName,
MathFunction mathFunction)
Perform a mathematical operation (such as sin(x), ceil(x), exp(x) etc) on a column
|
TransformProcess.Builder |
TransformProcess.Builder.doubleMathOp(String columnName,
MathOp mathOp,
double scalar)
Perform a mathematical operation (add, subtract, scalar max etc) on the specified double column, with a scalar
|
TransformProcess.Builder |
TransformProcess.Builder.duplicateColumn(String column,
String newName)
Duplicate a single column
|
TransformProcess.Builder |
TransformProcess.Builder.duplicateColumns(List<String> columnNames,
List<String> newNames)
Duplicate a set of columns
|
TransformProcess.Builder |
TransformProcess.Builder.filter(Condition condition)
Add a filter operation, based on the specified condition.
|
TransformProcess.Builder |
TransformProcess.Builder.filter(Filter filter)
Add a filter operation to be executed after the previously-added operations have been executed
|
TransformProcess.Builder |
TransformProcess.Builder.firstDigitTransform(String inputColumn,
String outputColumn)
FirstDigitTransform converts a column to a categorical column, with values being the first digit of the number.
For example, "3.1415" becomes "3" and "2.0" becomes "2". Negative numbers ignore the sign: "-7.123" becomes "7". Note that two FirstDigitTransform.Modes are supported, which determines how non-numerical entries should be handled:EXCEPTION_ON_INVALID: output has 10 category values ("0", ..., "9"), and any non-numerical values result in an exception INCLUDE_OTHER_CATEGORY: output has 11 category values ("0", ..., "9", "Other"), all non-numerical values are mapped to "Other" FirstDigitTransform is useful (combined with CategoricalToOneHotTransform and Reductions) to implement
Benford's law. |
TransformProcess.Builder |
TransformProcess.Builder.firstDigitTransform(String inputColumn,
String outputColumn,
FirstDigitTransform.Mode mode)
FirstDigitTransform converts a column to a categorical column, with values being the first digit of the number.
For example, "3.1415" becomes "3" and "2.0" becomes "2". Negative numbers ignore the sign: "-7.123" becomes "7". Note that two FirstDigitTransform.Modes are supported, which determines how non-numerical entries should be handled:EXCEPTION_ON_INVALID: output has 10 category values ("0", ..., "9"), and any non-numerical values result in an exception INCLUDE_OTHER_CATEGORY: output has 11 category values ("0", ..., "9", "Other"), all non-numerical values are mapped to "Other" FirstDigitTransform is useful (combined with CategoricalToOneHotTransform and Reductions) to implement
Benford's law. |
TransformProcess.Builder |
TransformProcess.Builder.floatColumnsMathOp(String newColumnName,
MathOp mathOp,
String... columnNames)
Calculate and add a new float column by performing a mathematical operation on a number of existing columns.
|
TransformProcess.Builder |
TransformProcess.Builder.floatMathFunction(String columnName,
MathFunction mathFunction)
Perform a mathematical operation (such as sin(x), ceil(x), exp(x) etc) on a column
|
TransformProcess.Builder |
TransformProcess.Builder.floatMathOp(String columnName,
MathOp mathOp,
float scalar)
Perform a mathematical operation (add, subtract, scalar max etc) on the specified double column, with a scalar
|
TransformProcess.Builder |
TransformProcess.Builder.integerColumnsMathOp(String newColumnName,
MathOp mathOp,
String... columnNames)
Calculate and add a new integer column by performing a mathematical operation on a number of existing columns.
|
TransformProcess.Builder |
TransformProcess.Builder.integerMathOp(String column,
MathOp mathOp,
int scalar)
Perform a mathematical operation (add, subtract, scalar max etc) on the specified integer column, with a scalar
|
TransformProcess.Builder |
TransformProcess.Builder.integerToCategorical(String columnName,
List<String> categoryStateNames)
Convert the specified column from an integer representation (assume values 0 to numCategories-1) to
a categorical representation, given the specified state names
|
TransformProcess.Builder |
TransformProcess.Builder.integerToCategorical(String columnName,
Map<Integer,String> categoryIndexNameMap)
Convert the specified column from an integer representation to a categorical representation, given the specified
mapping between integer indexes and state names
|
TransformProcess.Builder |
TransformProcess.Builder.integerToOneHot(String columnName,
int minValue,
int maxValue)
Convert an integer column to a set of 1 hot columns, based on the value in integer column
|
TransformProcess.Builder |
TransformProcess.Builder.longColumnsMathOp(String newColumnName,
MathOp mathOp,
String... columnNames)
Calculate and add a new long column by performing a mathematical operation on a number of existing columns.
|
TransformProcess.Builder |
TransformProcess.Builder.longMathOp(String columnName,
MathOp mathOp,
long scalar)
Perform a mathematical operation (add, subtract, scalar max etc) on the specified long column, with a scalar
|
TransformProcess.Builder |
TransformProcess.Builder.ndArrayColumnsMathOpTransform(String newColumnName,
MathOp mathOp,
String... columnNames)
Perform an element wise mathematical operation (such as add, subtract, multiply) on NDArray columns.
|
TransformProcess.Builder |
TransformProcess.Builder.ndArrayDistanceTransform(String newColumnName,
Distance distance,
String firstCol,
String secondCol)
Calculate a distance (cosine similarity, Euclidean, Manhattan) on two equal-sized NDArray columns.
|
TransformProcess.Builder |
TransformProcess.Builder.ndArrayMathFunctionTransform(String columnName,
MathFunction mathFunction)
Apply an element wise mathematical function (sin, tanh, abs etc) to an NDArray column.
|
TransformProcess.Builder |
TransformProcess.Builder.ndArrayScalarOpTransform(String columnName,
MathOp op,
double value)
Element-wise NDArray math operation (add, subtract, etc) on an NDArray column
|
TransformProcess.Builder |
TransformProcess.Builder.normalize(String column,
Normalize type,
DataAnalysis da)
Normalize the specified column with a given type of normalization
|
TransformProcess.Builder |
TransformProcess.Builder.offsetSequence(List<String> columnsToOffset,
int offsetAmount,
SequenceOffsetTransform.OperationType operationType)
Perform a sequence of operation on the specified columns.
|
TransformProcess.Builder |
TransformProcess.Builder.reduce(IAssociativeReducer reducer)
Reduce (i.e., aggregate/combine) a set of examples (typically by key).
|
TransformProcess.Builder |
TransformProcess.Builder.reduceSequence(IAssociativeReducer reducer)
Reduce (i.e., aggregate/combine) a set of sequence examples - for each sequence individually.
|
TransformProcess.Builder |
TransformProcess.Builder.reduceSequenceByWindow(IAssociativeReducer reducer,
WindowFunction windowFunction)
Reduce (i.e., aggregate/combine) a set of sequence examples - for each sequence individually - using a window function.
|
TransformProcess.Builder |
TransformProcess.Builder.removeAllColumnsExceptFor(Collection<String> columnNames)
Remove all columns, except for those that are specified here
|
TransformProcess.Builder |
TransformProcess.Builder.removeAllColumnsExceptFor(String... columnNames)
Remove all columns, except for those that are specified here
|
TransformProcess.Builder |
TransformProcess.Builder.removeColumns(Collection<String> columnNames)
Remove all of the specified columns, by name
|
TransformProcess.Builder |
TransformProcess.Builder.removeColumns(String... columnNames)
Remove all of the specified columns, by name
|
TransformProcess.Builder |
TransformProcess.Builder.renameColumn(String oldName,
String newName)
Rename a single column
|
TransformProcess.Builder |
TransformProcess.Builder.renameColumns(List<String> oldNames,
List<String> newNames)
Rename multiple columns
|
TransformProcess.Builder |
TransformProcess.Builder.reorderColumns(String... newOrder)
Reorder the columns using a partial or complete new ordering.
|
TransformProcess.Builder |
TransformProcess.Builder.replaceStringTransform(String columnName,
Map<String,String> mapping)
Replace one or more String values in the specified column that match regular expressions.
|
TransformProcess.Builder |
TransformProcess.Builder.sequenceMovingWindowReduce(String columnName,
int lookback,
ReduceOp op)
SequenceMovingWindowReduceTransform: Adds a new column, where the value is derived by:
(a) using a window of the last N values in a single column, (b) Apply a reduction op on the window to calculate a new value for example, this transformer can be used to implement a simple moving average of the last N values, or determine the minimum or maximum values in the last N time steps. |
TransformProcess.Builder |
TransformProcess.Builder.splitSequence(SequenceSplit split)
Split sequences into 1 or more other sequences.
|
TransformProcess.Builder |
TransformProcess.Builder.stringMapTransform(String columnName,
Map<String,String> mapping)
Replace one or more String values in the specified column with new values.
|
TransformProcess.Builder |
TransformProcess.Builder.stringRemoveWhitespaceTransform(String columnName)
Remove all whitespace characters from the values in the specified String column
|
TransformProcess.Builder |
TransformProcess.Builder.stringToCategorical(String columnName,
List<String> stateNames)
Convert the specified String column to a categorical column.
|
TransformProcess.Builder |
TransformProcess.Builder.stringToTimeTransform(String column,
String format,
org.joda.time.DateTimeZone dateTimeZone)
Convert a String column (containing a date/time String) to a time column (by parsing the date/time String)
|
TransformProcess.Builder |
TransformProcess.Builder.stringToTimeTransform(String column,
String format,
org.joda.time.DateTimeZone dateTimeZone,
Locale locale)
Convert a String column (containing a date/time String) to a time column (by parsing the date/time String)
|
TransformProcess.Builder |
TransformProcess.Builder.timeMathOp(String columnName,
MathOp mathOp,
long timeQuantity,
TimeUnit timeUnit)
Perform a mathematical operation (add, subtract, scalar min/max only) on the specified time column
|
TransformProcess.Builder |
TransformProcess.Builder.transform(Transform transform)
Add a transformation to be executed after the previously-added operations have been executed
|
TransformProcess.Builder |
TransformProcess.Builder.trimOrPadSequenceToLength(int length,
@NonNull List<Writable> pad)
Trim or pad the sequence to the specified length (number of sequence steps).
Sequences longer than the specified maximum will be trimmed to exactly the maximum. |
TransformProcess.Builder |
TransformProcess.Builder.trimSequence(int numStepsToTrim,
boolean trimFromStart)
SequenceTrimTranform removes the first or last N values in a sequence.
|
TransformProcess.Builder |
TransformProcess.Builder.trimSequenceToLength(int maxLength)
Trim the sequence to the specified length (number of sequence steps).
Sequences longer than the specified maximum will be trimmed to exactly the maximum. |
Copyright © 2022. All rights reserved.