Package org.datavec.api.split
Class TransformSplit
- java.lang.Object
-
- org.datavec.api.split.BaseInputSplit
-
- org.datavec.api.split.TransformSplit
-
- All Implemented Interfaces:
InputSplit
public class TransformSplit extends BaseInputSplit
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description static interface
TransformSplit.URITransform
-
Field Summary
-
Fields inherited from class org.datavec.api.split.BaseInputSplit
iterationOrder, length, uriStrings
-
-
Constructor Summary
Constructors Constructor Description TransformSplit(@NonNull BaseInputSplit sourceSplit, @NonNull TransformSplit.URITransform transform)
Apply a given transformation to the raw URI objects
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Modifier and Type Method Description void
bootStrapForWrite()
Bootstrap this input split for writing.boolean
needsBootstrapForWrite()
Returns true if thisInputSplit
needs bootstrapping for writing.static TransformSplit
ofSearchReplace(@NonNull BaseInputSplit sourceSplit, @NonNull String search, @NonNull String replace)
Static factory method, replace the string version of the URI with a simple search-replace pairInputStream
openInputStreamFor(String location)
Open anInputStream
for the given location.OutputStream
openOutputStreamFor(String location)
Open anOutputStream
for the given location.void
reset()
Reset the InputSplit without reinitializing it from scratch.boolean
resetSupported()
void
updateSplitLocations(boolean reset)
Refreshes the split locations if needed in memory.-
Methods inherited from class org.datavec.api.split.BaseInputSplit
addNewLocation, addNewLocation, canWriteToLocation, length, locations, locationsIterator, locationsPathIterator, sample
-
-
-
-
Constructor Detail
-
TransformSplit
public TransformSplit(@NonNull @NonNull BaseInputSplit sourceSplit, @NonNull @NonNull TransformSplit.URITransform transform) throws URISyntaxException
Apply a given transformation to the raw URI objects- Parameters:
sourceSplit
- the split with URIs to transformtransform
- transform operation that returns a new URI based on an input URI- Throws:
URISyntaxException
- thrown if the transformed URI is malformed
-
-
Method Detail
-
ofSearchReplace
public static TransformSplit ofSearchReplace(@NonNull @NonNull BaseInputSplit sourceSplit, @NonNull @NonNull String search, @NonNull @NonNull String replace) throws URISyntaxException
Static factory method, replace the string version of the URI with a simple search-replace pair- Parameters:
sourceSplit
- the split with URIs to transformsearch
- the string to searchreplace
- the string to replace with- Throws:
URISyntaxException
- thrown if the transformed URI is malformed
-
updateSplitLocations
public void updateSplitLocations(boolean reset)
Description copied from interface:InputSplit
Refreshes the split locations if needed in memory. (Think a few file gets added)
-
needsBootstrapForWrite
public boolean needsBootstrapForWrite()
Description copied from interface:InputSplit
Returns true if thisInputSplit
needs bootstrapping for writing. A simple example of needing bootstrapping is forFileSplit
where there is only a directory existing, but no file to write to- Returns:
- true if this input split needs bootstrapping for writing to or not
-
bootStrapForWrite
public void bootStrapForWrite()
Description copied from interface:InputSplit
Bootstrap this input split for writing. This is for use withRecordWriter
-
openOutputStreamFor
public OutputStream openOutputStreamFor(String location) throws Exception
Description copied from interface:InputSplit
Open anOutputStream
for the given location. Note that the user is responsible for closing the associated output stream.- Parameters:
location
- the location to open the output stream for- Returns:
- the output input stream
- Throws:
Exception
-
openInputStreamFor
public InputStream openInputStreamFor(String location) throws Exception
Description copied from interface:InputSplit
Open anInputStream
for the given location. Note that the user is responsible for closing the associated input stream.- Parameters:
location
- the location to open the input stream for- Returns:
- the opened input stream
- Throws:
Exception
-
reset
public void reset()
Description copied from interface:InputSplit
Reset the InputSplit without reinitializing it from scratch. In many cases, this is a no-op. For InputSplits that have randomization: reset should shuffle the order.
-
resetSupported
public boolean resetSupported()
- Returns:
- True if the reset() method is supported (or is a no-op), false otherwise. If false is returned, reset() may throw an exception
-
-