org.datavec.api.transform.sequence.window

Class TimeWindowFunction

java.lang.Object

org.datavec.api.transform.sequence.window.TimeWindowFunction

All Implemented Interfaces:

Serializable, WindowFunction

public class TimeWindowFunction
extends Object
implements WindowFunction

A windowing function based on time, with non-overlapping windows. Time for each entry in the sequence is provided by a Time column
Functionality here: Calculate windows of data based on a fixed window size (1 minute, 1 hour, etc), with an optional offset.
Specifically, window start times T are calculated such that (T + timeZoneOffset + offset) % windowSize == 0 timeZoneOffset comes from the Time column metadata; offset allows for the window to be shifted one way or another, for example to allow for windowing like (10:15 to 11:15) instead of (10:00 to 11:00), using an offset of 15 minutes

Note that the windows generated by this window function need not contain any data - i.e., it can generate empty an empty window if no data occurs in the specified time period.

Author:

Alex Black

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	TimeWindowFunction.Builder

Constructor Summary

Constructors

Constructor and Description
TimeWindowFunction(String timeColumn, long windowSize, TimeUnit windowSizeUnit) Constructor with zero offset
TimeWindowFunction(String timeColumn, long windowSize, TimeUnit windowSizeUnit, boolean addWindowStartTimeColumn, boolean addWindowEndTimeColumn) Constructor with zero offset, and supports adding columns containing the start and/or end times of the window
TimeWindowFunction(String timeColumn, long windowSize, TimeUnit windowSizeUnit, long offset, TimeUnit offsetUnit) Constructor with optional offset
TimeWindowFunction(String timeColumn, long windowSize, TimeUnit windowSizeUnit, long offset, TimeUnit offsetUnit, boolean addWindowStartTimeColumn, boolean addWindowEndTimeColumn, boolean excludeEmptyWindows) Constructor with optional offset

Method Summary

Modifier and Type	Method and Description
List<List<List<Writable>>>	applyToSequence(List<List<Writable>> sequence) Apply the windowing function to the given sequence
Schema	getInputSchema()
long	getWindowEndTimeForTime(long time) Calculates the end time of the window for which the specified time belongs, in unix epoch (millisecond) format.
long	getWindowStartTimeForTime(long time) Calculates the start time of the window for which the specified time belongs, in unix epoch (millisecond) format For example, if the window size is 1 hour with offset 0, then a time 10:17 would return 10:00, as the 1 hour window is for 10:00:00.000 to 10:59:59.999 inclusive, or 10:00:00.000 (inclusive) to 11:00:00.000 (exclusive)
void	setInputSchema(Schema schema)
String	toString()
Schema	transform(Schema inputSchema) Get the output schema, given the input schema.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

TimeWindowFunction

public TimeWindowFunction(String timeColumn,
                          long windowSize,
                          TimeUnit windowSizeUnit)

Constructor with zero offset

Parameters:

timeColumn - Name of the column that contains the time values (must be a time column)

windowSize - Numerical quantity for the size of the time window (used in conjunction with windowSizeUnit)

windowSizeUnit - Unit of the time window

TimeWindowFunction

public TimeWindowFunction(String timeColumn,
                          long windowSize,
                          TimeUnit windowSizeUnit,
                          boolean addWindowStartTimeColumn,
                          boolean addWindowEndTimeColumn)

Constructor with zero offset, and supports adding columns containing the start and/or end times of the window

Parameters:

timeColumn - Name of the column that contains the time values (must be a time column)

windowSize - Numerical quantity for the size of the time window (used in conjunction with windowSizeUnit)

windowSizeUnit - Unit of the time window

addWindowStartTimeColumn - If true: add a time column (name: "windowStartTime") that contains the start time of the window

addWindowStartTimeColumn - If true: add a time column (name: "windowEndTime") that contains the end time of the window

TimeWindowFunction

public TimeWindowFunction(String timeColumn,
                          long windowSize,
                          TimeUnit windowSizeUnit,
                          long offset,
                          TimeUnit offsetUnit)

Constructor with optional offset

Parameters:

timeColumn - Name of the column that contains the time values (must be a time column)

windowSize - Numerical quantity for the size of the time window (used in conjunction with windowSizeUnit)

windowSizeUnit - Unit of the time window

offset - Optional offset amount, to shift start/end of the time window forward or back

offsetUnit - Optional offset unit for the offset amount.

TimeWindowFunction

public TimeWindowFunction(String timeColumn,
                          long windowSize,
                          TimeUnit windowSizeUnit,
                          long offset,
                          TimeUnit offsetUnit,
                          boolean addWindowStartTimeColumn,
                          boolean addWindowEndTimeColumn,
                          boolean excludeEmptyWindows)

Constructor with optional offset

Parameters:

timeColumn - Name of the column that contains the time values (must be a time column)

windowSize - Numerical quantity for the size of the time window (used in conjunction with windowSizeUnit)

windowSizeUnit - Unit of the time window

offset - Optional offset amount, to shift start/end of the time window forward or back

offsetUnit - Optional offset unit for the offset amount.

addWindowStartTimeColumn - If true: add a column (at the end) with the window start time

addWindowEndTimeColumn - If true: add a column (at the end) with the window end time

excludeEmptyWindows - If true: exclude any windows that don't have any values in them

Method Detail

setInputSchema

public void setInputSchema(Schema schema)

Specified by:

setInputSchema in interface WindowFunction

getInputSchema

public Schema getInputSchema()

Specified by:

getInputSchema in interface WindowFunction

Returns:

transform

public Schema transform(Schema inputSchema)

Description copied from interface: WindowFunction

Get the output schema, given the input schema. Typically the output schema is the same as the input schema, but not necessarily (for example, if the window function adds columns for the window start/end times)

Specified by:

transform in interface WindowFunction

Parameters:

inputSchema - Schema of the input data

Returns:

Schema of the output windows

toString

public String toString()

Overrides:

toString in class Object

applyToSequence

public List<List<List<Writable>>> applyToSequence(List<List<Writable>> sequence)

Description copied from interface: WindowFunction

Apply the windowing function to the given sequence

Specified by:

applyToSequence in interface WindowFunction

Parameters:

sequence - the input sequence

Returns:

the sequence with the window function applied

getWindowStartTimeForTime

public long getWindowStartTimeForTime(long time)

Calculates the start time of the window for which the specified time belongs, in unix epoch (millisecond) format
For example, if the window size is 1 hour with offset 0, then a time 10:17 would return 10:00, as the 1 hour window is for 10:00:00.000 to 10:59:59.999 inclusive, or 10:00:00.000 (inclusive) to 11:00:00.000 (exclusive)

Parameters:

time - Time at which to determine the window start time (milliseconds epoch format)

getWindowEndTimeForTime

public long getWindowEndTimeForTime(long time)

Calculates the end time of the window for which the specified time belongs, in unix epoch (millisecond) format. Note: this value is not included in the interval. Put another way, it is the start time of the next interval: i.e., is equivalent to getWindowStartTimeForTime(long) + interval (in milliseconds).
To get the last inclusive time for the interval, subtract 1L (1 millisecond) from the value returned by this method.
For example, if the window size is 1 hour with offset 0, then a time 10:17 would return 11:00, as the 1 hour window is for 10:00:00.000 (inclusive) to 11:00:00.000 (exclusive)

Parameters:

time - Time at which to determine the window start time

Returns: