String streamName
The name of the stream.
com.amazonaws.internal.SdkInternalMap<K,V> tags
The set of key-value pairs to use to create the tags.
String streamName
A name to identify the stream. The stream name is scoped to the AWS account used by the application that creates the stream. It is also scoped by region. That is, two streams in two different AWS accounts can have the same name, and two streams in the same AWS account but in two different regions can have the same name.
Integer shardCount
The number of shards that the stream will use. The throughput of the stream is a function of the number of shards; more shards are required for greater provisioned throughput.
DefaultShardLimit;
String streamName
The name of the stream to delete.
StreamDescription streamDescription
The current status of the stream, the stream ARN, an array of shard objects that comprise the stream, and states whether there are more shards available.
String streamName
The name of the Amazon Kinesis stream for which to disable enhanced monitoring.
com.amazonaws.internal.SdkInternalList<T> shardLevelMetrics
List of shard-level metrics to disable.
The following are the valid shard-level metrics. The value "ALL
" disables every metric.
IncomingBytes
IncomingRecords
OutgoingBytes
OutgoingRecords
WriteProvisionedThroughputExceeded
ReadProvisionedThroughputExceeded
IteratorAgeMilliseconds
ALL
For more information, see Monitoring the Amazon Kinesis Streams Service with Amazon CloudWatch in the Amazon Kinesis Streams Developer Guide.
String streamName
The name of the Amazon Kinesis stream.
com.amazonaws.internal.SdkInternalList<T> currentShardLevelMetrics
Represents the current state of the metrics that are in the enhanced state before the operation.
com.amazonaws.internal.SdkInternalList<T> desiredShardLevelMetrics
Represents the list of all the metrics that would be in the enhanced state after the operation.
String streamName
The name of the stream for which to enable enhanced monitoring.
com.amazonaws.internal.SdkInternalList<T> shardLevelMetrics
List of shard-level metrics to enable.
The following are the valid shard-level metrics. The value "ALL
" enables every metric.
IncomingBytes
IncomingRecords
OutgoingBytes
OutgoingRecords
WriteProvisionedThroughputExceeded
ReadProvisionedThroughputExceeded
IteratorAgeMilliseconds
ALL
For more information, see Monitoring the Amazon Kinesis Streams Service with Amazon CloudWatch in the Amazon Kinesis Streams Developer Guide.
String streamName
The name of the Amazon Kinesis stream.
com.amazonaws.internal.SdkInternalList<T> currentShardLevelMetrics
Represents the current state of the metrics that are in the enhanced state before the operation.
com.amazonaws.internal.SdkInternalList<T> desiredShardLevelMetrics
Represents the list of all the metrics that would be in the enhanced state after the operation.
com.amazonaws.internal.SdkInternalList<T> shardLevelMetrics
List of shard-level metrics.
The following are the valid shard-level metrics. The value "ALL
" enhances every metric.
IncomingBytes
IncomingRecords
OutgoingBytes
OutgoingRecords
WriteProvisionedThroughputExceeded
ReadProvisionedThroughputExceeded
IteratorAgeMilliseconds
ALL
For more information, see Monitoring the Amazon Kinesis Streams Service with Amazon CloudWatch in the Amazon Kinesis Streams Developer Guide.
String shardIterator
The position in the shard from which you want to start sequentially reading data records. A shard iterator specifies this position using the sequence number of a data record in the shard.
Integer limit
The maximum number of records to return. Specify a value of up to 10,000. If you specify a value that is greater
than 10,000, GetRecords throws InvalidArgumentException
.
com.amazonaws.internal.SdkInternalList<T> records
The data records retrieved from the shard.
String nextShardIterator
The next position in the shard from which to start sequentially reading data records. If set to null
, the shard has been closed and the requested iterator will not return any more data.
Long millisBehindLatest
The number of milliseconds the GetRecords response is from the tip of the stream, indicating how far behind current time the consumer is. A value of zero indicates record processing is caught up, and there are no new records to process at this moment.
String streamName
The name of the Amazon Kinesis stream.
String shardId
The shard ID of the Amazon Kinesis shard to get the iterator for.
String shardIteratorType
Determines how the shard iterator is used to start reading data records from the shard.
The following are the valid Amazon Kinesis shard iterator types:
StartingSequenceNumber
.StartingSequenceNumber
.Timestamp
.String startingSequenceNumber
The sequence number of the data record in the shard from which to start reading. Used with shard iterator type AT_SEQUENCE_NUMBER and AFTER_SEQUENCE_NUMBER.
Date timestamp
The timestamp of the data record from which to start reading. Used with shard iterator type AT_TIMESTAMP. A
timestamp is the Unix epoch date with precision in milliseconds. For example,
2016-04-04T19:58:46.480-00:00
or 1459799926.480
. If a record with this exact timestamp
does not exist, the iterator returned is for the next (later) record. If the timestamp is older than the current
trim horizon, the iterator returned is for the oldest untrimmed data record (TRIM_HORIZON).
String shardIterator
The position in the shard from which to start reading data records sequentially. A shard iterator specifies this position using the sequence number of a data record in a shard.
com.amazonaws.internal.SdkInternalList<T> streamNames
The names of the streams that are associated with the AWS account making the ListStreams
request.
Boolean hasMoreStreams
If set to true
, there are more streams available to list.
String streamName
The name of the stream.
String exclusiveStartTagKey
The key to use as the starting point for the list of tags. If this parameter is set,
ListTagsForStream
gets all tags that occur after ExclusiveStartTagKey
.
Integer limit
The number of tags to return. If this number is less than the total number of tags associated with the stream,
HasMoreTags
is set to true
. To list additional tags, set
ExclusiveStartTagKey
to the last key in the response.
com.amazonaws.internal.SdkInternalList<T> tags
A list of tags associated with StreamName
, starting with the first tag after
ExclusiveStartTagKey
and up to the specified Limit
.
Boolean hasMoreTags
If set to true
, more tags are available. To request additional tags, set
ExclusiveStartTagKey
to the key of the last tag returned.
String streamName
The name of the stream to put the data record into.
ByteBuffer data
The data blob to put into the record, which is base64-encoded when the blob is serialized. When the data blob (the payload before base64-encoding) is added to the partition key size, the total size must not exceed the maximum record size (1 MB).
String partitionKey
Determines which shard in the stream the data record is assigned to. Partition keys are Unicode strings with a maximum length limit of 256 characters for each key. Amazon Kinesis uses the partition key as input to a hash function that maps the partition key and associated data to a specific shard. Specifically, an MD5 hash function is used to map partition keys to 128-bit integer values and to map associated data records to shards. As a result of this hashing mechanism, all data records with the same partition key map to the same shard within the stream.
String explicitHashKey
The hash value used to explicitly determine the shard the data record is assigned to by overriding the partition key hash.
String sequenceNumberForOrdering
Guarantees strictly increasing sequence numbers, for puts from the same client and to the same partition key.
Usage: set the SequenceNumberForOrdering
of record n to the sequence number of record
n-1 (as returned in the result when putting record n-1). If this parameter is not set, records will
be coarsely ordered based on arrival time.
String shardId
The shard ID of the shard where the data record was placed.
String sequenceNumber
The sequence number identifier that was assigned to the put data record. The sequence number for the record is unique across all records in the stream. A sequence number is the identifier associated with every record put into the stream.
com.amazonaws.internal.SdkInternalList<T> records
The records associated with the request.
String streamName
The stream name associated with the request.
ByteBuffer data
The data blob to put into the record, which is base64-encoded when the blob is serialized. When the data blob (the payload before base64-encoding) is added to the partition key size, the total size must not exceed the maximum record size (1 MB).
String explicitHashKey
The hash value used to determine explicitly the shard that the data record is assigned to by overriding the partition key hash.
String partitionKey
Determines which shard in the stream the data record is assigned to. Partition keys are Unicode strings with a maximum length limit of 256 characters for each key. Amazon Kinesis uses the partition key as input to a hash function that maps the partition key and associated data to a specific shard. Specifically, an MD5 hash function is used to map partition keys to 128-bit integer values and to map associated data records to shards. As a result of this hashing mechanism, all data records with the same partition key map to the same shard within the stream.
Integer failedRecordCount
The number of unsuccessfully processed records in a PutRecords
request.
com.amazonaws.internal.SdkInternalList<T> records
An array of successfully and unsuccessfully processed record results, correlated with the request by natural
ordering. A record that is successfully added to a stream includes SequenceNumber
and
ShardId
in the result. A record that fails to be added to a stream includes ErrorCode
and ErrorMessage
in the result.
String sequenceNumber
The sequence number for an individual record result.
String shardId
The shard ID for an individual record result.
String errorCode
The error code for an individual record result. ErrorCodes
can be either
ProvisionedThroughputExceededException
or InternalFailure
.
String errorMessage
The error message for an individual record result. An ErrorCode
value of
ProvisionedThroughputExceededException
has an error message that includes the account ID, stream
name, and shard ID. An ErrorCode
value of InternalFailure
has the error message
"Internal Service Failure"
.
String sequenceNumber
The unique identifier of the record in the stream.
Date approximateArrivalTimestamp
The approximate time that the record was inserted into the stream.
ByteBuffer data
The data blob. The data in the blob is both opaque and immutable to the Amazon Kinesis service, which does not inspect, interpret, or change the data in the blob in any way. When the data blob (the payload before base64-encoding) is added to the partition key size, the total size must not exceed the maximum record size (1 MB).
String partitionKey
Identifies which shard in the stream the data record is assigned to.
String streamName
The name of the stream.
com.amazonaws.internal.SdkInternalList<T> tagKeys
A list of tag keys. Each corresponding tag is removed from the stream.
String shardId
The unique identifier of the shard within the stream.
String parentShardId
The shard ID of the shard's parent.
String adjacentParentShardId
The shard ID of the shard adjacent to the shard's parent.
HashKeyRange hashKeyRange
The range of possible hash key values for the shard, which is a set of ordered contiguous positive integers.
SequenceNumberRange sequenceNumberRange
The range of possible sequence numbers for the shard.
String streamName
The name of the stream for the shard split.
String shardToSplit
The shard ID of the shard to split.
String newStartingHashKey
A hash key value for the starting hash key of one of the child shards created by the split. The hash key range
for a given shard constitutes a set of ordered contiguous positive integers. The value for
NewStartingHashKey
must be in the range of hash keys being mapped into the shard. The
NewStartingHashKey
hash key value and all higher hash key values in hash key range are distributed
to one of the child shards. All the lower hash key values in the range are distributed to the other child shard.
String streamName
The name of the stream being described.
String streamARN
The Amazon Resource Name (ARN) for the stream being described.
String streamStatus
The current status of the stream being described. The stream status is one of the following states:
CREATING
- The stream is being created. Amazon Kinesis immediately returns and sets
StreamStatus
to CREATING
.DELETING
- The stream is being deleted. The specified stream is in the DELETING
state until Amazon Kinesis completes the deletion.ACTIVE
- The stream exists and is ready for read and write operations or deletion. You should
perform read and write operations only on an ACTIVE
stream.UPDATING
- Shards in the stream are being merged or split. Read and write operations continue to
work while the stream is in the UPDATING
state.com.amazonaws.internal.SdkInternalList<T> shards
The shards that comprise the stream.
Boolean hasMoreShards
If set to true
, more shards in the stream are available to describe.
Integer retentionPeriodHours
The current retention period, in hours.
com.amazonaws.internal.SdkInternalList<T> enhancedMonitoring
Represents the current enhanced monitoring settings of the stream.
String key
A unique identifier for the tag. Maximum length: 128 characters. Valid characters: Unicode letters, digits, white space, _ . / = + - % @
String value
An optional string, typically used to describe or define the tag. Maximum length: 256 characters. Valid characters: Unicode letters, digits, white space, _ . / = + - % @
String applicationName
Name of your existing Amazon Kinesis Analytics application to which you want to add the streaming source.
Long currentApplicationVersionId
Current version of your Amazon Kinesis Analytics application. You can use the DescribeApplication operation to find the current application version.
Input input
String applicationName
Name of the application to which you want to add the output configuration.
Long currentApplicationVersionId
Version of the application to which you want add the output configuration. You can use the
DescribeApplication operation to get the current application version. If the version specified is not the
current version, the ConcurrentModificationException
is returned.
Output output
An array of objects, each describing one output configuration. In the output configuration, you specify the name of an in-application stream, a destination (that is, an Amazon Kinesis stream or an Amazon Kinesis Firehose delivery stream), and record the formation to use when writing to the destination.
String applicationName
Name of an existing application.
Long currentApplicationVersionId
Version of the application for which you are adding the reference data source. You can use the
DescribeApplication operation to get the current application version. If the version specified is not the
current version, the ConcurrentModificationException
is returned.
ReferenceDataSource referenceDataSource
The reference data source can be an object in your Amazon S3 bucket. Amazon Kinesis Analytics reads the object and copies the data into the in-application table that is created. You provide an S3 bucket, object key name, and the resulting in-application table that is created. You must also provide an IAM role with the necessary permissions that Amazon Kinesis Analytics can assume to read the object from your S3 bucket on your behalf.
String applicationName
Name of the application.
String applicationDescription
Description of the application.
String applicationARN
ARN of the application.
String applicationStatus
Status of the application.
Date createTimestamp
Timestamp when the application version was created.
Date lastUpdateTimestamp
Timestamp when the application was last updated.
List<E> inputDescriptions
Describes the application input configuration. For more information, see Configuring Application Input.
List<E> outputDescriptions
Describes the application output configuration. For more information, see Configuring Application Output.
List<E> referenceDataSourceDescriptions
Describes reference data sources configured for the application. For more information, see Configuring Application Input.
String applicationCode
Returns the application code that you provided to perform data analysis on any of the in-application streams in your application.
Long applicationVersionId
Provides the current application version.
List<E> inputUpdates
Describes application input configuration updates.
String applicationCodeUpdate
Describes application code updates.
List<E> outputUpdates
Describes application output configuration updates.
List<E> referenceDataSourceUpdates
Describes application reference data source updates.
String applicationName
Name of your Amazon Kinesis Analytics application (for example, sample-app
).
String applicationDescription
Summary description of the application.
List<E> inputs
Use this parameter to configure the application input.
You can configure your application to receive input from a single streaming source. In this configuration, you map this streaming source to an in-application stream that is created. Your application code can then query the in-application stream like a table (you can think of it as a constantly updating table).
For the streaming source, you provide its Amazon Resource Name (ARN) and format of data on the stream (for example, JSON, CSV, etc). You also must provide an IAM role that Amazon Kinesis Analytics can assume to read this stream on your behalf.
To create the in-application stream, you need to specify a schema to transform your data into a schematized version used in SQL. In the schema, you provide the necessary mapping of the data elements in the streaming source to record columns in the in-app stream.
List<E> outputs
You can configure application output to write data from any of the in-application streams to up to five destinations.
These destinations can be Amazon Kinesis streams, Amazon Kinesis Firehose delivery streams, or both.
In the configuration, you specify the in-application stream name, the destination stream Amazon Resource Name (ARN), and the format to use when writing data. You must also provide an IAM role that Amazon Kinesis Analytics can assume to write to the destination stream on your behalf.
In the output configuration, you also provide the output stream Amazon Resource Name (ARN) and the format of data in the stream (for example, JSON, CSV). You also must provide an IAM role that Amazon Kinesis Analytics can assume to write to this stream on your behalf.
String applicationCode
One or more SQL statements that read input data, transform it, and generate output. For example, you can write a SQL statement that reads input data and generates a running average of the number of advertisement clicks by vendor.
You can also provide a series of SQL statements, where output of one statement can be used as the input for the next statement.
Note that the application code must create the streams with names specified in the Outputs
. For
example, if your Outputs
defines output streams named ExampleOutputStream1
and
ExampleOutputStream2
, then your application code must create these streams.
ApplicationSummary applicationSummary
In response to your CreateApplication
request, Amazon Kinesis Analytics returns a response with a
summary of the application it created, including the application Amazon Resource Name (ARN), name, and status.
String applicationName
Amazon Kinesis Analytics application name.
Long currentApplicationVersionId
Amazon Kinesis Analytics application version. You can use the DescribeApplication operation to get the
current application version. If the version specified is not the current version, the
ConcurrentModificationException
is returned.
String outputId
The ID of the configuration to delete. Each output configuration that is added to the application, either when
the application is created or later using the AddApplicationOutput operation, has a unique ID. You need to
provide the ID to uniquely identify the output configuration that you want to delete from the application
configuration. You can use the DescribeApplication operation to get the specific OutputId
.
String applicationName
Name of an existing application.
Long currentApplicationVersionId
Version of the application. You can use the DescribeApplication operation to get the current application
version. If the version specified is not the current version, the ConcurrentModificationException
is
returned.
String referenceId
ID of the reference data source. When you add a reference data source to your application using the AddApplicationReferenceDataSource, Amazon Kinesis Analytics assigns an ID. You can use the DescribeApplication operation to get the reference ID.
String applicationName
Name of the application.
ApplicationDetail applicationDetail
Provides a description of the application, such as the application Amazon Resource Name (ARN), status, latest version, and input and output configuration details.
String recordFormatType
Specifies the format of the records on the output stream.
String resourceARN
Amazon Resource Name (ARN) of the streaming source.
String roleARN
ARN of the IAM role that Amazon Kinesis Analytics can assume to access the stream on your behalf.
InputStartingPositionConfiguration inputStartingPositionConfiguration
Point at which you want Amazon Kinesis Analytics to start reading records from the specified streaming source discovery purposes.
SourceSchema inputSchema
Schema inferred from the streaming source. It identifies the format of the data in the streaming source and how each data element maps to corresponding columns in the in-application stream that you can create.
List<E> parsedInputRecords
An array of elements, where each element corresponds to a row in a stream record (a stream record can have more than one row).
List<E> rawInputRecords
Raw stream data that was sampled to infer the schema.
String namePrefix
Name prefix to use when creating in-application stream. Suppose you specify a prefix "MyInApplicationStream".
Kinesis Analytics will then create one or more (as per the InputParallelism
count you specified)
in-application streams with names "MyInApplicationStream_001", "MyInApplicationStream_002" and so on.
KinesisStreamsInput kinesisStreamsInput
If the streaming source is an Amazon Kinesis stream, identifies the stream's Amazon Resource Name (ARN) and an IAM role that enables Amazon Kinesis Analytics to access the stream on your behalf.
KinesisFirehoseInput kinesisFirehoseInput
If the streaming source is an Amazon Kinesis Firehose delivery stream, identifies the Firehose delivery stream's ARN and an IAM role that enables Amazon Kinesis Analytics to access the stream on your behalf.
InputParallelism inputParallelism
Describes the number of in-application streams to create.
Data from your source will be routed to these in-application input streams.
SourceSchema inputSchema
Describes the format of the data in the streaming source, and how each data element maps to corresponding columns in the in-application stream that is being created.
Also used to describe the format of the reference data source.
String id
Input source ID. You can get this ID by calling the DescribeApplication operation.
InputStartingPositionConfiguration inputStartingPositionConfiguration
Point at which you want the application to start processing records from the streaming source.
String inputId
Input ID associated with the application input. This is the ID that Amazon Kinesis Analytics assigns to each input configuration you add to your application.
String namePrefix
In-application name prefix.
List<E> inAppStreamNames
Returns the in-application stream names that are mapped to the stream source.
KinesisStreamsInputDescription kinesisStreamsInputDescription
If an Amazon Kinesis stream is configured as streaming source, provides Amazon Kinesis stream's ARN and an IAM role that enables Amazon Kinesis Analytics to access the stream on your behalf.
KinesisFirehoseInputDescription kinesisFirehoseInputDescription
If an Amazon Kinesis Firehose delivery stream is configured as a streaming source, provides the Firehose delivery stream's Amazon Resource Name (ARN) and an IAM role that enables Amazon Kinesis Analytics to access the stream on your behalf.
SourceSchema inputSchema
InputParallelism inputParallelism
Describes the configured parallelism (number of in-application streams mapped to the streaming source).
InputStartingPositionConfiguration inputStartingPositionConfiguration
Point at which the application is configured to read from the input stream.
Integer countUpdate
Number of in-application streams to create for the specified streaming source.
RecordFormat recordFormatUpdate
Specifies the format of the records on the streaming source.
String recordEncodingUpdate
Specifies the encoding of the records in the streaming source. For example, UTF-8.
List<E> recordColumnUpdates
A list of RecordColumn
objects. Each object describes the mapping of the streaming source element to
the corresponding column in the in-application stream.
String inputStartingPosition
The starting position on the stream.
LATEST
- Start reading just after the most recent record in the stream.
TRIM_HORIZON
- Start reading at the last untrimmed record in the stream, which is the oldest record
available in the stream. This option is not available for an Amazon Kinesis Firehose delivery stream.
LAST_STOPPED_POINT
- Resume reading from where the application last stopped reading.
String inputId
Input ID of the application input to be updated.
String namePrefixUpdate
Name prefix for in-application stream(s) that Kinesis Analytics creates for the specific streaming source.
KinesisStreamsInputUpdate kinesisStreamsInputUpdate
If a Amazon Kinesis stream is the streaming source to be updated, provides an updated stream ARN and IAM role ARN.
KinesisFirehoseInputUpdate kinesisFirehoseInputUpdate
If an Amazon Kinesis Firehose delivery stream is the streaming source to be updated, provides an updated stream Amazon Resource Name (ARN) and IAM role ARN.
InputSchemaUpdate inputSchemaUpdate
Describes the data format on the streaming source, and how record elements on the streaming source map to columns of the in-application stream that is created.
InputParallelismUpdate inputParallelismUpdate
Describes the parallelism updates (the number in-application streams Kinesis Analytics creates for the specific streaming source).
String recordRowPath
Path to the top-level parent that contains the records.
For example, consider the following JSON record:
In the RecordRowPath
, "$"
refers to the root and path "$.vehicle.Model"
refers to the specific "Model"
key in the JSON.
String resourceARNUpdate
ARN of the input Amazon Kinesis Firehose delivery stream to read.
String roleARNUpdate
Amazon Resource Name (ARN) of the IAM role that Amazon Kinesis Analytics can assume to access the stream on your behalf. You need to grant necessary permissions to this role.
String resourceARNUpdate
Amazon Resource Name (ARN) of the Amazon Kinesis Firehose delivery stream to write to.
String roleARNUpdate
ARN of the IAM role that Amazon Kinesis Analytics can assume to access the stream on your behalf. You need to grant necessary permissions to this role.
String resourceARNUpdate
Amazon Resource Name (ARN) of the input Amazon Kinesis stream to read.
String roleARNUpdate
ARN of the IAM role that Amazon Kinesis Analytics can assume to access the stream on your behalf. You need to grant the necessary permissions to this role.
String resourceARNUpdate
Amazon Resource Name (ARN) of the Amazon Kinesis stream where you want to write the output.
String roleARNUpdate
ARN of the IAM role that Amazon Kinesis Analytics can assume to access the stream on your behalf. You need to grant the necessary permissions to this role.
Integer limit
Maximum number of applications to list.
String exclusiveStartApplicationName
Name of the application to start the list with. When using pagination to retrieve the list, you don't need to specify this parameter in the first request. However, in subsequent requests, you add the last application name from the previous response to get the next page of applications.
JSONMappingParameters jSONMappingParameters
Provides additional mapping information when JSON is the record format on the streaming source.
CSVMappingParameters cSVMappingParameters
Provides additional mapping information when the record format uses delimiters (for example, CSV).
String name
Name of the in-application stream.
KinesisStreamsOutput kinesisStreamsOutput
Identifies an Amazon Kinesis stream as the destination.
KinesisFirehoseOutput kinesisFirehoseOutput
Identifies an Amazon Kinesis Firehose delivery stream as the destination.
DestinationSchema destinationSchema
String outputId
A unique identifier for the output configuration.
String name
Name of the in-application stream configured as output.
KinesisStreamsOutputDescription kinesisStreamsOutputDescription
Describes Amazon Kinesis stream configured as the destination where output is written.
KinesisFirehoseOutputDescription kinesisFirehoseOutputDescription
Describes the Amazon Kinesis Firehose delivery stream configured as the destination where output is written.
DestinationSchema destinationSchema
Data format used for writing data to the destination.
String outputId
Identifies the specific output configuration that you want to update.
String nameUpdate
If you want to specify a different in-application stream for this output configuration, use this field to specify the new in-application stream name.
KinesisStreamsOutputUpdate kinesisStreamsOutputUpdate
Describes an Amazon Kinesis stream as the destination for the output.
KinesisFirehoseOutputUpdate kinesisFirehoseOutputUpdate
Describes a Amazon Kinesis Firehose delivery stream as the destination for the output.
DestinationSchema destinationSchemaUpdate
String name
Name of the column created in the in-application input stream or reference table.
String mapping
Reference to the data element in the streaming input of the reference data source.
String sqlType
Type of column created in the in-application input stream or reference table.
String recordFormatType
The type of record format.
MappingParameters mappingParameters
String tableName
Name of the in-application table to create.
S3ReferenceDataSource s3ReferenceDataSource
SourceSchema referenceSchema
String referenceId
ID of the reference data source. This is the ID that Amazon Kinesis Analytics assigns when you add the reference data source to your application using the AddApplicationReferenceDataSource operation.
String tableName
The in-application table name created by the specific reference data source configuration.
S3ReferenceDataSourceDescription s3ReferenceDataSourceDescription
Provides the S3 bucket name, the object key name that contains the reference data. It also provides the Amazon Resource Name (ARN) of the IAM role that Amazon Kinesis Analytics can assume to read the Amazon S3 object and populate the in-application reference table.
SourceSchema referenceSchema
String referenceId
ID of the reference data source being updated. You can use the DescribeApplication operation to get this value.
String tableNameUpdate
In-application table name that is created by this update.
S3ReferenceDataSourceUpdate s3ReferenceDataSourceUpdate
Describes the S3 bucket name, object key name, and IAM role that Amazon Kinesis Analytics can assume to read the Amazon S3 object on your behalf and populate the in-application reference table.
SourceSchema referenceSchemaUpdate
String bucketARN
Amazon Resource Name (ARN) of the S3 bucket.
String fileKey
Object key name containing reference data.
String referenceRoleARN
ARN of the IAM role that the service can assume to read data on your behalf. This role must have permission for
the s3:GetObject
action on the object and trust policy that allows Amazon Kinesis Analytics service
principal to assume this role.
String bucketARN
Amazon Resource Name (ARN) of the S3 bucket.
String fileKey
Amazon S3 object key name.
String referenceRoleARN
ARN of the IAM role that Amazon Kinesis Analytics can assume to read the Amazon S3 object on your behalf to populate the in-application reference table.
String bucketARNUpdate
Amazon Resource Name (ARN) of the S3 bucket.
String fileKeyUpdate
Object key name.
String referenceRoleARNUpdate
ARN of the IAM role that Amazon Kinesis Analytics can assume to read the Amazon S3 object and populate the in-application.
RecordFormat recordFormat
Specifies the format of the records on the streaming source.
String recordEncoding
Specifies the encoding of the records in the streaming source. For example, UTF-8.
List<E> recordColumns
A list of RecordColumn
objects.
String applicationName
Name of the application.
List<E> inputConfigurations
Identifies the specific input, by ID, that the application starts consuming. Amazon Kinesis Analytics starts reading the streaming source associated with the input. You can also specify where in the streaming source you want Amazon Kinesis Analytics to start reading.
String applicationName
Name of the running application to stop.
String applicationName
Name of the Kinesis Analytics application to update.
Long currentApplicationVersionId
The current application version ID. You can use the DescribeApplication operation to get this value.
ApplicationUpdate applicationUpdate
Describes application updates.
Integer sizeInMBs
Buffer incoming data to the specified size, in MBs, before delivering it to the destination. The default value is 5.
We recommend setting SizeInMBs to a value greater than the amount of data you typically ingest into the delivery stream in 10 seconds. For example, if you typically ingest data at 1 MB/sec set SizeInMBs to be 10 MB or higher.
Integer intervalInSeconds
Buffer incoming data for the specified period of time, in seconds, before delivering it to the destination. The default value is 300.
Boolean enabled
Enables or disables CloudWatch logging.
String logGroupName
The CloudWatch group name for logging. This value is required if Enabled is true.
String logStreamName
The CloudWatch log stream name for logging. This value is required if Enabled is true.
String dataTableName
The name of the target table. The table must already exist in the database.
String dataTableColumns
A comma-separated list of column names.
String copyOptions
Optional parameters to use with the Amazon Redshift COPY
command. For more information, see the
"Optional Parameters" section of Amazon
Redshift COPY command. Some possible examples that would apply to Firehose are as follows.
delimiter '\t' lzop;
- fields are delimited with "\t" (TAB character) and compressed using lzop.
delimiter '|
- fields are delimited with "|" (this is the default delimiter).
delimiter '|' escape
- the delimiter should be escaped.
fixedwidth 'venueid:3,venuename:25,venuecity:12,venuestate:2,venueseats:6'
- fields are fixed width
in the source, with each width specified after every column in the table.
JSON 's3://mybucket/jsonpaths.txt'
- data is in JSON format, and the path specified is the format of
the data.
For more examples, see Amazon Redshift COPY command examples.
String deliveryStreamName
The name of the delivery stream.
S3DestinationConfiguration s3DestinationConfiguration
The destination in Amazon S3. This value must be specified if ElasticsearchDestinationConfiguration or RedshiftDestinationConfiguration is specified (see restrictions listed above).
RedshiftDestinationConfiguration redshiftDestinationConfiguration
The destination in Amazon Redshift. This value cannot be specified if Amazon S3 or Amazon Elasticsearch is the desired destination (see restrictions listed above).
ElasticsearchDestinationConfiguration elasticsearchDestinationConfiguration
The destination in Amazon ES. This value cannot be specified if Amazon S3 or Amazon Redshift is the desired destination (see restrictions listed above).
String deliveryStreamARN
The ARN of the delivery stream.
String deliveryStreamName
The name of the delivery stream.
String deliveryStreamName
The name of the delivery stream.
String deliveryStreamARN
The Amazon Resource Name (ARN) of the delivery stream.
String deliveryStreamStatus
The status of the delivery stream.
String versionId
Used when calling the UpdateDestination operation. Each time the destination is updated for the delivery stream, the VersionId is changed, and the current VersionId is required when updating the destination. This is so that the service knows it is applying the changes to the correct version of the delivery stream.
Date createTimestamp
The date and time that the delivery stream was created.
Date lastUpdateTimestamp
The date and time that the delivery stream was last updated.
List<E> destinations
The destinations.
Boolean hasMoreDestinations
Indicates whether there are more destinations available to list.
String deliveryStreamName
The name of the delivery stream.
Integer limit
The limit on the number of destinations to return. Currently, you can have one destination per delivery stream.
String exclusiveStartDestinationId
Specifies the destination ID to start returning the destination information. Currently Firehose supports one destination per delivery stream.
DeliveryStreamDescription deliveryStreamDescription
Information about the delivery stream.
String destinationId
The ID of the destination.
S3DestinationDescription s3DestinationDescription
The Amazon S3 destination.
RedshiftDestinationDescription redshiftDestinationDescription
The destination in Amazon Redshift.
ElasticsearchDestinationDescription elasticsearchDestinationDescription
The destination in Amazon ES.
Integer intervalInSeconds
Buffer incoming data for the specified period of time, in seconds, before delivering it to the destination. The default value is 300 (5 minutes).
Integer sizeInMBs
Buffer incoming data to the specified size, in MBs, before delivering it to the destination. The default value is 5.
We recommend setting SizeInMBs to a value greater than the amount of data you typically ingest into the delivery stream in 10 seconds. For example, if you typically ingest data at 1 MB/sec, set SizeInMBs to be 10 MB or higher.
String roleARN
The ARN of the IAM role to be assumed by Firehose for calling the Amazon ES Configuration API and for indexing documents. For more information, see Amazon S3 Bucket Access.
String domainARN
The ARN of the Amazon ES domain. The IAM role must have permission for DescribeElasticsearchDomain
,
DescribeElasticsearchDomains
, and DescribeElasticsearchDomainConfig
after assuming
RoleARN.
String indexName
The Elasticsearch index name.
String typeName
The Elasticsearch type name.
String indexRotationPeriod
The Elasticsearch index rotation period. Index rotation appends a timestamp to the IndexName to facilitate
expiration of old data. For more information, see Index Rotation for
Amazon Elasticsearch Service Destination. Default value is OneDay
.
ElasticsearchBufferingHints bufferingHints
Buffering options. If no value is specified, ElasticsearchBufferingHints object default values are used.
ElasticsearchRetryOptions retryOptions
Configures retry behavior in the event that Firehose is unable to deliver documents to Amazon ES. Default value is 300 (5 minutes).
String s3BackupMode
Defines how documents should be delivered to Amazon S3. When set to FailedDocumentsOnly, Firehose writes any documents that could not be indexed to the configured Amazon S3 destination, with elasticsearch-failed/ appended to the key prefix. When set to AllDocuments, Firehose delivers all incoming records to Amazon S3, and also writes failed documents with elasticsearch-failed/ appended to the prefix. For more information, see Amazon S3 Backup for Amazon Elasticsearch Service Destination. Default value is FailedDocumentsOnly.
S3DestinationConfiguration s3Configuration
CloudWatchLoggingOptions cloudWatchLoggingOptions
Describes CloudWatch logging options for your delivery stream.
String roleARN
The ARN of the AWS credentials.
String domainARN
The ARN of the Amazon ES domain.
String indexName
The Elasticsearch index name.
String typeName
The Elasticsearch type name.
String indexRotationPeriod
The Elasticsearch index rotation period
ElasticsearchBufferingHints bufferingHints
Buffering options.
ElasticsearchRetryOptions retryOptions
Elasticsearch retry options.
String s3BackupMode
Amazon S3 backup mode.
S3DestinationDescription s3DestinationDescription
CloudWatchLoggingOptions cloudWatchLoggingOptions
CloudWatch logging options.
String roleARN
The ARN of the IAM role to be assumed by Firehose for calling the Amazon ES Configuration API and for indexing documents. For more information, see Amazon S3 Bucket Access.
String domainARN
The ARN of the Amazon ES domain. The IAM role must have permission for DescribeElasticsearchDomain, DescribeElasticsearchDomains , and DescribeElasticsearchDomainConfig after assuming RoleARN.
String indexName
The Elasticsearch index name.
String typeName
The Elasticsearch type name.
String indexRotationPeriod
The Elasticsearch index rotation period. Index rotation appends a timestamp to the IndexName to facilitate the
expiration of old data. For more information, see Index Rotation for
Amazon Elasticsearch Service Destination. Default value is OneDay
.
ElasticsearchBufferingHints bufferingHints
Buffering options. If no value is specified, ElasticsearchBufferingHints object default values are used.
ElasticsearchRetryOptions retryOptions
Configures retry behavior in the event that Firehose is unable to deliver documents to Amazon ES. Default value is 300 (5 minutes).
S3DestinationUpdate s3Update
CloudWatchLoggingOptions cloudWatchLoggingOptions
Describes CloudWatch logging options for your delivery stream.
Integer durationInSeconds
After an initial failure to deliver to Amazon ES, the total amount of time during which Firehose re-attempts delivery (including the first attempt). After this time has elapsed, the failed documents are written to Amazon S3. Default value is 300 seconds (5 minutes). A value of 0 (zero) results in no retries.
String noEncryptionConfig
Specifically override existing encryption information to ensure no encryption is used.
KMSEncryptionConfig kMSEncryptionConfig
The encryption key.
String aWSKMSKeyARN
The ARN of the encryption key. Must belong to the same region as the destination Amazon S3 bucket.
String recordId
The ID of the record.
ByteBuffer data
The data blob, which is base64-encoded when the blob is serialized. The maximum size of the data blob, before base64-encoding, is 1,000 KB.
String roleARN
The ARN of the AWS credentials.
String clusterJDBCURL
The database connection string.
CopyCommand copyCommand
The COPY
command.
String username
The name of the user.
String password
The user password.
RedshiftRetryOptions retryOptions
Configures retry behavior in the event that Firehose is unable to deliver documents to Amazon Redshift. Default value is 3600 (60 minutes).
S3DestinationConfiguration s3Configuration
The S3 configuration for the intermediate location from which Amazon Redshift obtains data. Restrictions are described in the topic for CreateDeliveryStream.
The compression formats SNAPPY
or ZIP
cannot be specified in
RedshiftDestinationConfiguration.S3Configuration because the Amazon Redshift COPY
operation
that reads from the S3 bucket doesn't support these compression formats.
CloudWatchLoggingOptions cloudWatchLoggingOptions
Describes CloudWatch logging options for your delivery stream.
String roleARN
The ARN of the AWS credentials.
String clusterJDBCURL
The database connection string.
CopyCommand copyCommand
The COPY
command.
String username
The name of the user.
RedshiftRetryOptions retryOptions
Configures retry behavior in the event that Firehose is unable to deliver documents to Amazon Redshift. Default value is 3600 (60 minutes).
S3DestinationDescription s3DestinationDescription
The Amazon S3 destination.
CloudWatchLoggingOptions cloudWatchLoggingOptions
Describes CloudWatch logging options for your delivery stream.
String roleARN
The ARN of the AWS credentials.
String clusterJDBCURL
The database connection string.
CopyCommand copyCommand
The COPY
command.
String username
The name of the user.
String password
The user password.
RedshiftRetryOptions retryOptions
Configures retry behavior in the event that Firehose is unable to deliver documents to Amazon Redshift. Default value is 3600 (60 minutes).
S3DestinationUpdate s3Update
The Amazon S3 destination.
The compression formats SNAPPY
or ZIP
cannot be specified in
RedshiftDestinationUpdate.S3Update because the Amazon Redshift COPY
operation that reads from
the S3 bucket doesn't support these compression formats.
CloudWatchLoggingOptions cloudWatchLoggingOptions
Describes CloudWatch logging options for your delivery stream.
Integer durationInSeconds
The length of time during which Firehose retries delivery after a failure, starting from the initial request and
including the first attempt. The default value is 3600 seconds (60 minutes). Firehose does not retry if the value
of DurationInSeconds
is 0 (zero) or if the first delivery attempt takes longer than the current
value.
String roleARN
The ARN of the AWS credentials.
String bucketARN
The ARN of the S3 bucket.
String prefix
The "YYYY/MM/DD/HH" time format prefix is automatically used for delivered S3 files. You can specify an extra prefix to be added in front of the time format prefix. Note that if the prefix ends with a slash, it appears as a folder in the S3 bucket. For more information, see Amazon S3 Object Name Format in the Amazon Kinesis Firehose Developer Guide.
BufferingHints bufferingHints
The buffering option. If no value is specified, BufferingHints object default values are used.
String compressionFormat
The compression format. If no value is specified, the default is UNCOMPRESSED
.
The compression formats SNAPPY
or ZIP
cannot be specified for Amazon Redshift
destinations because they are not supported by the Amazon Redshift COPY
operation that reads from
the S3 bucket.
EncryptionConfiguration encryptionConfiguration
The encryption configuration. If no value is specified, the default is no encryption.
CloudWatchLoggingOptions cloudWatchLoggingOptions
Describes CloudWatch logging options for your delivery stream.
String roleARN
The ARN of the AWS credentials.
String bucketARN
The ARN of the S3 bucket.
String prefix
The "YYYY/MM/DD/HH" time format prefix is automatically used for delivered S3 files. You can specify an extra prefix to be added in front of the time format prefix. Note that if the prefix ends with a slash, it appears as a folder in the S3 bucket. For more information, see Amazon S3 Object Name Format in the Amazon Kinesis Firehose Developer Guide.
BufferingHints bufferingHints
The buffering option. If no value is specified, BufferingHints object default values are used.
String compressionFormat
The compression format. If no value is specified, the default is NOCOMPRESSION
.
EncryptionConfiguration encryptionConfiguration
The encryption configuration. If no value is specified, the default is no encryption.
CloudWatchLoggingOptions cloudWatchLoggingOptions
Describes CloudWatch logging options for your delivery stream.
String roleARN
The ARN of the AWS credentials.
String bucketARN
The ARN of the S3 bucket.
String prefix
The "YYYY/MM/DD/HH" time format prefix is automatically used for delivered S3 files. You can specify an extra prefix to be added in front of the time format prefix. Note that if the prefix ends with a slash, it appears as a folder in the S3 bucket. For more information, see Amazon S3 Object Name Format in the Amazon Kinesis Firehose Developer Guide.
BufferingHints bufferingHints
The buffering option. If no value is specified, BufferingHints object default values are used.
String compressionFormat
The compression format. If no value is specified, the default is NOCOMPRESSION
.
The compression formats SNAPPY
or ZIP
cannot be specified for Amazon Redshift
destinations because they are not supported by the Amazon Redshift COPY
operation that reads from
the S3 bucket.
EncryptionConfiguration encryptionConfiguration
The encryption configuration. If no value is specified, the default is no encryption.
CloudWatchLoggingOptions cloudWatchLoggingOptions
Describes CloudWatch logging options for your delivery stream.
String deliveryStreamName
The name of the delivery stream.
String currentDeliveryStreamVersionId
Obtain this value from the VersionId result of the DeliveryStreamDescription operation. This value is required, and helps the service to perform conditional operations. For example, if there is a interleaving update and this value is null, then the update destination fails. After the update is successful, the VersionId value is updated. The service then performs a merge of the old configuration with the new configuration.
String destinationId
The ID of the destination.
S3DestinationUpdate s3DestinationUpdate
Describes an update for a destination in Amazon S3.
RedshiftDestinationUpdate redshiftDestinationUpdate
Describes an update for a destination in Amazon Redshift.
ElasticsearchDestinationUpdate elasticsearchDestinationUpdate
Describes an update for a destination in Amazon ES.
Copyright © 2016. All rights reserved.