required for AdpDataPipelineObject
required for AdpDataPipelineObject
Determines what AWS Data Pipeline does with pre-existing data in the target table that overlaps with rows in the data to be loaded. Valid values are KEEP_EXISTING, OVERWRITE_EXISTING, and TRUNCATE.
The SQL SELECT expression used to transform the input data.
Corresponds to the query_group setting in Amazon Redshift, which allows you to assign and prioritize concurrent activities based on their placement in queues. Amazon Redshift limits the number of simultaneous connections to 15.
Takes COPY parameters to pass to the Amazon Redshift data node.
The input data node. The data source can be Amazon S3, DynamoDB, or Amazon Redshift.
The output data node. The output location can be Amazon S3 or Amazon Redshift.
Required for AdpActivity
Required for AdpActivity
The timeout time interval for an object attempt.
The timeout time interval for an object attempt. If an attempt does not complete within the start time plus this time interval, AWS Data Pipeline marks the attempt as failed and your retry settings determine the next steps taken.
Takes COPY parameters to pass to the Amazon Redshift data node.
Required for AdpActivity
Required for AdpActivity
Determines whether pipeline object failures and rerun commands cascade through pipeline object dependencies
Determines whether pipeline object failures and rerun commands cascade through pipeline object dependencies
Possible values include cascade and none.
required for AdpDataPipelineObject
required for AdpDataPipelineObject
The input data node.
The input data node. The data source can be Amazon S3, DynamoDB, or Amazon Redshift.
Determines what AWS Data Pipeline does with pre-existing data in the target table that overlaps with rows in the data to be loaded.
Determines what AWS Data Pipeline does with pre-existing data in the target table that overlaps with rows in the data to be loaded. Valid values are KEEP_EXISTING, OVERWRITE_EXISTING, and TRUNCATE.
The time period in which the object run must start.
The time period in which the object run must start. If the object does not start within the scheduled start time plus this time interval, it is considered late
The maximum number of times to retry the action.
The maximum number of times to retry the action. The default value is 2, which results in 3 tries total (1 original attempt plus 2 retries). The maximum value is 5 (6 total attempts).
required for AdpDataPipelineObject
required for AdpDataPipelineObject
The SNS alarm to raise when the activity fails.
The SNS alarm to raise when the activity fails.
The SNS alarm to raise when the activity fails to start on time.
The SNS alarm to raise when the activity fails to start on time.
The SNS alarm to raise when the activity succeeds.
The SNS alarm to raise when the activity succeeds.
The output data node.
The output data node. The output location can be Amazon S3 or Amazon Redshift.
A condition that must be met before the object can run.
A condition that must be met before the object can run. To specify multiple conditions, add multiple precondition fields. The activity cannot run until all its conditions are met.
Corresponds to the query_group setting in Amazon Redshift, which allows you to assign and prioritize concurrent activities based on their placement in queues.
Corresponds to the query_group setting in Amazon Redshift, which allows you to assign and prioritize concurrent activities based on their placement in queues. Amazon Redshift limits the number of simultaneous connections to 15.
The timeout duration between two retry attempts.
The timeout duration between two retry attempts. The default is 10 minutes.
Required for AdpActivity
The SQL SELECT expression used to transform the input data.
The type of object.
The type of object. Use one of the predefined AWS Data Pipeline object types.
The worker group.
The worker group. This is used for routing tasks. If you provide a runsOn value and workerGroup exists, workerGroup is ignored.
ref: http://docs.aws.amazon.com/datapipeline/latest/DeveloperGuide/dp-object-redshiftcopyactivity.html
required for AdpDataPipelineObject
required for AdpDataPipelineObject
Determines what AWS Data Pipeline does with pre-existing data in the target table that overlaps with rows in the data to be loaded. Valid values are KEEP_EXISTING, OVERWRITE_EXISTING, and TRUNCATE.
The SQL SELECT expression used to transform the input data.
Corresponds to the query_group setting in Amazon Redshift, which allows you to assign and prioritize concurrent activities based on their placement in queues. Amazon Redshift limits the number of simultaneous connections to 15.
Takes COPY parameters to pass to the Amazon Redshift data node.
The input data node. The data source can be Amazon S3, DynamoDB, or Amazon Redshift.
The output data node. The output location can be Amazon S3 or Amazon Redshift.
Required for AdpActivity
Required for AdpActivity