Package org.apache.nifi.processors.standard

Class QueryDatabaseTable

java.lang.Object
org.apache.nifi.components.AbstractConfigurableComponent
org.apache.nifi.processor.AbstractSessionFactoryProcessor
org.apache.nifi.processors.standard.AbstractDatabaseFetchProcessor
org.apache.nifi.processors.standard.AbstractQueryDatabaseTable
org.apache.nifi.processors.standard.QueryDatabaseTable
All Implemented Interfaces:
org.apache.nifi.components.ConfigurableComponent, org.apache.nifi.processor.Processor
@TriggerSerially @InputRequirement(INPUT_FORBIDDEN) @Tags({"sql","select","jdbc","query","database"}) @SeeAlso({GenerateTableFetch.class,ExecuteSQL.class}) @CapabilityDescription("Generates a SQL select query, or uses a provided statement, and executes it to fetch all rows whose values in the specified Maximum Value column(s) are larger than the previously-seen maxima. Query result will be converted to Avro format. Expression Language is supported for several properties, but no incoming connections are permitted. The Environment/System properties may be used to provide values for any property containing Expression Language. If it is desired to leverage flow file attributes to perform these queries, the GenerateTableFetch and/or ExecuteSQL processors can be used for this purpose. Streaming is used so arbitrarily large result sets are supported. This processor can be scheduled to run on a timer or cron expression, using the standard scheduling methods. This processor is intended to be run on the Primary Node only. FlowFile attribute \'querydbtable.row.count\' indicates how many rows were selected.") @Stateful(scopes=CLUSTER, description="After performing a query on the specified table, the maximum values for the specified column(s) will be retained for use in future executions of the query. This allows the Processor to fetch only those records that have max values greater than the retained values. This can be used for incremental fetching, fetching of newly added rows, etc. To clear the maximum values, clear the state of the processor per the State Management documentation") @WritesAttribute(attribute="tablename",description="Name of the table being queried") @WritesAttribute(attribute="querydbtable.row.count",description="The number of rows selected by the query") @WritesAttribute(attribute="fragment.identifier",description="If \'Max Rows Per Flow File\' is set then all FlowFiles from the same query result set will have the same value for the fragment.identifier attribute. This can then be used to correlate the results.") @WritesAttribute(attribute="fragment.count",description="If \'Max Rows Per Flow File\' is set then this is the total number of FlowFiles produced by a single ResultSet. This can be used in conjunction with the fragment.identifier attribute in order to know how many FlowFiles belonged to the same incoming ResultSet. If Output Batch Size is set, then this attribute will not be populated.") @WritesAttribute(attribute="fragment.index",description="If \'Max Rows Per Flow File\' is set then the position of this FlowFile in the list of outgoing FlowFiles that were all derived from the same result set FlowFile. This can be used in conjunction with the fragment.identifier attribute to know which FlowFiles originated from the same query result set and in what order FlowFiles were produced") @WritesAttribute(attribute="maxvalue.*",description="Each attribute contains the observed maximum value of a specified \'Maximum-value Column\'. The suffix of the attribute is the name of the column. If Output Batch Size is set, then this attribute will not be populated.") @DynamicProperty(name="initial.maxvalue.<max_value_column>", value="Initial maximum value for the specified column", expressionLanguageScope=ENVIRONMENT, description="Specifies an initial max value for max value column(s). Properties should be added in the format `initial.maxvalue.<max_value_column>`. This value is only used the first time the table is accessed (when a Maximum Value Column is specified).") @PrimaryNodeOnly @DefaultSchedule(strategy=TIMER_DRIVEN, period="1 min") public class QueryDatabaseTable extends AbstractQueryDatabaseTable

  • Field Details

    • TABLE_NAME

      static final org.apache.nifi.components.PropertyDescriptor TABLE_NAME

    • PROPERTIES

      private static final List<org.apache.nifi.components.PropertyDescriptor> PROPERTIES

    • RELATIONSHIPS

      private static final Set<org.apache.nifi.processor.Relationship> RELATIONSHIPS

  • Constructor Details

    • QueryDatabaseTable

      public QueryDatabaseTable()

  • Method Details