Package org.apache.nifi.controller.queue

Interface FlowFileQueue

All Known Subinterfaces:
LoadBalancedFlowFileQueue
public interface FlowFileQueue

  • Method Details Link icon

    • getIdentifier Link icon

      String getIdentifier()
      Returns:
      the unique identifier for this FlowFileQueue

    • getPriorities Link icon

      List<FlowFilePrioritizer> getPriorities()
      Returns:
      list of processing priorities for this queue

    • recoverSwappedFlowFiles Link icon

      SwapSummary recoverSwappedFlowFiles()
      Reads any Swap Files that belong to this queue and returns a summary of what is swapped out. This will be called only during NiFi startup as an initialization step. This method is then responsible for returning a FlowFileSummary of the FlowFiles that are swapped out, or null if no FlowFiles are swapped out for this queue.
      Returns:
      a SwapSummary that describes the FlowFiles that exist in the queue but are swapped out.

    • purgeSwapFiles Link icon

      void purgeSwapFiles()
      Destroys any Swap Files that exist for this queue without updating the FlowFile Repository or Provenance Repository. This is done only on startup in the case of non-persistent repositories. In the case of non-persistent repositories, we may still have Swap Files because we may still need to overflow the FlowFiles from heap onto disk, even though we don't want to keep the FlowFiles on restart.

    • setPriorities Link icon

      void setPriorities(List<FlowFilePrioritizer> newPriorities)
      Resets the comparator used by this queue to maintain order.
      Parameters:
      newPriorities - the ordered list of prioritizers to use to determine order within this queue.
      Throws:
      NullPointerException - if arg is null

    • setBackPressureObjectThreshold Link icon

      void setBackPressureObjectThreshold(long maxQueueSize)
      Establishes this queue's preferred maximum work load.
      Parameters:
      maxQueueSize - the maximum number of flow files this processor recommends having in its work queue at any one time

    • getBackPressureObjectThreshold Link icon

      long getBackPressureObjectThreshold()
      Returns:
      maximum number of flow files that should be queued up at any one time

    • setBackPressureDataSizeThreshold Link icon

      void setBackPressureDataSizeThreshold(String maxDataSize)
      Parameters:
      maxDataSize - Establishes this queue's preferred maximum data size.

    • getBackPressureDataSizeThreshold Link icon

      String getBackPressureDataSizeThreshold()
      Returns:
      maximum data size that should be queued up at any one time

    • size Link icon

      org.apache.nifi.controller.queue.QueueSize size()

    • getTotalQueuedDuration Link icon

      long getTotalQueuedDuration(long fromTimestamp)
      Parameters:
      fromTimestamp - The timestamp in milliseconds from which to calculate durations. This will typically be the current timestamp.
      Returns:
      the sum in milliseconds of how long all FlowFiles within this queue have currently been in this queue.

    • getMinLastQueueDate Link icon

      long getMinLastQueueDate()
      Returns:
      The minimum lastQueueDate in milliseconds of all FlowFiles currently enqueued. If no FlowFile is enqueued, this returns 0.

    • isEmpty Link icon

      boolean isEmpty()
      Returns:
      true if no items queue; false otherwise

    • getFlowFileAvailability Link icon

      org.apache.nifi.controller.status.FlowFileAvailability getFlowFileAvailability()
      Returns:
      the FlowFile Availability for this queue

    • isActiveQueueEmpty Link icon

      boolean isActiveQueueEmpty()
      Returns:
      true if the queue is empty or contains only FlowFiles that already are being processed by others, false if the queue contains at least one FlowFile that is available for processing, regardless of whether that FlowFile(s) is in-memory or swapped out.

    • acknowledge Link icon

      void acknowledge(FlowFileRecord flowFile)

    • acknowledge Link icon

      void acknowledge(Collection<FlowFileRecord> flowFiles)

    • isUnacknowledgedFlowFile Link icon

      boolean isUnacknowledgedFlowFile()
      Returns:
      true if at least one FlowFile is unacknowledged, false if all FlowFiles that have been dequeued have been acknowledged

    • isFull Link icon

      boolean isFull()
      Returns:
      true if maximum queue size has been reached or exceeded; false otherwise

    • put Link icon

      void put(FlowFileRecord file)
      places the given file into the queue
      Parameters:
      file - to place into queue

    • putAll Link icon

      void putAll(Collection<FlowFileRecord> files)
      places the given files into the queue
      Parameters:
      files - to place into queue

    • poll Link icon

      FlowFileRecord poll(Set<FlowFileRecord> expiredRecords, PollStrategy pollStrategy)
      Parameters:
      expiredRecords - expired records
      pollStrategy - strategy of polling
      Returns:
      the next flow file on the queue; null if empty

    • poll Link icon

      FlowFileRecord poll(Set<FlowFileRecord> expiredRecords)

    • poll Link icon

      List<FlowFileRecord> poll(int maxResults, Set<FlowFileRecord> expiredRecords, PollStrategy pollStrategy)
      Parameters:
      maxResults - limits how many results can be polled
      expiredRecords - for expired records
      pollStrategy - strategy of polling
      Returns:
      the next flow files on the queue up to the max results; null if empty

    • poll Link icon

      List<FlowFileRecord> poll(int maxResults, Set<FlowFileRecord> expiredRecords)

    • poll Link icon

      List<FlowFileRecord> poll(org.apache.nifi.processor.FlowFileFilter filter, Set<FlowFileRecord> expiredRecords, PollStrategy pollStrategy)

    • poll Link icon

      List<FlowFileRecord> poll(org.apache.nifi.processor.FlowFileFilter filter, Set<FlowFileRecord> expiredRecords)

    • getFlowFileExpiration Link icon

      String getFlowFileExpiration()

    • getFlowFileExpiration Link icon

      long getFlowFileExpiration(TimeUnit timeUnit)

    • setFlowFileExpiration Link icon

      void setFlowFileExpiration(String flowExpirationPeriod)

    • dropFlowFiles Link icon

      DropFlowFileStatus dropFlowFiles(String requestIdentifier, String requestor)
      Initiates a request to drop all FlowFiles in this queue. This method returns a DropFlowFileStatus that can be used to determine the current state of the request. Additionally, the DropFlowFileStatus provides a request identifier that can then be passed to the getDropFlowFileStatus(String) and cancelDropFlowFileRequest(String) methods in order to obtain the status later or cancel a request
      Parameters:
      requestIdentifier - the identifier of the Drop FlowFile Request
      requestor - the entity that is requesting that the FlowFiles be dropped; this will be included in the Provenance Events that are generated.
      Returns:
      the status of the drop request.

    • getDropFlowFileStatus Link icon

      DropFlowFileStatus getDropFlowFileStatus(String requestIdentifier)
      Returns the current status of a Drop FlowFile Request that was initiated via the dropFlowFiles(String, String) method that has the given identifier
      Parameters:
      requestIdentifier - the identifier of the Drop FlowFile Request
      Returns:
      the status for the request with the given identifier, or null if no request status exists with that identifier

    • cancelDropFlowFileRequest Link icon

      DropFlowFileStatus cancelDropFlowFileRequest(String requestIdentifier)
      Cancels the request to drop FlowFiles that has the given identifier. After this method is called, the request will no longer be known by this queue, so subsequent calls to getDropFlowFileStatus(String) or cancelDropFlowFileRequest(String) will return null
      Parameters:
      requestIdentifier - the identifier of the Drop FlowFile Request
      Returns:
      the status for the request with the given identifier after it has been canceled, or null if no request status exists with that identifier

    • listFlowFiles Link icon

      ListFlowFileStatus listFlowFiles(String requestIdentifier, int maxResults)

      Initiates a request to obtain a listing of FlowFiles in this queue. This method returns a ListFlowFileStatus that can be used to obtain information about the FlowFiles that exist within the queue. Additionally, the ListFlowFileStatus provides a request identifier that can then be passed to the getListFlowFileStatus(String). The listing of FlowFiles will be returned ordered by the position of the FlowFile in the queue.

      Note that if maxResults is larger than the size of the "active queue" (i.e., the un-swapped queued, FlowFiles that are swapped out will not be returned.)

      Parameters:
      requestIdentifier - the identifier of the List FlowFile Request
      maxResults - the maximum number of FlowFileSummary objects to add to the ListFlowFileStatus
      Returns:
      the status for the request
      Throws:
      IllegalStateException - if either the source or the destination of the connection to which this queue belongs is currently running.

    • getListFlowFileStatus Link icon

      ListFlowFileStatus getListFlowFileStatus(String requestIdentifier)
      Returns the current status of a List FlowFile Request that was initiated via the listFlowFiles(String, int) method that has the given identifier
      Parameters:
      requestIdentifier - the identifier of the Drop FlowFile Request
      Returns:
      the current status of the List FlowFile Request with the given identifier or null if no request status exists with that identifier

    • cancelListFlowFileRequest Link icon

      ListFlowFileStatus cancelListFlowFileRequest(String requestIdentifier)
      Cancels the request to list FlowFiles that has the given identifier. After this method is called, the request will no longer be known by this queue, so subsequent calls to getListFlowFileStatus(String) or cancelListFlowFileRequest(String) will return null
      Parameters:
      requestIdentifier - the identifier of the Drop FlowFile Request
      Returns:
      the current status of the List FlowFile Request with the given identifier or null if no request status exists with that identifier

    • getFlowFile Link icon

      FlowFileRecord getFlowFile(String flowFileUuid) throws IOException
      Returns the FlowFile with the given UUID or null if no FlowFile can be found in this queue with the given UUID
      Parameters:
      flowFileUuid - the UUID of the FlowFile to retrieve
      Returns:
      the FlowFile with the given UUID or null if no FlowFile can be found in this queue with the given UUID
      Throws:
      IOException - if unable to read FlowFiles that are stored on some external device

    • verifyCanList Link icon

      void verifyCanList() throws IllegalStateException
      Ensures that a listing can be performed on the queue
      Throws:
      IllegalStateException - if the queue is not in a state in which a listing can be performed

    • getQueueDiagnostics Link icon

      QueueDiagnostics getQueueDiagnostics()
      Returns diagnostic information about the queue

    • lock Link icon

      void lock()

    • unlock Link icon

      void unlock()

    • setLoadBalanceStrategy Link icon

      void setLoadBalanceStrategy(LoadBalanceStrategy strategy, String partitioningAttribute)

    • offloadQueue Link icon

      void offloadQueue()
      Offloads the flowfiles in the queue to other nodes. This disables the queue from partition flowfiles locally.

      This operation is a no-op if the node that contains this queue is not in a cluster.

    • resetOffloadedQueue Link icon

      void resetOffloadedQueue()
      Resets a queue that has previously been offloaded. This allows the queue to partition flowfiles locally, and has no other effect on processors or remote process groups.

      This operation is a no-op if the queue is not currently offloaded or the node that contains this queue is not clustered.

    • getLoadBalanceStrategy Link icon

      LoadBalanceStrategy getLoadBalanceStrategy()

    • setLoadBalanceCompression Link icon

      void setLoadBalanceCompression(LoadBalanceCompression compression)

    • getLoadBalanceCompression Link icon

      LoadBalanceCompression getLoadBalanceCompression()

    • getPartitioningAttribute Link icon

      String getPartitioningAttribute()

    • startLoadBalancing Link icon

      void startLoadBalancing()

    • stopLoadBalancing Link icon

      void stopLoadBalancing()

    • isActivelyLoadBalancing Link icon

      boolean isActivelyLoadBalancing()
      Returns:
      true if the queue is actively transferring data to another node, false otherwise