org.apache.spark.sql.execution
SnapshotConnectionListener
Related Docs:
object SnapshotConnectionListener
| package execution
org.apache.spark.sql.execution
This is a TaskContext listener (for both success and failure of the task) that handles startup, commit and rollback of snapshot transactions for the task. It also provides a common connection that can be shared by all plans executing in the task. In conjunction with the apply methods of the companion object, it ensures that only one instance of this listener is attached in a TaskContext which is automatically removed at the end of the task execution.
This is the preferred way for all plans that need connections and/or snapshot transactions so that handling transaction start/commit for any level of plan nesting etc can be dealt with cleanly for the entire duration of the task. Additionally cases where an EXCHANGE gets inserted between two plans are also handled as expected where separate transactions and connections will be used for the two plans. Both generated code and non-generated code (including RDD.compute) should use the apply methods of the companion object to obtain an instance of the listener, then use its connection() method to obtain the connection.
One of the overloads of the apply method also allows one to send a custom connection creator instead of using the default one, but it is also assumed to return SnappyData connection only (either embedded or thin) for snapshot transactions to work. Typical usage of custom creator is for smart connector RDDs to use direct URLs without load-balance to the preferred hosts for the buckets being targeted instead of the default creator that will always use the locator.