This conversion is necessary as Sink documentation disallows an operation on incoming dataframe.
This conversion is necessary as Sink documentation disallows an operation on incoming dataframe. Otherwise it will break incremental planning of streaming dataframes. See http://apache-spark-developers-list.1001551.n3.nabble.com/ Structured-Streaming-Sink-in-2-0-collect-foreach-restrictions-added-in- SPARK-16020-td18118.html for a detailed discussion.