Package io.pravega.connectors.flink.util

Class FlinkPravegaUtils

  • public class FlinkPravegaUtils
extends java.lang.Object

    • Nested Class Summary

      Nested Classes 
      Modifier and Type Class Description
      static class  FlinkPravegaUtils.FlinkDeserializer<T>
      A Pravega Serializer that wraps around a Flink DeserializationSchema.

    • Method Summary

      All Methods Static Methods Concrete Methods 
      Modifier and Type Method Description
      static <T> io.pravega.client.stream.EventStreamReader<T> createPravegaReader​(java.lang.String readerId, java.lang.String readerGroupName, org.apache.flink.api.common.serialization.DeserializationSchema<T> deserializationSchema, io.pravega.client.stream.ReaderConfig readerConfig, io.pravega.client.EventStreamClientFactory eventStreamClientFactory)
      Creates a Pravga EventStreamReader.
      static java.lang.String generateRandomReaderGroupName()
      Generates a random reader group name.
      static java.lang.String getReaderName​(java.lang.String taskName, int index, int total)
      Utility method that derives the reader name from taskName, index and parallelism.
      static <T> org.apache.flink.streaming.api.datastream.DataStreamSink<T> writeToPravegaInEventTimeOrder​(org.apache.flink.streaming.api.datastream.DataStream<T> stream, FlinkPravegaWriter<T> writer, int parallelism)
      Writes a stream of elements to a Pravega stream with event time ordering.

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    • Method Detail

      • writeToPravegaInEventTimeOrder

        public static <T> org.apache.flink.streaming.api.datastream.DataStreamSink<T> writeToPravegaInEventTimeOrder​(org.apache.flink.streaming.api.datastream.DataStream<T> stream,
                                                                                                             FlinkPravegaWriter<T> writer,
                                                                                                             int parallelism)
        Writes a stream of elements to a Pravega stream with event time ordering.

        This method returns a sink that automatically reorders events by their event timestamp. The ordering is preserved by Pravega using the associated routing key.

        Type Parameters:
        T - The type of the event.
        Parameters:
        stream - the stream to read.
        writer - the Pravega writer to use.
        parallelism - the degree of parallelism for the writer.
        Returns:
        a sink.

      • getReaderName

        public static java.lang.String getReaderName​(java.lang.String taskName,
                                             int index,
                                             int total)
        Utility method that derives the reader name from taskName, index and parallelism.
        Parameters:
        taskName - the original task name
        index - the index of the subtask
        total - the total parallelism of the subtask
        Returns:
        the generated default reader name.

      • generateRandomReaderGroupName

        public static java.lang.String generateRandomReaderGroupName()
        Generates a random reader group name.
        Returns:
        generated reader group name.

      • createPravegaReader

        public static <T> io.pravega.client.stream.EventStreamReader<T> createPravegaReader​(java.lang.String readerId,
                                                                                    java.lang.String readerGroupName,
                                                                                    org.apache.flink.api.common.serialization.DeserializationSchema<T> deserializationSchema,
                                                                                    io.pravega.client.stream.ReaderConfig readerConfig,
                                                                                    io.pravega.client.EventStreamClientFactory eventStreamClientFactory)
        Creates a Pravga EventStreamReader.
        Type Parameters:
        T - The type of the event.
        Parameters:
        readerId - The id of the Pravega reader.
        readerGroupName - The reader group name.
        deserializationSchema - The implementation to deserialize events from pravega streams.
        readerConfig - The reader configuration.
        eventStreamClientFactory - The eventStreamClientFactory used to create the EventStreamReader
        Returns:
        the create Pravega reader.