Class/Object

it.agilelab.bigdata.wasp.models

TopicModel

Related Docs: object TopicModel | package models

Permalink

case class TopicModel(name: String, creationTime: Long, partitions: Int, replicas: Int, topicDataType: String, keyFieldName: Option[String], headersFieldName: Option[String], valueFieldsNames: Option[Seq[String]], useAvroSchemaManager: Boolean, schema: BsonDocument, topicCompression: TopicCompression = TopicCompression.Disabled, subjectStrategy: SubjectStrategy = SubjectStrategy.None, keySchema: Option[String] = None) extends DatastoreModel with Product with Serializable

A model for a topic, that is, a message queue of some sort. Right now this means just Kafka topics.

name

the name of the topic, and doubles as the unique identifier for the model in the models database

creationTime

marks the time at which the model was generated.

partitions

the number of partitions used for the topic when wasp creates it

replicas

the number of replicas used for the topic when wasp creates it

topicDataType

field specifies the format to use when encoding/decoding data to/from messages, allowed values are: avro, plaintext, json, binary

keyFieldName

optionally specify a field whose contents will be used as a message key when writing to Kafka. The field must be of type string or binary. The original field will be left as-is, so you schema must handle it (or you can use valueFieldsNames).

headersFieldName

allows you to optionally specify a field whose contents will be used as message headers when writing to Kafka. The field must contain an array of non-null objects which must have a non-null field headerKey of type string and a field headerValue of type binary. The original field will be left as-is, so your schema must handle it (or you can use valueFieldsNames).

valueFieldsNames

allows you to specify a list of field names to be used to filter the fields that get passed to the value encoding; with this you can filter out fields that you don't need in the value, obviating the need to handle them in the schema. This is especially useful when specifying the keyFieldName or headersFieldName. For the avro and json topic data type this is optional; for the plaintext and binary topic data types this field is mandatory and the list must contain a single value field name that has the proper type (string for plaintext and binary for binary).

useAvroSchemaManager

if a schema registry should be used or not to handle the schema evolution (it makes sense only for avro message datatype)

schema

the Avro schema to use when encoding the value, for plaintext and binary this field is ignored. For json and avro the field names need to match 1:1 with the valueFieldsNames or the schema output of the strategy

topicCompression

to use to compress messages

subjectStrategy

subject strategy to use when registering the schema to the schema registry for the schema registry implementations that support it. This property makes sense only for avro and only if useAvroSchemaManager is set to true

keySchema

the schema to be used to encode the key as avro

Linear Supertypes
Serializable, Serializable, Product, Equals, DatastoreModel, Model, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. TopicModel
  2. Serializable
  3. Serializable
  4. Product
  5. Equals
  6. DatastoreModel
  7. Model
  8. AnyRef
  9. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new TopicModel(name: String, creationTime: Long, partitions: Int, replicas: Int, topicDataType: String, keyFieldName: Option[String], headersFieldName: Option[String], valueFieldsNames: Option[Seq[String]], useAvroSchemaManager: Boolean, schema: BsonDocument, topicCompression: TopicCompression = TopicCompression.Disabled, subjectStrategy: SubjectStrategy = SubjectStrategy.None, keySchema: Option[String] = None)

    Permalink

    name

    the name of the topic, and doubles as the unique identifier for the model in the models database

    creationTime

    marks the time at which the model was generated.

    partitions

    the number of partitions used for the topic when wasp creates it

    replicas

    the number of replicas used for the topic when wasp creates it

    topicDataType

    field specifies the format to use when encoding/decoding data to/from messages, allowed values are: avro, plaintext, json, binary

    keyFieldName

    optionally specify a field whose contents will be used as a message key when writing to Kafka. The field must be of type string or binary. The original field will be left as-is, so you schema must handle it (or you can use valueFieldsNames).

    headersFieldName

    allows you to optionally specify a field whose contents will be used as message headers when writing to Kafka. The field must contain an array of non-null objects which must have a non-null field headerKey of type string and a field headerValue of type binary. The original field will be left as-is, so your schema must handle it (or you can use valueFieldsNames).

    valueFieldsNames

    allows you to specify a list of field names to be used to filter the fields that get passed to the value encoding; with this you can filter out fields that you don't need in the value, obviating the need to handle them in the schema. This is especially useful when specifying the keyFieldName or headersFieldName. For the avro and json topic data type this is optional; for the plaintext and binary topic data types this field is mandatory and the list must contain a single value field name that has the proper type (string for plaintext and binary for binary).

    useAvroSchemaManager

    if a schema registry should be used or not to handle the schema evolution (it makes sense only for avro message datatype)

    schema

    the Avro schema to use when encoding the value, for plaintext and binary this field is ignored. For json and avro the field names need to match 1:1 with the valueFieldsNames or the schema output of the strategy

    topicCompression

    to use to compress messages

    subjectStrategy

    subject strategy to use when registering the schema to the schema registry for the schema registry implementations that support it. This property makes sense only for avro and only if useAvroSchemaManager is set to true

    keySchema

    the schema to be used to encode the key as avro

Value Members

  1. final def !=(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int

    Permalink
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  4. final def asInstanceOf[T0]: T0

    Permalink
    Definition Classes
    Any
  5. def clone(): AnyRef

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  6. val creationTime: Long

    Permalink

    marks the time at which the model was generated.

  7. def datastoreProduct: DatastoreProduct

    Permalink
    Definition Classes
    TopicModelDatastoreModel
  8. final def eq(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  9. def finalize(): Unit

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  10. final def getClass(): Class[_]

    Permalink
    Definition Classes
    AnyRef → Any
  11. def getJsonSchema: String

    Permalink
  12. val headersFieldName: Option[String]

    Permalink

    allows you to optionally specify a field whose contents will be used as message headers when writing to Kafka.

    allows you to optionally specify a field whose contents will be used as message headers when writing to Kafka. The field must contain an array of non-null objects which must have a non-null field headerKey of type string and a field headerValue of type binary. The original field will be left as-is, so your schema must handle it (or you can use valueFieldsNames).

  13. final def isInstanceOf[T0]: Boolean

    Permalink
    Definition Classes
    Any
  14. val keyFieldName: Option[String]

    Permalink

    optionally specify a field whose contents will be used as a message key when writing to Kafka.

    optionally specify a field whose contents will be used as a message key when writing to Kafka. The field must be of type string or binary. The original field will be left as-is, so you schema must handle it (or you can use valueFieldsNames).

  15. val keySchema: Option[String]

    Permalink

    the schema to be used to encode the key as avro

  16. val name: String

    Permalink

    the name of the topic, and doubles as the unique identifier for the model in the models database

    the name of the topic, and doubles as the unique identifier for the model in the models database

    Definition Classes
    TopicModelModel
  17. final def ne(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  18. final def notify(): Unit

    Permalink
    Definition Classes
    AnyRef
  19. final def notifyAll(): Unit

    Permalink
    Definition Classes
    AnyRef
  20. val partitions: Int

    Permalink

    the number of partitions used for the topic when wasp creates it

  21. val replicas: Int

    Permalink

    the number of replicas used for the topic when wasp creates it

  22. val schema: BsonDocument

    Permalink

    the Avro schema to use when encoding the value, for plaintext and binary this field is ignored.

    the Avro schema to use when encoding the value, for plaintext and binary this field is ignored. For json and avro the field names need to match 1:1 with the valueFieldsNames or the schema output of the strategy

  23. val subjectStrategy: SubjectStrategy

    Permalink

    subject strategy to use when registering the schema to the schema registry for the schema registry implementations that support it.

    subject strategy to use when registering the schema to the schema registry for the schema registry implementations that support it. This property makes sense only for avro and only if useAvroSchemaManager is set to true

  24. final def synchronized[T0](arg0: ⇒ T0): T0

    Permalink
    Definition Classes
    AnyRef
  25. val topicCompression: TopicCompression

    Permalink

    to use to compress messages

  26. val topicDataType: String

    Permalink

    field specifies the format to use when encoding/decoding data to/from messages, allowed values are: avro, plaintext, json, binary

  27. val useAvroSchemaManager: Boolean

    Permalink

    if a schema registry should be used or not to handle the schema evolution (it makes sense only for avro message datatype)

  28. val valueFieldsNames: Option[Seq[String]]

    Permalink

    allows you to specify a list of field names to be used to filter the fields that get passed to the value encoding; with this you can filter out fields that you don't need in the value, obviating the need to handle them in the schema.

    allows you to specify a list of field names to be used to filter the fields that get passed to the value encoding; with this you can filter out fields that you don't need in the value, obviating the need to handle them in the schema. This is especially useful when specifying the keyFieldName or headersFieldName. For the avro and json topic data type this is optional; for the plaintext and binary topic data types this field is mandatory and the list must contain a single value field name that has the proper type (string for plaintext and binary for binary).

  29. final def wait(): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  30. final def wait(arg0: Long, arg1: Int): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  31. final def wait(arg0: Long): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

Inherited from Serializable

Inherited from Serializable

Inherited from Product

Inherited from Equals

Inherited from DatastoreModel

Inherited from Model

Inherited from AnyRef

Inherited from Any

Ungrouped