tensorflow

Type Members

1. class ClassifierDatasetEncoder extends Serializable

   

2. case class ClassifierDatasetEncoderParams(tags: Array[String]) extends Product with Serializable

   

3. case class DatasetEncoderParams(tags: List[String], chars: List[Char], emptyVector: List[Float], embeddingsDim: Int, defaultTag: String = "O") extends Product with Serializable

   

4. trait Logging extends AnyRef

   

5. class NerBatch extends AnyRef

   

   Batch that contains data in Tensorflow input format.

6. class NerDatasetEncoder extends Serializable

   

7. trait ReadTensorflowModel extends AnyRef

   

8. case class SentenceGrouper[T](getLength: (T) ⇒ Int, sizes: Array[Int] = Array(5, 10, 20, 50))(implicit evidence$1: ClassTag[T]) extends Product with Serializable

   

9. class TensorResources extends AnyRef

   

10. class TensorflowAlbert extends Serializable

    

    This class is used to calculate ALBERT embeddings for For Sequence Batches of WordpieceTokenizedSentence.

    This class is used to calculate ALBERT embeddings for For Sequence Batches of WordpieceTokenizedSentence. Input for this model must be tokenzied with a SentencePieceModel,

    This Tensorflow model is using the weights provided by https://tfhub.dev/google/albert_base/3 * sequence_output: representations of every token in the input sequence with shape [batch_size, max_sequence_length, hidden_size].

    ALBERT: A LITE BERT FOR SELF-SUPERVISED LEARNING OF LANGUAGE REPRESENTATIONS - Google Research, Toyota Technological Institute at Chicago This these embeddings represent the outputs generated by the Albert model. All offical Albert releases by google in TF-HUB are supported with this Albert Wrapper:

    TF-HUB Models : albert_base = https://tfhub.dev/google/albert_base/3 | 768-embed-dim, 12-layer, 12-heads, 12M parameters albert_large = https://tfhub.dev/google/albert_large/3 | 1024-embed-dim, 24-layer, 16-heads, 18M parameters albert_xlarge = https://tfhub.dev/google/albert_xlarge/3 | 2048-embed-dim, 24-layer, 32-heads, 60M parameters albert_xxlarge = https://tfhub.dev/google/albert_xxlarge/3 | 4096-embed-dim, 12-layer, 64-heads, 235M parameters

    This model requires input tokenization with SentencePiece model, which is provided by Spark NLP

    For additional information see : https://arxiv.org/pdf/1909.11942.pdf https://github.com/google-research/ALBERT https://tfhub.dev/s?q=albert

    Tips:

    ALBERT uses repeating layers which results in a small memory footprint, however the computational cost remains similar to a BERT-like architecture with the same number of hidden layers as it has to iterate through the same number of (repeating) layers.

11. class TensorflowBert extends Serializable

    

12. class TensorflowClassifier extends Serializable with Logging

    

13. class TensorflowElmo extends Serializable

    

    This class is used to calculate ELMO embeddings for For Sequence Batches of TokenizedSentences.

    This class is used to calculate ELMO embeddings for For Sequence Batches of TokenizedSentences.

    https://tfhub.dev/google/elmo/3 * word_emb: the character-based word representations with shape [batch_size, max_length, 512]. == word_emb * lstm_outputs1: the first LSTM hidden state with shape [batch_size, max_length, 1024]. === lstm_outputs1 * lstm_outputs2: the second LSTM hidden state with shape [batch_size, max_length, 1024]. === lstm_outputs2 * elmo: the weighted sum of the 3 layers, where the weights are trainable. This tensor has shape [batch_size, max_length, 1024] == elmo

14. class TensorflowNer extends Serializable with Logging

    

15. class TensorflowUSE extends Serializable

    

16. class TensorflowWrapper extends Serializable

    

17. class TensorflowXlnet extends Serializable

    

18. case class Variables(variables: Array[Byte], index: Array[Byte]) extends Product with Serializable

    

19. trait WriteTensorflowModel extends AnyRef