Package org.deeplearning4j.iterator.bert
Interface BertSequenceMasker
-
- All Known Implementing Classes:
BertMaskedLMMasker
public interface BertSequenceMasker
-
-
Method Summary
All Methods Instance Methods Abstract Methods Modifier and Type Method Description org.nd4j.common.primitives.Pair<List<String>,boolean[]>
maskSequence(List<String> input, String maskToken, List<String> vocabWords)
-
-
-
Method Detail
-
maskSequence
org.nd4j.common.primitives.Pair<List<String>,boolean[]> maskSequence(List<String> input, String maskToken, List<String> vocabWords)
- Parameters:
input
- Input sequence of tokensmaskToken
- Token to use for masking - usually something like "[MASK]"vocabWords
- Vocabulary, as a list- Returns:
- Pair: The new input tokens (after masking out), along with a boolean[] for whether the token is masked or not (same length as number of tokens). boolean[i] is true if token i was masked.
-
-