Package net.morimekta.util.json

Class JsonTokenizer

java.lang.Object

java.io.Reader

net.morimekta.util.io.LineBufferedReader

net.morimekta.util.json.JsonTokenizer

All Implemented Interfaces:

java.io.Closeable, java.lang.AutoCloseable, java.lang.Readable

public class JsonTokenizer
extends LineBufferedReader

Class for tokenizing a JSON document and return one token at a time. It is optimized for high volume JSON parsing, and avoids things like buffer copying and utf-8 parsing. The tokens are just references back to the original input buffer, so are not safe to use as reference (except for token type) after the next token has been read.

Field Summary

Fields inherited from class net.morimekta.util.io.LineBufferedReader

buffer, bufferLimit, bufferLineEnd, bufferOffset, lastChar, lineNo, linePos, preLoaded, reader

Fields inherited from class java.io.Reader

lock

Constructor Summary

Constructors

Constructor	Description
JsonTokenizer(java.io.InputStream in)	Create a JSON tokenizer that reads from the input steam.
JsonTokenizer(java.io.InputStream in, int bufferSize)	Create a JSON tokenizer that reads from the input steam.
JsonTokenizer(java.io.Reader in)	Create a JSON tokenizer that reads from the char reader.
JsonTokenizer(java.io.Reader in, int bufferSize)	Create a JSON tokenizer that reads from the char reader.

Method Summary

Modifier and Type	Method	Description
JsonToken	expect(java.lang.String message)	Expect a new JSON token on the stream.
JsonToken	expectNumber(java.lang.String message)	Expect a JSON number.
JsonToken	expectString(java.lang.String message)	Expect a string literal JSON token.
char	expectSymbol(java.lang.String message, char... symbols)	Expect a string literal JSON token.
java.lang.String	getLastLine()	Deprecated.
boolean	hasNext()	Whether there is another token on the stream.
JsonToken	next()	Returns the next token on the stream, or null if there are no more JSON tokens on the stream.
JsonToken	peek(java.lang.String message)	Return the next token or throw an exception.
java.lang.String	restOfLine()	Deprecated.

Methods inherited from class net.morimekta.util.io.LineBufferedReader

close, getLine, getLineNo, getLinePos, getRemainingLines, getRestOfLine, maybeConsolidateBuffer, read, read, readNextChar

Methods inherited from class java.io.Reader

mark, markSupported, nullReader, read, read, ready, reset, skip, transferTo

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

JsonTokenizer

public JsonTokenizer(java.io.InputStream in)

Create a JSON tokenizer that reads from the input steam. It will only read as far as requested, and no bytes further. It has no checking of whether the document follows the JSON standard, but will only accept JSON formatted tokens. Note that the content is assumed to be separated with newlines, which means that if multiple JSON contents are read from the same stream, they MUST have a separating newline. A single JSON object may still have newlines in it's stream.

Parameters:

in - Input stream to parse from.

JsonTokenizer

public JsonTokenizer(java.io.InputStream in,
                     int bufferSize)

Create a JSON tokenizer that reads from the input steam. It will only read as far as requested, and no bytes further. It has no checking of whether the document follows the JSON standard, but will only accept JSON formatted tokens. Note that the content is assumed to be separated with newlines, which means that if multiple JSON contents are read from the same stream, they MUST have a separating newline. A single JSON object may still have newlines in it's stream.

Parameters:

in - Input stream to parse from.

bufferSize - The size of the char buffer. Default is 2048 chars (4096 bytes).

JsonTokenizer

public JsonTokenizer(java.io.Reader in)

Create a JSON tokenizer that reads from the char reader. It will only read as far as requested, and no bytes further. It has no checking of whether the document follows the JSON standard, but will only accept JSON formatted tokens. Note that the content is assumed to be separated with newlines, which means that if multiple JSON contents are read from the same stream, they MUST have a separating newline. A single JSON object may still have newlines in it's stream.

Parameters:

in - Reader of content to parse.

JsonTokenizer

public JsonTokenizer(java.io.Reader in,
                     int bufferSize)

Create a JSON tokenizer that reads from the char reader. It will only read as far as requested, and no bytes further. It has no checking of whether the document follows the JSON standard, but will only accept JSON formatted tokens. Note that the content is assumed to be separated with newlines, which means that if multiple JSON contents are read from the same stream, they MUST have a separating newline. A single JSON object may still have newlines in it's stream.

Parameters:

in - Reader of content to parse.

bufferSize - The size of the char buffer. Default is 2048 chars (4096 bytes).

Method Detail

expect

@Nonnull
public JsonToken expect(java.lang.String message)
                 throws JsonException,
                        java.io.IOException

Expect a new JSON token on the stream.

Parameters:

message - Message to add to exception if there are no more JSON tokens on the stream.

Returns:

The next token.

Throws:

JsonException - If no more tokens, or the token is illegally formatted.

java.io.IOException - If unable to read from stream.

expectString

@Nonnull
public JsonToken expectString(java.lang.String message)
                       throws java.io.IOException,
                              JsonException

Expect a string literal JSON token. A string literal is a double-quote delimited UTF-8 encoded, JSON-escaped string.

Parameters:

message - Message to add to exception if there are no more JSON tokens on the stream.

Returns:

The string literal token.

Throws:

JsonException - If no more tokens, or the token is illegally formatted.

java.io.IOException - If unable to read from stream.

expectNumber

@Nonnull
public JsonToken expectNumber(java.lang.String message)
                       throws java.io.IOException,
                              JsonException

Expect a JSON number. A number is a 64 bit integer (long), or a 64-bit real (double) number. The long can be decimal (base-10), octal (base-8) with '0' prefix, or hexadecimal (base-16) encoded with '0x' prefix, and the double can be a decimal point number (-0.0), or a scientific notation number (-0.0e-1).

Parameters:

message - Message to add to exception if there are no more JSON tokens on the stream.

Returns:

The number token.

Throws:

JsonException - If no more tokens, or the token is illegally formatted.

java.io.IOException - If unable to read from stream.

expectSymbol

public char expectSymbol(java.lang.String message,
                         char... symbols)
                  throws java.io.IOException,
                         JsonException

Expect a string literal JSON token. A string literal is a double-quote delimited UTF-8 encoded, JSON-escaped string.

Parameters:

message - Message to add to exception if there are no more JSON tokens on the stream.

symbols - List of symbol characters to expect. If

Returns:

The symbol that was encountered.

Throws:

JsonException - If no more tokens, or the token is illegally formatted.

java.io.IOException - If unable to read from stream.

hasNext

public boolean hasNext()
                throws java.io.IOException,
                       JsonException

Whether there is another token on the stream. This will read up until it finds a JSON token, or until the stream ends.

Returns:

True if (and only if) there is at least one more token on the stream.

Throws:

JsonException - If the next token is illegally formatted.

java.io.IOException - If unable to read from stream.

restOfLine

@Nonnull
@Deprecated
public java.lang.String restOfLine()
                            throws java.io.IOException

Deprecated.

Return the rest of the current line. This is handy for handling unwanted content after the last

Returns:

The rest of the last read line. Not including leading and ending whitespaces, since these are allowed.

Throws:

java.io.IOException - If unable to read the rest of the line.

peek

@Nonnull
public JsonToken peek(java.lang.String message)
               throws java.io.IOException,
                      JsonException

Return the next token or throw an exception. Though it does not consume that token.

Parameters:

message - Message to add to exception if there are no more JSON tokens on the stream.

Returns:

The next token.

Throws:

JsonException - If the next token is illegally formatted.

java.io.IOException - If unable to read from stream.

next

@Nullable
public JsonToken next()
               throws java.io.IOException,
                      JsonException

Returns the next token on the stream, or null if there are no more JSON tokens on the stream.

Returns:

The next token, or null.

Throws:

JsonException - If the next token is illegally formatted.

java.io.IOException - If unable to read from stream.

getLastLine

@Nonnull
@Deprecated
public java.lang.String getLastLine()

Deprecated.

Returns the last line in the buffer. Or empty string if not usable.

Returns:

The line string, not including the line-break.