org.springframework.cloud.dataflow.core.dsl

Class Tokens

java.lang.Object

org.springframework.cloud.dataflow.core.dsl.Tokens

public class Tokens
extends Object

Class that converts an expression into a list of tokens. Furthermore, this class provides methods to process the tokens and keeps track of the current token being processed.

Author:

Andy Clement, Patrick Peralta

Method Summary

Modifier and Type	Method and Description
protected void	checkpoint() Indicate that the current token has been successfully processed.
protected int	decrementPosition() Decrement the current token position and return the new position.
protected Token	eat() Consume the next token.
protected Token	eat(TokenKind expectedKind) Consume the next token if it matches the indicated token kind; otherwise throw CheckPointedParseException.
protected String	getExpression() Return the expression string converted to tokens.
int	getLine(Token token)
List<Token>	getTokenStream() Return an immutable list of tokens
protected boolean	hasNext() Return true if there are more tokens to process.
protected boolean	isNextAdjacent() Return true if the first character of the token at the current position is the same as the last character of the token at the previous position.
protected boolean	lookAhead(int distance, TokenKind desiredTokenKind) Return true if the token in the position indicated by position + distance matches the token indicated by desiredTokenKind.
protected Token	next() Return the next Token and advance the current token position.
protected Token	peek() Return the token at the current position.
protected Token	peek(int howFarAhead) Return the token at a specified distance beyond current position.
protected boolean	peek(TokenKind desiredTokenKind) Return true if the indicated token matches the current token position.
protected int	position() Return the current token position.
protected void	raiseException(int position, DSLMessage message, Object... inserts) Throw a new CheckPointedParseException based on the current and last successfully processed token position.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

getExpression

protected String getExpression()

Return the expression string converted to tokens.

Returns:

expression string

decrementPosition

protected int decrementPosition()

Decrement the current token position and return the new position.

Returns:

new token position

position

protected int position()

Return the current token position.

Returns:

current token position

getTokenStream

public List<Token> getTokenStream()

Return an immutable list of tokens

Returns:

list of tokens

lookAhead

protected boolean lookAhead(int distance,
                            TokenKind desiredTokenKind)

Return true if the token in the position indicated by position + distance matches the token indicated by desiredTokenKind.

Parameters:

distance - number of token positions past the current position

desiredTokenKind - the token to check for

Returns:

true if the token at the indicated position matches desiredTokenKind

hasNext

protected boolean hasNext()

Return true if there are more tokens to process.

Returns:

true if there are more tokens to process

peek

protected Token peek()

Return the token at the current position. If there are no more tokens, return null.

Returns:

token at current position or null if there are no more tokens

peek

protected Token peek(int howFarAhead)

Return the token at a specified distance beyond current position. If that is off the end of the list of known tokens, return null.

Parameters:

howFarAhead - how many character positions to peek ahead.

Returns:

token at specified distance beyond current position or null if there are no more tokens

peek

protected boolean peek(TokenKind desiredTokenKind)

Return true if the indicated token matches the current token position.

Parameters:

desiredTokenKind - token to match

Returns:

true if the current token kind matches the provided token kind

next

protected Token next()

Return the next Token and advance the current token position.

Returns:

next Token

eat

protected Token eat(TokenKind expectedKind)

Consume the next token if it matches the indicated token kind; otherwise throw CheckPointedParseException.

Parameters:

expectedKind - the expected token kind

Returns:

the next token

Throws:

CheckPointedParseException - if the next token does not match the expected token kind

eat

protected Token eat()

Consume the next token. Throw CheckPointedParseException if there is nothing to consume.

Returns:

the next token

Throws:

CheckPointedParseException - if there are no more tokens

isNextAdjacent

protected boolean isNextAdjacent()

Return true if the first character of the token at the current position is the same as the last character of the token at the previous position.

Returns:

true if the first character of the current token matches the last character of the previous token

checkpoint

protected void checkpoint()

Indicate that the current token has been successfully processed.

See Also:

lastGoodPosition

raiseException

protected void raiseException(int position,
                              DSLMessage message,
                              Object... inserts)

Throw a new CheckPointedParseException based on the current and last successfully processed token position.

Parameters:

position - position where parse error occurred

message - parse exception message

inserts - variables that may be inserted in the error message

getLine

public int getLine(Token token)

Parameters:

token - the token for which to determine the line

Returns:

which line the token is on, starting from 0