Package com.tonic.analysis.source.parser

Class Token

java.lang.Object

com.tonic.analysis.source.parser.Token

public final class Token
extends Object

An immutable lexer token - its type, source text, decoded literal value and position.

Constructor Summary

Constructors

Constructor	Description
Token(TokenType type, String text, SourcePosition position)	Creates a token with no decoded literal value.
Token(TokenType type, String text, Object value, SourcePosition position)	Creates a token carrying a decoded literal value.

Method Summary

Modifier and Type	Method	Description
char	charValue()	Reads the literal value as a char.
double	doubleValue()	Reads the literal value as a double, converting any numeric value.
boolean	equals(Object o)
float	floatValue()	Reads the literal value as a float, converting any numeric value.
int	getColumn()
int	getLine()
SourcePosition	getPosition()
String	getText()
TokenType	getType()
Object	getValue()
int	hashCode()
int	intValue()	Reads the literal value as an int, narrowing any numeric value.
boolean	is(TokenType type)	Tests the token type.
boolean	isKeyword()
boolean	isLiteral()
boolean	isModifier()
boolean	isOneOf(TokenType... types)	Tests the token type against several candidates.
boolean	isPrimitiveType()
long	longValue()	Reads the literal value as a long, widening or narrowing any numeric value.
String	stringValue()	Reads the literal value as a string.
String	toString()

Methods inherited from class java.lang.Object

getClass, notify, notifyAll, wait, wait, wait

Constructor Detail

Token

public Token(TokenType type,
             String text,
             Object value,
             SourcePosition position)

Creates a token carrying a decoded literal value.

Parameters:

type - the token type

text - the raw source text

value - the decoded literal value, null for non-literals

position - the source position

Token

public Token(TokenType type,
             String text,
             SourcePosition position)

Creates a token with no decoded literal value.

Parameters:

type - the token type

text - the raw source text

position - the source position

Method Detail

getType

public TokenType getType()

Returns:

the type

getText

public String getText()

Returns:

the text

getValue

public Object getValue()

Returns:

the value

getPosition

public SourcePosition getPosition()

Returns:

the position

is

public boolean is(TokenType type)

Tests the token type.

Parameters:

type - the type to compare against

Returns:

true if the types match

isOneOf

public boolean isOneOf(TokenType... types)

Tests the token type against several candidates.

Parameters:

types - the types to compare against

Returns:

true if any type matches

isKeyword

public boolean isKeyword()

Returns:

true if the token type is a Java keyword

isLiteral

public boolean isLiteral()

Returns:

true if the token type is a literal

isModifier

public boolean isModifier()

Returns:

true if the token type is a modifier keyword

isPrimitiveType

public boolean isPrimitiveType()

Returns:

true if the token type is a primitive type keyword

getLine

public int getLine()

Returns:

the one-based source line

getColumn

public int getColumn()

Returns:

the one-based source column

intValue

public int intValue()

Reads the literal value as an int, narrowing any numeric value.

Returns:

the int value

Throws:

IllegalStateException - if the token carries no numeric value

longValue

public long longValue()

Reads the literal value as a long, widening or narrowing any numeric value.

Returns:

the long value

Throws:

IllegalStateException - if the token carries no numeric value

doubleValue

public double doubleValue()

Reads the literal value as a double, converting any numeric value.

Returns:

the double value

Throws:

IllegalStateException - if the token carries no numeric value

floatValue

public float floatValue()

Reads the literal value as a float, converting any numeric value.

Returns:

the float value

Throws:

IllegalStateException - if the token carries no numeric value

stringValue

public String stringValue()

Reads the literal value as a string.

Returns:

the string value

Throws:

IllegalStateException - if the token carries no string value

charValue

public char charValue()

Reads the literal value as a char.

Returns:

the char value

Throws:

IllegalStateException - if the token carries no char value

equals

public boolean equals(Object o)

Overrides:

equals in class Object

hashCode

public int hashCode()

Overrides:

hashCode in class Object

toString

public String toString()

Overrides:

toString in class Object