org.egothor.parser.plain

Class SimpleTokenManager

java.lang.Object

org.egothor.parser.plain.SimpleTokenManager

All Implemented Interfaces:

SimpleConstants

public class SimpleTokenManager
extends Object
implements SimpleConstants

Field Summary

Fields

Modifier and Type	Field and Description
PrintStream	debugStream
static String[]	jjstrLiteralImages
static String[]	lexStateNames

Fields inherited from interface org.egothor.parser.plain.SimpleConstants

ACRONYM, CJK_LETTER, CRAP, DATE, DEFAULT, DIGIT, EOF, GRAM, LETTER, MARK, NOISE, NUM, PUNCT, tokenImage, WORD

Constructor Summary

Constructors

Constructor and Description
SimpleTokenManager(CharStream stream)
SimpleTokenManager(CharStream stream, int lexState)

Method Summary

Modifier and Type	Method and Description
Token	getNextToken()
void	ReInit(CharStream stream)
void	ReInit(CharStream stream, int lexState)
void	setDebugStream(PrintStream ds)
void	SwitchTo(int lexState)

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

debugStream

public PrintStream debugStream

jjstrLiteralImages

public static final String[] jjstrLiteralImages

lexStateNames

public static final String[] lexStateNames

Constructor Detail

SimpleTokenManager

public SimpleTokenManager(CharStream stream)

SimpleTokenManager

public SimpleTokenManager(CharStream stream,
                          int lexState)

Method Detail

setDebugStream

public void setDebugStream(PrintStream ds)

ReInit

public void ReInit(CharStream stream)

ReInit

public void ReInit(CharStream stream,
                   int lexState)

SwitchTo

public void SwitchTo(int lexState)

getNextToken

public Token getNextToken()