plume

Class StrTok

java.lang.Object

plume.StrTok

public class StrTok
extends java.lang.Object

Provides a somewhat simpler interface for tokenizing strings than does StreamTokenizer. All tokenizing is done by StreamTokenizer.

The major difference from StreamTokenizer is that all tokens are returned as strings. EOF returns a null, EOL returns an empty string. A delimiter is returned as a one-character string. Words and numbers are returned as strings. Quoted strings are also returned as strings including their quote characters (so they can easily be differentiated from words and numbers).

Other differences are:

• Only handles tokenizing strings, not other sources of characters.
• Automatic setup for tokenizing strings.
• Underscores are included in identifiers (words).
• I/O errors (which should be impossible when tokenizing strings) are converted to RuntimeExceptions so that every call doesn't have to be included in a try block.
• Convenience functions isWord(), isQString(), and need().
• Returned string tokens are interned for easier comparisons.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	StrTok.ErrorHandler Default class for error handling.

Field Summary

Fields

Modifier and Type	Field and Description
java.io.StreamTokenizer	stok The StreamTokenizer to which all work is delegated.

Constructor Summary

Constructors

Constructor and Description
StrTok(java.lang.String s) Creates a tokenizer for the specified string.
StrTok(java.lang.String s, StrTok.ErrorHandler e) Creates a tokenizer for the specified string with the specified error handler.

Method Summary

Modifier and Type	Method and Description
void	commentChar(int ch) Specifies the single line comment character.
boolean	isQString() Returns true if the current token is a quoted string.
boolean	isWord() Returns true if the current token is a word (identifier).
java.lang.String	need_word() Reads the next token and checks to make sure that it is a word (id).
void	need(java.lang.String tok) Reads the next token and checks that it matches tok.
java.lang.String	nextToken() Returns the next token as a string.
void	pushBack() Causes the next call to nextToken() to return the current token.
void	quoteChar(int ch) Specifies that matching pairs of this character delimit string constants.
void	set_error_handler(StrTok.ErrorHandler errorHandler) Sets the error handler.
java.lang.String	token() Return the current token.
int	ttype() Return the type of the current token.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

stok

public java.io.StreamTokenizer stok

The StreamTokenizer to which all work is delegated.

Constructor Detail

StrTok

public StrTok(java.lang.String s)

Creates a tokenizer for the specified string.

Parameters:

s - string to tokenize

StrTok

public StrTok(java.lang.String s,
              StrTok.ErrorHandler e)

Creates a tokenizer for the specified string with the specified error handler.

Parameters:

s - string to tokenize

e - error handler

Method Detail

nextToken

public java.lang.String nextToken()

Returns the next token as a string. EOF returns a null, EOL returns an empty string. Delimiters are returned as one character strings. Quoted strings and words are returned as strings.

Returns:

the next token

pushBack

public void pushBack()

Causes the next call to nextToken() to return the current token.

token

public java.lang.String token()

Return the current token.

Returns:

the current token

See Also:

nextToken()

commentChar

public void commentChar(int ch)

Specifies the single line comment character.

Parameters:

ch - the comment character

See Also:

StreamTokenizer.commentChar(int)

quoteChar

public void quoteChar(int ch)

Specifies that matching pairs of this character delimit string constants.

Parameters:

ch - the quoting character

See Also:

StreamTokenizer.quoteChar(int)

ttype

public int ttype()

Return the type of the current token.

Returns:

the type of the current token

See Also:

StreamTokenizer.ttype

isWord

public boolean isWord()

Returns true if the current token is a word (identifier).

Returns:

true iff the current token is a word (identifier)

isQString

public boolean isQString()

Returns true if the current token is a quoted string.

Returns:

true iff the current token is a quoted string

set_error_handler

public void set_error_handler(StrTok.ErrorHandler errorHandler)

Sets the error handler. The default error handler will throw a runtime exception on errors.

Parameters:

errorHandler - the new error handler

See Also:

StrTok.ErrorHandler

need

public void need(java.lang.String tok)

Reads the next token and checks that it matches tok. If it does not match, calls the current error handling routine (see set_error_handler()). If it does match, just returns.

Parameters:

tok - string to check next token against

need_word

public java.lang.String need_word()

Reads the next token and checks to make sure that it is a word (id). If it is not a word, calls the error handling routine. If it is, returns the string of the word.

Returns:

next token, if if it is a word; otherwise calls the error handling routine