Package emissary.util.search

Class ByteTokenizer

java.lang.Object

emissary.util.search.ByteTokenizer

All Implemented Interfaces:

Iterator<String>

public class ByteTokenizer
extends Object
implements Iterator<String>

The byte tokenizer class allows an application to break a byte buffer into tokens. This was modified from the java.util.StringTokenizer implementation. Note that all characters in the delimiter set are considered to be characters in the range 0 - 255. In other words the ISO8859-1 encoding is used to match the delimiters to the byte array.

Constructor Summary

Constructors

Constructor	Description
ByteTokenizer(byte[] bytes)	Constructs a byte tokenizer for the specified byte array.
ByteTokenizer(byte[] bytes, int start, int len)	Constructs a byte tokenizer for the specified byte array.
ByteTokenizer(byte[] bytes, int start, int len, String delim)	Constructs a byte tokenizer for the specified byte array.
ByteTokenizer(byte[] bytes, int start, int len, String delim, boolean returnDelims)	Constructs a byte tokenizer for the specified byte array.
ByteTokenizer(byte[] bytes, int start, int len, String delim, boolean returnDelims, String encoding)	Constructs a byte tokenizer for the specified byte array using the specified encoding.
ByteTokenizer(byte[] bytes, int start, int len, String delim, String encoding)	Constructs a byte tokenizer for the specified byte array using the specified encoding.
ByteTokenizer(byte[] bytes, String delim)	Constructs a byte tokenizer for the specified byte array.
ByteTokenizer(byte[] bytes, String delim, boolean returnDelims)	Constructs a byte tokenizer for the specified byte array.
ByteTokenizer(byte[] bytes, String delim, boolean returnDelims, String encoding)	Constructs a byte tokenizer for the specified byte array using the specified encoding.
ByteTokenizer(byte[] bytes, String delim, String encoding)	Constructs a byte tokenizer for the specified byte array using the specified encoding.

Method Summary

Modifier and Type	Method	Description
int	countTokens()	Calculates the number of times that this tokenizer's next method can be called before it generates an exception.
boolean	hasNext()	Tests if there are more tokens available from this tokenizer's string.
String	next()	Returns the next token from this string tokenizer.
String	next(String delim)	Returns the next token in this string tokenizer's string.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface java.util.Iterator

forEachRemaining, remove

Constructor Detail

ByteTokenizer

public ByteTokenizer(byte[] bytes,
                     int start,
                     int len,
                     String delim,
                     boolean returnDelims)

Constructs a byte tokenizer for the specified byte array. All characters in the delim argument are the delimiters for separating tokens. Characters must be in the range of 0 - 255.

If the returnDelims flag is true, then the delimiter characters are also returned as tokens. Each delimiter is returned as a byte[] or String of length one. If the flag is false, the delimiter characters are skipped and only serve as separators between tokens.

Parameters:

bytes - a byte array to be parsed.

start - the first byte in the array

len - the number of bytes to parse

delim - the delimiters.

returnDelims - flag indicating whether to return the delimiters as tokens.

ByteTokenizer

public ByteTokenizer(byte[] bytes,
                     int start,
                     int len,
                     String delim,
                     boolean returnDelims,
                     String encoding)
              throws UnsupportedEncodingException

Constructs a byte tokenizer for the specified byte array using the specified encoding.

Parameters:

bytes - a byte array to be parsed.

start - the first byte in the array

len - the number of bytes to parse

delim - the delimiters.

returnDelims - flag indicating whether to return the delimiters as tokens.

encoding - the encoding for which to return the bytes.

Throws:

UnsupportedEncodingException - thrown if the supplied encoding is unsupported.

ByteTokenizer

public ByteTokenizer(byte[] bytes,
                     int start,
                     int len,
                     String delim)

Constructs a byte tokenizer for the specified byte array. The characters in the delim argument are the delimiters for separating tokens. Delimiter characters themselves will not be treated as tokens.

Parameters:

bytes - a byte array to be parsed.

start - the first byte in the array

len - the number of bytes to parse

delim - the delimiters.

ByteTokenizer

public ByteTokenizer(byte[] bytes,
                     int start,
                     int len,
                     String delim,
                     String encoding)
              throws UnsupportedEncodingException

Constructs a byte tokenizer for the specified byte array using the specified encoding.

Parameters:

bytes - a byte array to be parsed.

start - the first byte in the array

len - the number of bytes to parse

delim - the delimiters.

encoding - the encoding for which to return the bytes.

Throws:

UnsupportedEncodingException - thrown if the supplied encoding is unsupported.

ByteTokenizer

public ByteTokenizer(byte[] bytes,
                     int start,
                     int len)

Constructs a byte tokenizer for the specified byte array. The tokenizer uses the default delimiter set, which is " \t\n\r\f": the space character, the tab character, the newline character, the carriage-return character, and the form-feed character. Delimiter characters themselves will not be treated as tokens.

Parameters:

bytes - a byte array to be parsed.

start - the first byte in the array

len - the number of bytes to parse

ByteTokenizer

public ByteTokenizer(byte[] bytes,
                     String delim,
                     boolean returnDelims)

Constructs a byte tokenizer for the specified byte array. All characters in the delim argument are the delimiters for separating tokens. Characters must be in the range of 0 - 255.

If the returnDelims flag is true, then the delimiter characters are also returned as tokens. Each delimiter is returned as a byte[] or String of length one. If the flag is false, the delimiter characters are skipped and only serve as separators between tokens.

Parameters:

bytes - a byte array to be parsed.

delim - the delimiters.

returnDelims - flag indicating whether to return the delimiters as tokens.

ByteTokenizer

public ByteTokenizer(byte[] bytes,
                     String delim,
                     boolean returnDelims,
                     String encoding)
              throws UnsupportedEncodingException

Constructs a byte tokenizer for the specified byte array using the specified encoding.

Parameters:

bytes - a byte array to be parsed.

delim - the delimiters.

returnDelims - flag indicating whether to return the delimiters as tokens.

encoding - the encoding for which to return the bytes.

Throws:

UnsupportedEncodingException - thrown if the supplied encoding is unsupported.

ByteTokenizer

public ByteTokenizer(byte[] bytes,
                     String delim)

Constructs a byte tokenizer for the specified byte array. The characters in the delim argument are the delimiters for separating tokens. Delimiter characters themselves will not be treated as tokens.

Parameters:

bytes - a byte array to be parsed.

delim - the delimiters.

ByteTokenizer

public ByteTokenizer(byte[] bytes,
                     String delim,
                     String encoding)
              throws UnsupportedEncodingException

Constructs a byte tokenizer for the specified byte array using the specified encoding.

Parameters:

bytes - a byte array to be parsed.

delim - the delimiters.

encoding - the encoding for which to return the bytes.

Throws:

UnsupportedEncodingException - thrown if the supplied encoding is unsupported.

ByteTokenizer

public ByteTokenizer(byte[] bytes)

Constructs a byte tokenizer for the specified byte array. The tokenizer uses the default delimiter set, which is " \t\n\r\f": the space character, the tab character, the newline character, the carriage-return character, and the form-feed character. Delimiter characters themselves will not be treated as tokens.

Parameters:

bytes - a byte array to be parsed.

Method Detail

hasNext

public boolean hasNext()

Tests if there are more tokens available from this tokenizer's string. If this method returns true, then a subsequent call to next with no argument will successfully return a token.

Specified by:

hasNext in interface Iterator<String>

Returns:

true if and only if there is at least one token in the string after the current position; false otherwise.

next

public String next()

Returns the next token from this string tokenizer.

Specified by:

next in interface Iterator<String>

Returns:

the next token from this string tokenizer.

Throws:

NoSuchElementException - if there are no more tokens in this tokenizer's string.

next

public String next(String delim)

Returns the next token in this string tokenizer's string. First, the set of characters considered to be delimiters by this ByteTokenizer object is changed to be the characters in the string delim. Then the next token in the string after the current position is returned. The current position is advanced beyond the recognized token. The new delimiter set remains the default after this call.

Parameters:

delim - the new delimiters.

Returns:

the next token, after switching to the new delimiter set.

Throws:

NoSuchElementException - if there are no more tokens in this tokenizer's string.

countTokens

public int countTokens()

Calculates the number of times that this tokenizer's next method can be called before it generates an exception. The current position is not advanced.

Returns:

the number of tokens remaining in the string using the current delimiter set.

See Also:

next()