Package emissary.util

Class LineTokenizer


  • public class LineTokenizer
    extends Object
    Used the default way, this is equivalent to StringTokenizer st = new StringTokenizer(new String(theData),"\n",false); except that a token is returned for blank lines as well. There seems no way to tell the StringTokenizer to do that.
    • Constructor Summary

      Constructors 
      Constructor Description
      LineTokenizer​(byte[] theData)
      Create a line tokenizer to operate on some data
      LineTokenizer​(byte[] theData, byte delim)
      Create a line tokenizer to operate on some data
      LineTokenizer​(byte[] theData, byte delim, String charset)
      Create a line tokenizer to operate on some data
      LineTokenizer​(byte[] theData, byte delim, Charset charset)
      Create a line tokenizer to operate on some data
      LineTokenizer​(byte[] theData, String charset)
      Create a line tokenizer to operate on some data
      LineTokenizer​(byte[] theData, Charset charset)
      Create a line tokenizer to operate on some data
    • Field Detail

      • previousIndex

        protected int previousIndex
      • index

        protected int index
      • delim

        protected byte delim
      • tokenCount

        protected int tokenCount
      • data

        protected byte[] data
      • charset

        @Nullable
        protected Charset charset
    • Constructor Detail

      • LineTokenizer

        public LineTokenizer​(byte[] theData)
        Create a line tokenizer to operate on some data
        Parameters:
        theData - byte array of data
      • LineTokenizer

        public LineTokenizer​(byte[] theData,
                             byte delim)
        Create a line tokenizer to operate on some data
        Parameters:
        theData - byte array of data
        delim - the delimiter to mark off lines
      • LineTokenizer

        public LineTokenizer​(byte[] theData,
                             byte delim,
                             @Nullable
                             String charset)
        Create a line tokenizer to operate on some data
        Parameters:
        theData - byte array of data
        delim - the delimiter to mark off lines
        charset - the character set to use the outputting tokens as strings
      • LineTokenizer

        public LineTokenizer​(byte[] theData,
                             byte delim,
                             Charset charset)
        Create a line tokenizer to operate on some data
        Parameters:
        theData - byte array of data
        delim - the delimiter to mark off lines
        charset - the character set to use the outputting tokens as strings
      • LineTokenizer

        public LineTokenizer​(byte[] theData,
                             @Nullable
                             String charset)
        Create a line tokenizer to operate on some data
        Parameters:
        theData - byte array of data
        charset - the character set to use the outputting tokens as strings
      • LineTokenizer

        public LineTokenizer​(byte[] theData,
                             Charset charset)
        Create a line tokenizer to operate on some data
        Parameters:
        theData - byte array of data
        charset - the character set to use the outputting tokens as strings
    • Method Detail

      • setCharset

        public void setCharset​(@Nullable
                               String charset)
        Set the character set to use when outputting tokens as strings default is 8859_1
        Parameters:
        charset - the java charset value
      • setCharset

        public void setCharset​(Charset charset)
        Set the character set to use when outputting tokens as strings default is 8859_1
        Parameters:
        charset - the java charset value
      • hasMoreTokens

        public boolean hasMoreTokens()
        Indicate if there are more lines
        Returns:
        true if there are more lines
      • countTokens

        public int countTokens()
        Current count of tokens remaining
        Returns:
        count of tokens remaining
      • getCurrentPosition

        public int getCurrentPosition()
        Current byte offset in the data Caller can use this on their copy of the original data buffer to extract data of interest
        Returns:
        current byte offset
      • nextToken

        public String nextToken()
        Next token as a string The string is created using the charset specified in the constructor or in the setCharset(String s) method
        Returns:
        the next line as a string
      • nextTokenBytes

        @Nullable
        public byte[] nextTokenBytes()
        Next token as an array of bytes
        Returns:
        the next line as an array of bytes
      • pushBack

        public void pushBack()
        Push back a single token onto the stack We only take a single push back. This just moves our pointers to the previous index