antlr
Class TokenBuffer

java.lang.Object
  |
  +--antlr.TokenBuffer

public class TokenBuffer
extends Object

Untamed: A Stream of Token objects fed to the parser from a Tokenizer that can be rewound via mark()/rewind() methods.

A dynamic array is used to buffer up all the input tokens. Normally, "k" tokens are stored in the buffer. More tokens may be stored during guess mode (testing syntactic predicate), or when LT(i>k) is referenced. Consumption of tokens is deferred. In other words, reading the next token is not done by conume(), but deferred until needed by LA or LT.

See Also:
antlr.Token, antlr.Tokenizer, antlr.TokenQueue

Field Summary
protected  TokenStream input
           
(package private)  int markerOffset
           
(package private)  int nMarkers
           
(package private)  int numToConsume
           
(package private)  TokenQueue queue
           
 
Constructor Summary
TokenBuffer(TokenStream input_)
          Enabled: Create a token buffer
 
Method Summary
 void consume()
          Enabled: Mark another token for deferred consumption
private  void fill(int amount)
          Ensure that the token buffer is sufficiently full
 TokenStream getInput()
          Enabled: return the Tokenizer (needed by ParseView)
 int LA(int i)
          Enabled: Get a lookahead token value
 Token LT(int i)
          Enabled: Get a lookahead token
 int mark()
          Enabled: Return an integer marker that can be used to rewind the buffer to its current state.
 void rewind(int mark)
          Enabled: Rewind the token buffer to a marker.
private  void syncConsume()
          Sync up deferred consumption
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

input

protected TokenStream input

nMarkers

int nMarkers

markerOffset

int markerOffset

numToConsume

int numToConsume

queue

TokenQueue queue
Constructor Detail

TokenBuffer

public TokenBuffer(TokenStream input_)
Enabled: Create a token buffer

Method Detail

consume

public final void consume()
Enabled: Mark another token for deferred consumption


fill

private final void fill(int amount)
                 throws TokenStreamException
Ensure that the token buffer is sufficiently full

TokenStreamException

getInput

public TokenStream getInput()
Enabled: return the Tokenizer (needed by ParseView)


LA

public final int LA(int i)
             throws TokenStreamException
Enabled: Get a lookahead token value

TokenStreamException

LT

public final Token LT(int i)
               throws TokenStreamException
Enabled: Get a lookahead token

TokenStreamException

mark

public final int mark()
Enabled: Return an integer marker that can be used to rewind the buffer to its current state.


rewind

public final void rewind(int mark)
Enabled: Rewind the token buffer to a marker.

Parameters:
mark - Marker returned previously from mark()

syncConsume

private final void syncConsume()
Sync up deferred consumption



comments?