org.quasiliteral.term
Class TermLexer

java.lang.Object
  |
  +--org.quasiliteral.syntax.BaseLexer
        |
        +--org.quasiliteral.term.TermLexer

All Implemented Interfaces:

LexerFace, Marker, PassByProxy

public class TermLexer

extends BaseLexer

Untamed: Breaks textually input into a stream of tokens according to the Term language's defined grammar.

Author:

Mark S. Miller

Field Summary

Fields inherited from class org.quasiliteral.syntax.BaseLexer

myBuilder, myChar, myContinueCount, myDelayedNextChar, myIndenter, myLData, myLTwine, myNoTabsFlag, myOptStartPos, myOptStartText, myPos, myQuasiFlag

Fields inherited from interface org.quasiliteral.syntax.LexerFace

EOFCHAR, EOFTOK

Fields inherited from interface org.erights.e.elib.serial.PassByProxy

HONORARY, HONORED_NAMES

Constructor Summary

TermLexer(LineFeeder input, boolean partialFlag, boolean quasiFlag, boolean noTabsFlag)
Enabled:

TermLexer(LineFeeder input, boolean partialFlag, boolean quasiFlag, boolean noTabsFlag, AstroBuilder builder)
Enabled:

Method Summary

protected Astro	getNextToken()
protected boolean	isRestBlank(int start) Consider '#' and '//' comments to be blank as well.
static boolean	isSegPart(char c) Enabled: Any character that may be a part of a Java identifier, or a '.' or '-'.
static boolean	isSegStart(char c) Enabled: Any character that may start a Java identifier.
static void	main(String[] args) Enabled: Just for testing.
static TermLexer	make(Twine sourceCode, boolean quasiFlag, boolean noTabsFlag) Enabled: @param sourceCode The source code itself
private Astro	tag() A tag is a sequence of tag-segments separated by colons.

Methods inherited from class org.quasiliteral.syntax.BaseLexer

charConstant, charLiteral, closeBracket, composite, docComment, endSpan, endToken, isEndOfFile, isJavaIdPart, isJavaIdStart, isWhite, leafEOL, leafTag, needMore, nextChar, nextToken, nextTopLevelUnit, numberLiteral, openBracket, openBracket, peekChar, peekChar, reset, setSource, skipLine, skipWhiteSpace, startToken, stopToken, stringLiteral, syntaxError, toString

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

TermLexer

public TermLexer(LineFeeder input,
                 boolean partialFlag,
                 boolean quasiFlag,
                 boolean noTabsFlag)
          throws IOException

Enabled:

TermLexer

public TermLexer(LineFeeder input,
                 boolean partialFlag,
                 boolean quasiFlag,
                 boolean noTabsFlag,
                 AstroBuilder builder)
          throws IOException

Enabled:

Method Detail

make

public static TermLexer make(Twine sourceCode,
                             boolean quasiFlag,
                             boolean noTabsFlag)
                      throws IOException

Enabled: @param sourceCode The source code itself

Parameters:

quasiFlag - Should doubled @ and $ be collapsed to singles?

noTabsFlag - Should tabs be rejected as valid whitespace? Assumed to correspond to the e.enable.notabs property.

IOException

isRestBlank

protected boolean isRestBlank(int start)

Consider '#' and '//' comments to be blank as well.

Overrides:

isRestBlank in class BaseLexer

getNextToken

protected Astro getNextToken()
                      throws IOException,
                             SyntaxException

Specified by:

getNextToken in class BaseLexer

IOException

SyntaxException

isSegStart

public static boolean isSegStart(char c)

Enabled: Any character that may start a Java identifier.

Note that a Java identifier may start with a '_' or '$'.

This includes all the characters that may start an XML-NS-NCName as well as '$'.

This also includes all the characters that may start a Java or E fully qualified name.

isSegPart

public static boolean isSegPart(char c)

Enabled: Any character that may be a part of a Java identifier, or a '.' or '-'.

Note that a Java identifier may contain '_'s and '$'s.

Not yet dealt with are the XML CombiningChars or the XML Extenders, both of which should be included in isSegStart. Once these are included, then this will include all the characters that can be part of an XML-NS-NCNameChar as well as '$'.

This also includes all the characters that may be a part of a Java or E fully qualified name.

tag

private Astro tag()
           throws IOException,
                  SyntaxException

A tag is a sequence of tag-segments separated by colons.

     <tag> ::= <segment> (':' <segment>)*
     <segment> ::= <segStart> <segPart>*
     |             '.' <segStart> <segPart>*
     |             '<' <uric>* '>'
 

Because of the second segment production, a segment may also begin with a '.', though neither an XML-NS-NCName nor a Java or E fully qualified name may begin with a '.'. Therefore, names that begin with a '.' (like ".bag.") can be used in a keyword-like way without conflicting with these other namespaces. Out of Fortran nostalgia, by convention such names end with a '.' as well.

IOException

SyntaxException

main

public static void main(String[] args)
                 throws IOException,
                        SyntaxException

Enabled: Just for testing. Reads an input file and prints one token per line to stdout.

IOException

SyntaxException