| Methods |
public
|
setInput(string $input): void
Sets the input data to be tokenized.
Sets the input data to be tokenized.
The Lexer is immediately reset and the new input tokenized.
Any unprocessed tokens from any previous input are lost.
Parameters
| $input |
The input to be tokenized.
|
|
#
|
public
|
reset(): void
Resets the lexer.
|
#
|
public
|
resetPeek(): void
Resets the peek pointer to 0.
Resets the peek pointer to 0.
|
#
|
public
|
resetPosition(int $position = 0): void
Resets the lexer position on the input to the given position.
Resets the lexer position on the input to the given position.
Parameters
| $position |
Position to place the lexical scanner.
|
|
#
|
public
|
getInputUntilPosition(int $position): string
Retrieve the original lexer's input until a given position.
Retrieve the original lexer's input until a given position.
|
#
|
public
|
isNextToken(T $type): bool
Checks whether a given token matches the current lookahead.
Checks whether a given token matches the current lookahead.
|
#
|
public
|
isNextTokenAny(list<T> $types): bool
Checks whether any of the given tokens matches the current lookahead.
Checks whether any of the given tokens matches the current lookahead.
|
#
|
public
|
moveNext(): bool
Moves to the next token in the input string.
Moves to the next token in the input string.
Overriden by
|
#
|
public
|
skipUntil(T $type): void
Tells the lexer to skip input tokens until it sees a token with the given value.
Tells the lexer to skip input tokens until it sees a token with the given value.
Parameters
| $type |
The token type to skip until.
|
|
#
|
public
|
isA(string $value, int|string $token): bool
Checks if given value is identical to the given token.
Checks if given value is identical to the given token.
|
#
|
public
|
peek(): mixed[]|null
Moves the lookahead token forward.
Moves the lookahead token forward.
Returns
The next token or NULL if there are no more tokens ahead.
|
#
|
public
|
glimpse(): mixed[]|null
Peeks at the next token, returns it and immediately resets the peek.
Peeks at the next token, returns it and immediately resets the peek.
Returns
The next token or NULL if there are no more tokens ahead.
|
#
|
protected
|
scan(string $input): void
Scans the input string for tokens.
Scans the input string for tokens.
Parameters
|
#
|
public
|
getLiteral(T $token): int|string
Gets the literal for a given token.
Gets the literal for a given token.
|
#
|
protected
|
getModifiers(): string
Regex modifiers
|
#
|
abstract
protected
|
getCatchablePatterns(): string[]
Lexical catchable patterns.
Lexical catchable patterns.
Implemented by
|
#
|
abstract
protected
|
getNonCatchablePatterns(): string[]
Lexical non-catchable patterns.
Lexical non-catchable patterns.
Implemented by
|
#
|
abstract
protected
|
getType(string &$value): T|null
Retrieve token type. Also processes the token value if necessary.
Retrieve token type. Also processes the token value if necessary.
Implemented by
|
#
|