This module implements a base object of a lexer with efficient buffer handling. Only at line endings checks are necessary if the buffer needs refilling.
Types
BaseLexer = object of RootObj bufpos*: int ## the current position within the buffer when defined(js): buf*: string else: buf*: cstring bufLen*: int ## length of buffer in characters input: Stream ## the input stream lineNumber*: int ## the current line number sentinel: int lineStart: int refillChars: set[char]
- the base lexer. Inherit your lexer from this object. Source Edit
Procs
proc close(L: var BaseLexer) {.
raises: [Exception], tags: [].}- closes the base lexer. This closes L's associated stream too. Source Edit
proc handleCR(L: var BaseLexer; pos: int): int {.
raises: [Exception], tags: [ReadIOEffect].}- Call this if you scanned over 'c' in the buffer; it returns the the position to continue the scanning from. pos must be the position of the 'c'. Source Edit
proc handleLF(L: var BaseLexer; pos: int): int {.
raises: [Exception], tags: [ReadIOEffect].}- Call this if you scanned over 'L' in the buffer; it returns the the position to continue the scanning from. pos must be the position of the 'L'. Source Edit
proc handleRefillChar(L: var BaseLexer; pos: int): int {.
raises: [Exception], tags: [ReadIOEffect].}- To be documented. Source Edit
proc open(L: var BaseLexer; input: Stream; bufLen: int = 8192; refillChars: set[char] = NewLines) {.
raises: [Exception], tags: [ReadIOEffect].}- inits the BaseLexer with a stream to read from. Source Edit
proc getColNumber(L: BaseLexer; pos: int): int {.
raises: [], tags: [].}- retrieves the current column. Source Edit
proc getCurrentLine(L: BaseLexer; marker: bool = true): string {.
raises: [], tags: [].}- retrieves the current line. Source Edit