2.9 KiB
asciigoat's core library
This package contains the basics for writing simple parsers of text languages heavily inspired by Rob Pike's talk on Lexical Scanning in Go in 2011 which you can watch online to get better understanding of the ideas behind asciigoat.
asciigoat is MIT licensed.
Lexer
lexer.Reader
The lexer package provides lexer.Reader which is
actually an io.RuneScanner
that buffers accepted runes until you are ready to
emit or
discard.
lexer.Position
lexer.Position
is a (Line, Column) pair with methods to facilitate tracking
your position on the source Reader.
lexer.Error
lexer.Error
is an unwrappable error with a
token position and hint attached.
lexer.StateFn
At the heart of asciigoat we have state functions as proposed on Rob Pike's famous talk which return the next state function parsing is done.
Additionally there is a Run() helper that implements the loop.
rune checkers
Rune checkers are simple functions that tell if a rune is of a class or it's not.
Fundamental checkers are provided by the unicode package.
Our lexer.Reader uses them on its Accept() and AcceptAll() methods to
make it easier to consume the source document.
To facilitate the declaration of rune classes in the context of asciigoat powered parsers we include a series of rune checker factories.
NewIsIn(string)NewIsInRunes(...rune)NewIsNot(checker)NewIsOneOf(...checker)
Others
ReadCloser
ReadCloser assists in providing a io.Closer to Readers or buffers without on, or unearthing one if available so io.ReadCloser can be fulfilled.