Parser.Advanced
Core
Context-aware parsers with custom problem types.
Types
Parser
type Parser context problem value = Parser (State context -> PStep context problem valueAn advanced Parser gives two ways to improve your error messages:
- problem - Instead of all errors being a String, you can create a custom type like type Problem = BadIndent | BadKeyword String and track problems much more precisely. - context - Error messages can be further improved when precise problems are paired with information about where you ran into trouble. By tracking the context, instead of saying “I found a bad keyword” you can say “I found a bad keyword when parsing a list” and give folks a better idea of what the parser thinks it is doing.
I recommend starting with the simpler Parser module though, and when you feel comfortable and want better error messages, you can create a type alias like this:
Examples
import Parser.Advanced
type alias MyParser a =
Parser.Advanced.Parser Context Problem a
type Context = Definition String | List | Record
type Problem = BadIndent | BadKeyword StringAll of the functions from Parser should exist in Parser.Advanced in some form, allowing you to switch over pretty easily.
DeadEnd
type alias DeadEnd context problem =
{ row : Int
, col : Int
, problem : problem
, contextStack : List { row : Int
, col : Int
, context : context
}
}Say you are parsing a function named viewHealthData that contains a list. You might get a DeadEnd like this:
Examples
{ row = 18
, col = 22
, problem = UnexpectedComma
, contextStack =
[ { row = 14
, col = 1
, context = Definition "viewHealthData"
}
, { row = 15
, col = 4
, context = List
}
]
}We have a ton of information here! So in the error message, we can say that “I ran into an issue when parsing a list in the definition of viewHealthData. It looks like there is an extra comma.” Or maybe something even better!
Furthermore, many parsers just put a mark where the problem manifested. By tracking the row and col of the context, we can show a much larger region as a way of indicating “I thought I was parsing this thing that starts over here.” Otherwise you can get very confusing error messages on a missing ] or } or ) because “I need more indentation” on something unrelated.
**Note:** Rows and columns are counted like a text editor. The beginning is row=1 and col=1. The col increments as characters are chomped. When a \n is chomped, row is incremented and col starts over again at 1.
Step
type Step state a
= Loop state
| Done aJust like Parser.Step
Examples
Loop state -- keep going
Done value -- finishToken
type Token x = Token String xWith the simpler Parser module, you could just say symbol "," and parse all the commas you wanted. But now that we have a custom type for our problems, we actually have to specify that as well. So anywhere you just used a String in the simpler module, you now use a Token Problem in the advanced module:
Examples
type Problem
= ExpectingComma
| ExpectingListEnd
comma : Token Problem
comma =
Token "," ExpectingComma
listEnd : Token Problem
listEnd =
Token "]" ExpectingListEndYou can be creative with your custom type. Maybe you want a lot of detail. Maybe you want looser categories. It is a custom type. Do what makes sense for you!
Trailing
type Trailing
= Forbidden
| Optional
| MandatoryWhat’s the deal with trailing commas? Are they Forbidden? Are they Optional? Are they Mandatory? Welcome to shapes club!
Examples
{ trailing = Optional } -- allow [1,2] and [1,2,]Nestable
type Nestable
= NotNestable
| NestableWorks just like Parser.Nestable to help distinguish between unnestable /* */ comments like in JS and nestable {- -} comments like {- -}.
Examples
multiComment open close NestableValues
|=
infix left 5 (|=) = keeperKeep the value from a parser in a pipeline.
Examples
run (succeed identity |= int ExpectingInt InvalidNumber) "42" == Ok 42|.
infix left 6 (|.) = ignorerSkip the value from a parser in a pipeline.
Examples
run (succeed () |. symbol (Token "," ExpectingComma)) "," == Ok ()run
run : Parser c x a -> String -> Result (List (DeadEnd c x)) aThis works just like Parser.run. The only difference is that when it fails, it has much more precise information for each dead end.
Examples
run (succeed 42) "" == Ok 42succeed
succeed : a -> Parser c x aJust like Parser.succeed
Examples
run (succeed 90210) "mississippi" == Ok 90210problem
problem : x -> Parser c x aJust like Parser.problem except you provide a custom type for your problem.
Examples
run (problem BadZip) "" == Err ...map
map : (a -> b) -> Parser c x a -> Parser c x bJust like Parser.map
Examples
run (map String.fromInt (int ExpectingInt InvalidNumber)) "123" == Ok "123"andThen
andThen : (a -> Parser c x b) -> Parser c x a -> Parser c x bJust like Parser.andThen
Examples
run (int ExpectingInt InvalidNumber |> andThen (\n -> succeed (n + 1))) "1" == Ok 2lazy
lazy : (() -> Parser c x a) -> Parser c x aJust like Parser.lazy
Examples
lazy (\_ -> succeed ())oneOf
oneOf : List (Parser c x a) -> Parser c x aJust like Parser.oneOf
Examples
run (oneOf [ succeed 1, succeed 2 ]) "" == Ok 1loop
loop : state -> (state -> Parser c x (Step state a)) -> Parser c x aJust like Parser.loop
Examples
loop [] (\_ -> succeed (Done []))backtrackable
backtrackable : Parser c x a -> Parser c x aJust like Parser.backtrackable
Examples
backtrackable (token (Token "let" ExpectingLet))commit
commit : a -> Parser c x aJust like Parser.commit
Examples
run (commit ()) "" == Ok ()symbol
symbol : Token x -> Parser c x ()Just like Parser.symbol except you provide a Token to clearly indicate your custom type of problems:
Examples
comma : Parser Context Problem ()
comma =
symbol (Token "," ExpectingComma)keyword
keyword : Token x -> Parser c x ()Just like Parser.keyword except you provide a Token to clearly indicate your custom type of problems:
Examples
let_ : Parser Context Problem ()
let_ =
symbol (Token "let" ExpectingLet)Note that this would fail to chomp letter because of the subsequent characters. Use token if you do not want that last letter check.
token
token : Token x -> Parser c x ()Just like Parser.token except you provide a Token specifying your custom type of problems.
Examples
run (token (Token "," ExpectingComma)) "," == Ok ()int
int : x -> x -> Parser c x IntJust like Parser.int where you have to handle negation yourself. The only difference is that you provide a two potential problems:
Examples
int : x -> x -> Parser c x Int
int expecting invalid =
number
{ int = Ok identity
, hex = Err invalid
, octal = Err invalid
, binary = Err invalid
, float = Err invalid
, invalid = invalid
, expecting = expecting
}You can use problems like ExpectingInt and InvalidNumber.
float
float : x -> x -> Parser c x FloatJust like Parser.float where you have to handle negation yourself. The only difference is that you provide a two potential problems:
Examples
float : x -> x -> Parser c x Float
float expecting invalid =
number
{ int = Ok toFloat
, hex = Err invalid
, octal = Err invalid
, binary = Err invalid
, float = Ok identity
, invalid = invalid
, expecting = expecting
}You can use problems like ExpectingFloat and InvalidNumber.
number
number : { int : Result x (Int -> a) , hex : Result x (Int -> a) , octal : Result x (Int -> a) , binary : Result x (Int -> a) , float : Result x (Float -> a) , invalid : x , expecting : x } -> Parser c x aJust like Parser.number where you have to handle negation yourself. The only difference is that you provide all the potential problems.
Examples
run (int ExpectingInt InvalidNumber) "123" == Ok 123end
end : x -> Parser c x ()Just like Parser.end except you provide the problem that arises when the parser is not at the end of the input.
Examples
run (succeed () |. end ExpectingEnd) "" == Ok ()getChompedString
getChompedString : Parser c x a -> Parser c x StringJust like Parser.getChompedString
Examples
run (getChompedString (chompWhile Char.isDigit)) "123" == Ok "123"mapChompedString
mapChompedString : (String -> a -> b) -> Parser c x a -> Parser c x bJust like Parser.mapChompedString
Examples
mapChompedString (\str n -> (str, n)) (int ExpectingInt InvalidNumber)chompIf
chompIf : (Char -> Bool) -> x -> Parser c x ()Just like Parser.chompIf except you provide a problem in case a character cannot be chomped.
Examples
run (chompIf Char.isUpper UnexpectedChar) "T" == Ok ()chompWhile
chompWhile : (Char -> Bool) -> Parser c x ()Just like Parser.chompWhile
Examples
run (chompWhile (\c -> c == ' ')) " x" == Ok ()chompUntil
chompUntil : Token x -> Parser c x ()Just like Parser.chompUntil except you provide a Token in case you chomp all the way to the end of the input without finding what you need.
Examples
run (chompUntil (Token "*/" ExpectingEndComment)) "hi */" == Ok ()chompUntilEndOr
chompUntilEndOr : String -> Parser c x ()Just like Parser.chompUntilEndOr
Examples
run (chompUntilEndOr "\n") "hi" == Ok ()inContext
inContext : context -> Parser context x a -> Parser context x aThis is how you mark that you are in a certain context. For example, here is a rough outline of some code that uses inContext to mark when you are parsing a specific definition:
Examples
import Char
import Parser.Advanced exposing (..)
import Set
type Context
= Definition String
| List
definition : Parser Context Problem Expr
definition =
functionName
|> andThen definitionBody
definitionBody : String -> Parser Context Problem Expr
definitionBody name =
inContext (Definition name) <|
succeed (Function name)
|= arguments
|. symbol (Token "=" ExpectingEquals)
|= expression
functionName : Parser c Problem String
functionName =
variable
{ start = Char.isLower
, inner = Char.isAlphaNum
, reserved = Set.fromList ["let","in"]
, expecting = ExpectingFunctionName
}First we parse the function name, and then we parse the rest of the definition. Importantly, we call inContext so that any dead end that occurs in definitionBody will get this extra context information. That way you can say things like, “I was expecting an equals sign in the view definition.” Context!
getIndent
getIndent : Parser c x IntJust like Parser.getIndent
Examples
run getIndent "" == Ok 0withIndent
withIndent : Int -> Parser c x a -> Parser c x aJust like Parser.withIndent
Examples
run (withIndent 4 getIndent) "" == Ok 4getPosition
getPosition : Parser c x (Int, Int)Just like Parser.getPosition
Examples
run getPosition "" == Ok (1, 1)getRow
getRow : Parser c x IntJust like Parser.getRow
Examples
run getRow "" == Ok 1getCol
getCol : Parser c x IntJust like Parser.getCol
Examples
run getCol "" == Ok 1getOffset
getOffset : Parser c x IntJust like Parser.getOffset
Examples
run getOffset "" == Ok 0getSource
getSource : Parser c x StringJust like Parser.getSource
Examples
run getSource "hi" == Ok "hi"variable
variable : { start : Char -> Bool , inner : Char -> Bool , reserved : Set.Set String , expecting : x } -> Parser c x StringJust like Parser.variable except you specify the problem yourself.
Examples
variable { start = Char.isLower, inner = Char.isAlphaNum, reserved = Set.empty, expecting = ExpectingVar }sequence
sequence : { start : Token x , separator : Token x , end : Token x , spaces : Parser c x () , item : Parser c x a , trailing : Trailing } -> Parser c x (List a)Just like Parser.sequence except with a Token for the start, separator, and end. That way you can specify your custom type of problem for when something is not found.
Examples
sequence { start = Token "[" ExpectingListStart, separator = Token "," ExpectingComma, end = Token "]" ExpectingListEnd, spaces = spaces, item = int ExpectingInt InvalidNumber, trailing = Forbidden }spaces
spaces : Parser c x ()Just like Parser.spaces
Examples
run spaces " \n" == Ok ()lineComment
lineComment : Token x -> Parser c x ()Just like Parser.lineComment except you provide a Token describing the starting symbol.
Examples
run (lineComment (Token "--" ExpectingComment)) "-- hi" == Ok ()multiComment
multiComment : Token x -> Token x -> Nestable -> Parser c x ()Just like Parser.multiComment except with a Token for the open and close symbols.
Examples
run (multiComment (Token "/*" ExpectingComment) (Token "*/" ExpectingComment) NotNestable) "/* hi */" == Ok ()