Parser.Advanced

Core

Context-aware parsers with custom problem types.

Types

Parser

type Parser context problem value = Parser (State context -> PStep context problem value

An advanced Parser gives two ways to improve your error messages:

- problem - Instead of all errors being a String, you can create a custom type like type Problem = BadIndent | BadKeyword String and track problems much more precisely. - context - Error messages can be further improved when precise problems are paired with information about where you ran into trouble. By tracking the context, instead of saying “I found a bad keyword” you can say “I found a bad keyword when parsing a list” and give folks a better idea of what the parser thinks it is doing.

I recommend starting with the simpler Parser module though, and when you feel comfortable and want better error messages, you can create a type alias like this:

Examples

import Parser.Advanced

type alias MyParser a =
  Parser.Advanced.Parser Context Problem a

type Context = Definition String | List | Record

type Problem = BadIndent | BadKeyword String

All of the functions from Parser should exist in Parser.Advanced in some form, allowing you to switch over pretty easily.

DeadEnd

type alias DeadEnd context problem =
{ row : Int
, col : Int
, problem : problem
, contextStack : List { row : Int
, col : Int
, context : context
}
}

Say you are parsing a function named viewHealthData that contains a list. You might get a DeadEnd like this:

Examples

{ row = 18
, col = 22
, problem = UnexpectedComma
, contextStack =
    [ { row = 14
      , col = 1
      , context = Definition "viewHealthData"
      }
    , { row = 15
      , col = 4
      , context = List
      }
    ]
}

We have a ton of information here! So in the error message, we can say that “I ran into an issue when parsing a list in the definition of viewHealthData. It looks like there is an extra comma.” Or maybe something even better!

Furthermore, many parsers just put a mark where the problem manifested. By tracking the row and col of the context, we can show a much larger region as a way of indicating “I thought I was parsing this thing that starts over here.” Otherwise you can get very confusing error messages on a missing ] or } or ) because “I need more indentation” on something unrelated.

**Note:** Rows and columns are counted like a text editor. The beginning is row=1 and col=1. The col increments as characters are chomped. When a \n is chomped, row is incremented and col starts over again at 1.

Step

type Step state a
= Loop state
| Done a

Just like Parser.Step

Examples

Loop state  -- keep going
Done value  -- finish

Token

type Token x = Token String x

With the simpler Parser module, you could just say symbol "," and parse all the commas you wanted. But now that we have a custom type for our problems, we actually have to specify that as well. So anywhere you just used a String in the simpler module, you now use a Token Problem in the advanced module:

Examples

type Problem
  = ExpectingComma
  | ExpectingListEnd

comma : Token Problem
comma =
  Token "," ExpectingComma

listEnd : Token Problem
listEnd =
  Token "]" ExpectingListEnd

You can be creative with your custom type. Maybe you want a lot of detail. Maybe you want looser categories. It is a custom type. Do what makes sense for you!

Trailing

type Trailing
= Forbidden
| Optional
| Mandatory

What’s the deal with trailing commas? Are they Forbidden? Are they Optional? Are they Mandatory? Welcome to shapes club!

Examples

{ trailing = Optional }  -- allow [1,2] and [1,2,]

Nestable

type Nestable
= NotNestable
| Nestable

Works just like Parser.Nestable to help distinguish between unnestable /* */ comments like in JS and nestable {- -} comments like {- -}.

Examples

multiComment open close Nestable

Values

|=

infix left 5 (|=) = keeper

Keep the value from a parser in a pipeline.

Examples

run (succeed identity |= int ExpectingInt InvalidNumber) "42" == Ok 42

|.

infix left 6 (|.) = ignorer

Skip the value from a parser in a pipeline.

Examples

run (succeed () |. symbol (Token "," ExpectingComma)) "," == Ok ()

run

run : Parser c x a -> String -> Result (List (DeadEnd c x)) a

This works just like Parser.run. The only difference is that when it fails, it has much more precise information for each dead end.

Examples

run (succeed 42) "" == Ok 42

succeed

succeed : a -> Parser c x a

Just like Parser.succeed

Examples

run (succeed 90210) "mississippi" == Ok 90210

problem

problem : x -> Parser c x a

Just like Parser.problem except you provide a custom type for your problem.

Examples

run (problem BadZip) "" == Err ...

map

map : (a -> b) -> Parser c x a -> Parser c x b

Just like Parser.map

Examples

run (map String.fromInt (int ExpectingInt InvalidNumber)) "123" == Ok "123"

andThen

andThen : (a -> Parser c x b) -> Parser c x a -> Parser c x b

Just like Parser.andThen

Examples

run (int ExpectingInt InvalidNumber |> andThen (\n -> succeed (n + 1))) "1" == Ok 2

lazy

lazy : (() -> Parser c x a) -> Parser c x a

Just like Parser.lazy

Examples

lazy (\_ -> succeed ())

oneOf

oneOf : List (Parser c x a) -> Parser c x a

Just like Parser.oneOf

Examples

run (oneOf [ succeed 1, succeed 2 ]) "" == Ok 1

loop

loop : state -> (state -> Parser c x (Step state a)) -> Parser c x a

Just like Parser.loop

Examples

loop [] (\_ -> succeed (Done []))

backtrackable

backtrackable : Parser c x a -> Parser c x a

Examples

backtrackable (token (Token "let" ExpectingLet))

commit

commit : a -> Parser c x a

Just like Parser.commit

Examples

run (commit ()) "" == Ok ()

symbol

symbol : Token x -> Parser c x ()

Just like Parser.symbol except you provide a Token to clearly indicate your custom type of problems:

Examples

comma : Parser Context Problem ()
comma =
  symbol (Token "," ExpectingComma)

keyword

keyword : Token x -> Parser c x ()

Just like Parser.keyword except you provide a Token to clearly indicate your custom type of problems:

Examples

let_ : Parser Context Problem ()
let_ =
  symbol (Token "let" ExpectingLet)

Note that this would fail to chomp letter because of the subsequent characters. Use token if you do not want that last letter check.

token

token : Token x -> Parser c x ()

Just like Parser.token except you provide a Token specifying your custom type of problems.

Examples

run (token (Token "," ExpectingComma)) "," == Ok ()

int

int : x -> x -> Parser c x Int

Just like Parser.int where you have to handle negation yourself. The only difference is that you provide a two potential problems:

Examples

int : x -> x -> Parser c x Int
int expecting invalid =
  number
    { int = Ok identity
    , hex = Err invalid
    , octal = Err invalid
    , binary = Err invalid
    , float = Err invalid
    , invalid = invalid
    , expecting = expecting
    }

You can use problems like ExpectingInt and InvalidNumber.

float

float : x -> x -> Parser c x Float

Just like Parser.float where you have to handle negation yourself. The only difference is that you provide a two potential problems:

Examples

float : x -> x -> Parser c x Float
float expecting invalid =
  number
    { int = Ok toFloat
    , hex = Err invalid
    , octal = Err invalid
    , binary = Err invalid
    , float = Ok identity
    , invalid = invalid
    , expecting = expecting
    }

You can use problems like ExpectingFloat and InvalidNumber.

number

number : { int : Result x (Int -> a) , hex : Result x (Int -> a) , octal : Result x (Int -> a) , binary : Result x (Int -> a) , float : Result x (Float -> a) , invalid : x , expecting : x } -> Parser c x a

Just like Parser.number where you have to handle negation yourself. The only difference is that you provide all the potential problems.

Examples

run (int ExpectingInt InvalidNumber) "123" == Ok 123

end

end : x -> Parser c x ()

Just like Parser.end except you provide the problem that arises when the parser is not at the end of the input.

Examples

run (succeed () |. end ExpectingEnd) "" == Ok ()

getChompedString

getChompedString : Parser c x a -> Parser c x String

Examples

run (getChompedString (chompWhile Char.isDigit)) "123" == Ok "123"

mapChompedString

mapChompedString : (String -> a -> b) -> Parser c x a -> Parser c x b

Examples

mapChompedString (\str n -> (str, n)) (int ExpectingInt InvalidNumber)

chompIf

chompIf : (Char -> Bool) -> x -> Parser c x ()

Just like Parser.chompIf except you provide a problem in case a character cannot be chomped.

Examples

run (chompIf Char.isUpper UnexpectedChar) "T" == Ok ()

chompWhile

chompWhile : (Char -> Bool) -> Parser c x ()

Examples

run (chompWhile (\c -> c == ' ')) "  x" == Ok ()

chompUntil

chompUntil : Token x -> Parser c x ()

Just like Parser.chompUntil except you provide a Token in case you chomp all the way to the end of the input without finding what you need.

Examples

run (chompUntil (Token "*/" ExpectingEndComment)) "hi */" == Ok ()

chompUntilEndOr

chompUntilEndOr : String -> Parser c x ()

Examples

run (chompUntilEndOr "\n") "hi" == Ok ()

inContext

inContext : context -> Parser context x a -> Parser context x a

This is how you mark that you are in a certain context. For example, here is a rough outline of some code that uses inContext to mark when you are parsing a specific definition:

Examples

import Char
import Parser.Advanced exposing (..)
import Set

type Context
  = Definition String
  | List

definition : Parser Context Problem Expr
definition =
  functionName
    |> andThen definitionBody

definitionBody : String -> Parser Context Problem Expr
definitionBody name =
  inContext (Definition name) <|
    succeed (Function name)
      |= arguments
      |. symbol (Token "=" ExpectingEquals)
      |= expression

functionName : Parser c Problem String
functionName =
  variable
    { start = Char.isLower
    , inner = Char.isAlphaNum
    , reserved = Set.fromList ["let","in"]
    , expecting = ExpectingFunctionName
    }

First we parse the function name, and then we parse the rest of the definition. Importantly, we call inContext so that any dead end that occurs in definitionBody will get this extra context information. That way you can say things like, “I was expecting an equals sign in the view definition.” Context!

getIndent

getIndent : Parser c x Int

Examples

run getIndent "" == Ok 0

withIndent

withIndent : Int -> Parser c x a -> Parser c x a

Examples

run (withIndent 4 getIndent) "" == Ok 4

getPosition

getPosition : Parser c x (Int, Int)

Examples

run getPosition "" == Ok (1, 1)

getRow

getRow : Parser c x Int

Just like Parser.getRow

Examples

run getRow "" == Ok 1

getCol

getCol : Parser c x Int

Just like Parser.getCol

Examples

run getCol "" == Ok 1

getOffset

getOffset : Parser c x Int

Examples

run getOffset "" == Ok 0

getSource

getSource : Parser c x String

Examples

run getSource "hi" == Ok "hi"

variable

variable : { start : Char -> Bool , inner : Char -> Bool , reserved : Set.Set String , expecting : x } -> Parser c x String

Just like Parser.variable except you specify the problem yourself.

Examples

variable { start = Char.isLower, inner = Char.isAlphaNum, reserved = Set.empty, expecting = ExpectingVar }

sequence

sequence : { start : Token x , separator : Token x , end : Token x , spaces : Parser c x () , item : Parser c x a , trailing : Trailing } -> Parser c x (List a)

Just like Parser.sequence except with a Token for the start, separator, and end. That way you can specify your custom type of problem for when something is not found.

Examples

sequence { start = Token "[" ExpectingListStart, separator = Token "," ExpectingComma, end = Token "]" ExpectingListEnd, spaces = spaces, item = int ExpectingInt InvalidNumber, trailing = Forbidden }

spaces

spaces : Parser c x ()

Just like Parser.spaces

Examples

run spaces "  \n" == Ok ()

lineComment

lineComment : Token x -> Parser c x ()

Just like Parser.lineComment except you provide a Token describing the starting symbol.

Examples

run (lineComment (Token "--" ExpectingComment)) "-- hi" == Ok ()

multiComment

multiComment : Token x -> Token x -> Nestable -> Parser c x ()

Just like Parser.multiComment except with a Token for the open and close symbols.

Examples

run (multiComment (Token "/*" ExpectingComment) (Token "*/" ExpectingComment) NotNestable) "/* hi */" == Ok ()