Function token

  • Parse exactly the given string, without any regard to what comes next.

    A potential pitfall when parsing keywords is getting tricked by variables that start with a keyword, like let in letters or import in important. This is especially likely if you have a whitespace parser that can consume zero characters. So the keyword parser is defined with token and a trick to peek ahead a bit:

    const isVarChar = (char: string) => {
    return Helpers.isAlphaNum(char) || char === "_";
    };

    const checkEnding =
    (kwd: string) =>
    (isBadEnding: boolean): P.Parser<P.Unit> => {
    if (isBadEnding) {
    return P.problem("expecting the `" + kwd + "` keyword");
    } else {
    return P.commit(P.Unit);
    }
    };

    const keyword = (kwd: string): P.Parser<P.Unit> => {
    return P.succeed((v: P.Unit) => v)
    .skip(P.backtrackable(P.token(kwd)))
    .keep(P.oneOf(P.backtrackable(P.chompIf(isVarChar)), P.succeed(false)))
    .andThen(checkEnding(kwd));
    };

    This definition is specially designed so that (1) if you really see let you commit to that path and (2) if you see letters instead you can backtrack and try other options. If I had just put a backtrackable around the whole thing you would not get (1) anymore.

    Parameters

    • token: string

    Returns Simple.Parser<Symbol>