Remove space and atWord tokens

`atWord` wasn't used at all and spaces should be eaten by the lexer

And added `unspecified` token
This commit is contained in:
Luis Ángel San Martín
2021-01-14 08:39:16 +01:00
parent ddb140d430
commit f09c5955d8
4 changed files with 8 additions and 23 deletions

View File

@ -9,10 +9,9 @@ public:
enum class Type {
eof,
opcode,
atWord,
word,
quotedWord,
space
undefined
};
Token(Type type, std::string lexeme = "")
@ -49,7 +48,6 @@ private:
char get();
Token single(Token::Type type);
Token space();
Token word();
Token quotedWord();