VapourSynth-llvmexpr
Loading...
Searching...
No Matches
infix2postfix::preprocessor_detail::PreprocessorTokenizer Class Reference

Public Member Functions

 PreprocessorTokenizer (std::string_view source)
std::vector< Tokentokenize ()

Detailed Description

Definition at line 115 of file Preprocessor.cpp.

Constructor & Destructor Documentation

◆ PreprocessorTokenizer()

infix2postfix::preprocessor_detail::PreprocessorTokenizer::PreprocessorTokenizer ( std::string_view source)
inlineexplicit

Definition at line 117 of file Preprocessor.cpp.

117: source(source) {}

Member Function Documentation

◆ tokenize()

std::vector< Token > infix2postfix::preprocessor_detail::PreprocessorTokenizer::tokenize ( )
inline

Definition at line 119 of file Preprocessor.cpp.

119 {
120 std::vector<Token> tokens;
121 while (!eof()) {
122 Token tok = nextToken();
123 tokens.push_back(tok);
124 if (tok.type == TokenType::EndOfFile) {
125 break;
126 }
127 }
128 return tokens;
129 }
preprocessor_detail::Token Token
TokenType type

References infix2postfix::preprocessor_detail::Token::type.

Referenced by infix2postfix::Preprocessor::Impl::addPredefinedMacro(), and infix2postfix::Preprocessor::Impl::process().


The documentation for this class was generated from the following file: