zenilib  0.5.3.0
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Properties Friends Macros Groups Pages
pp::Tokenizer Class Reference

#include <Tokenizer.h>

Inherits pp::Lexer.

Classes

struct  Context
 

Public Member Functions

 Tokenizer (Diagnostics *diagnostics)
 
 ~Tokenizer ()
 
bool init (size_t count, const char *const string[], const int length[])
 
virtual void lex (Token *token)
 
void setFileNumber (int file)
 
void setLineNumber (int line)
 
- Public Member Functions inherited from pp::Lexer
virtual ~Lexer ()
 

Static Public Attributes

static const std::size_t kMaxTokenLength = 256
 

Detailed Description

Definition at line 19 of file Tokenizer.h.

Constructor & Destructor Documentation

pp::Tokenizer::Tokenizer ( Diagnostics diagnostics)

Definition at line 2297 of file Tokenizer.cpp.

References pp::Tokenizer::Context::diagnostics.

pp::Tokenizer::~Tokenizer ( )

Definition at line 2302 of file Tokenizer.cpp.

Member Function Documentation

bool pp::Tokenizer::init ( size_t  count,
const char *const  string[],
const int  length[] 
)

Definition at line 2307 of file Tokenizer.cpp.

References pp::Tokenizer::Context::input.

void pp::Tokenizer::setFileNumber ( int  file)

Definition at line 2315 of file Tokenizer.cpp.

References ppset_column().

void pp::Tokenizer::setLineNumber ( int  line)

Definition at line 2322 of file Tokenizer.cpp.

References ppset_lineno().

Member Data Documentation

const size_t pp::Tokenizer::kMaxTokenLength = 256
static

Definition at line 35 of file Tokenizer.h.

Referenced by lex().


The documentation for this class was generated from the following files: