PREFIX_FlexScannerBase Class Reference

This class implements basic functionality for flex-based scanners. More...

#include <scannerbase.h>

Collaboration diagram for PREFIX_FlexScannerBase:

[legend]List of all members.

Public Member Functions
ssize_t	_yy_input (char *buf, size_t max_size)
	Called by the lexer to get more input.
	PREFIX_FlexScannerBase ()
virtual	~PREFIX_FlexScannerBase ()
Protected Member Functions
void	_pi_tok (size_t l)
	To be called after having read a token which does not contain a newline.
void	_pi_tab ()
	To be called when "eating" a tab character.
void	_pi_line ()
	To be called after having read a newline.
void	_pi_char (char c)
	To be called after having read the passed character.
void	_pi_str (const char *str, size_t len)
	To be called after having read the passed string which possibly contains special chars (newline, tab, return).
void	_pi_start ()
	Make current pos the beginning of a token. Call after whitespace and comments.
void	_pi_eof ()
	Called when EOF (on current input) is "read".
SCLocation	_MakeCurrLoc () const
	Get current (start) location as SCLocation.
SCLocation	_MakeCurrEndLoc () const
	Get current start location as SCLocation.
SCLocationRange	_MakeCurrLocRange () const
	Get current location range as SCLocationRange.
virtual ssize_t	ReadInput (char *buf, size_t max_size)
	Read more input to be fed into the lexer.
virtual TokenEntry *	AllocTokenEntry ()
	Allocate a new TokenEntry.
void	_RawYYLex (TokenEntry *dest)
	Raw lexing routine as provided by flex(1).
void	_reset ()
	PREFIX_FlexScannerBase (const PREFIX_FlexScannerBase &)
	Do not use.
void	operator= (const PREFIX_FlexScannerBase &)
	Do not use.
Protected Attributes
uint32	magic
	For simple detection of memory corruption or evil pointer mess.
void *	scanner
	The scanner used for scanning; only one; NULL if inactive.
TokenEntry *	lex_token
int	file_depth
	Current include hierarchy depth:.
int	n_errors
	Accumulate number of errors.
XYPos	p0
	Current token: start position; while parsing: previous position.
XYPos	p1
	Current token: end position; while parsing: current position.
SourcePositionArchive *	pos_arch
	Used to generate source position objects; see there.
bool	pos_arch_allocated
bool	loc_use_lpos
int	tab_width
	(Config:) Tab with in number of characters for correct lpos.
Static Protected Attributes
const uint32	MyMagic = 0xdeadbeefU
	What the magic value should be:.
Friends
int	PREFIX_lex (void *)

---

Detailed Description

This class implements basic functionality for flex-based scanners.

Author:

Wolfgang Wieser (wwieser@gmx.de)

Please read the docu for scannerbase.h on how to use it. Really! It's not easy to use.

In order to attach some information to the token (such as the read integer value or the YYSTYPE union in bison-based parsers, use a type derived from TokenEntry and supply a suitable AllocTokenEntry() function and implement the TokenEntry::MayBeCleared() function as well as TokenEntry::clear()!

Class is (of couse) not C++-safe.

Definition at line 117 of file scannerbase.h.

---

Constructor & Destructor Documentation

PREFIX_FlexScannerBase::PREFIX_FlexScannerBase (  const PREFIX_FlexScannerBase &   )  [protected]

Do not use.

PREFIX_FlexScannerBase::PREFIX_FlexScannerBase (   )

virtual PREFIX_FlexScannerBase::~PREFIX_FlexScannerBase (   )  [virtual]

---

Member Function Documentation

SCLocation PREFIX_FlexScannerBase::_MakeCurrEndLoc (   )  const [inline, protected]

Get current start location as SCLocation. Definition at line 241 of file scannerbase.h. References SourcePositionArchive::GetPos(), PREFIX_FlexScannerBase::XYPos::line, loc_use_lpos, PREFIX_FlexScannerBase::XYPos::lpos, p1, and pos_arch. Referenced by _MakeCurrLocRange().

SCLocation PREFIX_FlexScannerBase::_MakeCurrLoc (   )  const [inline, protected]

Get current (start) location as SCLocation. Definition at line 237 of file scannerbase.h. References SourcePositionArchive::GetPos(), PREFIX_FlexScannerBase::XYPos::line, loc_use_lpos, PREFIX_FlexScannerBase::XYPos::lpos, p0, and pos_arch. Referenced by _MakeCurrLocRange().

SCLocationRange PREFIX_FlexScannerBase::_MakeCurrLocRange (   )  const [inline, protected]

Get current location range as SCLocationRange. Definition at line 245 of file scannerbase.h. References _MakeCurrEndLoc(), and _MakeCurrLoc().

void PREFIX_FlexScannerBase::_pi_char (  char  c  )  [protected]

To be called after having read the passed character.

void PREFIX_FlexScannerBase::_pi_eof (   )  [protected]

Called when EOF (on current input) is "read".

void PREFIX_FlexScannerBase::_pi_line (   )  [inline, protected]

To be called after having read a newline. Definition at line 215 of file scannerbase.h. References PREFIX_FlexScannerBase::XYPos::line, PREFIX_FlexScannerBase::XYPos::lpos, and p1.

void PREFIX_FlexScannerBase::_pi_start (   )  [inline, protected]

Make current pos the beginning of a token. Call after whitespace and comments. Definition at line 224 of file scannerbase.h. References p0, and p1.

void PREFIX_FlexScannerBase::_pi_str (  const char *  str, size_t  len )  [protected]

To be called after having read the passed string which possibly contains special chars (newline, tab, return).

void PREFIX_FlexScannerBase::_pi_tab (   )  [inline, protected]

To be called when "eating" a tab character. Definition at line 212 of file scannerbase.h. References PREFIX_FlexScannerBase::XYPos::lpos, p1, and tab_width.

void PREFIX_FlexScannerBase::_pi_tok (  size_t  l  )  [inline, protected]

To be called after having read a token which does not contain a newline. Definition at line 209 of file scannerbase.h. References PREFIX_FlexScannerBase::XYPos::lpos, and p1.

void PREFIX_FlexScannerBase::_RawYYLex (  TokenEntry *  dest  )  [protected]

Raw lexing routine as provided by flex(1). Only some additional code is added for position handling. Token is stored in passed token entry struct.

void PREFIX_FlexScannerBase::_reset (   )  [protected]

Reset scanner; like after construction but not changing config/tuning.

ssize_t PREFIX_FlexScannerBase::_yy_input (  char *  buf, size_t  max_size )  [inline]

Called by the lexer to get more input. Definition at line 232 of file scannerbase.h. References ReadInput(), and ssize_t.

virtual TokenEntry* PREFIX_FlexScannerBase::AllocTokenEntry (   )  [protected, virtual]

Allocate a new TokenEntry. This must be overridden if you use your own derived version.

void PREFIX_FlexScannerBase::operator= (  const PREFIX_FlexScannerBase &   )  [protected]

Do not use.

virtual ssize_t PREFIX_FlexScannerBase::ReadInput (  char *  buf, size_t  max_size )  [protected, virtual]

Read more input to be fed into the lexer. See also _yy_input(). There is a suitable default implementation for the streams and normally no need to re-implement this. Returns: Number of bytes actually read; use 0 for "error" because error has to be handeled externally. Referenced by _yy_input().

---

Friends And Related Function Documentation

int PREFIX_lex (  void *   )  [friend]

---

Member Data Documentation

int PREFIX_FlexScannerBase::file_depth [protected]

Current include hierarchy depth:. Definition at line 181 of file scannerbase.h.

TokenEntry* PREFIX_FlexScannerBase::lex_token [protected]

Exclusively used to pass current TokenEntry() to the flex-based lex() routine (yylex); may be accessed to get access to token information (e.g. put parsed integer value of integer token there). Definition at line 178 of file scannerbase.h.

bool PREFIX_FlexScannerBase::loc_use_lpos [protected]

(Config:) Use lpos (position in line) in postition objects? (Using "no" here will decrease memory footprint.) Definition at line 200 of file scannerbase.h. Referenced by _MakeCurrEndLoc(), and _MakeCurrLoc().

uint32 PREFIX_FlexScannerBase::magic [protected]

For simple detection of memory corruption or evil pointer mess. Definition at line 166 of file scannerbase.h.

const uint32 PREFIX_FlexScannerBase::MyMagic = 0xdeadbeefU [static, protected]

What the magic value should be:. Definition at line 168 of file scannerbase.h.

int PREFIX_FlexScannerBase::n_errors [protected]

Accumulate number of errors. Definition at line 184 of file scannerbase.h.

XYPos PREFIX_FlexScannerBase::p0 [protected]

Current token: start position; while parsing: previous position. Definition at line 187 of file scannerbase.h. Referenced by _MakeCurrLoc(), and _pi_start().

XYPos PREFIX_FlexScannerBase::p1 [protected]

Current token: end position; while parsing: current position. Definition at line 189 of file scannerbase.h. Referenced by _MakeCurrEndLoc(), _pi_line(), _pi_start(), _pi_tab(), and _pi_tok().

SourcePositionArchive* PREFIX_FlexScannerBase::pos_arch [protected]

Used to generate source position objects; see there. Definition at line 192 of file scannerbase.h. Referenced by _MakeCurrEndLoc(), and _MakeCurrLoc().

bool PREFIX_FlexScannerBase::pos_arch_allocated [protected]

If _we_ allocated the pos archive or if it was passed to us via some input setting function. Definition at line 196 of file scannerbase.h.

void* PREFIX_FlexScannerBase::scanner [protected]

The scanner used for scanning; only one; NULL if inactive. Definition at line 173 of file scannerbase.h.

int PREFIX_FlexScannerBase::tab_width [protected]

(Config:) Tab with in number of characters for correct lpos. Definition at line 203 of file scannerbase.h. Referenced by _pi_tab().

---

The documentation for this class was generated from the following file:

• /ray/src/lib/lex/scannerbase.h

---