Lexer

The Lexer class provides several methods that return objects of class Understand::Lexeme. A lexeme is the smallest unit of language in a source file that has a meaning. For example, a comment, a string, a keyword, or an operator.

The $ent->lexer() method returns a lexical stream as an object of the Understand::Lexer class. A lexical stream contains all the lexemes for the specified entity. A lexical stream may be generated for a file entity only if the original file exists and is unchanged from the last database reparse.

Individual lexemes (tokens) may be accessed from a lexer object, either sequentially with $lexer->first() and $lexeme->next(), or as an array with $lexer->lexemes().

Each lexeme object indicates its token kind ($lexeme->token()), its text ($lexeme->text()), its referenced entity ($lexeme->ent()) and its line and column position.

# lookup file entity, create lexer 
$file = $db->lookup("test.cpp"); 
$lexer = $file->lexer(); 
# regenerate source file from lexemes 
# add a '@' after each entity name 
foreach $lexeme ($lexer->lexemes()) { 
   print $lexeme->text(); 
   if ($lexeme->ent()) { 
      print "@"; 
   } 
} 

See Understand::Lexeme class and Understand::Lexer class for detailed information.


Scientific Toolworks, Inc.
http://www.scitools.com