What is token , lexeme and pattern?
Token
A token is a pain consisting of a token name and an optional attribute value. The token name is an abstract symbol representing a kind of lexical unit.A particular denoting an identifier.The token names are the input symbol that are the parser process.
Typically tokens are ,
1) Identifiers
2) Keywords
3) Operator
4) special symbol 5)constant
Pattern
A pattern is a description of the form that the lexeme’s of a token may take. In the case of keyword as a token , the pattern is just the sequence of characters that form the keyword. For identifiers and some other tokens. The pattern is more complex structure.That is matched by many string.
Lexeme
A lexeme is a sequence of character in the source program that matches pattern for a token and identified by the lexical analyzer as an instance of that token.
Example : while( y > = t ) = y -3
Will be represented by the set of pairs.
Lexeme
|
token
|
while
|
while
|
(
|
lparen
|
y
|
identifier
|
>=
|
Comparison
|
t
|
identifier
|
)
|
Rparen
|
y
|
identifier
|
=
|
Assignment
|
y
|
identifier
|
-
|
Arithmetic
|
3
|
integer
|
;
|
Finish of a statement
|
>> What is application software and system software
>> incremental model with details.
>> Software Engineering | Classification of Software Requirements
>>What is operating system
>>What is traffic monitoring system
>> what is computer virus and names of virus.
>>What is embedded control system
>> What is compiler .
>> What is linker.
>> What is Interpreter
>> Modern Principles Of Software Development
>> Types of Software Testing
>> Software Testing | Basics
>> Software Engineering | Debugging Approaches
>> Software Engineering | Classification of Software.
>> Deference between software engineering and computer science.
>> What is cost of software?.
>> which have been explicitly designed to support process iteration ?
>> Software engineering some question.
>> Deference between software engineering and computer science.
>> What is cost of software?.
>> which have been explicitly designed to support process iteration ?
>> Software engineering some question.
0 Comments