词法分析

一种改进的词法分析程序及其类C语言实现

Improved Lexical Analysis and Its Implementation in C-like Language

执行词法分析的程序称为词法分析程序或扫描程序（Scanner）。

Perform lexical analysis program called lexical analyzer or scanner ( Scanner ) .

Internet实时浏览翻译系统中的词法分析模型

Morphology Analysis Model for Internet Real-time Browsing Translation System

探讨了SIP消息处理过程，重点介绍了词法分析以及用于数据加密的IDEA算法的设计实现方式。

The process of message processing is researched .

软PLC的编译程序包括词法分析、语法分析、语义分析和目标代码生成四个阶段。

The compiled system includes the phrase analysis , syntax analysis , language analysis and generate target code .

Fortran语言语法分析程序和词法分析程序自动生成的尝试

The Experiment on The Automatic Generation of Parser of Fortran Language

在词法分析和解析方面已经没有任何已知bug了！

No known lexing / parsing bugs !

用LEX构造数控编译器词法分析程序的研究

Research on the Construct of Lexical Analyzer of NC Compiler by LEX

第一期的工作是完成ATLAS语言的词法分析和语法分析；

The work in the first phase is to complete the lexical analysis and syntax analysis of ATLAS .

数控代码诊断模块从词法分析、语法分析和错误处理三个方面实现了对NC代码程序的诊断纠错。串口通讯模块实现了手持终端与上下位机的通信功能。

Then , the NC code is processed by the lexical analysis , syntax analysis and error is displayed on screen in the NC code diagnostic module .

介绍了一种以编译技术，主要是以词法分析和语法分析为基础的开发程序可视化工具的方法，并给出了C语言程序执行可视化工具的实例，以说明该方法的实用性。

This paper introduces a method of developing visualizing tool based on compiling technology mainly on lexical analysis and parsing , and also describes an implementation of program visualization tool of C language program .

IDL编译器的实现&词法分析部分

Implementation of IDL compiler & lexical analysis

N-gram技术在中文词法分析中的应用研究

N-gram Technology Application Study in Computer Processing of Chinese Language

类似于P：：RD，那些语法使用简单的规则来进行词法分析，然后使用更复杂的规则来解析经过词法分析后的输入内容。

Like P : : RD , those grammars use simple rules to do lexing , and then use more complex rules to parse the lexed input .

将编译技术引入NC代码分析处理过程中，把NC代码编译过程分为词法分析、语法分析和代码转换三个阶段。

Having introduced the compile technology into the process of NC code analysis , the process of NC code compile is divided into three stages : Lexical Analysis , Syntactic Analysis and NC Code Conversion .

Erlang还允许我们编程访问经过词法分析（tokenized）、语法分析之后、抽象形式的表达式。

Erlang also gives us programmatic access to the tokenized , parsed and abstract forms of an expression .

作为一种专用的编译工具，LEX的源程序简单，编译后生成的词法分析程序效率高。

Being a special compile tool , the source program of LEX is simple . The compiled lexical analyzer constructed by LEX has high efficiency .

本文完成的工作主要有：首先，对国内外C程序安全规则集及检查工具进行了分析比较；分析了GCC编译器的编译流程及其前端词法分析、语法语义分析的主要数据结构和工作过程。

The main work of the paper can be concluded as follows : Firstly , the C programs safety rules and corresponding checking tools are analyzed and summarized . The structure of GCC compiler is described .

将浏览器的HTML解析过程实现为分词和词法分析两部分，实现了解析模块的分词算法和词法分析算法，分词算法具有简单的容错功能。

This dissertation separates the process of parsing into two process : HTML segmentation and wording analysis . The HTML segmentation algorithm , which has simple fault tolerance ability , and parse module are of realization .

根据XML特点研制开发的适用于XML网络管理一致性测试的XML解析器可对XML文档进行词法分析、语法分析和语义分析，并为测试系统提供数据信息。

According to XML characteristic , XML parser suitable for XML network management conformance test is developed . This parser can carry on morphological analysis , syntactic analysis and semantic analysis to XML file , and can offer the data information for test system .

词法分析的主要任务是：从左到右逐个字符地扫描源程序，产生一个个单词（Token），同时检查源程序中的词法错误。

Lexical analysis of the main tasks are : to scan source code character by character from left to right , resulting in one word ( Token ), also check the source of lexical errors .

最后举例说明Z到C自动求精的过程，在这里对Z词法分析后输出各类单词及其内码值，对Z语法分析后如有错给出相应的错误处理，最终通过求精规则转化成C++程序代码。

Finally design Z to C compiler , output each kind of word and its code value in Z lexical analysis , leave corresponding wrong processing after the Z grammar analysis , finally through refinement rule to transform the C + + procedure code .

在下节（“词法分析”）中使用的BNF都是词法定义，以后的章节是句法定义。

All uses of BNF in the next chapter ( 'Lexical Analysis ' ) are lexical definitions ; uses in subsequent chapters are syntactic definitions .

从定义来看，词法分析是将输入内容分解为有意义的单词（也叫做词法分析记号（lexingtokens）或词位（lexemes））的动作。

Lexing , to give one definition , is the act of breaking up input into meaningful words ( also called lexing tokens or lexemes ) .

词法分析模块实现了完成标识符识别工作的确定有限自动机，把ESQL源文件作为输入流，识别关键字、标识符、C变量等，并将识别出来的标识符输出给语法分析模块。

Lexical Analysis Module implemented the deterministic finite automation , which can recognize the identifiers . It inputs the ESQL source file and recognizes the keywords , identifiers and C variables , output the recognized identifiers to Syntax Analysis Module .

词法分析和语法分析部分又分为输入命令数据流以及Fortran源程序数据流的词法分析和语法分析，后一部分是本文分析的重点，它以词法分析程序生成器grammarlexer。

The accidence and syntax analysis parts consist of the analysis of data stream of the input directive and the Fortran program . The latter is the emphasis of this article . Its hard core is the accidence analysis builder grammar_lexer .

介绍了一种数控NC代码解释器的新型开发方法，这种方法以微软推出的GRETA正则表达式类库为匹配和分析工具，构造了解释器的语法和词法分析模块。

It introduces a new method for the development of NC code interpreter which is suitable for GRETA formula expressions library developed by Microsoft , designs the model to analyze the words and sentence structure of the interpreter .

中文分词采用的中国科学院计算技术研究所汉语词法分析系统ICTCLAS的开源代码。

The code of Chinese word segmentation is adopted open source codes of ICTCLAS of Institute of Computing Technology , Chinese Academy of Science .

结合编译原理，详细阐述了词法分析和语法分析的过程，分析了Flex和Bison两个编译器工具，为整个软件提供了理论基础。（2）采用分层架构作为总体设计。

With the compiler theory , illustrate the parsing principles and Flex and Bison which are two compiler tools . They provide a theoretical basis for the entire software . ( 2 ) Use layered architecture for the overall design .

本文用正规表达式描述SIPROM语言的单词符号，并利用有穷自动机原理完成词法分析任务。

This paper describes SIPROM words with the regular expression and youqong automatic machine principle complete the phrase analysis .