Levels of Source Code

• Lexical characters, lines, Lexer
• Documentary Comments, whitespace
• Syntactic statements, part of AST, Parser
• Semantic Behavior, actual execution, execution of intermediate representation

Regex Examples

• Search for a keyword
• Search for a comment line
• Search for a statement
• Search for an underflow error

Regex & Levels

AST Examples

• Search for a keyword?
• Search for a comment line?
• Search for a statement?
• Search for an underflow error?

AST & Levels

So Far

Document View

Preserve all text in the original file, including whitespace and newlines, and use markup to add syntactic and AST information

• Standard markup languages, e.g., XML, have mature tools for search, query, and transformation
• Preserving all original text means the original file can be extracted from the format
• srcML

Document View Examples

• Search for a keyword?
• Search for a comment line?
• Search for a statement?
• Search for an underflow error?

Comparison