pysrc2cpg
API
Generated with
pysrc2cpg/io.joern/io.joern.pysrc2cpg/PythonAstVisitor

PythonAstVisitor

io.joern.pysrc2cpg.PythonAstVisitor
See thePythonAstVisitor companion object
class PythonAstVisitor(relFileName: String, val nodeToCode: NodeToCode, version: PythonVersion, enableFileContent: Boolean)(implicit withSchemaValidation: ValidationMode) extends AstCreatorBase, PythonAstVisitorHelpers

Attributes

Companion
object
Graph
Supertypes
class AstCreatorBase
class Object
trait Matchable
class Any

Members list

Value members

Concrete methods

def convert(astNode: iast): NewNode
def convert(mod: imod): NewNode
def convert(module: Module): NewNode
def convert(stmt: istmt): NewNode
def convert(functionDef: FunctionDef): NewNode
def convert(functionDef: AsyncFunctionDef): NewNode
def convert(classDef: ClassDef): NewNode
def convert(ret: Return): NewNode
def convert(delete: Delete): NewNode
def convert(assign: Assign): NewNode
def convert(annotatedAssign: AnnAssign): NewNode
def convert(augAssign: AugAssign): NewNode
def convert(forStmt: For): NewNode
def convert(forStmt: AsyncFor): NewNode
def convert(astWhile: While): NewNode
def convert(astIf: If): NewNode
def convert(withStmt: With): NewNode
def convert(withStmt: AsyncWith): NewNode
def convert(matchStmt: Match): NewNode
def convert(raise: Raise): NewNode
def convert(tryStmt: Try): NewNode
def convert(assert: Assert): NewNode
def convert(importStmt: Import): NewNode
def convert(importFrom: ImportFrom): NewNode
def convert(global: Global): NewNode
def convert(nonLocal: Nonlocal): NewNode
def convert(expr: Expr): NewNode
def convert(pass: Pass): NewNode
def convert(astBreak: Break): NewNode
def convert(astContinue: Continue): NewNode
def convert(raise: RaiseP2): NewNode
def convert(errorStatement: ErrorStatement): NewNode
def convert(expr: iexpr): NewNode
def convert(boolOp: BoolOp): NewNode
def convert(namedExpr: NamedExpr): NewNode
def convert(binOp: BinOp): NewNode
def convert(unaryOp: UnaryOp): NewNode
def convert(lambda: Lambda): NewNode
def convert(ifExp: IfExp): NewNode
def convert(dict: Dict): NewNode

Lowering of {x:1, y:2, **z}: { tmp = {} tmp[x] = 1 tmp[y] = 2 tmp.update(z) tmp }

Lowering of {x:1, y:2, **z}: { tmp = {} tmp[x] = 1 tmp[y] = 2 tmp.update(z) tmp }

Attributes

def convert(set: Set): NewNode
def convert(listComp: ListComp): NewNode

Lowering of [x for y in l for x in y]: { tmp = [] ( for y in l: for x in y: tmp.append(x) ) tmp }

Lowering of [x for y in l for x in y]: { tmp = [] ( for y in l: for x in y: tmp.append(x) ) tmp }

Attributes

def convert(setComp: SetComp): NewNode

Lowering of {x for y in l for x in y}: { tmp = {} ( for y in l: for x in y: tmp.add(x) ) tmp }

Lowering of {x for y in l for x in y}: { tmp = {} ( for y in l: for x in y: tmp.add(x) ) tmp }

Attributes

def convert(dictComp: DictComp): NewNode

Lowering of {k:v for y in l for k, v in y}: { tmp = {} ( for y in l: for k, v in y: tmp[k] = v ) tmp }

Lowering of {k:v for y in l for k, v in y}: { tmp = {} ( for y in l: for k, v in y: tmp[k] = v ) tmp }

Attributes

def convert(generatorExp: GeneratorExp): NewNode

Lowering of (x for y in l for x in y): { tmp = .genExp ( for y in l: for x in y: tmp.append(x) ) tmp } This lowering is not quite correct as it ignores the lazy evaluation of the generator expression. Instead it just mimics the list comprehension lowering but for now this is good enough.

Lowering of (x for y in l for x in y): { tmp = .genExp ( for y in l: for x in y: tmp.append(x) ) tmp } This lowering is not quite correct as it ignores the lazy evaluation of the generator expression. Instead it just mimics the list comprehension lowering but for now this is good enough.

Attributes

def convert(await: Await): NewNode
def convert(yieldExpr: Yield): NewNode
def convert(yieldFrom: YieldFrom): NewNode
def convert(compare: Compare): NewNode
def convert(call: Call): NewNode

TODO For now this function compromises on the correctness of the lowering in order to get some data flow tracking going.

TODO For now this function compromises on the correctness of the lowering in order to get some data flow tracking going.

  1. For constructs like x.func() we assume x to be the instance which is passed into func. This is not true since the instance method object gets the instance already bound/captured during function access. This becomes relevant for constructs like: x.func = y.func <- y.func is class method object x.func() In this case the instance passed into func is y and not x. We cannot represent this in th CPG and thus stick to the assumption that the part before the "." and the bound/captured instance will be the same. For reference see: https://docs.python.org/3/reference/datamodel.html#the-standard-type-hierarchy search for "Instance methods"

Attributes

def convert(formattedValue: FormattedValue): NewNode
def convert(joinedString: JoinedString): NewNode
def convert(constant: Constant): NewNode
def convert(attribute: Attribute): NewNode

TODO We currently ignore possible attribute access provider/interception mechanisms like getattr, getattribute and get.

TODO We currently ignore possible attribute access provider/interception mechanisms like getattr, getattribute and get.

Attributes

def convert(subscript: Subscript): NewNode
def convert(starred: Starred): NewNode
def convert(name: Name): NewNode
def convert(list: List): NewNode
def convert(tuple: Tuple): NewNode
def convert(slice: Slice): NewNode
def convert(stringExpList: StringExpList): NewNode
def convert(exceptHandler: ExceptHandler): NewNode
def convert(parameters: Arguments, startIndex: Int): Iterable[NewMethodParameterIn]
def convert(keyword: Keyword): NewNode
def convert(alias: Alias): NewNode
def convert(typeIgnore: TypeIgnore): NewNode
def convertKeywordOnlyArg(arg: Arg): NewMethodParameterIn
def convertKwArg(arg: Arg): NewMethodParameterIn
def convertNormalArg(arg: Arg, index: AutoIncIndex): NewMethodParameterIn
def convertPosOnlyArg(arg: Arg, index: AutoIncIndex): NewMethodParameterIn
def convertVarArg(arg: Arg, index: AutoIncIndex): NewMethodParameterIn
def createArguments(arguments: Arguments, lineAndColumn: LineAndColumn): (Iterable[NewNode], Iterable[(String, NewNode)])
override def createAst(): DiffGraphBuilder

Attributes

Definition Classes
AstCreatorBase
def lowerComparatorChain(lhsNode: NewNode, compOperators: Iterable[icompop], comparators: Iterable[iexpr], lineAndColumn: LineAndColumn): Iterable[NewNode]
def wrapMethodRefWithDecorators(methodRefNode: NewNode, decoratorList: Iterable[iexpr]): NewNode

Inherited methods

def absolutePath(filename: String): String

Absolute path for the given file name

Absolute path for the given file name

Attributes

Inherited from:
AstCreatorBase
protected def addAstChildNodes(parentNode: NewNode, startIndex: Int, childNodes: NewNode*): Unit

Attributes

Inherited from:
PythonAstVisitorHelpers
protected def addAstChildNodes(parentNode: NewNode, startIndex: AutoIncIndex, childNodes: NewNode*): Unit

Attributes

Inherited from:
PythonAstVisitorHelpers
protected def addAstChildNodes(parentNode: NewNode, startIndex: Int, childNodes: Iterable[NewNode]): Unit

Attributes

Inherited from:
PythonAstVisitorHelpers
protected def addAstChildNodes(parentNode: NewNode, startIndex: AutoIncIndex, childNodes: Iterable[NewNode]): Unit

Attributes

Inherited from:
PythonAstVisitorHelpers
protected def addAstChildrenAsArguments(parentNode: NewNode, startIndex: Int, childNodes: NewNode*): Unit

Attributes

Inherited from:
PythonAstVisitorHelpers
protected def addAstChildrenAsArguments(parentNode: NewNode, startIndex: AutoIncIndex, childNodes: NewNode*): Unit

Attributes

Inherited from:
PythonAstVisitorHelpers
protected def addAstChildrenAsArguments(parentNode: NewNode, startIndex: Int, childNodes: Iterable[NewNode]): Unit

Attributes

Inherited from:
PythonAstVisitorHelpers
protected def addAstChildrenAsArguments(parentNode: NewNode, startIndex: AutoIncIndex, childNodes: Iterable[NewNode]): Unit

Attributes

Inherited from:
PythonAstVisitorHelpers
def annotationAssignmentAst(assignmentValueName: String, code: String, assignmentAst: Ast): Ast

Creates an AST that represents an annotation assignment with a name for the assigned value, its overall code, and the respective assignment AST.

Creates an AST that represents an annotation assignment with a name for the assigned value, its overall code, and the respective assignment AST.

Attributes

Inherited from:
AstCreatorBase
def annotationAst(annotation: NewAnnotation, children: Seq[Ast]): Ast

Creates an AST that represents an annotation, including its content (annotation parameter assignments).

Creates an AST that represents an annotation, including its content (annotation parameter assignments).

Attributes

Inherited from:
AstCreatorBase
def blockAst(blockNode: NewBlock, statements: List[Ast]): Ast

For a given block node and statement ASTs, create an AST that represents the block. The main purpose of this method is to increase the readability of the code which creates block asts.

For a given block node and statement ASTs, create an AST that represents the block. The main purpose of this method is to increase the readability of the code which creates block asts.

Attributes

Inherited from:
AstCreatorBase
def callAst(callNode: NewCall, arguments: Seq[Ast], base: Option[Ast], receiver: Option[Ast]): Ast

Create an abstract syntax tree for a call, including CPG-specific edges required for arguments and the receiver.

Create an abstract syntax tree for a call, including CPG-specific edges required for arguments and the receiver.

Our call representation is inspired by ECMAScript, that is, in addition to arguments, a call has a base and a receiver. For languages other than Javascript, leave receiver empty for now.

Value parameters

arguments

arguments (without the base argument (instance))

base

the value to use as this in the method call.

callNode

the node that represents the entire call

receiver

the object in which the property lookup is performed

Attributes

Inherited from:
AstCreatorBase
protected def codeOf(node: NewNode): String

Attributes

Inherited from:
PythonAstVisitorHelpers
def controlStructureAst(controlStructureNode: NewControlStructure, condition: Option[Ast], children: Seq[Ast], placeConditionLast: Boolean): Ast

For a given node, condition AST and children ASTs, create an AST that represents the control structure. The main purpose of this method is to automatically assign the correct condition edges.

For a given node, condition AST and children ASTs, create an AST that represents the control structure. The main purpose of this method is to automatically assign the correct condition edges.

Attributes

Inherited from:
AstCreatorBase
protected def createAssignment(lhsNode: NewNode, rhsNode: NewNode, lineAndColumn: LineAndColumn): NewNode

Attributes

Inherited from:
PythonAstVisitorHelpers
protected def createAssignmentToIdentifier(identifierName: String, rhsNode: NewNode, lineAndColumn: LineAndColumn): NewNode

Attributes

Inherited from:
PythonAstVisitorHelpers
protected def createAugAssignment(lhsNode: NewNode, operatorCode: String, rhsNode: NewNode, operatorFullName: String, lineAndColumn: LineAndColumn): NewNode

Attributes

Inherited from:
PythonAstVisitorHelpers
protected def createBinaryOperatorCall(lhsNode: NewNode, opCodeAndFullName: () => (String, String), rhsNode: NewNode, lineAndColumn: LineAndColumn): NewCall

Attributes

Inherited from:
PythonAstVisitorHelpers
protected def createBinding(methodNode: NewMethod, typeDeclNode: NewTypeDecl): NewBinding

Attributes

Inherited from:
PythonAstVisitorHelpers
protected def createBlock(blockElements: Iterable[NewNode], lineAndColumn: LineAndColumn): NewNode

Attributes

Inherited from:
PythonAstVisitorHelpers
protected def createCall(receiverNode: NewNode, name: String, lineAndColumn: LineAndColumn, argumentNodes: Iterable[NewNode], keywordArguments: Iterable[(String, NewNode)]): NewCall

Attributes

Inherited from:
PythonAstVisitorHelpers
protected def createComprehensionLowering(tmpVariableName: String, containerInitAssignNode: NewNode, innerMostLoopNode: NewNode, comprehensions: Iterable[Comprehension], lineAndColumn: LineAndColumn): NewNode

Attributes

Inherited from:
PythonAstVisitorHelpers
protected def createFieldAccess(baseNode: NewNode, fieldName: String, lineAndColumn: LineAndColumn): NewCall

Attributes

Inherited from:
PythonAstVisitorHelpers
protected def createIdentifierNode(name: String, memOp: MemoryOperation, lineAndColumn: LineAndColumn): NewIdentifier

Attributes

Inherited from:
PythonAstVisitorHelpers
protected def createIndexAccess(baseNode: NewNode, indexNode: NewNode, lineAndColumn: LineAndColumn): NewNode

Attributes

Inherited from:
PythonAstVisitorHelpers
protected def createIndexAccessChain(rootNode: NewNode, accessChain: List[Int], lineAndColumn: LineAndColumn): NewNode

Attributes

Inherited from:
PythonAstVisitorHelpers
protected def createInstanceCall(receiverNode: NewNode, instanceNode: NewNode, name: String, lineAndColumn: LineAndColumn, argumentNodes: Iterable[NewNode], keywordArguments: Iterable[(String, NewNode)]): NewCall

Attributes

Inherited from:
PythonAstVisitorHelpers
protected def createLiteralOperatorCall(codeStart: String, codeEnd: String, opFullName: String, lineAndColumn: LineAndColumn, operands: NewNode*): NewCall

Attributes

Inherited from:
PythonAstVisitorHelpers
protected def createNAryOperatorCall(opCodeAndFullName: () => (String, String), operands: Iterable[NewNode], lineAndColumn: LineAndColumn): NewNode

Attributes

Inherited from:
PythonAstVisitorHelpers
protected def createReturn(returnExprOption: Option[NewNode], codeOption: Option[String], lineAndColumn: LineAndColumn): NewReturn

Attributes

Inherited from:
PythonAstVisitorHelpers
protected def createStarredUnpackOperatorCall(unpackOperand: NewNode, lineAndColumn: LineAndColumn): NewNode

Attributes

Inherited from:
PythonAstVisitorHelpers
protected def createStaticCall(name: String, methodFullName: String, lineAndColumn: LineAndColumn, argumentNodes: Iterable[NewNode], keywordArguments: Iterable[(String, NewNode)]): NewNode

Attributes

Inherited from:
PythonAstVisitorHelpers
protected def createTransformedImport(from: String, names: Iterable[Alias], lineAndCol: LineAndColumn): NewNode

Attributes

Inherited from:
PythonAstVisitorHelpers
protected def createTry(body: Iterable[NewNode], handlers: Iterable[NewNode], finalBlock: Iterable[NewNode], orElseBlock: Iterable[NewNode], lineAndColumn: LineAndColumn): NewNode

Attributes

Inherited from:
PythonAstVisitorHelpers
protected def createTypeRef(typeName: String, typeFullName: String, lineAndColumn: LineAndColumn): NewTypeRef

Attributes

Inherited from:
PythonAstVisitorHelpers
protected def createValueToTargetsDecomposition(targets: Iterable[iexpr], valueNode: NewNode, lineAndColumn: LineAndColumn): Iterable[NewNode]

Attributes

Inherited from:
PythonAstVisitorHelpers
protected def createXDotYCall(x: () => NewNode, y: String, xMayHaveSideEffects: Boolean, lineAndColumn: LineAndColumn, argumentNodes: Iterable[NewNode], keywordArguments: Iterable[(String, NewNode)]): NewNode

Attributes

Inherited from:
PythonAstVisitorHelpers
def doWhileAst(condition: Option[Ast], body: Seq[Ast], code: Option[String], lineNumber: Option[Integer], columnNumber: Option[Integer]): Ast

Attributes

Inherited from:
AstCreatorBase
def forAst(forNode: NewControlStructure, locals: Seq[Ast], initAsts: Seq[Ast], conditionAsts: Seq[Ast], updateAsts: Seq[Ast], bodyAsts: Seq[Ast]): Ast

Attributes

Inherited from:
AstCreatorBase
def forAst(forNode: NewControlStructure, locals: Seq[Ast], initAsts: Seq[Ast], conditionAsts: Seq[Ast], updateAsts: Seq[Ast], bodyAst: Ast): Ast

Attributes

Inherited from:
AstCreatorBase
protected def getTargetsWithAccessChains(target: iexpr): Iterable[(iexpr, List[Int])]

Attributes

Inherited from:
PythonAstVisitorHelpers
protected def getUnusedName(prefix: String): String

Attributes

Inherited from:
PythonAstVisitorHelpers
def globalNamespaceBlock(): NewNamespaceBlock

Create a global namespace block for the given filename

Create a global namespace block for the given filename

Attributes

Inherited from:
AstCreatorBase
protected def lineAndColOf(node: iattributes): LineAndColumn

Attributes

Inherited from:
PythonAstVisitorHelpers
def methodAst(method: NewMethod, parameters: Seq[Ast], body: Ast, methodReturn: NewMethodReturn, modifiers: Seq[NewModifier]): Ast

Creates an AST that represents an entire method, including its content.

Creates an AST that represents an entire method, including its content.

Attributes

Inherited from:
AstCreatorBase
def methodAstWithAnnotations(method: NewMethod, parameters: Seq[Ast], body: Ast, methodReturn: NewMethodReturn, modifiers: Seq[NewModifier], annotations: Seq[Ast]): Ast

Creates an AST that represents an entire method, including its content and with support for both method and parameter annotations.

Creates an AST that represents an entire method, including its content and with support for both method and parameter annotations.

Attributes

Inherited from:
AstCreatorBase
def methodStubAst(method: NewMethod, parameters: Seq[Ast], methodReturn: NewMethodReturn, modifiers: Seq[NewModifier]): Ast

Creates an AST that represents a method stub, containing information about the method, its parameters, and the return type.

Creates an AST that represents a method stub, containing information about the method, its parameters, and the return type.

Attributes

Inherited from:
AstCreatorBase
def nextClosureName(): String

Attributes

Returns

the next available name for a closure in this context

Inherited from:
AstCreatorBase
def returnAst(returnNode: NewReturn, arguments: Seq[Ast]): Ast

For a given return node and arguments, create an AST that represents the return instruction. The main purpose of this method is to automatically assign the correct argument indices.

For a given return node and arguments, create an AST that represents the return instruction. The main purpose of this method is to automatically assign the correct argument indices.

Attributes

Inherited from:
AstCreatorBase
def setArgumentIndices(arguments: Seq[Ast]): Unit

Attributes

Inherited from:
AstCreatorBase
def staticInitMethodAst(initAsts: List[Ast], fullName: String, signature: Option[String], returnType: String, fileName: Option[String], lineNumber: Option[Integer], columnNumber: Option[Integer]): Ast

Attributes

Inherited from:
AstCreatorBase
def tryCatchAst(tryNode: NewControlStructure, tryBodyAst: Ast, catchAsts: Seq[Ast], finallyAst: Option[Ast]): Ast

For the given try body, catch ASTs and finally AST, create a try-catch-finally AST with orders set correctly for the ossdataflow engine.

For the given try body, catch ASTs and finally AST, create a try-catch-finally AST with orders set correctly for the ossdataflow engine.

Attributes

Inherited from:
AstCreatorBase
def whileAst(condition: Option[Ast], body: Seq[Ast], code: Option[String], lineNumber: Option[Integer], columnNumber: Option[Integer]): Ast

Attributes

Inherited from:
AstCreatorBase
def withArgumentIndex[T <: ExpressionNew](node: T, argIdxOpt: Option[Int]): T

Attributes

Inherited from:
AstCreatorBase
def withArgumentName[T <: ExpressionNew](node: T, argNameOpt: Option[String]): T

Attributes

Inherited from:
AstCreatorBase
def withIndex[T, X](nodes: Array[T])(f: (T, Int) => X): Seq[X]

Attributes

Inherited from:
AstCreatorBase
def withIndex[T, X](nodes: Seq[T])(f: (T, Int) => X): Seq[X]

Attributes

Inherited from:
AstCreatorBase
def wrapMultipleInBlock(asts: Seq[Ast], lineNumber: Option[Integer]): Ast

Attributes

Inherited from:
AstCreatorBase
Generated with