Skip to content

[SwiftLexicalScopes][GSoC] Add simple scope queries and initial name lookup API structure #2696

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 13 commits into from
Jul 5, 2024
Merged
Show file tree
Hide file tree
Changes from 7 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions Package.swift
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ let package = Package(
.library(name: "SwiftSyntaxMacros", targets: ["SwiftSyntaxMacros"]),
.library(name: "SwiftSyntaxMacroExpansion", targets: ["SwiftSyntaxMacroExpansion"]),
.library(name: "SwiftSyntaxMacrosTestSupport", targets: ["SwiftSyntaxMacrosTestSupport"]),
.library(name: "SwiftLexicalScopes", targets: ["SwiftLexicalScopes"]),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should talk about the name here a little more. I think I suggested SwiftLexicalScopes initially, but given that the "scope" part is actually getting somewhat downplayed... should we call this SwiftLexicalLookup? All of these operations are, essentially, lexical lookups for various kinds of things---lookups for declarations, labeled statements, where a fallthrough ends up---and most will have "lookup" in their names.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I also like the idea. I changed it to SwiftLexicalLookup. It’s more broad and not limiting the library to just scope lookup could pave the way for including a broader range of APIs, like the break and continue lookup you’ve mentioned yesterday that is not necessarily part of ASTScope as far as I remember. Maybe it would be a good idea to use this library in the long run to consolidate this sort of logic in one codebase?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I think this is the right place to consolidate all of the lexical lookup logic over time.

.library(
name: "SwiftSyntaxMacrosGenericTestSupport",
targets: ["SwiftSyntaxMacrosGenericTestSupport"]
Expand Down Expand Up @@ -243,6 +244,18 @@ let package = Package(
]
),

// MARK: SwiftLexicalScopes

.target(
name: "SwiftLexicalScopes",
dependencies: ["SwiftSyntax"]
),

.testTarget(
name: "SwiftLexicalScopesTest",
dependencies: ["_SwiftSyntaxTestSupport", "SwiftLexicalScopes"]
),

// MARK: SwiftSyntaxMacrosGenericTestSupport

.target(
Expand Down
14 changes: 0 additions & 14 deletions Sources/SwiftBasicFormat/BasicFormat.swift
Original file line number Diff line number Diff line change
Expand Up @@ -676,17 +676,3 @@ fileprivate extension TokenSyntax {
}
}
}

fileprivate extension SyntaxProtocol {
/// Returns this node or the first ancestor that satisfies `condition`.
func ancestorOrSelf<T>(mapping map: (Syntax) -> T?) -> T? {
var walk: Syntax? = Syntax(self)
while let unwrappedParent = walk {
if let mapped = map(unwrappedParent) {
return mapped
}
walk = unwrappedParent.parent
}
return nil
}
}
36 changes: 36 additions & 0 deletions Sources/SwiftLexicalScopes/LexicalScopes.swift
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
//===----------------------------------------------------------------------===//
//
// This source file is part of the Swift.org open source project
//
// Copyright (c) 2014 - 2024 Apple Inc. and the Swift project authors
// Licensed under Apache License v2.0 with Runtime Library Exception
//
// See https://swift.org/LICENSE.txt for license information
// See https://swift.org/CONTRIBUTORS.txt for the list of Swift project authors
//
//===----------------------------------------------------------------------===//

import Foundation
import SwiftSyntax

extension SyntaxProtocol {
/// Given syntax node position, returns all available labeled statements.
@_spi(Compiler) @_spi(Testing) public func lookupLabeledStmts() -> [LabeledStmtSyntax] {
guard let scope else { return [] }
return scope.lookupLabeledStmts(at: self)
}

/// Given syntax node position, returns the current switch case and it's fallthrough destination.
@_spi(Compiler) @_spi(Testing) public func lookupFallthroughSourceAndDest()
-> (source: SwitchCaseSyntax?, destination: SwitchCaseSyntax?)
{
guard let scope else { return (nil, nil) }
return scope.lookupFallthroughSourceAndDestination(at: self)
}

/// Given syntax node position, returns the closest ancestor catch node.
@_spi(Compiler) @_spi(Testing) public func lookupCatchNode() -> Syntax? {
guard let scope else { return nil }
return scope.lookupCatchNode(at: Syntax(self))
}
}
185 changes: 185 additions & 0 deletions Sources/SwiftLexicalScopes/Scope.swift
Original file line number Diff line number Diff line change
@@ -0,0 +1,185 @@
//===----------------------------------------------------------------------===//
//
// This source file is part of the Swift.org open source project
//
// Copyright (c) 2014 - 2024 Apple Inc. and the Swift project authors
// Licensed under Apache License v2.0 with Runtime Library Exception
//
// See https://swift.org/LICENSE.txt for license information
// See https://swift.org/CONTRIBUTORS.txt for the list of Swift project authors
//
//===----------------------------------------------------------------------===//

import Foundation
import SwiftSyntax

extension SyntaxProtocol {
/// Scope at the syntax node. Could be inherited from parent or introduced at the node.
var scope: Scope? {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This property is effectively always returning a FileScope for the outermost SourceFileSyntax.

switch self.syntaxNodeType {
case is SourceFileSyntax.Type:
FileScope(syntax: self)
default:
parent?.scope
}
}
}

/// Provide common functionality for specialized scope implementatations.
protocol Scope {
/// The parent of this scope.
var parent: Scope? { get }

/// Syntax node that introduces this protocol.
var sourceSyntax: SyntaxProtocol { get }
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As far as I can tell, Scope isn't doing anything any more. Can we remove it, and put the various operations on SyntaxProtocol directly?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It was a leftover from the original API structure I wanted to implement. I moved now all the query methods to SyntaxProtocol and removed parts related to the Scope protocol.


extension Scope {
/// Recursively walks up syntax tree and finds the closest scope other than this scope.
func getParentScope(forSyntax syntax: SyntaxProtocol?) -> Scope? {
if let lookedUpScope = syntax?.scope, lookedUpScope.sourceSyntax.id == syntax?.id {
return getParentScope(forSyntax: sourceSyntax.parent)
} else {
return syntax?.scope
}
}

// MARK: - lookupLabeledStmts

/// Given syntax node position, returns all available labeled statements.
func lookupLabeledStmts(at syntax: SyntaxProtocol) -> [LabeledStmtSyntax] {
return walkParentTreeUpToFunctionBoundary(
at: syntax.parent,
collect: LabeledStmtSyntax.self
)
}

// MARK: - lookupFallthroughSourceAndDest

/// Given syntax node position, returns the current switch case and it's fallthrough destination.
func lookupFallthroughSourceAndDestination(at syntax: SyntaxProtocol) -> (SwitchCaseSyntax?, SwitchCaseSyntax?) {
guard
let originalSwitchCase = walkParentTreeUpToFunctionBoundary(
at: Syntax(syntax),
collect: SwitchCaseSyntax.self
)
else {
return (nil, nil)
}

let nextSwitchCase = lookupNextSwitchCase(at: originalSwitchCase)

return (originalSwitchCase, nextSwitchCase)
}

/// Given a switch case, returns the case that follows according to the parent.
private func lookupNextSwitchCase(at switchCaseSyntax: SwitchCaseSyntax) -> SwitchCaseSyntax? {
guard let switchCaseListSyntax = switchCaseSyntax.parent?.as(SwitchCaseListSyntax.self) else { return nil }

var visitedOriginalCase = false

for child in switchCaseListSyntax.children(viewMode: .sourceAccurate) {
if let thisCase = child.as(SwitchCaseSyntax.self) {
if thisCase.id == switchCaseSyntax.id {
visitedOriginalCase = true
} else if visitedOriginalCase {
return thisCase
}
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this currently doesn’t handle #if clauses at all. @DougGregor Have you thought about how lexical scopes should related to #if clauses. Because really, we need to evaluate the #if conditions to be able to tell which case the switch falls through to. The same issue also comes up for name lookup.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you think it would be possible to e.g. evaluate the #if clauses and then perform the lookup on the resulting source code?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@DougGregor has a PR up to evaluate #if in swift-syntax but I think it’s not quite ready yet. #1816

Let’s wait until Doug is back to discuss how we want to approach this.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had in fact not considered how to handle #if clauses. I don't want to rely on any kind of syntactic preprocessing of the syntax tree to remove the #ifs: we need this query to work on the original, unmodified syntax tree.

So, it seems like we will need something like #1816 to have any chance of making lookupNextSwitchCase(at:) work when there's an #if around the case. When we do get #1816, we'll need to parameterize this operation by a BuildConfiguration. I'm okay with waiting until #1816 lands to make that change, and just... pick the first one or something... if there are #ifs around the following case.

}

return nil
}

// MARK: - lookupCatchNode

/// Given syntax node position, returns the closest ancestor catch node.
func lookupCatchNode(at syntax: Syntax) -> Syntax? {
return lookupCatchNodeHelper(at: syntax, traversedCatchClause: false)
}

/// Given syntax node location, finds where an error could be caught. If set to `true`, `traverseCatchClause`lookup will skip the next do statement.
private func lookupCatchNodeHelper(at syntax: Syntax?, traversedCatchClause: Bool) -> Syntax? {
guard let syntax else { return nil }

switch syntax.as(SyntaxEnum.self) {
case .doStmt:
if traversedCatchClause {
return lookupCatchNodeHelper(at: syntax.parent, traversedCatchClause: false)
} else {
return syntax
}
case .catchClause:
return lookupCatchNodeHelper(at: syntax.parent, traversedCatchClause: true)
case .tryExpr(let tryExpr):
if tryExpr.questionOrExclamationMark != nil {
return syntax
} else {
return lookupCatchNodeHelper(at: syntax.parent, traversedCatchClause: traversedCatchClause)
}
case .functionDecl, .accessorDecl, .initializerDecl:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also deinits and closures.

return syntax
default:
return lookupCatchNodeHelper(at: syntax.parent, traversedCatchClause: traversedCatchClause)
}
}

/// Callect the first syntax node matching the collection type up to a function boundary.
func walkParentTreeUpToFunctionBoundary<T: SyntaxProtocol>(
at syntax: Syntax?,
collect: T.Type
) -> T? {
walkParentTreeUpToFunctionBoundary(at: syntax, collect: collect, stopWithFirstMatch: true).first
}

/// Callect syntax nodes matching the collection type up to a function boundary.
func walkParentTreeUpToFunctionBoundary<T: SyntaxProtocol>(
at syntax: Syntax?,
collect: T.Type,
stopWithFirstMatch: Bool = false
) -> [T] {
walkParentTree(
upTo: [
MemberBlockSyntax.self,
FunctionDeclSyntax.self,
InitializerDeclSyntax.self,
DeinitializerDeclSyntax.self,
AccessorDeclSyntax.self,
ClosureExprSyntax.self,
],
at: syntax,
collect: collect,
stopWithFirstMatch: stopWithFirstMatch
)
}

/// Callect syntax nodes matching the collection type up until encountering one of the specified syntax nodes.
func walkParentTree<T: SyntaxProtocol>(
upTo stopAt: [SyntaxProtocol.Type],
at syntax: Syntax?,
collect: T.Type,
stopWithFirstMatch: Bool = false
) -> [T] {
guard let syntax, !stopAt.contains(where: { syntax.is($0) }) else { return [] }
if let matchedSyntax = syntax.as(T.self) {
if stopWithFirstMatch {
return [matchedSyntax]
} else {
return [matchedSyntax]
+ walkParentTree(
upTo: stopAt,
at: syntax.parent,
collect: collect,
stopWithFirstMatch: stopWithFirstMatch
)
}
} else {
return walkParentTree(
upTo: stopAt,
at: syntax.parent,
collect: collect,
stopWithFirstMatch: stopWithFirstMatch
)
}
}
}
24 changes: 24 additions & 0 deletions Sources/SwiftLexicalScopes/ScopeConcreteImplementations.swift
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
//===----------------------------------------------------------------------===//
//
// This source file is part of the Swift.org open source project
//
// Copyright (c) 2014 - 2024 Apple Inc. and the Swift project authors
// Licensed under Apache License v2.0 with Runtime Library Exception
//
// See https://swift.org/LICENSE.txt for license information
// See https://swift.org/CONTRIBUTORS.txt for the list of Swift project authors
//
//===----------------------------------------------------------------------===//

import Foundation
import SwiftSyntax

class FileScope: Scope {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FileScope isn't doing anything. I think it should be removed.

var parent: Scope? = nil

var sourceSyntax: SyntaxProtocol

init(syntax: SyntaxProtocol) {
self.sourceSyntax = syntax
}
}
12 changes: 0 additions & 12 deletions Sources/SwiftParserDiagnostics/SyntaxExtensions.swift
Original file line number Diff line number Diff line change
Expand Up @@ -119,18 +119,6 @@ extension SyntaxProtocol {
}
}

/// Returns this node or the first ancestor that satisfies `condition`.
func ancestorOrSelf<T>(mapping map: (Syntax) -> T?) -> T? {
var walk: Syntax? = Syntax(self)
while let unwrappedParent = walk {
if let mapped = map(unwrappedParent) {
return mapped
}
walk = unwrappedParent.parent
}
return nil
}

/// Returns `true` if the next token's leading trivia should be made leading trivia
/// of this mode, when it is switched from being missing to present.
var shouldBeInsertedAfterNextTokenTrivia: Bool {
Expand Down
14 changes: 13 additions & 1 deletion Sources/SwiftSyntax/SyntaxProtocol.swift
Original file line number Diff line number Diff line change
Expand Up @@ -186,7 +186,7 @@ extension SyntaxProtocol {
}
}

// MARK: Children / parent
// MARK: Children / parent / ancestor

extension SyntaxProtocol {
/// A sequence over the children of this node.
Expand Down Expand Up @@ -238,6 +238,18 @@ extension SyntaxProtocol {
public var previousToken: TokenSyntax? {
return self.previousToken(viewMode: .sourceAccurate)
}

/// Returns this node or the first ancestor that satisfies `condition`.
public func ancestorOrSelf<T>(mapping map: (Syntax) -> T?) -> T? {
var walk: Syntax? = Syntax(self)
while let unwrappedParent = walk {
if let mapped = map(unwrappedParent) {
return mapped
}
walk = unwrappedParent.parent
}
return nil
}
}

// MARK: Accessing tokens
Expand Down
Loading