Description
With nested macro expansion, the inner macro sees an expression generated by the outer macro which may include Expr(:escape). However, macro writers generally test macros only with a single level of expansion, not including Expr(:escape).
This means that macros which pattern match their input are incorrect by default when used in a nested expansion.
As a simple example of how pervasive this problem is, consider that Base.@view cannot generally be used within the AST generated by another macro:
julia> macro m(ex)
quote
@view $(esc(ex))
end
end
julia> A = [1,2,3]
julia> @m A[1:2]
ERROR: LoadError: ArgumentError: Invalid use of @view macro: argument must be a reference expression A[...].
Stacktrace:
[1] @view(::LineNumberNode, ::Module, ::Any) at ./views.jl:123
in expression starting at REPL[9]:3The problem here is that @view gets provided with esc(:(A[1:2])) as an argument, which is not an Expr(:ref) as naturally expected by the authors of @view
This problem occurs whenever macros try to pattern match their input rather than simply substituting it into a larger expression. The pattern matching must be aware that Expr(:escape) could occur anywhere. Anybody writing macros directly against the Expr API (by using the head field, etc) is going to handle this incorrectly.
This usability issue has also been discussed at length in #23221. However that issue doesn't describe the problem very clearly as a problem of usability, so I thought I'd restate it here.
Here's another interesting case:
julia> macro m2(ex)
quote
@show $(esc(ex))
end
end
julia> @m2 1
$(Expr(:escape, 1)) = 1What to do?
A possible way forward is to treat this as an Expr API problem: if pattern matching within macros is incorrect by default, maybe we need better ways to pattern match expressions — for example as in MacroTools or MLStyle — ensuring that any appearance of Expr(:escape) doesn't break the matching process, and returning matched pieces with a correctly nested level of escape.
A larger overhaul of the macro system as in #6910 has also been mentioned in relation to this. In that PR, quoteed code created within macros is transformed during lowering, such that every quoted symbol made by the macro is unescaped with Expr(:hygenic, sym). I'm not sure whether it solves the problem completely or simply shifts it around to create new and exciting footguns for macro writers.