`ProcLiteral#call` for macros

[HertzDevil]

#8835 proposes a way to have user-definable methods in the macro language that accept and return AST nodes:

macro def foo(x : StringLiteral) : StringLiteral
  "foo#{x.id}"
end

{{ foo "x" }} # => "foox"

There is a similar concept that would complement this kind of code reuse. We only need to implement ProcLiteral#call, interpreting the literal's entire body as a macro expression:

Foo = ->(x : StringLiteral) : StringLiteral do
  "foo#{x.id}"
end

{{ Foo.call "x" }} # => "foox"

Parameter and return types are optional as long as the constant is not referenced in normal code; if present, they are assumed to be AST node names and correspondingly typechecked. (Unlike defs, we do not have to care about overloading here.) This is also doable:

Fib = ->(x) { x <= 2 ? 1 : Fib.call(x - 1) + Fib.call(x - 2) }

{{ (1..10).map { |v| Fib.call(v) } }} # => [1, 1, 2, 3, 5, 8, 13, 21, 34, 55]

This requires no changes to the language grammar and can be further enhanced by features that make them closer to Procs in normal code, e.g. closures and macro ProcPointers.

---

Add a 👍 reaction to issues you find important.

[Blacksmoke16]

Copying my sentiment over here from Discord for posterity. I find this to be quite usable as a pretty simple way to add some flexibility to the macro language while #8835 is being ironed out. I'd use this more internally for code-reuse reasons. macro def would still be the most optimal approach from a public API perspective (mainly how they are used and can be more easily documented). But until if/when that happens this would already be a big win in my books.

[straight-shoota]

I appreciate the simplicity of this approach. The implementation is a very short diff.
But I share the concern that this feels like not as neat as we'd like.

The major issue is probably that these proc literals look like normal code (except the uncommon type restrictions) but are actually intended for macro code. I think that macros should be more obviously different and recognizable. Maybe that could be addressed at the syntax level, but I'm not sure it would be worth it.

A more static, method-like definition still seems a more promising approach to me.

I'm also wondering how you intend to integrate closures into this concept. What would be the context for closured variables? Procs assigned to constants are defined outside any macro scope.

[HertzDevil]

The major issue is probably that these proc literals look like normal code (except the uncommon type restrictions) but are actually intended for macro code.

Constants that are used exclusively in macros are not exactly something new. The compiler already has Crystal::Repl::Instructions for example, and some array literals (e.g. in spec/std/float_printer/ryu_printf_test_cases.cr) used an explicit [...] of _ even though the literal would be perfectly valid normal code otherwise. So if there is a need to distinguish them from normal code constructs, I would consider them equally and not single out ProcLiterals here.

macro followed by a constant is currently a syntax error. Maybe that would be a good extension to have? (This would still be a Const node with an extra attribute indicating whether the constant is macro-only, so the abstract syntax is backwards-compatible.)

macro Crystal::Repl::Instructions = {
  ...
}

macro Fact = ->(x) { x <= 1 ? x : x * Fact.call(x - 1) }

---

I'm also wondering how you intend to integrate closures into this concept. What would be the context for closured variables? Procs assigned to constants are defined outside any macro scope.

A closured macro variable might look like one of these:

{% begin %}
  {% x = 0 %}
  Counter = -> { x += 1 }
{% end %}
{% Counter.call %} # should this be 1?
{% Counter.call %} # should this be 2?

{%
  x = 0
  counter = -> { x += 1 }
  counter.call # should this be 1?
  counter.call # should this be 2?
%}

Contrast the latter with:

{%
  x = 0
  [nil].each { x += 1 }
  x # => 1
  {nil}.each { x += 1 }
  x # => 2
%}

Here unlike normal code, the macro interpreter never "captures" the blocks passed to AST node methods, and the block is simply evaluated inline. The possibility of passing counter around means it might need to form a true macro closure here.

[Blacksmoke16]

I still think this would be worth having, just for the possibilities it enables. E.g. could pass a proc literal in an annotation and use/call it in-conjunction with other values off the annotation. Having this user-provided flexibility is not something we really have in the macro world at the moment. But if I had to choose between this and #8835, I think the latter would be the most impactful. Mainly because it's just a better experience for the use case IMO.

I like the idea of macro constants tho! Ideally it would raise an error if you tried to access them at runtime. I ran into a few interesting bugs because of that.