Elixir - Metaprogramming


quoted
= playbook project

  1. https://medium.com/@andreichernykh/elixir-a-bit-about-macros-behaviours-84fd3de1595d
  2. https://dockyard.com/blog/2017/12/07/macro-madness-how-to-use-use-well

quoted expressions

https://elixir-lang.org/getting-started/meta/quote-and-unquote.html

When quoting more complex expressions, we can see that the code is represented in such tuples, which are often nested inside each other in a structure resembling a tree. Many languages would call such representations an Abstract Syntax Tree (AST). Elixir calls them quoted expressions.

=> AST = quoted expressions.

https://dockyard.com/blog/2016/08/16/the-minumum-knowledge-you-need-to-start-metaprogramming-in-elixir

Macros receive AST as arguments and provide AST as return values.

https://elixir-lang.org/getting-started/meta/macros.html

That’s what macros are all about. They are about receiving quoted expressions and transforming them into something else.

AST literals

  1. https://elixir-lang.org/getting-started/meta/quote-and-unquote.html

there are five Elixir literals that, when quoted, return themselves (and not a tuple). They are:

:sum         #=> Atoms
1.0          #=> Numbers
[1, 2]       #=> Lists
"strings"    #=> Strings
{key, value} #=> Tuples with two elements

NOTE: say, keyword list (list of tuples) turns out to be AST literal as well (though second elements of tuples can be not AST literals already).

=> quoted AST literals are AST literals themselves so they are:

quoting

macro arguments

https://elixir-lang.org/getting-started/meta/macros.html

arguments to a function call are evaluated before calling the function. macros do not evaluate their arguments - they receive the arguments as quoted expressions which are then transformed into other quoted expressions.

=> all macro arguments are quoted before being passed to macro (except for AST literals - see AST literals section above).

https://elixirforum.com/t/pass-module-to-a-macro/978/2

You don’t get values into a macro but ASTs.

The AST generated by the term Plug is {:aliases, [alias: false], [:Plug]}. To get the actual value out of it, you need to expand it.

if it’s necessary to get the actual value of quoted expression (which is not AST literal) inside macro before unquoting it, use Macro.expand/2:

# lib/lain/page/conversation/loader.ex:
defmodule Lain.Page.Conversation.Loader do
  use Lain.CQRS.Loader, schema: Lain.Page.Conversation
end

# lib/lain/cqrs/loader.ex:
defmodule Lain.CQRS.Loader do
  defmacro __using__(opts) do
    # opts[:schema] =>
    #   {:__aliases__, [alias: false], [:Lain, :Page, :Conversation]}
    # schema =>
    #   Lain.Page.Conversation
    schema = Macro.expand(opts[:schema], __ENV__)

    quote do
      alias unquote(:"#{schema}.Query")
    end
  end
end

still in most cases:

https://groups.google.com/d/msg/elixir-lang-core/g6kAJjuWDOc/iVbcTlmb1t4J

Most of the work should be done inside a quote instead. Some arguments should be really handled as an AST expression… Those can be passed down to the quote with escaping.

quote/2 vs. Macro.escape/2

  1. https://elixirforum.com/t/understand-macro-escape/405/2

https://hexdocs.pm/elixir/Macro.html#escape/2-comparison-to-kernel-quote-2

Macro.escape/2 is used to escape values (either directly passed or variable bound), while Kernel.SpecialForms.quote/2 produces syntax trees for expressions.

that is Macro.escape/2 first evaluates passed expression and returns AST of the result while quote/2 returns AST of passed expression as is:

# Macro.escape/1 returns :"Elixir.Lain.Page.Conversation" which is
# is printed as Lain.Page.Conversation in IEx (atom is AST literal)
Macro.escape(Lain.Page.Conversation)
# => Lain.Page.Conversation

quote do: Lain.Page.Conversation
# => {:__aliases__, [alias: false], [:Lain, :Page, :Conversation]}

bind_quoted option of quote/2

  1. https://dockyard.com/blog/2016/08/16/the-minumum-knowledge-you-need-to-start-metaprogramming-in-elixir
# <https://hexdocs.pm/elixir/Kernel.SpecialForms.html#quote/2-quote-and-macros>

defmacro squared(x) do
  quote do
    x = unquote(x)
    x * x
  end
end

# is equal to

defmacro squared(x) do
  quote bind_quoted: [x: x] do
    x * x
  end
end

so bind_quoted option does 2 things:

https://hexdocs.pm/elixir/Kernel.SpecialForms.html

:bind_quoted - passes a binding to the macro. Whenever a binding is given, unquote/1 is automatically disabled.

=> moreover unquoting is prohibited when using bind_quoted option (because unquote: false option is added automatically in this case):

iex> defmodule Foo do
...>   defmacro my_macro(name) do
...>     quote bind_quoted: [name: name] do
...>       IO.puts(unquote(name))
...>     end
...>   end
...> end
iex> require Foo
iex> Foo.my_macro("foo")
** (CompileError) iex:6: unquote called outside quote
    expanding macro: Foo.my_macro/1
    iex:6: (file)

all these rules (about bind_quoted and unquoting) don’t apply to the functions dynamically generated inside macros - you always can and must use unquote/1 inside them:

defmacro my_macro(name) do
  quote bind_quoted: [name: name] do
    IO.puts(name) # name is properly unquoted thanks to bind_quoted

    def foo do
      IO.puts(unquote(name)) # name must be always unquoted explicitly
    end
  end
end

but execution of unquote/1 inside foo/0 (when using bind_quoted option) is deferred until runtime: without unquote: false option (which always comes with bind_quoted option) unquote/1 inside foo/0 would be executed during compilation.

unquoting

  1. https://elixirforum.com/t/how-to-quote-regular-expression/6311

only quoted expressions (or AST literals) can be unquoted with unquote/1 => not quoted expressions (which are not AST literals) must be converted to quoted expressions with Macro.escape/2 before being unquoted inside a macro:

# https://github.com/elixir-lang/elixir/blob/c76c5d6d188bbb787d6afefab74a31080695d1f2/lib/elixir/lib/gen_server.ex#L702

quote location: :keep, bind_quoted: [opts: opts] do
  # ...
  # opts is a keyword list here (=> not AST literal) so it must
  # be converted to quoted expression first before being unquoted
  Supervisor.child_spec(default, unquote(Macro.escape(opts)))
  # ...
end

note that there’ll be no error at compile time if you unquote not quoted experssion inside a macro - error will occur when you’ll try to evaluate resulting quoted expression:

iex> quote do: "123" == unquote(%{a: 1})
{:==, [context: Elixir, import: Kernel], ["123", %{a: 1}]}
iex> Code.eval_quoted(quote do: "123" == unquote(%{a: 1}))
** (CompileError) nofile: invalid quoted expression: %{a: 1}
    (stdlib) lists.erl:1354: :lists.mapfoldl/3
    (stdlib) lists.erl:1355: :lists.mapfoldl/3
iex> Code.eval_quoted(quote do: "123" == unquote(Macro.escape(%{a: 1})))
{false, []}

use macro

  1. https://elixir-lang.org/getting-started/alias-require-and-import.html#use
  2. https://dockyard.com/blog/2017/12/07/macro-madness-how-to-use-use-well

example

# https://github.com/elixir-lang/elixir/blob/master/lib/elixir/lib/gen_server.ex
defmodule MyApp.CQRS.Loader do
  defmacro __using__(opts) do
    # https://elixirforum.com/t/pass-module-to-a-macro/978
    # opts are quoted (converted to AST) when passed to macro
    schema = Macro.expand(opts[:schema], __ENV__)

    quote do
      alias MyApp.Repo
      alias unquote(schema)
      alias unquote(:"#{schema}.Query")

      @schema unquote(opts[:schema])

      def all(clauses) do
        Repo.all(@schema)
      end
    end
  end
end

wrapping functions

it can be useful to do some housekeeping such as:

custom def macro

  1. https://kr00lix.com/wrap-methods-for-logging-in-elixir.html
  2. https://elixir-lang.org/getting-started/meta/quote-and-unquote.html#quoting

in a nutshell it’s just defining your function (say, call) but using custom def macro - it can be named, say, def_with_log.

downsides of this approach:

example:

defmodule Common.Helpers.Operation do
  defmacro def_with_tagging(head, do: body) do
    {_name, _metadata, args} = head

    quote do
      def unquote(head) do
        Appsignal.Transaction.set_sample_data("params", unquote(args))
        unquote(body)
      end
    end
  end
end

wrapper function

  1. https://medium.com/@andreichernykh/elixir-a-bit-about-macros-behaviours-84fd3de1595d

in a nutshell it’s just adding another wrapper function (say, call_with_log) that calls original function (say, call) and does some housekeeping but that another function is defined with use macro.

downsides of this approach:

module templates

  1. https://github.com/elixir-lang/elixir/issues/638
  2. https://hexdocs.pm/elixir/Kernel.html#defoverridable/1
  3. https://dockyard.com/blog/2017/12/07/macro-madness-how-to-use-use-well

module (service, worker, etc.) template can be extracted using macro and then customized by implementing callbacks in client modules. those callbacks might either have default implementations in macro or not. in the former case it’s necessary to make those callbacks overridable with Kernel.defoverridable/1.

it’s like having a base class in Ruby with or without default implementations of some methods (the very callbacks) and adding those implementations in child classes inheriting from a base one.

template module:

defmodule Template do
  require Logger

  # IMO it's not necessary to declare foo/0 and bar/0 as callbacks
  @callback foo() :: any
  @callback bar() :: any
  @callback baz() :: any

  defmacro __using__(_opts) do
    quote do
      @behaviour Template

      @impl Template
      def foo, do: Template.foo(__MODULE__)

      @impl Template
      def bar, do: Template.bar(__MODULE__)

      # it's not necessary to make all callbacks overridable but it
      # can be useful if you, say, provide default implementation of
      # baz/0 which is meant to be customized in some cases
      defoverridable Template
      # use this line if foo/0 and bar/0 are not declared as callbacks
      #defoverridable foo: 0, bar: 0
    end
  end

  def foo(module) do
    Logger.info("Calling foo...", module: module)
    # always use `module` to call all callbacks and functions in
    # `quote` block (that is `module.bar()` - not `bar(module)`)
    # - in this case overridden versions (if any) will be used
    module.bar()
  end

  def bar(module) do
    Logger.info("Calling bar...", module: module)
    module.baz()
  end
end

client module:

defmodule Client do
  use Template

  @impl Template
  def baz, do: "baz"
end

usage:

Client.foo()

alternative implementation without macro

alternatively it’s possible to avoid using macro altogether by passing module implementing Template behaviour directly to Template.foo/1.

template module:

defmodule Template do
  require Logger

  @callback baz() :: any

  def foo(module) do
    Logger.info("Calling foo...", module: module)
    # now it's possible to use `bar(module)` since there
    # is no macro and functions cannot be overridden
    bar(module)
  end

  def bar(module) do
    Logger.info("Calling bar...", module: module)
    module.baz()
  end
end

client module:

defmodule Client do
  use Template

  @impl Template
  def baz, do: "baz"
end

usage:

Template.foo(Client)