Macros in Julia

February 4, 2021January 19, 2021 Editorial Staff

julia> @foofoo “rocks”

MLG:rocks

You might be thinking how is machine learning related to Macros. Or perhaps you’re still haunted by the Base.@kwdef we used a while ago, or maybe you still want to grok @epochs. Today we shall assuage your fears and enlighten your guts by shining a light on what @ does, and the how and the why behind it.

Introduction
Today we’ll finally talk in-depth about the homoiconicity we’ve been touting so much with. So, Lisp fans, please take my words with a pinch of salt, and the rest, please read the following as an opinionated overview rather than a point-by-point play.

First things first, what exactly is the meaning of homoiconicity? It essentially means that the Julia code you write, is itself an object in Julia. This means that you can manipulate your code with itself, and start off an infinite recursion(if you so please).

What
Lets take a boolean array for example:

julia> foobar=[true,false,true,true]
4-element Array{Bool,1}:
1
0
1
1
What is the boolean array? Is it foobar? Of course not, that’s the variable, it’s the value [true,false,true,true], that’s assigned to that variable that we’re actually after. The variable is there, as far as we are concerned, solely for convenience. Actually, all julia code is simply an Expr object (well, technically its all a string that’s parsed into that object). But what exactly is an Expr object ? Its short for an expression, and its composed of Symbol that specifies the ‘type’ of the expression, and (an array of) arguments.

So, to emulate rendering foobar from scratch, Julia would interpret the code in our .jl file as a string:

julia> foo=”[true,false,true,true]”
“[true,false,true,true]”
Then it would intern its identifier(for internal efficiency) and parse the ‘code’:

julia> ex1=Meta.parse(foo)
:([true, false, true, true])
Note how our array is inside parentheses suffixed by a colon, this makes into an expression:

julia> typeof(ex1)
Expr
The Symbol is in a field called head:

julia> ex1.head
:vect
While the arguments reside in args, which is an Array of base-type Any:

julia> ex1.args
4-element Array{Any,1}:
true
false
true
true
The astute reader(or LISP fan) might have realised that put together, this looks just like prefix notation. Indeed, we can use precisely that to construct an equivalent expression:

julia> ex1==Expr(:vect,true,false,true,true)
true
So now we have a data structure, of type Expr that contains all information about each expression that we code. This should also now make clear to how Julia uses semi-colons, they demarcate expressions in cases where EOL doesn’t. Now, if you wanted to have a quick look at an expression without having to deconstruct it step-by-step, you’d use dump:

julia> dump(Meta.parse(“[true,false&&false,true,true]”))
Expr
head: Symbol vect
args: Array{Any}((4,))
1: Bool true
2: Expr
head: Symbol &&
args: Array{Any}((2,))
1: Bool false
2: Bool false
3: Bool true
4: Bool true
But if you’re a LISPer, I feel you’ll much prefer show_sexpr, standing for show S-expression:

julia> Meta.show_sexpr(Meta.parse(“[true,false&&false,true,true]”))
(:vect, true, (:&&, false, false), true, true)
Ahh, but we also know that short-hand for constructing expression right?:

julia> Meta.show_sexpr(:([true,false&&false,true,true]))
(:vect, true, (:&&, false, false), true, true)
Enough talk about construction though, the other thing you can do with expressions is evaluate them:

julia> eval(:([true,false&&false,true,true]))
4-element Array{Bool,1}:
1
0
1
1
As we see, the evaluation creates a boolean array and we’ve come full circle. Now that basic introductions are done, we can actually talk about Macros.

How @
Macros are can be thought of as functions that create an expression and evaluates them. To be precise, you simply use the keyword macro and write a ‘function’ that returns an expression. Do know that you can very well write a ‘normal’ function that returns Expr, but you’d then need to also evaluate that expression, which not only is inelegant, but also has performance penalties. Without further ado, let’s write a very simple macro:

julia> macro MLG() :(“Hello from MachineLearningGeek.com”) end
@MLG (macro with 1 method)
To invoke it, we use the @ symbol. This is an old tradition, for example, Vim too invokes macros using this key. Anyhow, this is how we invoke the above macro:

julia> @MLG
“Hello from MachineLearningGeek.com”
You can also define macros that take an argument:

julia> macro foofoo(foo)
return 🙁 println(“MLG:”, $foo) )
end
@foofoo (macro with 1 method)
And can either call them like you call traditional functions:

julia> @foofoo(“rocks”)
MLG:rocks
Or via the more compact syntax:

julia> @foofoo “rocks”
MLG:rocks
For debugging purposes, you often want to check what expression the macro is evaluating without having it actually evaluated. For this purpose, you use the macroexpand macro:

julia> @macroexpand @foofoo “rocks”
:(Main.println(“MLG:”, “rocks”))
Here we see that the argument is correctly interpolated.

Why @
Well, now you know everything about macros. But one thing that you might still be thinking is why would you use macros instead of usual functions ? There are many reasons, but the most common ones are performance and elegance. Sometimes having bespoke code is either much faster than writing generic functions, or is much more readable. For example, using @epochs macro made our code in the last article look much better than using a loop, and maintained faster performance than using map. We shall visit its actual source code in the next article.

But the gist of the matter is that macros let you insert custom code when your program is parsed, and then execute some other code when it is actually executed. Thus, macros are actually ‘better’ than functions, they can do everything a function can do, and then some more. Take the following example, plucked right from Julia’s documentation:

macro twostep(arg)
println(“I execute at parse time. The argument is: “, arg)
return :(println(“I execute at runtime. The argument is: “, $arg))
end
when it is just expanded, which is a step that happens when julia parses your code:

julia> ex = macroexpand(Main, :(@twostep :(1, 2, 3)) );
I execute at parse time. The argument is: :((1, 2, 3))
Only the body of the macro executes. But when it is evaluated, i.e. it’s called:

eval(ex)
I execute at runtime. The argument is: (1, 2, 3)
The rest of the macro is completed. Thus you can see that macros are much more granular than a normal function.

References
https://docs.julialang.org/en/v1/manual/metaprogramming/#man-macros