Skip to content

Commit

Permalink
Fixup more line styles in src/macros
Browse files Browse the repository at this point in the history
  • Loading branch information
Veykril committed Jul 11, 2021
1 parent fac0a25 commit 460b634
Show file tree
Hide file tree
Showing 17 changed files with 313 additions and 411 deletions.
1 change: 1 addition & 0 deletions src/macros.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ Following up the two introductions it offers some generally very useful [pattern

Should the information presented here not suffice, then there is also the [Macros chapter of the Rust Book] which is a more approachable, high-level explanation as well as the reference [chapter](https://doc.rust-lang.org/reference/macros-by-example.html) which goes more into the precise details of things.

> **Note**: This book will usually use the term *mbe*(**M**acro-**B**y-**E**xample), *mbe macro* or `macro_rules!` macro when talking about `macro_rules!` macros.
[mbe]: https://doc.rust-lang.org/reference/macros-by-example.html
[Macros chapter of the Rust Book]: https://doc.rust-lang.org/book/ch19-06-macros.html
Expand Down
2 changes: 1 addition & 1 deletion src/macros/building-blocks.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
# Building Blocks

Reusable snippets of macro code.
Reusable snippets of `macro_rules!` macro code.
37 changes: 16 additions & 21 deletions src/macros/building-blocks/abacus-counting.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,9 @@
# Abacus Counters

> **Provisional**: needs a more compelling example. Matching nested groups that are *not* denoted by
> Rust groups is sufficiently unusual that it may not merit inclusion.
> **Provisional**: needs a more compelling example.
> Matching nested groups that are *not* denoted by Rust groups is sufficiently unusual that it may not merit inclusion.
> **Note**: this section assumes understanding of [push-down accumulation](#push-down-accumulation)
> and [incremental TT munchers](#incremental-tt-munchers).
> **Note**: this section assumes understanding of [push-down accumulation](#push-down-accumulation) and [incremental TT munchers](#incremental-tt-munchers).
```rust
macro_rules! abacus {
Expand Down Expand Up @@ -32,16 +31,15 @@ fn main() {
}
```

This technique can be used in cases where you need to keep track of a varying counter that starts at
or near zero, and must support the following operations:
This technique can be used in cases where you need to keep track of a varying counter that starts at or near zero, and must support the following operations:

* Increment by one.
* Decrement by one.
* Compare to zero (or any other fixed, finite value).

A value of *n* is represented by *n* instances of a specific token stored in a group. Modifications
are done using recursion and [push-down accumulation](#push-down-accumulation). Assuming the token
used is `x`, the operations above are implemented as follows:
A value of *n* is represented by *n* instances of a specific token stored in a group.
Modifications are done using recursion and [push-down accumulation](#push-down-accumulation).
Assuming the token used is `x`, the operations above are implemented as follows:

* Increment by one: match `($($count:tt)*)`, substitute `(x $($count)*)`.
* Decrement by one: match `(x $($count:tt)*)`, substitute `($($count)*)`.
Expand All @@ -58,12 +56,11 @@ In this way, operations on the counter are like flicking tokens back and forth l
In fairness, it could *also* have been called ["unary counting"](https://en.wikipedia.org/wiki/Unary_numeral_system).

In cases where you want to represent negative values, *-n* can be represented as *n* instances of a
*different* token. In the example given above, *+n* is stored as *n* `+` tokens, and *-m* is stored
as *m* `-` tokens.
*different* token.
In the example given above, *+n* is stored as *n* `+` tokens, and *-m* is stored as *m* `-` tokens.

In this case, the operations become slightly more complicated; increment and decrement effectively
reverse their usual meanings when the counter is negative. To which given `+` and `-` for the
positive and negative tokens respectively, the operations change to:
In this case, the operations become slightly more complicated; increment and decrement effectively reverse their usual meanings when the counter is negative.
To which given `+` and `-` for the positive and negative tokens respectively, the operations change to:

* Increment by one:
* match `()`, substitute `(+)`.
Expand All @@ -80,11 +77,10 @@ positive and negative tokens respectively, the operations change to:
* Compare to -2: match `(--)`.
* *(and so on...)*

Note that the example at the top combines some of the rules together (for example, it combines
increment on `()` and `($($count:tt)+)` into an increment on `($($count:tt)*)`).
Note that the example at the top combines some of the rules together (for example, it combines increment on `()` and `($($count:tt)+)` into an increment on `($($count:tt)*)`).

If you want to extract the actual *value* of the counter, this can be done using a regular
[counter macro](./counting.md). For the example above, the terminal rules can be replaced with the following:
If you want to extract the actual *value* of the counter, this can be done using a regular [counter macro](./counting.md).
For the example above, the terminal rules can be replaced with the following:

```rust,ignore
macro_rules! abacus {
Expand All @@ -106,9 +102,8 @@ macro_rules! count_tts {
}
```

> **<abbr title="Just for this example">JFTE</abbr>**: strictly speaking, the above formulation of
> `abacus!` is needlessly complex. It can be implemented much more efficiently using repetition,
> provided you *do not* need to match against the counter's value in a macro:
> **<abbr title="Just for this example">JFTE</abbr>**: strictly speaking, the above formulation of `abacus!` is needlessly complex.
> It can be implemented much more efficiently using repetition, provided you *do not* need to match against the counter's value in a macro:
>
> ```ignore
> macro_rules! abacus {
Expand Down
13 changes: 5 additions & 8 deletions src/macros/building-blocks/ast-coercion.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,8 @@
# AST Coercion

The Rust parser is not very robust in the face of `tt` substitutions. Problems can arise when the
parser is expecting a particular grammar construct and *instead* finds a lump of substituted `tt`
tokens. Rather than attempt to parse them, it will often just *give up*. In these cases, it is
necessary to employ an AST coercion.
The Rust parser is not very robust in the face of `tt` substitutions.
Problems can arise when the parser is expecting a particular grammar construct and *instead* finds a lump of substituted `tt` tokens.
Rather than attempt to parse them, it will often just *give up*. In these cases, it is necessary to employ an AST coercion.

```rust
# #![allow(dead_code)]
Expand All @@ -20,10 +19,8 @@ fn main() {
}
```

These coercions are often used with [push-down accumulation] macros in order to get the parser to
treat the final `tt` sequence as a particular kind of grammar construct.
These coercions are often used with [push-down accumulation] macros in order to get the parser to treat the final `tt` sequence as a particular kind of grammar construct.

Note that this specific set of macros is determined by what macros are allowed to expand to, *not*
what they are able to capture.
Note that this specific set of macros is determined by what macros are allowed to expand to, *not* what they are able to capture.

[push-down accumulation]: ../patterns/push-down-acc.md
60 changes: 25 additions & 35 deletions src/macros/building-blocks/counting.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,8 @@

## Repetition with replacement

Counting things in a macro is a surprisingly tricky task. The simplest way is to use replacement
with a repetition match.
Counting things in a macro is a surprisingly tricky task.
The simplest way is to use replacement with a repetition match.

```rust
macro_rules! replace_expr {
Expand All @@ -19,15 +19,14 @@ macro_rules! count_tts {
# }
```

This is a fine approach for smallish numbers, but will likely *crash the compiler* with inputs of
around 500 or so tokens. Consider that the output will look something like this:
This is a fine approach for smallish numbers, but will likely *crash the compiler* with inputs of around 500 or so tokens.
Consider that the output will look something like this:

```rust,ignore
0usize + 1usize + /* ~500 `+ 1usize`s */ + 1usize
```

The compiler must parse this into an AST, which will produce what is effectively a perfectly
unbalanced binary tree 500+ levels deep.
The compiler must parse this into an AST, which will produce what is effectively a perfectly unbalanced binary tree 500+ levels deep.

## Recursion

Expand All @@ -44,15 +43,13 @@ macro_rules! count_tts {
# }
```

> **Note**: As of `rustc` 1.2, the compiler has *grievous* performance problems when large numbers
> of integer literals of unknown type must undergo inference. We are using explicitly
> `usize`-typed literals here to avoid that.
> **Note**: As of `rustc` 1.2, the compiler has *grievous* performance problems when large numbers of integer literals of unknown type must undergo inference.
> We are using explicitly `usize`-typed literals here to avoid that.
>
> If this is not suitable (such as when the type must be substitutable), you can help matters by
> using `as` (*e.g.* `0 as $ty`, `1 as $ty`, *etc.*).
> If this is not suitable (such as when the type must be substitutable), you can help matters by using `as` (*e.g.* `0 as $ty`, `1 as $ty`, *etc.*).
This *works*, but will trivially exceed the recursion limit. Unlike the repetition approach, you can
extend the input size by matching multiple tokens at once.
This *works*, but will trivially exceed the recursion limit.
Unlike the repetition approach, you can extend the input size by matching multiple tokens at once.

```rust
macro_rules! count_tts {
Expand Down Expand Up @@ -147,12 +144,10 @@ macro_rules! count_idents {
# }
```

This method does have two drawbacks. First, as implied above, it can *only* count valid identifiers
(which are also not keywords), and it does not allow those identifiers to repeat.
This method does have two drawbacks.
First, as implied above, it can *only* count valid identifiers(which are also not keywords), and it does not allow those identifiers to repeat.

Secondly, this approach is *not* hygienic, meaning that if whatever identifier you use in place of
`__CountIdentsLast` is provided as input, the macro will fail due to the duplicate variants in the
`enum`.
Secondly, this approach is *not* hygienic, meaning that if whatever identifier you use in place of `__CountIdentsLast` is provided as input, the macro will fail due to the duplicate variants in the `enum`.

## Bit twiddling

Expand All @@ -170,34 +165,29 @@ macro_rules! count_tts {
# }
```

This approach is pretty smart as it effectively halves its input whenever its even and then
multiplying the counter by 2 (or in this case shifting 1 bit to the left which is equivalent). If
the input is uneven it simply takes one token tree from the input `or`s the token tree to the
previous counter which is equivalent to adding 1 as the lowest bit has to be a 0 at this point due
to the previous shifting. Rinse and repeat until we hit the base rule `() => 0`.
This approach is pretty smart as it effectively halves its input whenever its even and then multiplying the counter by 2 (or in this case shifting 1 bit to the left which is equivalent).
If the input is uneven it simply takes one token tree from the input `or`s the token tree to the previous counter which is equivalent to adding 1 as the lowest bit has to be a 0 at this point due to the previous shifting.
Rinse and repeat until we hit the base rule `() => 0`.

The benefit of this is that the constructed AST expression that makes up the counter value will grow
with a complexity of `O(log(n))` instead of `O(n)` like the other approaches. Be aware that you can
still hit the recursion limit with this if you try hard enough. Credits for this method go to Reddit
user [`YatoRust`](https://www.reddit.com/r/rust/comments/d3yag8/the_little_book_of_rust_macros/).
The benefit of this is that the constructed AST expression that makes up the counter value will grow with a complexity of `O(log(n))` instead of `O(n)` like the other approaches.
Be aware that you can still hit the recursion limit with this if you try hard enough.
Credits for this method go to Reddit user [`YatoRust`](https://www.reddit.com/r/rust/comments/d3yag8/the_little_book_of_rust_macros/).


Let's go through the procedure by hand once:

```rust,ignore
count_tts!(0 0 0 0 0 0 0 0 0 0);
```
This invocation will match the third rule due to the fact that we have an even number of token trees
(10). The matcher names the odd token trees in the sequence `$a` and the even ones `$even` but the
expansion only makes use of `$a`, which means it effectively discards all the even elements cutting
the input in half. So the invocation now becomes:
This invocation will match the third rule due to the fact that we have an even number of token trees(10).
The matcher names the odd token trees in the sequence `$a` and the even ones `$even` but the expansion only makes use of `$a`, which means it effectively discards all the even elements cutting the input in half.
So the invocation now becomes:
```rust,ignore
count_tts!(0 0 0 0 0) << 1;
```
This invocation will now match the second rule as its input is an uneven amount of token trees. In
this case the first token tree is discarded to make the input even again, then we also do the
halving step in this invocation again since we know the input would be even now anyways. Therefor we
can count 1 for the uneven discard and multiply by 2 again since we also halved.
This invocation will now match the second rule as its input is an uneven amount of token trees.
In this case the first token tree is discarded to make the input even again, then we also do the halving step in this invocation again since we know the input would be even now anyways.
Therefor we can count 1 for the uneven discard and multiply by 2 again since we also halved.
```rust,ignore
((count_tts!(0 0) << 1) | 1) << 1;
```
Expand Down
38 changes: 14 additions & 24 deletions src/macros/building-blocks/parsing.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,11 @@
# Parsing Rust

Parsing some of Rust's items can be useful in certain situations. This section will show a few
macros that can parse some of Rust's more complex items like structs and functions to a certain extent.
The goal of these macros is not to be able to parse the entire grammar of the items but to parse
parts that are in general quite useful without being too complex to parse. This means we ignore
things like generics and such.
Parsing some of Rust's items can be useful in certain situations.
This section will show a few macros that can parse some of Rust's more complex items like structs and functions to a certain extent.
The goal of these macros is not to be able to parse the entire grammar of the items but to parse parts that are in general quite useful without being too complex to parse. This means we ignore things like generics and such.

The main points of interest of these macros are their `matchers`. The transcribers are only there
for example purposes and are usually not that impressive.
The main points of interest of these macros are their `matchers`.
The transcribers are only there for example purposes and are usually not that impressive.

## Function

Expand Down Expand Up @@ -42,23 +40,18 @@ macro_rules! function_item_matcher {
# }
```

A simple function matcher that ignores qualifiers like `unsafe`, `async`, ... as well a generics and
where clauses. If parsing those is required it is likely that you are better off using a proc-macro
instead.
A simple function matcher that ignores qualifiers like `unsafe`, `async`, ... as well a generics and where clauses.
If parsing those is required it is likely that you are better off using a proc-macro instead.

This lets you for example, inspect the function signature, generate some extra things from it and
then re-emit the entire function again. Kind of like a `Derive` proc-macro but weaker and for
functions.
This lets you for example, inspect the function signature, generate some extra things from it and then re-emit the entire function again.
Kind of like a `Derive` proc-macro but weaker and for functions.

> Ideally we would like to use a pattern fragment specifier instead of an ident for the arguments
> but this is currently not allowed. Fortunately people don't use patterns in function signatures
> that often so this is okay.
> Ideally we would like to use a pattern fragment specifier instead of an ident for the arguments but this is currently not allowed.
> Fortunately people don't use non-identifier patterns in function signatures that often so this is okay(a shame, really).
### Method

The macro for parsing basic functions is nice and all, but sometimes we would like to also parse
methods, functions that refer to their object via some form of `self` usage. This makes things a bit
trickier:
The macro for parsing basic functions is nice and all, but sometimes we would like to also parse methods, functions that refer to their object via some form of `self` usage. This makes things a bit trickier:

> WIP
Expand Down Expand Up @@ -146,11 +139,8 @@ macro_rules! struct_item_matcher {

# Enum

Parsing enums is a bit more complex than structs so we will finally make use of some of the
[patterns] we have discussed, [Incremental TT Muncher] and [Internal Rules]. Instead of just
building the parsed enum again we will merely visit all the tokens of the enum, as rebuilding the
enum would require us to collect all the parsed tokens temporarily again via a
[Push Down Accumulator].
Parsing enums is a bit more complex than structs so we will finally make use of some of the [patterns] we have discussed, [Incremental TT Muncher] and [Internal Rules].
Instead of just building the parsed enum again we will merely visit all the tokens of the enum, as rebuilding the enum would require us to collect all the parsed tokens temporarily again via a [Push Down Accumulator].

```rust
macro_rules! enum_item_matcher {
Expand Down
Loading

0 comments on commit 460b634

Please sign in to comment.