Skip to content

Commit

Permalink
Format documents
Browse files Browse the repository at this point in the history
  • Loading branch information
juchiast authored and Geal committed Oct 6, 2018
1 parent 5d69046 commit 87d7fe1
Show file tree
Hide file tree
Showing 4 changed files with 36 additions and 56 deletions.
40 changes: 10 additions & 30 deletions doc/how_nom_macros_work.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
% How nom macros work
# How nom macros work

nom uses Rust macros heavily to provide a nice syntax and generate parsing code. This has multiple advantages:

Expand All @@ -11,20 +11,12 @@ nom uses Rust macros heavily to provide a nice syntax and generate parsing code.
Let's take the `opt!` macro as example: `opt!` returns `IResult<I,Option<O>>`, producing a `Some(o)` if the child parser succeeded, and None otherwise. Here is how you could use it:

```rust
# #[macro_use] extern crate nom;
# use nom::{IResult,digit};
# fn main() {
# }
named!(opt_tag<Option<&[u8]>>, opt!(digit));
```

And here is how it is defined:

```rust
# #[macro_use] extern crate nom;
# use nom::IResult;
# fn main() {
# }
#[macro_export]
macro_rules! opt(
($i:expr, $submac:ident!( $($args:tt)* )) => ({
Expand All @@ -42,7 +34,7 @@ macro_rules! opt(

To define a Rust macro, you indicate the name of the macro, then each pattern it is meant to apply to:

```ignore
```rust
macro_rules! my_macro (
(<pattern1>) => ( <generated code for pattern1> );
(<pattern2>) => ( <generated code for pattern2> );
Expand All @@ -53,33 +45,25 @@ macro_rules! my_macro (

The first thing you can see in `opt!` is that the pattern have an additional parameter that you do not use:

```ignore
```rust
($i:expr, $f:expr)
```

while you call:

```ignore
```rust
opt!(digit)
```

This is the first trick of nom macros: the first parameter, usually `$i` or `$input`, is the input data, passed by the parent parser. The expression using `named!` will translate like this:

```rust
# #[macro_use] extern crate nom;
# use nom::digit;
# fn main() {
# }
named!(opt_tag<Option<&[u8]>>, opt!(digit));
```

to

```rust
# #[macro_use] extern crate nom;
# use nom::{IResult,digit};
# fn main() {
# }
fn opt_tag(input:&[u8]) -> IResult<&[u8], Option<&[u8]>> {
opt!(input, digit)
}
Expand All @@ -90,10 +74,6 @@ This is how combinators hide all the plumbing: they receive the input automatica
When you have multiple submacros, such as this example, the input is always passed to the first, top level combinator:

```rust
# #[macro_use] extern crate nom;
# use nom::IResult;
# fn main() {
# }
macro_rules! multispaced (
($i:expr, $submac:ident!( $($args:tt)* )) => (
delimited!($i, opt!(multispace), $submac!($($args)*), opt!(multispace));
Expand All @@ -110,7 +90,7 @@ Here, `delimited!` will apply `opt!(multispace)` on the input, and if successful

The second trick you can see is the two patterns:

```ignore
```rust
#[macro_export]
macro_rules! opt(
($i:expr, $submac:ident!( $($args:tt)* )) => (
Expand All @@ -124,19 +104,19 @@ macro_rules! opt(

The first pattern is used to receive a macro as child parser, like this:

```ignore
```rust
opt!(tag!("abcd"))
```

The second pattern can receive a function, and transforms it in a macro, then calls itself again. This is done to avoid repeating code. Applying `opt!` with `digit` as argument would be transformed from this:

```ignore
```rust
opt!(digit)
```

transformed with the second pattern:

```ignore
```rust
opt!(call!(digit))
```

Expand All @@ -146,7 +126,7 @@ The `call!` macro transforms `call!(input, f)` into `f(input)`. If you need to p

The macro argument is decomposed into `$submac:ident!`, the macro's name and a bang, and `( $($args:tt)* )`, the tokens contained between the parenthesis of the macro call.

```ignore
```rust
($i:expr, $submac:ident!( $($args:tt)* )) => ({
match $submac!($i, $($args)*) {
Ok((i,o)) => Ok((i, Some(o))),
Expand All @@ -164,7 +144,7 @@ of `Ok((i,o))`: it is the input remaining after the first parser was applied.

As an example, see how the `preceded!` macro works:

```ignore
```rust
($i:expr, $submac:ident!( $($args:tt)* ), $submac2:ident!( $($args2:tt)* )) => (
{
match $submac!($i, $($args)*) {
Expand Down
10 changes: 5 additions & 5 deletions doc/upgrading_to_nom_1.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ Here are a few tips to update your code to run with nom 1.0:

`nom::Err` now depends on two generic types, the position `P` and the error type `E`:

```ignore
```rust
pub enum Err<P,E=u32>{
Code(ErrorKind<E>),
Node(ErrorKind<E>, Box<Err<P,E>>),
Expand All @@ -19,20 +19,20 @@ pub enum Err<P,E=u32>{

The default error type is `u32` to keep some compatibility with older code. To update your code, the first step is to **replace all usages of `nom::ErrorCode` by `nom::ErrorKind`**. `ErrorKind` is now an enum that contains the same instances as the previous `ErrorCode`, with an additional generic parameter:

```ignore
```rust
pub enum ErrorKind<E=u32> {
Custom(E),
Tag,
MapRes,
MapOpt,
Alt,
[...]
[...]
}
```

`ErrorKind::Custom` is where you will store your custom error type. Note that default nom parsers like `alphabetic` use `u32` as custom type, so you may need to translate the error types coming from those parsers like this:

```ignore
```rust
fix_error!(CustomErrorType, alphabetic)
```

Expand All @@ -42,7 +42,7 @@ Since the error type is now an enum instead of a `u32`, you can now **replace an

The error type is now completely generic over the input type, so the lifetime that appeared in `IResult` is not necessary anymore. It changes function declarations like this:

```ignore
```rust
fn parse_status<'a>(i: &'a [u8]) -> IResult<'a, &'a [u8], Status>

// To this:
Expand Down
18 changes: 9 additions & 9 deletions doc/upgrading_to_nom_2.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ Not everybody uses that feature, so it was moved behind a compilation feature ca

For the parsers using it, you will probably get something like the following compilation error:

```ignore
```
error: no associated item named `Code` found for type `nom::ErrorKind<_>` in the current scope
--> src/metadata/parser.rs:309:31
|
Expand Down Expand Up @@ -55,20 +55,20 @@ nom = { version = "^2.0.0", features = ["verbose-errors"] }

If you only use `Err::Code` to make your custom error codes, you could switch to the simple errors, since it replaces the `Err<Input,E=u32>` enum, which contained an `ErrorKind<E=u32>`, with the `ErrorKind<E=u32>` type directly.

## the eof function was removed
## The eof function was removed

The eof implementation was linked too much to the input type. This is now a macro combinator, called `eof!()`.

If you see the following error, remove the `eof` import and replace all `eof` calls by `eof!()`.
```ìgnore
```
error[E0432]: unresolved import `nom::eof`
--> src/parser.rs:1:20
|
1 | use nom::{IResult, eof, line_ending, not_line_ending, space};
| ^^^ no `eof` in `nom`. Did you mean to use `eol`?
```

## parsers returning `Incomplete` instead of an error on empty input
## Parsers returning `Incomplete` instead of an error on empty input

`alpha`, `digit`, `alphanumeric`, `hex_digit`, `oct_digit`, `space`, `multispace`, `sized_buffer` will now return `Incomplete` if they get an empty input. If you get the following error message, you can wrap those calls with `complete!`, a combinator that transforms `Incomplete` to `Error`.

Expand All @@ -83,7 +83,7 @@ This change was implemented to make these basic parsers more consistent. Please

The input types must now conform to a trait which requires changes to `take_till!`. If you get the following error:

```ignore
```
error[E0308]: mismatched types
--> src/linux/parser.rs:32:1
|
Expand Down Expand Up @@ -112,7 +112,7 @@ The "length-value" pattern usually indicates that we get a length from the input
- the new `length_value!` macros takes a slice of the size obtained by the first child parser, then applies the second child parser on this slice. If the second parser returns incomplete, the parser fails
- `length_data!` gets a length from its child parser, then returns a subslice of that length

```ignore
```
error[E0308]: mismatched types
--> src/tls.rs:378:37
|
Expand Down Expand Up @@ -156,15 +156,15 @@ It is fixed by:
map_res!(flat_map!(take_s!(1), digit), FromStr::from_str))));
```

## the `offset()` method was moved to the `Offset` trait
## The `offset()` method was moved to the `Offset` trait

There is now an implementation of `Offset` for `&str`. The `HexDisplay` trait is now reserved for `&[u8]`.

## `AsChar::is_0_to_9` is now `AsChar::is_dec_digit`

This makes the method naming more consistent.

## the number parsing macros with configurable endianness now take an enum as argument instead of a boolean
## The number parsing macros with configurable endianness now take an enum as argument instead of a boolean

Using a boolean to specify endianness was confusing, there is now the `nom::Endianness` enum:

Expand All @@ -175,6 +175,6 @@ Using a boolean to specify endianness was confusing, there is now the `nom::Endi
+ named!(le_tst32<u32>, u32!(Endianness::Little));
```

## end of line parsing
## End of line parsing

There were different, incompatible ways to parse line endings. Now, the `eol`, `line_ending` and `not_line_ending` all have the same behaviour. First, test for '\n', then if it is not the right character, test for "\r\n". This fixes the length issues.
24 changes: 12 additions & 12 deletions doc/upgrading_to_nom_4.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,13 +6,13 @@ The nom 4.0 is a nearly complete rewrite of nom's internal structures, along wit

Previous versions of nom all generated parsers with the following signature:

```rust,ignore
```rust
fn parser(input: I) -> IResult<I,O> { ... }
```

With the following definition for `IResult`:

```rust,ignore
```rust
pub enum IResult<I,O,E=u32> {
/// remaining input, result value
Done(I,O),
Expand Down Expand Up @@ -47,7 +47,7 @@ pub enum Err<P,E=u32>{

The new design uses the `Result` type from the standard library:

```rust,ignore
```rust
pub type IResult<I, O, E = u32> = Result<(I, O), Err<I, E>>;

pub enum Err<I, E = u32> {
Expand Down Expand Up @@ -107,7 +107,7 @@ result (like in a unit test), you will have to perform the following changes:

For the correct result case:

```rust,ignore
```rust
IResult::Done(i, o)

// becomes
Expand All @@ -118,15 +118,15 @@ Ok((i, o))
For the error case (note that argument position for `error_position` and other sibling macros was changed
for the sake of consistency with the rest of the code):

```rust,ignore
```rust
IResult::Error(error_position!(ErrorKind::OneOf, input)),

// becomes

Err(Err::Error(error_position!(input, ErrorKind::OneOf)))
```

```rust,ignore
```rust
IResult::Incomplete(Needed::Size(1))

// becomes
Expand All @@ -137,7 +137,7 @@ Err(Err::Incomplete(Needed::Size(1)))
For pattern matching, you now need to handle the `Failure` case as well, which works like the error
case:

```rust,ignore
```rust
match result {
Ok((remaining, value)) => { ... },
Err(Err::Incomplete(needed)) => { ... },
Expand Down Expand Up @@ -188,23 +188,23 @@ Thanks to the new `AtEof` trait for input types, nom now provides the `CompleteB
With these types, no need to put a `complete!()` combinator everywhere, you can juste apply those types
like this:

```rust,ignore
```rust
named!(parser<&str,ReturnType>, ... );

// becomes

named!(parser<CompleteStr,ReturnType>, ... );
```

```rust,ignore
```rust
named!(parser<&str,&str>, ... );

// becomes

named!(parser<CompleteStr,CompleteStr>, ... );
```

```rust,ignore
```rust
named!(parser, ... );

// becomes
Expand All @@ -214,7 +214,7 @@ named!(parser<CompleteByteSlice,CompleteByteSlice>, ... );

And as an example, for a unit test:

```rust,ignore
```rust
assert_eq!(parser("abcd123"), Ok(("123", "abcd"));

// becomes
Expand All @@ -228,7 +228,7 @@ empty line or not, as seen in [one of the examples](https://github.com/Geal/nom/
If those types feel a bit long to write everywhere in the parsers, it's possible
to alias them like this:

```rust,ignore
```rust
use nom::types::CompleteByteSlice as Input;
```

Expand Down

0 comments on commit 87d7fe1

Please sign in to comment.