Adding xgrammar

[zbowling]

Checklist

• Title of this PR is meaningful: e.g. "Adding my_nifty_package", not "updated meta.yaml".
• License file is packaged (see here for an example).
• Source is from official source.
• Package does not vendor other packages. (If a package uses the source of another package, they should be separate packages or the licenses of all packages need to be packaged).
• If static libraries are linked in, the license of the static library is packaged.
• Package does not ship static libraries. If static libraries are needed, follow CFEP-18.
• Build number is 0.
• A tarball (url) rather than a repo (e.g. git_url) is used in your recipe (see here for more details).
• GitHub users listed in the maintainer section have posted a comment confirming they are willing to be listed there.
• When in trouble, please check our knowledge base documentation before pinging a team.

[github-actions]

Hi! This is the staged-recipes linter and your PR looks excellent! 🚀

[conda-forge-admin]

Hi! This is the friendly automated conda-forge-linting service.

I wanted to let you know that I linted all conda-recipes in your PR (recipes/xgrammar/recipe.yaml) and found some lint.

Here's what I've got...

For recipes/xgrammar/recipe.yaml:

• ❌ The homepage item is expected in the about section.
• ❌ The summary item is expected in the about section.

For recipes/xgrammar/recipe.yaml:

• ℹ️ No valid build backend found for Python recipe for package xgrammar using pip. Python recipes using pip need to explicitly specify a build backend in the host section. If your recipe has built with only pip in the host section in the past, you likely should add setuptools to the host section of your recipe.

_{This message was generated by GitHub Actions workflow run https://github.com/conda-forge/conda-forge-webservices/actions/runs/13209596163. Examine the logs at this URL for more detail.}

[conda-forge-admin]

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipes/xgrammar/recipe.yaml) and found it was in an excellent condition.

I do have some suggestions for making it better though...

For recipes/xgrammar/recipe.yaml:

• ℹ️ No valid build backend found for Python recipe for package xgrammar using pip. Python recipes using pip need to explicitly specify a build backend in the host section. If your recipe has built with only pip in the host section in the past, you likely should add setuptools to the host section of your recipe.

_{This message was generated by GitHub Actions workflow run https://github.com/conda-forge/conda-forge-webservices/actions/runs/13209621265. Examine the logs at this URL for more detail.}

[conda-forge-admin]

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipes/xgrammar/recipe.yaml) and found it was in an excellent condition.

[zbowling]

@conda-forge/help-python-c @conda-forge/help-python This should be ready for review now.

[maresb]

Nice!!!

Is it problematic to be vendoring these? https://github.com/mlc-ai/xgrammar/tree/v0.1.8/3rdparty

[zbowling]

Is it problematic to be vendoring these?

Already handled. With dlpack I get it from conda and use the -Isystem line to fix the included since it's not included in the tarball because of the submodule so it works. It's a header only include that sets up a shared struct that has to be forward abi compatible from any version with any other library that uses it. The other is googletest but since we are not running tests we don't need it.

[maresb]

Oh very nice!!! Thanks a lot for the explanation!

What about picojson? Should that be handled similarly to here?

I don't do C, and I was looking at this before, so it's really educational for me.

[zbowling]

Ah, good catch. Because it wasn't a submodule, I didn't think much of it. Header-only includes libs are funny things since they don't link anything, statically or dynamically. The gain by unvendoring in that case is only that if the source is fixed in some way all the dependent packages can get rebuilt.

That other case in that other feedstock was funny in that it was trying to prevent picking up the unvendored version so I don't know what they were doing there. Maybe it's a case that they conditionally not use it or maybe it's used a feature also disabled.

[zbowling]

Ok, I dug into this after trying to use the conda-forge version. It turns out that picojson stopped being maintained in 2015, and now a series of hard forks exist across a dozen or so projects. A whole bunch of ML frameworks seem to be copying different updated versions from each other, making their own changes, and they are all slightly incompatible with each other now. So that explains why jwt-cpp-feedstock explicitly doesn't want to use the conda-forge version. Since it's a header only include you can just make it work by disabling warnings without disabling them everywhere.

This looks like one of the latest forks folks are copying from, but the upstream project is dead-dead and has compile issues with modern C++ compilers because of const correctness.
https://github.com/apache/tvm/tree/main/3rdparty/picojson

So, in this case, we can look at it as a fork and not a vendor version for the sake of being vendored.

The upstream project is graveyard of 10+ years of ignored PRs to fix it and these changes all exist in everyone else's forks: https://github.com/kazuho/picojson/pulls?q=sort%3Aupdated-desc+is%3Apr+is%3Aopen