Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Define TORCHAUDIO_API for better visbility control #2504

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

mthrok
Copy link
Collaborator

@mthrok mthrok commented Jun 24, 2022

Context:

TorchAudio uses dual-binding (PyBind11 and TorchBind) to make custom operations available in Python.
The both binding eventually calls the same implementation contained in libtorchaudio[_XXX].so.
The ones bound via PyBind11 (the ones in torchtext._torchaudio[_XXX]) calls into libtorch[_XXX].so.

(The equivalent illustration from pytorch/text#1806)
Untitled drawing

This means that libtorchaudio[_XXX].so has to make the symbols (APIs) used by torchtext._torchtaudio[_XXX] visible.

However, the default visibility of symbols in shared libraries are different in Windows.
On Windows all the symbols are by default hidden. To work around this, we use CMAKE_WINDOWS_EXPORT_ALL_SYMBOLS to expose all the symbols.
There is an upper limit of visible symbols one library fine can contain, and this can be problematic in the future.

A better approach is to selectively mark the symbols that should be visible as visible.

Summary of the change set

This commit introduces TORCHAUDIO_API macro which annotates functions with proper visibility.
The core logic was taken from https://github.com/pytorch/pytorch/blob/bcc02769bef1d7b89bec724223284958b7c5b564/c10/macros/Export.h

The behavior is as follow;

For non-Windows: It is always __attribute__((__visibility__("default")))
For Windows:
If the header is included from the compilation unit of libtorchtext, then it resolves to __declspec(dllexport). otherwise it resolves to __declspec(dllimport).

This allows to remove CMAKE_WINDOWS_EXPORT_ALL_SYMBOLS.

@facebook-github-bot
Copy link
Contributor

@mthrok has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

@facebook-github-bot
Copy link
Contributor

@mthrok has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

@mthrok mthrok force-pushed the public-api branch 2 times, most recently from 9e06448 to d219166 Compare July 5, 2022 17:32
@facebook-github-bot
Copy link
Contributor

Hi @mthrok!

Thank you for your pull request.

We require contributors to sign our Contributor License Agreement, and yours needs attention.

You currently have a record in our system, but the CLA is no longer valid, and will need to be resubmitted.

Process

In order for us to review and merge your suggested changes, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA.

Once the CLA is signed, our tooling will perform checks and validations. Afterwards, the pull request will be tagged with CLA signed. The tagging process may take up to 1 hour after signing. Please give it that time before contacting us about it.

If you have received this in error or have any questions, please contact us at [email protected]. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants