Skip to content

OpenBSD port tracking #78437

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
3405691582 opened this issue Jan 4, 2025 · 6 comments
Open

OpenBSD port tracking #78437

3405691582 opened this issue Jan 4, 2025 · 6 comments
Assignees
Labels

Comments

@3405691582
Copy link
Member

3405691582 commented Jan 4, 2025

Description

Instead of using the forums, use this Github issue to track OpenBSD porting status and notes that are beneficial for other people trying to build Swift on this platform but not necessarily appropriate for putting into prs yet for official documentation.

Notes

Last edit: 2025-05-17.

Building Swift from source without an existing prebuilt toolchain requires a bootstrapping process.

Install required packages:

doas pkg_add bash ninja python%3 e2fsprogs git

Save the following json file for the update-checkout script (compacted for simplicity):

{"ssh-clone-pattern":"[email protected]:%s.git","https-clone-pattern":"https://github.com/%s.git","repos":{"swift":{"remote":{"id":"swiftlang/swift"}},"llvm-project":{"remote":{"id":"swiftlang/llvm-project"}},"cmark":{"remote":{"id":"swiftlang/swift-cmark"}},"llbuild":{"remote":{"id":"swiftlang/swift-llbuild"}},"swift-build":{"remote":{"id":"swiftlang/swift-build"}},"swift-argument-parser":{"remote":{"id":"apple/swift-argument-parser"}},"swift-collections":{"remote":{"id":"apple/swift-collections"}},"swift-crypto":{"remote":{"id":"apple/swift-crypto"}},"swift-certificates":{"remote":{"id":"apple/swift-certificates"}},"swift-asn1":{"remote":{"id":"apple/swift-asn1"}},"swift-driver":{"remote":{"id":"swiftlang/swift-driver"}},"swift-toolchain-sqlite":{"remote":{"id":"swiftlang/swift-toolchain-sqlite"}},"swift-tools-support-core":{"remote":{"id":"swiftlang/swift-tools-support-core"}},"swiftpm":{"remote":{"id":"swiftlang/swift-package-manager"}},"swift-syntax":{"remote":{"id":"swiftlang/swift-syntax"}},"swift-system":{"remote":{"id":"apple/swift-system"}},"swift-corelibs-xctest":{"remote":{"id":"swiftlang/swift-corelibs-xctest"}},"swift-corelibs-foundation":{"remote":{"id":"swiftlang/swift-corelibs-foundation"}},"swift-foundation-icu":{"remote":{"id":"swiftlang/swift-foundation-icu"}},"swift-foundation":{"remote":{"id":"swiftlang/swift-foundation"}},"swift-corelibs-libdispatch":{"remote":{"id":"swiftlang/swift-corelibs-libdispatch"}},"swift-experimental-string-processing":{"remote":{"id":"swiftlang/swift-experimental-string-processing"}}},"default-branch-scheme":"main","branch-schemes":{"main":{"aliases":["swift/main","main","stable/20240723"],"repos":{"llvm-project":"stable/20240723","swift":"main","cmark":"gfm","llbuild":"main","swift-build":"main","swift-toolchain-sqlite":"1.0.1","swift-tools-support-core":"main","swiftpm":"main","swift-argument-parser":"1.4.0","swift-collections":"main","swift-crypto":"main","swift-certificates":"main","swift-asn1":"main","swift-driver":"main","swift-syntax":"main","swift-system":"main","swift-corelibs-xctest":"main","swift-corelibs-foundation":"main","swift-foundation-icu":"main","swift-foundation":"main","swift-corelibs-libdispatch":"main","swift-experimental-string-processing":"swift/main"}}}}

Check out swiftlang/swift.

Use the following update-checkout command:

./swift/utils/update-checkout --clone --config ./update-checkout-config.json --github-comment 'apple/swift-crypto#352'

Manual changes required:

  • In llvm-project on arm64 builds, apply Fix crash lowering stack guard on OpenBSD/aarch64. llvm/llvm-project#125416
  • In swift,
    • create an implementation for stdlib/public/Synchronization/Mutex.swift using pthread_mutex_t?, or use the 5.10 version of Foundation.
    • add the implementation and dependency in stdlib/public/Synchronization/CMakeLists.txt.
    • while this is not ideal to upstream, since pthread_mutex_t could be a value type with internal pointers, this should be safe enough for now since OpenBSD uses a pointer for pthread_mutex_t.
    • note the 5.10 version of swift-corelibs-foundation is the one prior to Mutex being adopted.
  • In swift,
    • edit utils/build-presets.ini: for presets bootstrap_stage0, bootstrap_stage1, and bootstrap_stage2,
    • remove lines starting llvm-install-components, install-llvm,
    • add lines skip-build-clang-tools-extra, skip-build-compiler-rt, and native_clang_tools_path=/usr/bin.
    • for bootstrap_stage2 only, add llvm-install-components=IndexStore.
    • remove compiler-rt and clangd from llvm_install_components in utils/build_swift/build_swift/defaults.py

Other manual changes not described here may be required and will eventually be upstreamed or discussed further here.

Generally, the build process will be

env ./swift/utils/build-script --preset=bootstrap_stage0 \
        build_subdir=bootstrap0 install_destdir=install0
env PATH=<path to...>/install0/usr/bin/:${PATH} ./swift/utils/build-script --preset=bootstrap_stage1 \
        build_subdir=bootstrap1 install_destdir=install1
env PATH=<path to...>/install1/usr/bin/:${PATH} ./swift/utils/build-script --preset=bootstrap_stage2 \
        build_subdir=bootstrap2 install_destdir=install2

This is not a fully refined process yet, so if you attempt these steps verbatim there may still be problems.

  • Needs investigation/known issues (roughly in priority order)
    • Blockers:
      • Currently the default configuration builds Swift with BTCFI disabled to work around a bug triggered by Concurrency. This should be remedied.
      • Dispatch queues can possibly trigger platform threading bugs that ld.so's allocator detects (but kills the program). Work around these by manually disabling threading for now.
    • Not blockers/unknown
      • A durable solution to Synchronization should be found.
      • threading can be problematic: swiftc -num-threads can be memory-hungry and cause lockups.
@3405691582 3405691582 added task triage needed This issue needs more specific labels labels Jan 4, 2025
@finagolfin finagolfin added platform support OpenBSD Platform: OpenBSD and removed triage needed This issue needs more specific labels labels Jan 6, 2025
@finagolfin
Copy link
Member

I've assigned you on this issue.

there are a number of issues with modules that prevent a successful build. Current suspicion is bad libc++ interaction with modules and/or existing upstream bugs with modules.

Have you tried turning off inline bridging mode, as currently done for Windows and as I used to do for Android?

I suggest you apply to be a committer if you plan to work on this port more. I can't say if you will get it, as I don't decide that, but based on your extensive track record, I suspect you will.

@finagolfin
Copy link
Member

Note that yams has been removed in trunk, #78826.

@3405691582
Copy link
Member Author

Have you tried turning off inline bridging mode, as currently done for Windows and as I used to do for Android?

I've experienced trouble with both PURE and INLINE bridging modes. Thankfully by turning interop off for now this sidesteps the issue and haven't been forced to deal with this problem further yet, though I likely will eventually.

@finagolfin
Copy link
Member

I've experienced trouble with both PURE and INLINE bridging modes.

OK, pure bridging also stopped working for me natively on Android last March in trunk, but worked for building 5.10. You may want to try it with 5.10 on OpenBSD and see if that works. At the very least, that could give you a working "standard toolchain" that you could use to bootstrap everything else.

Thankfully by turning interop off for now this sidesteps the issue and haven't been forced to deal with this problem further yet, though I likely will eventually.

Yes, you will need it, as more and more of the compiler is written in Swift and directly calls LLVM/Swift C++ APIs inside the compiler.

I think Swift's C++ interop doesn't work with libc++ platforms and needs to be provided for C++ interop to work.

It does work, as most Swift platforms currently use libc++, such as all Darwin platforms and Android. Egor even got it working on linux, #75589.

That said, it can be a bit flaky and required some small changes to work on Android.

(I think this is why there are errors about a missing Clang module for CxxStdlib).

That module is added in this repo, you may just need to enable OpenBSD there, as Alex did for Android.

Moreover, OpenBSD's C++ includes are missing a modulemap.

That's surprising, as libc++ has shipped with a module map for years now. If OpenBSD is not including it for some reason, try copying it over and using it.

@3405691582
Copy link
Member Author

3405691582 commented Feb 1, 2025

I have what I think is a janky standard toolchain already atop HEAD/6.2 with C++ interop turned off, but granted, the next step is testing it to make sure it actually properly incorporates the Swift compiler sources. These are good breadcrumbs for when I get back to trying to reenable interop, thanks!

@jakepetroules
Copy link
Contributor

@3405691582, thanks for your work on the OpenBSD port! We've recently open sourced Xcode's build system engine (Swift Build) which will eventually become part of the toolchain. I'm currently working on adding FreeBSD support there, and I expect OpenBSD support would be a similar change. Thought I'd mention it here so you can make sure you're aware of potential changes needed in this space. I filed an issue to track: swiftlang/swift-build#114

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

4 participants
@jakepetroules @finagolfin @3405691582 and others