Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Python and debian packages #203

Open
wants to merge 31 commits into
base: master
Choose a base branch
from
Open
Changes from 1 commit
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
8f57f67
preparation for python packages
N-Maas Nov 27, 2024
44e9496
CI for python wheels
N-Maas Nov 27, 2024
53ffea3
python build: install hwloc
N-Maas Nov 28, 2024
08c3f66
new handling of TBB installation
N-Maas Nov 28, 2024
bd9b79a
CI simplification
N-Maas Nov 28, 2024
d80a637
extended build
N-Maas Nov 28, 2024
91b4b52
new attempt
N-Maas Nov 28, 2024
d3d5571
macos
N-Maas Nov 28, 2024
d8dec7a
dyld_path
N-Maas Nov 28, 2024
88bad0b
we can't do 32 bit
N-Maas Nov 29, 2024
eae7a84
allow disabling hwloc
N-Maas Nov 29, 2024
f90e152
python build on release
N-Maas Dec 3, 2024
bf1bb8b
general guard for installation targets
N-Maas Dec 3, 2024
5b411f4
build debian package
N-Maas Dec 3, 2024
b6324c5
debian package ci
N-Maas Dec 3, 2024
78826a5
try get newer tbb
N-Maas Dec 3, 2024
ea9658a
new attempt
N-Maas Dec 3, 2024
09d21c2
correct package registry
N-Maas Dec 4, 2024
45a8795
python build extended instructions
N-Maas Dec 4, 2024
8117d55
try a full python run
N-Maas Dec 4, 2024
19eaed0
test pypi deployment
N-Maas Dec 4, 2024
4a3d39c
pyproject.toml
N-Maas Dec 4, 2024
fa4265b
and the dependencies
N-Maas Dec 4, 2024
e32150a
artifact names
N-Maas Dec 4, 2024
378fcfd
metadata and license update
N-Maas Dec 4, 2024
a4d1b38
full python ci
N-Maas Dec 4, 2024
87b59f1
automatic upload of debian packages?
N-Maas Dec 4, 2024
4dba43e
tag instead of release as trigger
N-Maas Dec 6, 2024
2d2f12e
add pypi classifiers
N-Maas Dec 6, 2024
71886a3
change tag expressions
N-Maas Dec 16, 2024
21099a3
update contact information
N-Maas Jan 10, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
full python ci
  • Loading branch information
N-Maas committed Dec 20, 2024
commit a4d1b38516eadf786eccef222c91890ff994e671
233 changes: 117 additions & 116 deletions .github/workflows/python_build_ci.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
name: Mt-KaHyPar Python Package

on:
push
release:
types: [published]

jobs:
package_linux_x86:
Expand Down Expand Up @@ -37,114 +38,114 @@ jobs:
path: ./wheels/*.whl


# package_linux_arm_1:
# name: Python Package Linux ARM 1

# runs-on: ubuntu-24.04
# env:
# CI_ACTIVE : 1
# TBB_INSTALL_DIR: /tmp/tbb_build

# steps:
# - uses: actions/checkout@v4
# - uses: actions/setup-python@v5

# - name: Set up QEMU
# uses: docker/setup-qemu-action@v3
# with:
# platforms: all

# - name: Install Dependencies
# run: |
# sudo apt-get install libtbb-dev libhwloc-dev libboost-program-options-dev
# python3 -m pip install cibuildwheel>=2.22.0

# - name: Build Python Wheels
# env:
# CIBW_ARCHS_LINUX: aarch64
# CIBW_BUILD: cp3{9,10,11}-manylinux*
# run: |
# # run cmake to create pyproject.toml from pyproject.toml.in
# cmake -B configure_python --preset=minimal -DKAHYPAR_SETUP_PYTHON=On -DKAHYPAR_TBB_DIR=$TBB_INSTALL_DIR -DKAHYPAR_DISABLE_HWLOC=Off
# rm -rf configure_python
# python3 -m cibuildwheel --output-dir wheels

# - uses: actions/upload-artifact@v4
# with:
# name: cibw-wheels-ubuntu-24.04-arm1
# path: ./wheels/*.whl

# package_linux_arm_2:
# name: Python Package Linux ARM 2

# runs-on: ubuntu-24.04
# env:
# CI_ACTIVE : 1
# TBB_INSTALL_DIR: /tmp/tbb_build

# steps:
# - uses: actions/checkout@v4
# - uses: actions/setup-python@v5

# - name: Set up QEMU
# uses: docker/setup-qemu-action@v3
# with:
# platforms: all

# - name: Install Dependencies
# run: |
# sudo apt-get install libtbb-dev libhwloc-dev libboost-program-options-dev
# python3 -m pip install cibuildwheel>=2.22.0

# - name: Build Python Wheels
# env:
# CIBW_ARCHS_LINUX: aarch64
# CIBW_BUILD: cp3*-manylinux*
# CIBW_PROJECT_REQUIRES_PYTHON: ">=3.12"
# run: |
# # run cmake to create pyproject.toml from pyproject.toml.in
# cmake -B configure_python --preset=minimal -DKAHYPAR_SETUP_PYTHON=On -DKAHYPAR_TBB_DIR=$TBB_INSTALL_DIR -DKAHYPAR_DISABLE_HWLOC=Off
# rm -rf configure_python
# python3 -m cibuildwheel --output-dir wheels

# - uses: actions/upload-artifact@v4
# with:
# name: cibw-wheels-ubuntu-24.04-arm2
# path: ./wheels/*.whl


# package_macos:
# name: Python Package MacOS

# runs-on: macos-latest
# env:
# CI_ACTIVE : 1
# TBB_INSTALL_DIR: /tmp/tbb_build

# steps:
# - uses: actions/checkout@v4
# - uses: actions/setup-python@v5

# - name: Install Dependencies
# run: |
# brew install tbb boost pipx

# - name: Build Python Wheels
# env:
# CIBW_ARCHS_MACOS: x86_64 arm64
# CIBW_BUILD: cp3*-macos*
# # TBB doesn't compile on older MacOS than 10.13
# MACOSX_DEPLOYMENT_TARGET: 10.13
# run: |
# # run cmake to create pyproject.toml from pyproject.toml.in
# cmake -B configure_python --preset=minimal -DKAHYPAR_SETUP_PYTHON=On -DKAHYPAR_TBB_DIR=$TBB_INSTALL_DIR -DKAHYPAR_DISABLE_HWLOC=On
# rm -rf configure_python
# pipx run cibuildwheel --output-dir wheels

# - uses: actions/upload-artifact@v4
# with:
# name: cibw-wheels-macos
# path: ./wheels/*.whl
package_linux_arm_1:
name: Python Package Linux ARM 1

runs-on: ubuntu-24.04
env:
CI_ACTIVE : 1
TBB_INSTALL_DIR: /tmp/tbb_build

steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5

- name: Set up QEMU
uses: docker/setup-qemu-action@v3
with:
platforms: all

- name: Install Dependencies
run: |
sudo apt-get install libtbb-dev libhwloc-dev libboost-program-options-dev
python3 -m pip install cibuildwheel>=2.22.0

- name: Build Python Wheels
env:
CIBW_ARCHS_LINUX: aarch64
CIBW_BUILD: cp3{9,10,11}-manylinux*
run: |
# run cmake to create pyproject.toml from pyproject.toml.in
cmake -B configure_python --preset=minimal -DKAHYPAR_SETUP_PYTHON=On -DKAHYPAR_TBB_DIR=$TBB_INSTALL_DIR -DKAHYPAR_DISABLE_HWLOC=Off
rm -rf configure_python
python3 -m cibuildwheel --output-dir wheels

- uses: actions/upload-artifact@v4
with:
name: cibw-wheels-ubuntu-24.04-arm1
path: ./wheels/*.whl

package_linux_arm_2:
name: Python Package Linux ARM 2

runs-on: ubuntu-24.04
env:
CI_ACTIVE : 1
TBB_INSTALL_DIR: /tmp/tbb_build

steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5

- name: Set up QEMU
uses: docker/setup-qemu-action@v3
with:
platforms: all

- name: Install Dependencies
run: |
sudo apt-get install libtbb-dev libhwloc-dev libboost-program-options-dev
python3 -m pip install cibuildwheel>=2.22.0

- name: Build Python Wheels
env:
CIBW_ARCHS_LINUX: aarch64
CIBW_BUILD: cp3*-manylinux*
CIBW_PROJECT_REQUIRES_PYTHON: ">=3.12"
run: |
# run cmake to create pyproject.toml from pyproject.toml.in
cmake -B configure_python --preset=minimal -DKAHYPAR_SETUP_PYTHON=On -DKAHYPAR_TBB_DIR=$TBB_INSTALL_DIR -DKAHYPAR_DISABLE_HWLOC=Off
rm -rf configure_python
python3 -m cibuildwheel --output-dir wheels

- uses: actions/upload-artifact@v4
with:
name: cibw-wheels-ubuntu-24.04-arm2
path: ./wheels/*.whl


package_macos:
name: Python Package MacOS

runs-on: macos-latest
env:
CI_ACTIVE : 1
TBB_INSTALL_DIR: /tmp/tbb_build

steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5

- name: Install Dependencies
run: |
brew install tbb boost pipx

- name: Build Python Wheels
env:
CIBW_ARCHS_MACOS: x86_64 arm64
CIBW_BUILD: cp3*-macos*
# TBB doesn't compile on older MacOS than 10.13
MACOSX_DEPLOYMENT_TARGET: 10.13
run: |
# run cmake to create pyproject.toml from pyproject.toml.in
cmake -B configure_python --preset=minimal -DKAHYPAR_SETUP_PYTHON=On -DKAHYPAR_TBB_DIR=$TBB_INSTALL_DIR -DKAHYPAR_DISABLE_HWLOC=On
rm -rf configure_python
pipx run cibuildwheel --output-dir wheels

- uses: actions/upload-artifact@v4
with:
name: cibw-wheels-macos
path: ./wheels/*.whl


build_sdist:
Expand All @@ -170,20 +171,20 @@ jobs:
path: dist/*.tar.gz

upload_pypi:
needs: [package_linux_x86, build_sdist] # package_linux_arm_1, package_linux_arm_2, package_macos, build_sdist]
needs: [package_linux_x86, package_linux_arm_1, package_linux_arm_2, package_macos, build_sdist]
runs-on: ubuntu-24.04
environment: testpypi
environment: pypi
permissions:
id-token: write
# if: github.event_name == 'release' && github.event.action == 'published'
if: github.event_name == 'release' && github.event.action == 'published'
steps:
- uses: actions/download-artifact@v4
- name: Collect Python Packages
uses: actions/download-artifact@v4
with:
# unpacks all CIBW artifacts into dist/
pattern: cibw-*
path: dist
merge-multiple: true

- uses: pypa/gh-action-pypi-publish@release/v1
with:
repository-url: https://test.pypi.org/legacy/
- name: Publish to PyPI
uses: pypa/gh-action-pypi-publish@release/v1