Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Easily available custom package server #12

Merged
merged 54 commits into from
May 29, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
54 commits
Select commit Hold shift + click to select a range
a4fbe30
Begin docmenting Elm package server API
changlinli Mar 30, 2024
bbf9ae2
Some more notes on the server API
changlinli Mar 31, 2024
cc519e0
Rename file
changlinli Mar 31, 2024
de77e2a
Rough draft of a web service
changlinli Apr 10, 2024
25e0331
Get all packages endpoint working
changlinli Apr 12, 2024
06fec0b
Get inserts and gets of packages working
changlinli Apr 12, 2024
b2a0c12
Add in sqlar table
changlinli Apr 12, 2024
fd04fef
Get rid of global import statements
changlinli Apr 12, 2024
aa1a838
Change up some documentation notes
changlinli Apr 12, 2024
0ac6b55
Get file uploads working
changlinli Apr 13, 2024
e7e1ee8
Get fully working package persistence
changlinli Apr 15, 2024
1a06dbe
Add sqids
changlinli Apr 15, 2024
4328ff1
Revert "Add sqids"
changlinli Apr 15, 2024
b616573
Finish out core API
changlinli Apr 16, 2024
f14ee6f
Add in dashboard endpoints
changlinli Apr 19, 2024
6f8c484
Finish dashboard get endpoint
changlinli Apr 23, 2024
e417031
Add session tokens
changlinli May 3, 2024
b7f746b
Add login sessions
changlinli May 4, 2024
ecefcc6
Get the compiler compiling
changlinli May 10, 2024
99faa9a
Fix serialization issues
changlinli May 10, 2024
0d16b46
Finish up all major publish code
changlinli May 10, 2024
7a0d8b8
Get things working end-to-end
changlinli May 11, 2024
f6812f8
Undo accidental commit of optimization changes
changlinli May 13, 2024
0cc8798
Round out remaining security edges
changlinli May 14, 2024
2ae2e1b
Frontend commit
changlinli May 15, 2024
8622a61
Add full test that tests the server end to end
changlinli May 15, 2024
8f68d57
Add static files
changlinli May 15, 2024
d95f9a7
Add in latest changes to frontend
changlinli May 16, 2024
2d62fbf
Get frontend finally working for packages
changlinli May 17, 2024
08e229b
Finish out the first rough iteration
changlinli May 19, 2024
5d11d6a
Build custom package server
changlinli May 19, 2024
dd87160
Update release.yaml to also build the package server
changlinli May 19, 2024
a8b3146
Upgrade from GHC 9.2 to 9.4
changlinli May 19, 2024
ea282ef
Add freeze file
changlinli May 19, 2024
3a6e258
Update CUSTOM_PACKAGE_SERVER_REFERENCE_NOTES.md
changlinli May 25, 2024
b57b804
Flesh out more design notes
changlinli May 25, 2024
6a266ba
Change to actual login screen
changlinli May 25, 2024
bca6f8e
Update workflow file to produce binaries
changlinli May 28, 2024
e8703e7
Add python to GitHub actions file
changlinli May 28, 2024
6ee406f
Change to python 3
changlinli May 28, 2024
bccac97
Change to explicit python3 binary
changlinli May 28, 2024
6aa498d
Try to really make sure python3 gets installed
changlinli May 28, 2024
a419c3b
Try using python3-dev instead
changlinli May 28, 2024
9c1f15b
More informative messages for frontend
changlinli May 28, 2024
60c5608
Add npx dependency
changlinli May 28, 2024
8537805
Make sure to add dependencies in latter section
changlinli May 28, 2024
d31f0d7
ls to see what's going on
changlinli May 28, 2024
dfb00bf
Change to top level output
changlinli May 28, 2024
1d4124c
Remove intermediate output
changlinli May 28, 2024
6cf6b1b
Use cabal build instead of install
changlinli May 28, 2024
f6756a9
Add in building of the Elm files for the frontend
changlinli May 28, 2024
a38f3d8
Fix more build configuration
changlinli May 28, 2024
0389684
Fix bug with deletions
changlinli May 28, 2024
8dae97e
Skip initial cabal build
changlinli May 28, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Skip initial cabal build
This is because otherwise we'll mess with the enable executable static
flag
  • Loading branch information
changlinli committed May 28, 2024
commit 8dae97ea82b93bf0e2828865a4b1a53a83415d6f
2 changes: 1 addition & 1 deletion .github/workflows/haskell.yml
Original file line number Diff line number Diff line change
Expand Up @@ -113,7 +113,7 @@ jobs:
mkdir -p /top-level-output
cabal build --enable-executable-static # We use build instead of install because our version of Cabal doesn't handle https://github.com/haskell/cabal/pull/9697, which prevents --enble-executable-static from being used with cabal install
OUTPUT_BIN=$(cabal list-bin myPackage)
python3 example_test.py
python3 example_test.py --skip-initial-cabal-build
cp $OUTPUT_BIN /top-level-output/zokka-custom-package-server
cd elm-src
npx zokka make frontend/Main.elm --output=elm.js
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -120,7 +120,7 @@ jobs:
mkdir -p /top-level-output
cabal build --enable-executable-static # We use build instead of install because our version of Cabal doesn't handle https://github.com/haskell/cabal/pull/9697, which prevents --enble-executable-static from being used with cabal install
OUTPUT_BIN=$(cabal list-bin myPackage)
python3 example_test.py
python3 example_test.py --skip-initial-cabal-build
cp $OUTPUT_BIN /top-level-output/zokka-custom-package-server
cd elm-src
npx zokka make frontend/Main.elm --output=elm.js
Expand Down
25 changes: 19 additions & 6 deletions custom-package-server/example_test.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
import subprocess
import threading
import time
import argparse

def run_command(command):
print("Executing command:", " ".join(command))
Expand Down Expand Up @@ -94,14 +95,19 @@ def get_dashboard(login_token):
SHOULD_SHUTDOWN_SERVER = False
SQLITE_DATABASE_FILENAME = "temp_testing_package_db.db"

def run_server_in_background():
command = [
f"cabal run myPackage -- --port=3000 --database-file={SQLITE_DATABASE_FILENAME} --initialization-script=initialize_tables.sql --external-website-url=https://localhost:3000",
]
def run_server_in_background(skip_initial_cabal_build=False):
# hack until we figure out what's causing things to hang
# this ensures that we don't have to wait for things to compile and
# therefore that our sleep of 3 seconds is sufficient
run_command(["cabal", "build"])
if not skip_initial_cabal_build:
run_command(["cabal", "build"])
if not skip_initial_cabal_build:
command = [
f"cabal run myPackage -- --port=3000 --database-file={SQLITE_DATABASE_FILENAME} --initialization-script=initialize_tables.sql --external-website-url=https://localhost:3000",
]
else:
location_of_zokka_executable = run_command(["cabal", "list-bin", "myPackage"]).rstrip()
command = [f"{location_of_zokka_executable} --port=3000 --database-file={SQLITE_DATABASE_FILENAME} --initialization-script=initialize_tables.sql --external-website-url=https://localhost:3000"]
def target(setup_event, shutdown_event):
process = subprocess.Popen(command, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE,)
print(f"Executing {command}")
Expand All @@ -128,8 +134,15 @@ def delete_database_file():
return output

if __name__ == "__main__":
parser = argparse.ArgumentParser(
prog="Test Script for Zokka Custom Package",
description="Tests the Zokka custom package server with basic smoke tests",
)
parser.add_argument("-s", "--skip-initial-cabal-build", action="store_true")
args = parser.parse_args()
try:
(server_thread, server_has_setup_event, should_shutdown_server_event) = run_server_in_background()
(server_thread, server_has_setup_event, should_shutdown_server_event) =\
run_server_in_background(args.skip_initial_cabal_build)
server_has_setup_event.wait()
create_user()
user_login_token = login_as_user()
Expand Down
Loading