interview-gaming-lab
- Docker desktop
- Dockerhub Token
- Trivy
- ACT
- Python
Run the following command
python gaming-app.py -u "https://news.ycombinator.com/" "https://fr.pinterest.com/" -o stdout
output:
Success: The URL scheme 'https' is allowed.
https://news.ycombinator.com/vote?id=42267725&how=up&goto=news
https://news.ycombinator.com/from?site=adamkarvonen.github.io
https://news.ycombinator.com/vote?id=42263178&how=up&goto=news
https://www.myphotos.site
https://www.ycombinator.com/apply/
https://news.ycombinator.com/item?id=42234323
https://news.ycombinator.com/hide?id=42242782&goto=news
https://news.ycombinator.com/user?id=signa11
https://news.ycombinator.com/from?site=newyorker.com
https://news.ycombinator.com/vote?id=42268580&how=up&goto=news
https://news.ycombinator.com/vote?id=42236903&how=up&goto=news
https://news.ycombinator.com/hide?id=42242392&goto=news
https://news.ycombinator.com/item?id=42271078
https://news.ycombinator.com/from?site=soniajoseph.ai
https://news.ycombinator.com/vote?id=42264427&how=up&goto=news
https://news.ycombinator.com/hide?id=42244791&goto=news
https://news.ycombinator.com/from?site=youtube.com
https://news.ycombinator.com/hide?id=42272927&goto=news
https://news.ycombinator.com/user?id=hyperknot
https://news.ycombinator.com/front
https://news.ycombinator.com/vote?id=42238603&how=up&goto=news
https://github.com/ilevd/cwp
https://news.ycombinator.com/user?id=pseudolus
https://news.ycombinator.com/user?id=leecoursey
https://news.ycombinator.com/from?site=myphotos.site
https://news.ycombinator.com/newsfaq.html
https://news.ycombinator.com/user?id=MrBuddyCasino
https://news.ycombinator.com/ask
https://news.ycombinator.com/item?id=42235718
https://news.ycombinator.com/from?site=theguardian.com
https://news.ycombinator.com/vote?id=42242782&how=up&goto=news
https://news.ycombinator.com/item?id=42228518
https://news.ycombinator.com/user?id=sebg
https://news.ycombinator.com/newcomments
https://news.ycombinator.com/user?id=gernest
https://news.ycombinator.com/vote?id=42260481&how=up&goto=news
https://news.ycombinator.com/item?id=42242392
https://news.ycombinator.com/item?id=42264427
https://news.ycombinator.com/hide?id=42236103&goto=news
https://news.ycombinator.com/vote?id=42272927&how=up&goto=news
https://news.ycombinator.com/item?id=42272927
https://news.ycombinator.com/item?id=42270389
https://news.ycombinator.com/item?id=42270424
https://news.ycombinator.com/item?id=42236903
https://ContinueAndPersist.org
https://harpers.org/archive/2024/12/in-the-rockets-red-glare-rachel-kushner/
https://news.ycombinator.com/hide?id=42268580&goto=news
https://www.soniajoseph.ai/multimodal-interpretability-in-2024/
https://interactionmagic.com/UX-LEGO-Interfaces/
https://news.ycombinator.com/user?id=bribri
https://news.ycombinator.com/lists
https://news.ycombinator.com/hide?id=42271078&goto=news
https://github.com/vinceanalytics/vince
https://www.welivesecurity.com/en/eset-research/bootkitty-analyzing-first-uefi-bootkit-linux/
https://news.ycombinator.com/hide?id=42234323&goto=news
https://news.ycombinator.com/user?id=f1shy
https://news.ycombinator.com/from?site=revolvermag.com
https://news.ycombinator.com/item?id=42242782
https://news.ycombinator.com/hide?id=42249609&goto=news
https://adamkarvonen.github.io/machine_learning/2024/06/11/sae-intuitions.html
https://onlinelibrary.wiley.com/doi/10.1111/maps.14288
https://news.ycombinator.com/vote?id=42242392&how=up&goto=news
https://news.ycombinator.com/user?id=tonghohin
https://news.ycombinator.com/hide?id=42236903&goto=news
https://techcrunch.com/2024/11/20/pdf-to-brainrot-study-tools-are-a-strange-iteration-on-a-tiktok-trend/
https://news.ycombinator.com/item?id=42264344
https://news.ycombinator.com/user?id=ZeljkoS
https://news.ycombinator.com/vote?id=42270424&how=up&goto=news
https://github.com/HackerNews/API
https://news.ycombinator.com/item?id=42268310
https://news.ycombinator.com/item?id=42269227
https://news.ycombinator.com/vote?id=42272813&how=up&goto=news
https://news.ycombinator.com/from?site=borgo-lang.github.io
https://news.ycombinator.com/from?site=alvaroduran.com
https://news.ycombinator.com/user?id=chmaynard
https://news.ycombinator.com/user?id=ilevd
https://news.ycombinator.com/item?id=42267725
https://news.ycombinator.com/user?id=zeristor
https://news.ycombinator.com/user?id=bookofjoe
https://news.ycombinator.com/user?id=tuumi
https://news.ycombinator.com/from?site=github.com/ilevd
https://news.ycombinator.com/security.html
https://news.ycombinator.com/item?id=42236103
https://news.ycombinator.com/from?site=github.com/tonghohin
https://news.ycombinator.com/vote?id=42269897&how=up&goto=news
https://news.ycombinator.com/vote?id=42234323&how=up&goto=news
https://news.ycombinator.com/hide?id=42268310&goto=news
https://121gigawatts.org/copper-sushi-power-flow-european-grid/
https://news.ycombinator.com/hide?id=42228518&goto=news
https://developer.spotify.com/blog/2024-11-27-changes-to-the-web-api
https://news.ycombinator.com/hide?id=42264345&goto=news
https://arxiv.org/abs/2411.16544
https://news.ycombinator.com/vote?id=42236103&how=up&goto=news
https://news.ycombinator.com/from?site=121gigawatts.org
https://www.ycombinator.com/legal/
https://news.ycombinator.com/vote?id=42264345&how=up&goto=news
https://news.ycombinator.com/item?id=42269897
https://news.ycombinator.com/item?id=42265533
https://news.ycombinator.com/user?id=adnanaga
https://news.ycombinator.com/from?site=neh.gov
https://news.ycombinator.com/hide?id=42270389&goto=news
https://www.theguardian.com/lifeandstyle/2024/nov/25/new-start-after-60-hummingbird-for-people-with-dementia-volunteering-care-home
https://news.ycombinator.com/vote?id=42271078&how=up&goto=news
https://news.ycombinator.com/from?site=continueandpersist.org
https://news.ycombinator.com/hide?id=42272813&goto=news
https://news.ycombinator.com/hide?id=42263178&goto=news
https://news.ycombinator.com/from?site=wiley.com
https://news.ycombinator.com/hide?id=42268304&goto=news
https://news.ycombinator.com/item?id=42272351
https://news.ycombinator.com/vote?id=42268304&how=up&goto=news
https://news.ycombinator.com
https://news.ycombinator.com/user?id=quincepie
https://news.ycombinator.com/from?site=imaginarytext.ca
https://news.ycombinator.com/vote?id=42272351&how=up&goto=news
https://www.theguardian.com/world/2024/nov/29/new-zealand-navy-ship-hmnzs-manawanui-sinking-autopilot-inquiry-ntwnfb
https://news.ycombinator.com/user?id=delichon
https://borgo-lang.github.io/
https://news.ycombinator.com/vote?id=42244791&how=up&goto=news
https://news.ycombinator.com/item?id=42263178
https://news.ycombinator.com/user?id=fagnerbrack
https://news.ycombinator.com/hide?id=42265533&goto=news
https://news.ycombinator.com/user?id=prismatic
https://news.ycombinator.com/from?site=github.com/vinceanalytics
https://news.ycombinator.com/vote?id=42270389&how=up&goto=news
https://www.youtube.com/watch?v=Fe1wYwGcjlo
https://news.ycombinator.com/from?site=lwn.net
https://news.ycombinator.com/hide?id=42269897&goto=news
https://news.ycombinator.com/hide?id=42238603&goto=news
https://news.ycombinator.com/hide?id=42264427&goto=news
https://news.ycombinator.com/item?id=42268461
https://www.theguardian.com/news/2024/nov/28/great-abandonment-what-happens-natural-world-people-disappear-bulgaria
https://news.ycombinator.com/from?site=neugierig.org
https://news.ycombinator.com/item?id=42268304
https://news.ycombinator.com/?p=2
https://news.ycombinator.com/item?id=42215520
https://www.revolvermag.com/music/author-punisher-how-tool-approved-industrial-metal-mad-scientist-found-hope-amid-doom/
https://www.newyorker.com/magazine/2024/12/02/naples-1925-martin-mittelmeier-book-review
https://news.ycombinator.com/vote?id=42249609&how=up&goto=news
https://news.ycombinator.com/user?id=apsec112
https://news.ycombinator.com/hide?id=42268461&goto=news
https://news.alvaroduran.com/p/engineers-do-not-get-to-make-startup
https://news.ycombinator.com/item?id=42249609
https://news.ycombinator.com/vote?id=42215520&how=up&goto=news
https://news.ycombinator.com/vote?id=42228518&how=up&goto=news
mailto:[email protected]
https://news.ycombinator.com/hide?id=42260481&goto=news
https://news.ycombinator.com/newest
https://news.ycombinator.com/hide?id=42215520&goto=news
https://cacm.acm.org/research-highlights/technical-perspective-mirror-mirror-on-the-wall-what-is-the-best-topology-of-them-all/
https://news.ycombinator.com/item?id=42260481
https://news.ycombinator.com/from?site=welivesecurity.com
https://news.ycombinator.com/show
https://news.ycombinator.com/item?id=42268580
https://www.neh.gov/humanities/2018/winter/feature/the-thinker-who-believed-in-doing-0
https://news.ycombinator.com/item?id=42238603
https://news.ycombinator.com/vote?id=42264344&how=up&goto=news
https://news.ycombinator.com/hide?id=42235718&goto=news
https://github.com/tonghohin/screen-sharing
https://imaginarytext.ca/posts/2024/pandoc-typst-tutorial/
https://lwn.net/SubscriberLink/998793/6c8d00bd1b2a7948/
https://news.ycombinator.com/user?id=Caiero
https://neugierig.org/software/blog/2020/05/ninja.html
https://news.ycombinator.com/vote?id=42265533&how=up&goto=news
https://news.ycombinator.com/vote?id=42235718&how=up&goto=news
https://news.ycombinator.com/from?site=harpers.org
https://news.ycombinator.com/item?id=42264345
https://news.ycombinator.com/news
https://news.ycombinator.com/item?id=42244791
https://news.ycombinator.com/vote?id=42269227&how=up&goto=news
https://news.ycombinator.com/user?id=hggh
https://news.ycombinator.com/from?site=spotify.com
https://lwn.net/Articles/997094/
https://news.ycombinator.com/hide?id=42269227&goto=news
https://news.ycombinator.com/hide?id=42264344&goto=news
https://news.ycombinator.com/user?id=doener
https://news.ycombinator.com/hide?id=42270424&goto=news
https://news.ycombinator.com/from?site=techcrunch.com
https://news.ycombinator.com/hide?id=42267725&goto=news
https://news.ycombinator.com/user?id=gabigrin
https://news.ycombinator.com/login?goto=news
https://news.ycombinator.com/user?id=rcdemski
https://news.ycombinator.com/from?site=interactionmagic.com
https://news.ycombinator.com/vote?id=42268310&how=up&goto=news
https://news.ycombinator.com/user?id=9dev
https://news.ycombinator.com/hide?id=42272351&goto=news
https://news.ycombinator.com/from?site=acm.org
https://news.ycombinator.com/jobs
https://news.ycombinator.com/user?id=laurenth
https://news.ycombinator.com/item?id=42272813
https://news.ycombinator.com/newsguidelines.html
https://news.ycombinator.com/vote?id=42268461&how=up&goto=news
https://news.ycombinator.com/from?site=arxiv.org
https://news.ycombinator.com/submit
https://news.ycombinator.com/user?id=rbanffy
Success: The URL scheme 'https' is allowed.
http://enable-javascript.com/
Run the following command
python gaming-app.py -u "https://news.ycombinator.com/" "https://fr.pinterest.com/" -o json
output:
Success: The URL scheme 'https' is allowed.
Success: The URL scheme 'https' is allowed.
{
"https://news.ycombinator.com": [
"/user",
"/user",
"/user",
"/hide",
"/submit",
"/news/2024/nov/28/great-abandonment-what-happens-natural-world-people-disappear-bulgaria",
"/vote",
"/item",
"/item",
"/user",
"/from",
"/from",
"/item",
"/lifeandstyle/2024/nov/25/new-start-after-60-hummingbird-for-people-with-dementia-volunteering-care-home",
"/ask",
"/watch",
"/from",
"/hide",
"/user",
"/vote",
"/vote",
"/user",
"/from",
"/hide",
"/tonghohin/screen-sharing",
"",
"/item",
"/hide",
"/item",
"/vote",
"[email protected]",
"/hide",
"/from",
"/from",
"/security.html",
"/user",
"",
"/from",
"/user",
"/from",
"/item",
"/vote",
"/hide",
"/item",
"/abs/2411.16544",
"/user",
"/hide",
"/from",
"/item",
"/hide",
"/from",
"/HackerNews/API",
"/world/2024/nov/29/new-zealand-navy-ship-hmnzs-manawanui-sinking-autopilot-inquiry-ntwnfb",
"/from",
"/vote",
"/item",
"/item",
"/user",
"/from",
"/software/blog/2020/05/ninja.html",
"/hide",
"/jobs",
"/item",
"/item",
"/vote",
"/vote",
"/hide",
"/newcomments",
"/hide",
"/vote",
"/hide",
"/item",
"/lists",
"/hide",
"/login",
"/vote",
"/vote",
"/from",
"/from",
"/from",
"/from",
"/item",
"/item",
"/user",
"/hide",
"/copper-sushi-power-flow-european-grid/",
"/music/author-punisher-how-tool-approved-industrial-metal-mad-scientist-found-hope-amid-doom/",
"/hide",
"/item",
"",
"/UX-LEGO-Interfaces/",
"/hide",
"/humanities/2018/winter/feature/the-thinker-who-believed-in-doing-0",
"/hide",
"/from",
"/item",
"/posts/2024/pandoc-typst-tutorial/",
"/hide",
"/hide",
"/item",
"/vote",
"/item",
"/doi/10.1111/maps.14288",
"/item",
"/vote",
"/vote",
"/vote",
"/vote",
"/research-highlights/technical-perspective-mirror-mirror-on-the-wall-what-is-the-best-topology-of-them-all/",
"/vote",
"/vote",
"/vote",
"/user",
"/user",
"/item",
"/item",
"/user",
"/vote",
"/vote",
"/from",
"/news",
"/vote",
"/multimodal-interpretability-in-2024/",
"/user",
"/vote",
"/blog/2024-11-27-changes-to-the-web-api",
"/vote",
"/user",
"/magazine/2024/12/02/naples-1925-martin-mittelmeier-book-review",
"/user",
"/vote",
"/hide",
"/hide",
"/item",
"/show",
"/2024/11/20/pdf-to-brainrot-study-tools-are-a-strange-iteration-on-a-tiktok-trend/",
"/vote",
"/item",
"/vote",
"/archive/2024/12/in-the-rockets-red-glare-rachel-kushner/",
"/en/eset-research/bootkitty-analyzing-first-uefi-bootkit-linux/",
"/newest",
"/",
"/hide",
"/user",
"/Articles/997094/",
"/p/engineers-do-not-get-to-make-startup",
"/user",
"/from",
"/user",
"/user",
"/vote",
"/vote",
"/item",
"/machine_learning/2024/06/11/sae-intuitions.html",
"/apply/",
"/from",
"/from",
"/item",
"/from",
"/hide",
"/item",
"/hide",
"/ilevd/cwp",
"/hide",
"/hide",
"/hide",
"/user",
"/from",
"/from",
"/from",
"/from",
"/user",
"/vote",
"/legal/",
"/front",
"/item",
"/vinceanalytics/vince",
"/item",
"/user",
"/user",
"/hide",
"/user",
"/user",
"/user",
"/item",
"/SubscriberLink/998793/6c8d00bd1b2a7948/",
"/hide",
"/hide",
"/user",
"/",
"/newsfaq.html",
"/newsguidelines.html",
"/user"
],
"https://fr.pinterest.com": [
"/"
]
}
Run the following command to run the docker image locally.
docker buildx build --no-cache -t gaming-app -f Dockerfile .
Run the following command to test the docker image locally.
docker run -it gaming-app -u https://news.ycombinator.com -o json
Run the following command to scan the docker image locally
trivy image gaming-app --output gaminapp-scan.file
The output of the scan is available at gaminapp-scan.file The scan clearly indicate that 2 library (requests and setuptools) used by the aplication are need to be updated as they facing security vulnerabilities.
I used ACT to run the github workflow locally so please ignore he error in the github action tab since it was never intended to run up there since the cluster is set locally.
The gaming-app-secrets.env file contain the necessary secret to make the workflow run correctly.
To work properly, the workflow need some credential:
- DOCKERHUB_USERNAME
- DOCKERHUB_TOKEN
Run the following command to run the workflow locally
First copy your kubeconfig file to local repository. ACT only got access to the repository where the .git directory is located.
cp ~/.kube/config .
Then run the workflow
act --network host --secret-file gaming-app-secrets.env
View the Github-action execution recording
To sanitize the domains list I used sed and AWK
⚠️ If you are using Linux based system: The AWK and SED command might produce different result!
sed -E 's#^(https?://)?(www\.)?##; s#\.$##; s#(.*\.)?([a-zA-Z0-9-]+\.[a-zA-Z]{2,})#\2#;' domains.txt | tr '[:upper:]' '[:lower:]' | sed 's/facebok/facebook/g' | sort | uniq
- Remove http()s and www protocal.
s#^(https?://)?(www\.)?##; s#\.$## - Sanitize the FQDN which end with a ".", its indicate the absolute path from the root DNS.
s#\.$s#\.$## - Remove the subdomain
s#\.$##; s#(.*\.)?([a-zA-Z0-9-]+\.[a-zA-Z]{2,})#\2# - Print url in lowercase
tr '[:upper:]' '[:lower:]' - Fixing missplelling
s/facebok/facebook/g - Remove duplication
sort | uniq
awk -F[/:] '{print tolower($NF)}' domains.txt | awk '{gsub(/\.$/, ""); print}' | awk '{gsub(/facebok/, "facebook"); print}' | awk -F. '{print $(NF-1)"."$NF}' | sort | uniq
- Remove the protocol that and print the url in lowercase letter.
awk -F[/:] '{print tolower($NF)}' - Remove the possible "." at the end of the url which is only use by the root DNS server.
awk '{gsub(/\.$/, ""); print}' - Replace facebok by facebook.
awk '{gsub(/facebok/, "facebook"); print}' - Extract the 2 last field of a domain divided by dots.
awk -F. '{print $(NF-1)"."$NF}' - Remove duplication after sorting to avoid the non consecutive duplication issue.
sort | uniq
In the Docker and security scan tasks, there is concepts related to container security, such as managing privilege escalation and enforcing access control. The principle of least privilege is respected, ensuring that the Docker container is run by a non-root user to limit potential vulnerabilities.
Automation and CI/CD refer to the processes of automating the build, testing, and deployment of the application. The automation and CD is done via github action only While the CI is done via github repository.
The task of extracting and sanitizing domains from URLs requires an understanding of regular expressions which is a subbranch of text processing.
The application can organize the data differently wether trough a json format of simply by a filestream wich is the way linux handle stdout.