This project offers an API for vectorizing images using OpenAI's CLIP model via BentoML and integrates Milvus for efficient vector storage and search

BentoML Service for CLIP Image Vectorization

Deploy BentoML to ECS on AWS using Terraform to create vectors quickly and easily.

Using Debezium to send Change Data Capture (CDC) events to HTTP endpoints

Run an Nginx service in an exsiting ECS Cluster using Terraform

Using Terraform to create an OpenSearch Ingestion pipeline with Amazon DynamoDB

A proxy to sit between an application an AWS Opensearch domain to log queries submitted to OpenSearch for debugging

Using Terraform to notify SQS when an object lands in S3

Create an AWS RDS instance, generate and store its admin password in AWS Secrets manager

This project sets up an AWS infrastructure including a VPC with public and private subnets, an ECS cluster, and associated resources for running containerized applications.

Create a VPC on AWS with public and private subnets using Terraform

Using Docker compose to start up Triton with the OpenClip model, to encode text in to vectors

An example of using Ansible to install a package on an EC2 instance using SSM

Using Packer, Ansible and Terraform to create a small Triton Inference cluster on AWS

A lambda function to list s3 buckets and their size

Using Terraform to create a Serverless OpenSearch cluster on AWS

Some useful scripts for reviewing AWS costs

Docker compose for trying out Debezium, RedPanda and RedPanda Console UI

Using Debezium with WarpStream as a Kafka alternative for CDC

Using OpenTofu to create an Apache Flink cluster, with Task Managers in an auto scaling group of Spot instances to help reduce costs

Using Open Tofu to create an AWS API gateway that submits data to Kinesis

A working example of using Debezium for CDC while excluding some columns to prevent consumption of personally private information (PII)

Trying out Apache Paimon with Apache Flink using Docker Compose

Using Apache Flink to store data in S3 using Apache Hudi

Using Apache Flink to write to s3 in Apache Iceberg format

Trying out Apache Doris for fast log analysis

Trying out Apache Flink with Apache Paimon and Apache SeaTunnel

A Python script to import my Github projects to Teamwork and log time for each commit