This is a sample data engineering project that tries to answer the question if cow best friends also take their meals together.
This project has been used for the "Path to Production: Databricks Project CICD FOR Seamless Inner to Outer Dev Loops" talk at DATA+AI Summit 2024 and the "Bridging the Production Gap: Develop and Deploy Code Easily With IDEs" talk at DATA+AI Summit 2023.
For the demo at DAIS 2024, use the dais24-main branch for the complete code and the dais24-stageX branches for snapshots. Please use Databricks CLI v0.223.0+ to deploy DAB from a web terminal in the Workspace.
For the demo at DAIS 2023, see the main branch for the complete code and the stageX branches for snapshots
The notebook /notebooks/Generate Cow Data.ipynb contains the code to generate the sample data. In order to create the sample data, clone this repo or import this notebook into your Databricks workspace and execute it there. Alternatively you can open the notebook in VS Code with the Databricks extension installed and execute the Databricks: Run File as Workflow on Databricks command.