Scan files (#12)

* add basic roadrunner + router

* add the host vault path to the env

* setup PSR worker

* working file scanner dumb shell

* rename to memorelia and move server

* add checksum

* Update .gitignore

* add scan features

* Delete Dummy.php

* remove debug from home controller

* add script to generate routes

* rename namespace to App

* add 404 and other goodies

* add a simple test

* add service provider

* scan endpoint dispatches scan job

* scan file progress

* no need for RPC here (yet)

* Update DirectoryScan.php

* add directory scan test

* use interface for logger

* extra checks

* file removed job

* extract queue mock

* remove null logger

* send hash to process, if available

* add file created action

* add dummy file-moved

* add the filemoved

* add file updated

* watch and tests working

* add FileRecreated scan action

* remove unused code

* fixes and refactoring

.env.example

@@ -0,0 +1,12 @@
+# By default, the vault storage is this ./vault folder
+# Change it to where your storage volume is mounted
+HOST_VAULT_PATH=vault
+
+POSTGRES_HOST=postgres
+POSTGRES_DB=postgres
+POSTGRES_USER=postgres
+POSTGRES_PASSWORD=mysecretpassword
+
+# ====================================================
+# Don't need to change these...
+VAULT_PATH=/vault

.gitignore

@@ -9,5 +9,13 @@ _site/
 .bundle/
 vendor/
 
+# Ignore cache files
+*.cache
+
 # Ignore useless files
 .DS_Store
+vault/
+.env
+app.log
+image.jpg
+*.log.gz

.vscode/launch.json

@@ -0,0 +1,18 @@
+{
+    // Use IntelliSense to learn about possible attributes.
+    // Hover to view descriptions of existing attributes.
+    // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
+    "version": "0.2.0",
+    "configurations": [
+        {
+            "name": "Listen for Xdebug",
+            "type": "php",
+            "request": "launch",
+            "port": 9003,
+            "pathMappings": {
+                "/var/www": "${workspaceFolder}/roadrunner"
+            },
+            "hostname": "localhost",
+        }
+    ]
+}

.vscode/pvp.code-workspace

@@ -0,0 +1,10 @@
+{
+	"folders": [
+		{
+			"path": ".."
+		}
+    ],
+    "settings": {
+        "intelephense.environment.phpVersion": "8.1.0"
+    }
+}

.vscode/settings.json

@@ -0,0 +1,3 @@
+{
+    "intelephense.environment.phpVersion": "8.1.0",
+}

README.md

@@ -1,28 +1,48 @@
-# PVP Documentation
+# Personal Vault Project PVP[^1]
+
+😌 Imagine having all your digital life, in your possession, indexed and searchable.
+🤖 Now, add AI on top and be able to ask it anything about your life. Find trends, detect habits, find the bright and the dark spots.
+🧳 Your digital legacy, completely yours. All your photos, notes, posts, projects, chats, everything.
+✨ This is a project I've been pondering for a while now, but actually giving it more thought now. I finally have words on paper.
 
 See the [Discussions section](https://github.com/dlucian/pvp/discussions) to ask, comment, suggest or contribute.
 
-## Building and previewing your site locally
+## Table of Contents
 
-Assuming [Jekyll] and [Bundler] are installed on your computer:
+- [View documentation](https://dlucian.github.io/pvp/) or [edit it](/docs)
 
-1.  Change your working directory to the root directory of your site.
+Components:
+- [`/storage`](storage/) - ZFS tools
+- [`/roadrunner`](roadrunner/) - Metadata extraction & processing tools
+- [`/database`](database/) - The relational database
+- [`/index`](index/) - _(future)_ Indexing tools (future)
+- [`/llm`](llm/) - _(future)_ LLM tools (future)
+- [`/viewer`](/viewer) - _(future)_ Web interface (future)
 
-2.  Run `bundle install`.
+## Development
 
-3.  Run `bundle exec jekyll serve` to build your site and preview it at `localhost:4000`.
+```shell
+# Destroy all conainers + volumes and rebuild from scratch
+docker compose down -v && docker compose up --build
 
-The built site is stored in the directory `_site`.
+# Destroy all conainers and rebuild from scratch
+docker compose down && docker compose up --build
 
-## Publishing your built site on a different platform
+# Run the scan script
+docker compose exec --workdir /var/www roadrunner /bin/bash -c "php src/scan.php"
+```
 
-Just upload all the files in the directory `_site`.
+Run the tests:
 
-## Customization
+```shell
+docker compose exec --workdir /var/www roadrunner /bin/bash -c "vendor/bin/phpunit --testdox"
+```
 
-You're free to customize sites that you create with this template, however you like!
+Watch and send:
 
-[Browse our documentation][Just the Docs] to learn more about how to use this theme.
+```
+fswatch -0 --event Created --event Updated --event Renamed --event Removed ./vault | xargs -0 -n 1 -I {} ./path-converter.sh {}
+```
 
 ## Licensing and Attribution
 
@@ -33,18 +53,7 @@ The documentation software [Just The Docs] is licensed under MIT. A copy of the
 The deployment GitHub Actions workflow is heavily based on GitHub's mixed-party [starter workflows]. A copy of their MIT License is available in [actions/starter-workflows].
 
 ----
-
-[^1]: [It can take up to 10 minutes for changes to your site to publish after you push the changes to GitHub](https://docs.github.com/en/pages/setting-up-a-github-pages-site-with-jekyll/creating-a-github-pages-site-with-jekyll#creating-your-site).
-
-[Jekyll]: https://jekyllrb.com
-[Just the Docs]: https://just-the-docs.github.io/just-the-docs/
-[GitHub Pages]: https://docs.github.com/en/pages
-[GitHub Pages / Actions workflow]: https://github.blog/changelog/2022-07-27-github-pages-custom-github-actions-workflows-beta/
-[Bundler]: https://bundler.io
-[use this template]: https://github.com/just-the-docs/just-the-docs-template/generate
-[`jekyll-default-layout`]: https://github.com/benbalter/jekyll-default-layout
-[`jekyll-seo-tag`]: https://jekyll.github.io/jekyll-seo-tag
-[MIT License]: https://en.wikipedia.org/wiki/MIT_License
+[^1]: Needs a better name, see [Issue #4](https://github.com/dlucian/pvp/issues/4)
 [starter workflows]: https://github.com/actions/starter-workflows/blob/main/pages/jekyll.yml
 [actions/starter-workflows]: https://github.com/actions/starter-workflows/blob/main/LICENSE
 [AGPL License]: https://www.gnu.org/licenses/agpl-3.0.en.html

database/Dockerfile

@@ -0,0 +1,20 @@
+# Use an official PostgreSQL runtime as a parent image
+FROM postgres:15-alpine3.18
+
+# Set the working directory in the container to /docker-entrypoint-initdb.d
+# This is the directory that Postgres runs scripts from during the container startup
+WORKDIR /docker-entrypoint-initdb.d
+
+# Copy the SQL script into the container
+# Your script must be in the same directory as your Dockerfile
+COPY ./init.sql ./init.sql
+
+# Copy the Postgres configuration file into the container
+COPY ./postgres.conf /etc/postgresql/postgresql.conf
+
+# Make sure the script is executable
+RUN chmod +x ./init.sql
+
+# When the container starts, this script will be run
+# followed by anything in /docker-entrypoint-initdb.d
+CMD ["postgres"]

database/init.sql

@@ -0,0 +1,17 @@
+CREATE TABLE files (
+    id SERIAL PRIMARY KEY,
+    hash VARCHAR(64) NOT NULL,
+    path TEXT UNIQUE NOT NULL,
+    filename TEXT,
+    filesize BIGINT,
+    mime TEXT,
+    date_created TIMESTAMP,
+    gps_lat REAL,
+    gps_lon REAL,
+    gps_alt REAL,
+    scan_version INTEGER,
+    scanned_at TIMESTAMP,
+    created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
+    updated_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
+    removed_at TIMESTAMP
+);

database/pgadmin/servers.json

@@ -0,0 +1,20 @@
+{
+    "Servers": {
+        "1": {
+            "Name": "postgres",
+            "Group": "Servers",
+            "Host": "postgres",
+            "Port": 5432,
+            "MaintenanceDB": "postgres",
+            "Username": "postgres",
+            "UseSSHTunnel": 0,
+            "TunnelPort": "22",
+            "TunnelAuthentication": 0,
+            "KerberosAuthentication": false,
+            "ConnectionParameters": {
+                "sslmode": "prefer",
+                "connect_timeout": 10
+            }
+        }
+    }
+}