Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Include SQLite3 schemas #52

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Include SQLite3 schemas #52

wants to merge 2 commits into from

Conversation

mattjoyce
Copy link

This is a simple addition to detect sqlite3 databases and include the schema of tables, views and indexes.
The sqlite database is detected if the file is binary and has the correct magic number header. (https://www.sqlite.org/fileformat.html). Only detects SQLite3 as that is what Python3 uses.

The schema is only include if the option --extract-sqlite is provided.

❯ python3 -m files_to_prompt ../example/test.db --extract-sqlite
../example/test.db
---
-- SQLite3 Database Schema
-- Tables
CREATE TABLE knowledge_records (
            id TEXT PRIMARY KEY,
            source_file TEXT,
            source_type TEXT,
            type TEXT,
            domain TEXT,
            author TEXT,
            content JSON,
            created_at TEXT,
            version TEXT,
            embedding BLOB,
            used_in_consensus BOOLEAN DEFAULT FALSE
        );
CREATE TABLE test_table (
    id INTEGER PRIMARY KEY,           -- Primary key (INTEGER)
    col_integer INTEGER,              -- INTEGER
    col_int INT,                      -- Alias for INTEGER
    col_smallint SMALLINT,            -- Small integer
    col_tinyint TINYINT,              -- Tiny integer
    col_bigint BIGINT,                -- Big integer
    col_real REAL,                    -- Floating point number
    col_double DOUBLE,                -- DOUBLE (treated as REAL)
    col_double_precision DOUBLE PRECISION, -- Another DOUBLE variant
    col_float FLOAT,                  -- FLOAT (also REAL)
    col_numeric NUMERIC,              -- NUMERIC type
    col_decimal DECIMAL(10,2),        -- DECIMAL with precision/scale
    col_boolean BOOLEAN,              -- BOOLEAN (stored as INTEGER 0/1)
    col_date DATE,                    -- DATE (stored as TEXT)
    col_datetime DATETIME,            -- DATETIME (stored as TEXT)
    col_text TEXT,                    -- TEXT
    col_varchar VARCHAR(100),         -- VARCHAR with a length limit
    col_blob BLOB                     -- Binary data
);

-- Views
CREATE VIEW test_view AS
SELECT id, col_text, col_date, col_boolean
FROM test_table;

-- Indexes
CREATE INDEX idx_col_date ON test_table(col_date);
CREATE INDEX idx_col_integer ON test_table(col_integer);
CREATE INDEX idx_col_text ON test_table(col_text);

---

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant