Skip to content

Commit 226f712

Browse files
author
Luke Hinds
committed
initial sqlc implementation with optimized structure
Implements the initial database layer using sqlc Core implementation: - Configure sqlc with Python code generation - Implement schema with version tracking support - Organize queries into domain-specific files - Add performance-focused indexes - Include analytics capabilities Structure: - sqlc.yaml: Base configuration with type safety options - schema.sql: Core schema with versioning support - queries/ * prompts.sql: Prompt management queries * outputs.sql: Output handling and analytics * alerts.sql: Alert system with statistics * settings.sql: Configuration management Features: - Schema version tracking - Settings management - Analytics queries - Optimized query patterns
1 parent de91231 commit 226f712

14 files changed

+1520
-2
lines changed

Makefile

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,9 @@ format:
2020
lint:
2121
poetry run ruff check .
2222

23+
sqlc:
24+
sqlc generate
25+
2326
test:
2427
poetry run pytest
2528

docs/sqlc.md

Lines changed: 199 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,199 @@
1+
# SQLC Documentation
2+
3+
## Overview
4+
5+
This project uses [sqlc](https://sqlc.dev/) with SQLite to generate type-safe Python code from SQL. The configuration is managed through `sqlc.yaml` in the root directory.
6+
7+
## Setup and Configuration
8+
9+
The project uses the following sqlc configuration:
10+
11+
```yaml
12+
version: "2"
13+
plugins:
14+
- name: "python"
15+
wasm:
16+
url: "https://downloads.sqlc.dev/plugin/sqlc-gen-python_1.2.0.wasm"
17+
sha256: "a6c5d174c407007c3717eea36ff0882744346e6ba991f92f71d6ab2895204c0e"
18+
19+
sql:
20+
- engine: "sqlite"
21+
schema: "sql/schema"
22+
queries: "sql/queries"
23+
codegen:
24+
- plugin: "python"
25+
out: "src/codegate/db"
26+
options:
27+
package: "codegate.db"
28+
emit_sync_querier: true
29+
emit_async_querier: true
30+
query_parameter_limit: 5
31+
```
32+
33+
## Directory Structure
34+
35+
```
36+
sql/
37+
├── queries/ # Contains SQL query files
38+
│ └── queries.sql
39+
└── schema/ # Contains database schema
40+
└── schema.sql
41+
```
42+
43+
## Generating Code
44+
45+
To generate Python code from your SQL files:
46+
47+
1. Install sqlc (if not already installed)
48+
```bash
49+
brew install sqlc
50+
```
51+
52+
2. Run the following command from the project root:
53+
```bash
54+
sqlc generate
55+
```
56+
57+
This will generate code in `src/codegate/db/` based on the schema and queries.
58+
59+
## Creating New Queries
60+
61+
Queries are defined in `sql/queries/queries.sql`. Each query must have a name and a command type annotation. Here are the supported command types:
62+
63+
- `:one` - Returns a single row
64+
- `:many` - Returns multiple rows
65+
- `:exec` - Executes a query without returning results
66+
67+
### Query Example
68+
69+
```sql
70+
-- name: CreatePrompt :one
71+
INSERT INTO prompts (
72+
id,
73+
timestamp,
74+
provider,
75+
system_prompt,
76+
user_prompt,
77+
type,
78+
status
79+
) VALUES (?, ?, ?, ?, ?, ?, ?) RETURNING *;
80+
81+
-- name: ListPrompts :many
82+
SELECT * FROM prompts
83+
ORDER BY timestamp DESC
84+
LIMIT ? OFFSET ?;
85+
```
86+
87+
### Query Naming Conventions
88+
89+
- Use PascalCase for query names
90+
- Prefix with action (Create, Get, List, Update, Delete)
91+
- Be descriptive about what the query does
92+
93+
## Using Generated Queries in Code
94+
95+
The generated code provides both synchronous and asynchronous query interfaces. Here are examples of how to use the generated queries:
96+
97+
### Synchronous Usage
98+
99+
```python
100+
from codegate.db.queries import Queries
101+
from sqlite3 import Connection
102+
103+
def create_prompt(conn: Connection,
104+
id: str,
105+
timestamp: datetime,
106+
provider: str,
107+
system_prompt: str,
108+
user_prompt: str,
109+
type: str,
110+
status: str):
111+
queries = Queries(conn)
112+
prompt = queries.create_prompt(
113+
id=id,
114+
timestamp=timestamp,
115+
provider=provider,
116+
system_prompt=system_prompt,
117+
user_prompt=user_prompt,
118+
type=type,
119+
status=status
120+
)
121+
return prompt
122+
123+
def list_prompts(conn: Connection, limit: int, offset: int):
124+
queries = Queries(conn)
125+
prompts = queries.list_prompts(limit=limit, offset=offset)
126+
return prompts
127+
```
128+
129+
### Asynchronous Usage
130+
131+
```python
132+
from codegate.db.queries import AsyncQuerier
133+
import aiosqlite
134+
135+
async def create_prompt_async(conn: aiosqlite.Connection,
136+
id: str,
137+
timestamp: datetime,
138+
provider: str,
139+
system_prompt: str,
140+
user_prompt: str,
141+
type: str,
142+
status: str):
143+
queries = AsyncQuerier(conn)
144+
prompt = await queries.create_prompt(
145+
id=id,
146+
timestamp=timestamp,
147+
provider=provider,
148+
system_prompt=system_prompt,
149+
user_prompt=user_prompt,
150+
type=type,
151+
status=status
152+
)
153+
return prompt
154+
155+
async def list_prompts_async(conn: aiosqlite.Connection, limit: int, offset: int):
156+
queries = AsyncQuerier(conn)
157+
prompts = await queries.list_prompts(limit=limit, offset=offset)
158+
return prompts
159+
```
160+
161+
## Best Practices
162+
163+
1. **Schema Changes**
164+
- Always update schema.sql when making database changes
165+
- Run `sqlc generate` after any schema changes
166+
- Commit both schema changes and generated code
167+
168+
2. **Query Organization**
169+
- Keep related queries together in the queries.sql file
170+
- Use clear, descriptive names for queries
171+
- Include comments for complex queries
172+
173+
3. **Error Handling**
174+
- Always handle database errors appropriately
175+
- Use transactions for operations that need to be atomic
176+
- Validate input parameters before executing queries
177+
178+
4. **Performance**
179+
- Use appropriate indexes (defined in schema.sql)
180+
- Be mindful of query complexity
181+
- Use LIMIT and OFFSET for pagination
182+
183+
## Current Implementation Review and Recommendations
184+
185+
### Type Safety Improvements
186+
187+
The current implementation uses `Any` for all model fields. Consider adding type hints:
188+
189+
```python
190+
@dataclasses.dataclass()
191+
class Prompt:
192+
id: str # Instead of Any
193+
timestamp: datetime # Instead of Any
194+
provider: Optional[str] # Instead of Optional[Any]
195+
system_prompt: Optional[str]
196+
user_prompt: str
197+
type: str
198+
status: str
199+
```

0 commit comments

Comments
 (0)