Core: calternal-db (SQLite layer, migrations, job queue, snapshots)

kayg commented

2026-09-24 07:51:06 +00:00

Owner

Codex job db.

Base: 79cae304a8

Codex job `db`. Base: 79cae304a8cfd52d5de562b7c9c65e7d57c3bbeb

kayg commented

2026-09-24 07:51:58 +00:00

Author

Owner

Starting calternal-db work on branch job/db at base 79cae304a8. The requested design sections and repository instructions are read; the worktree currently has no crate directory.

Starting calternal-db work on branch job/db at base 79cae304a8cfd52d5de562b7c9c65e7d57c3bbeb. The requested design sections and repository instructions are read; the worktree currently has no crate directory.

kayg commented

2026-09-24 07:57:13 +00:00

Author

Owner

Finding: rg --files found no Cargo.toml, Cargo.lock, crates/, or existing Rust source in this worktree; only CLAUDE.md, CONTEXT.md, docs/, and scripts/ are present. I will scaffold the standalone crates/calternal-db crate inside the owned path.

Finding: `rg --files` found no Cargo.toml, Cargo.lock, `crates/`, or existing Rust source in this worktree; only `CLAUDE.md`, `CONTEXT.md`, `docs/`, and `scripts/` are present. I will scaffold the standalone `crates/calternal-db` crate inside the owned path.

kayg commented

2026-09-24 08:07:58 +00:00

Author

Owner

Finding: the design contract requires filesystem access through calternal-fs, but the base worktree has no Rust crates or calternal-fs API to depend on. The snapshot API therefore accepts only a trusted, existing destination directory; crate docs state the server must resolve it before calling. The snapshot implementation uses VACUUM INTO and same-directory atomic rename as required.

Finding: the design contract requires filesystem access through `calternal-fs`, but the base worktree has no Rust crates or `calternal-fs` API to depend on. The snapshot API therefore accepts only a trusted, existing destination directory; crate docs state the server must resolve it before calling. The snapshot implementation uses `VACUUM INTO` and same-directory atomic rename as required.

kayg commented

2026-09-24 08:17:37 +00:00

Author

Owner

Finding: the first cargo check --all-targets showed that SQLx 0.9 rejects owned SQL strings unless code explicitly audits them (SqlSafeStr). I replaced dynamic query formatting with static SQL and marked plugin migration scripts with AssertSqlSafe only at the trusted Rust-code registration boundary; request data remains bound as parameters.

Finding: the first `cargo check --all-targets` showed that SQLx 0.9 rejects owned SQL strings unless code explicitly audits them (`SqlSafeStr`). I replaced dynamic query formatting with static SQL and marked plugin migration scripts with `AssertSqlSafe` only at the trusted Rust-code registration boundary; request data remains bound as parameters.

kayg commented

2026-09-24 08:34:19 +00:00

Author

Owner

Finding: cron review showed that when no cursor existed, enqueue_due compared only the current second with the next occurrence. A first poll off-tick therefore stored no state and could miss future runs. The queue now persists an initial cursor on the first poll; the scheduler then advances it through due occurrences, with the schedule id retaining one active dedup key.

Finding: cron review showed that when no cursor existed, `enqueue_due` compared only the current second with the next occurrence. A first poll off-tick therefore stored no state and could miss future runs. The queue now persists an initial cursor on the first poll; the scheduler then advances it through due occurrences, with the schedule id retaining one active dedup key.

kayg commented

2026-09-24 08:34:39 +00:00

Author

Owner

Finding: snapshot retention originally sorted names with a random UUID suffix, which does not order two snapshots created in the same millisecond. I changed the timestamp portion to increase monotonically within the target directory, so lexical pruning keeps the newest N snapshots even when the filesystem clock reports the same time.

kayg commented

2026-09-24 08:47:00 +00:00

Author

Owner

Finished calternal-db on branch job/db.

Head SHA: 4a53c855f49662fe1579ac85357aa66123c3b81e

Gate output:

cargo fmt --manifest-path crates/calternal-db/Cargo.toml --all -- --check exited 0 with no stdout.

cargo clippy --manifest-path crates/calternal-db/Cargo.toml --all-targets -- -D warnings:

    Checking calternal-db v0.1.0 (/home/kayg/Developer/calternal-wt/db/crates/calternal-db)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 1.26s

cargo test --manifest-path crates/calternal-db/Cargo.toml:

   Compiling calternal-db v0.1.0 (/home/kayg/Developer/calternal-wt/db/crates/calternal-db)
    Finished `test` profile [unoptimized + debuginfo] target(s) in 3.03s
     Running unittests src/lib.rs (crates/calternal-db/target/debug/deps/calternal_db-eb2a87f7a6181618)

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

     Running tests/queue.rs (crates/calternal-db/target/debug/deps/queue-9bfa65168b8d1f94)

running 9 tests
test enqueue_lease_throughput_microbenchmark ... ignored, manual enqueue plus lease throughput measurement
test expired_lease_is_recovered_and_old_owner_loses_lease ... ok
test deduplicates_pending_and_leased_jobs ... ok
test opens_wal_database_with_required_pragmas ... ok
test failure_backoff_uses_fake_clock_and_dead_letters_at_limit ... ok
test plugin_migrations_share_namespaces_and_check_applied_sql ... ok
test cron_enqueues_each_occurrence_once_and_keeps_one_active_job ... ok
test snapshot_restores_as_a_readable_database ... ok
test workers_do_not_execute_a_job_twice ... ok

test result: ok. 8 passed; 0 failed; 1 ignored; 0 measured; 0 filtered out; finished in 0.11s

   Doc-tests calternal_db

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

Manual benchmark: 1000 enqueue + 1000 lease operations: 1236 operation pairs/s (809.28747ms).

cargo clean --manifest-path crates/calternal-db/Cargo.toml removed 2031 files, 724.4MiB total. No Cargo.lock was committed. The worktree is clean. The issue remains open.

Finished calternal-db on branch `job/db`. Head SHA: `4a53c855f49662fe1579ac85357aa66123c3b81e` Gate output: `cargo fmt --manifest-path crates/calternal-db/Cargo.toml --all -- --check` exited 0 with no stdout. `cargo clippy --manifest-path crates/calternal-db/Cargo.toml --all-targets -- -D warnings`: ```text Checking calternal-db v0.1.0 (/home/kayg/Developer/calternal-wt/db/crates/calternal-db) Finished `dev` profile [unoptimized + debuginfo] target(s) in 1.26s ``` `cargo test --manifest-path crates/calternal-db/Cargo.toml`: ```text Compiling calternal-db v0.1.0 (/home/kayg/Developer/calternal-wt/db/crates/calternal-db) Finished `test` profile [unoptimized + debuginfo] target(s) in 3.03s Running unittests src/lib.rs (crates/calternal-db/target/debug/deps/calternal_db-eb2a87f7a6181618) running 0 tests test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s Running tests/queue.rs (crates/calternal-db/target/debug/deps/queue-9bfa65168b8d1f94) running 9 tests test enqueue_lease_throughput_microbenchmark ... ignored, manual enqueue plus lease throughput measurement test expired_lease_is_recovered_and_old_owner_loses_lease ... ok test deduplicates_pending_and_leased_jobs ... ok test opens_wal_database_with_required_pragmas ... ok test failure_backoff_uses_fake_clock_and_dead_letters_at_limit ... ok test plugin_migrations_share_namespaces_and_check_applied_sql ... ok test cron_enqueues_each_occurrence_once_and_keeps_one_active_job ... ok test snapshot_restores_as_a_readable_database ... ok test workers_do_not_execute_a_job_twice ... ok test result: ok. 8 passed; 0 failed; 1 ignored; 0 measured; 0 filtered out; finished in 0.11s Doc-tests calternal_db running 0 tests test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s ``` Manual benchmark: `1000 enqueue + 1000 lease operations: 1236 operation pairs/s (809.28747ms)`. `cargo clean --manifest-path crates/calternal-db/Cargo.toml` removed `2031 files, 724.4MiB total`. No Cargo.lock was committed. The worktree is clean. The issue remains open.

kayg referenced this issue from a commit

2026-09-24 08:49:13 +00:00

Merge job/db: SQLite layer, migrations, durable jobs, snapshots (#7)

kayg referenced this issue

2026-09-27 22:06:28 +00:00

AUDIT FOLLOW-UP: fold #237 in, fix clear inconsistencies, fresh capture on HEAD #261

kayg commented

2026-10-01 05:08:22 +00:00

Author

Owner

Completed on dev in 1b7452d544 (Merge job/db: SQLite layer, migrations, durable jobs, snapshots (#7)).

Completed on dev in 1b7452d5444dbed40c70596713a08222a428234b (Merge job/db: SQLite layer, migrations, durable jobs, snapshots (#7)).