Update dev with main #51
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add replication tests
Extends the on_conflict implementation to support Ecto.Query-based
updates, allowing keyword list syntax like:
on_conflict: [set: [name: "value"], inc: [count: 1]]
Changes:
- Add on_conflict pattern for %Ecto.Query{} in connection.ex
- Add update_all_for_on_conflict/1 helper for SQL generation
- Add 3 new tests for query-based on_conflict
- Document UPSERT operations in AGENTS.md and CHANGELOG.md
Closes el-ndz
feat: Add query-based on_conflict support for UPSERT operations
- Add Transaction Isolation tests to verify connections cannot access each other's transactions - Add Statement Isolation tests to verify prepared statements are scoped to connections - Add Cursor Isolation tests to verify cursors are scoped to connections - Add Savepoint Isolation tests to verify savepoints belong to owning transactions - Add Concurrent Access Safety tests for thread-safe operations - Add Resource Cleanup tests to verify cleanup on disconnect - Add Pool Isolation tests for multiple connections to same database - Add Cross-Connection Data Isolation tests for separate database files All 12 tests pass, verifying proper ownership tracking and security boundaries.
- Add support for 'strict: true' option in create_table() - STRICT tables enforce type checking (INT, INTEGER, BLOB, TEXT, REAL only) - Can be combined with RANDOM ROWID option - Generates 'STRICT' keyword at end of CREATE TABLE statement - Add tests for SQL generation (execution requires libSQL 3.37+) This implements part of el-z8u (STRICT Tables feature).
Implement full support for executing queries with named parameters (map-based arguments) instead of positional parameters (list-based). Supports all three SQLite syntaxes: - :name (colon prefix) - @name (at-sign prefix) - $name (dollar prefix) Changes: - Add normalize_arguments/3 helper to convert map params to positional list - Add map_to_positional_args/3 to introspect statement and reorder params - Add remove_param_prefix/1 helper to clean parameter names - Update handle_execute (both non-transactional and transactional paths) to normalize args - Add comprehensive test suite with 18 tests covering: - All three parameter syntaxes - Basic CRUD operations (INSERT, SELECT, UPDATE, DELETE) - Transactions (commit and rollback) - Prepared statements with parameter introspection - Edge cases (NULL values, extra params, missing params) - Backward compatibility with positional parameters Tests are thorough and include transaction isolation, error handling, and various parameter combinations. All tests clean up their database files after running. Issue: el-nqb
- Fix struct update pattern in security_test.exs (use map update after pattern match assertion) - Replace unreachable :deallocated clause with :halt in cursor test - Fix unused variable warnings (_state and _i) - Improve test cleanup to properly stop repo before file removal - Add -journal file cleanup to prevent stale files
Addresses CodeRabbit review comment: query_stmt and execute_stmt now properly normalise map arguments to positional lists using stmt introspection. - Add normalise_arguments_for_stmt/3 for prepared statement parameter conversion - Update execute_stmt/4 to normalise args before execution - Update query_stmt/3 to normalise args before query - Add comprehensive tests for named parameters with prepared statements - Update documentation to reflect both positional and named parameter support - Use British English spelling (normalise vs normalize) per project convention
Change the test to assert that using connection A's transaction on connection B fails with an error, rather than accepting both success and error outcomes. This ensures the security test actually validates that cross-connection transaction usage is rejected.
…test The test was disconnecting state_a which is the shared connection from setup, potentially causing issues. Now only per-test resources (state_b) are managed in the test, leaving the shared state to setup/teardown.
Security tests: - Remove manual disconnects of shared setup-provided state - Make cross-connection isolation test strict (assert error, not accept both) - Tests now properly manage only per-test resources Credo fixes: - Replace String.to_atom/1 with String.to_existing_atom/1 to avoid creating atoms at runtime (security best practice) - Add get_map_value_flexible/2 helper to support both atom and string keys in parameter maps - Parameter names now stored as strings internally, with flexible lookup
Fixes compiler warning that was causing CI to fail with --warnings-as-errors.
Use Enum.reduce instead of for comprehension to properly thread the state variable through each INSERT operation, consistent with patterns used elsewhere in the codebase.
- Remove unnecessary %{} pattern match from savepoint test that has no
shared setup context
- Fix state threading in concurrent access setup using Enum.reduce
Replace assert !String.contains?(sql, "STORED") with the more idiomatic ExUnit refute sql =~ "STORED" pattern.
Apply consistent anonymous `_` pattern for unused variables across test
files to satisfy Credo's consistency checks:
- security_test.exs: Simplify patterns like `{:error, _reason, _state}`
to `{:error, _, _}`, remove unused `db_path` from test context params
- fuzz_test.exs: Replace named unused vars (_state, _reason, _query, _e,
_count, _result, _final_state) with anonymous `_`, fix number format
(10000 → 10_000)
- named_parameters_execution_test.exs: Standardise `_state` → `_`
All 487 tests pass, Credo reports no consistency issues.
…tests
- Change rescue blocks to return 3-tuples {:error, :exception, state} matching
handle_execute/4's return type spec
- Update case patterns to match 3-tuple error forms {:error, _, _}
- Add {:disconnect, _, _} pattern matching for completeness
- Wrap binary blob data in {:blob, data} tuples so NIF treats them as BLOB
rather than TEXT (fixes null byte truncation issue where <<0>> became "")
The parameter name cache (SQL -> param_names mapping) was stored in persistent_term which has no size limit and can grow unboundedly with dynamic SQL workloads. Changes: - Replace persistent_term with ETS table for parameter name caching - Add maximum cache size of 1000 entries - Implement LRU eviction: when full, evict oldest 500 entries - Add thread-safe table creation with race condition handling - Add clear_param_cache/0 for testing and memory reclamation - Add param_cache_size/0 for monitoring cache usage - Update access times asynchronously to avoid blocking reads
…ling back
Previously, if statement_parameter_count/2 returned an error or unexpected
value, the code would silently fall back to treating it as 0 parameters.
This hid actual errors behind confusing SQL errors at runtime.
Changes:
- introspect_and_cache_params/3: Propagate {:error, reason} from
statement_parameter_count/2 instead of coercing to 0
- Clean up prepared statement before returning error
- Handle {:error, _reason} from statement_parameter_name/3 explicitly
- normalise_arguments_for_stmt/3: Use consistent count >= 0 pattern
and handle {:error, _reason} from statement_parameter_name/3
This makes error handling consistent with prepare failures which already
propagate as {:error, reason} and are converted to EctoLibSql.Error.
Bumps [actions/cache](https://github.com/actions/cache) from 4 to 5. - [Release notes](https://github.com/actions/cache/releases) - [Changelog](https://github.com/actions/cache/blob/main/RELEASES.md) - [Commits](actions/cache@v4...v5) --- updated-dependencies: - dependency-name: actions/cache dependency-version: '5' dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <[email protected]>
…ache-5 chore(deps): bump actions/cache from 4 to 5
Add sql_injection_with_chars_gen/0 that always generates strings containing injection characters, eliminating the need for filtering that caused StreamData to reject too many values.
Migrations, named parameters, security tests
Document the existing generated columns feature in AGENTS.md: - GENERATED ALWAYS AS syntax for virtual and stored columns - Options: generated: "expression" and stored: true - SQLite constraints (no DEFAULT, no PRIMARY KEY) - Examples showing VIRTUAL vs STORED column usage Closes el-ik6: Generated/Computed Columns
Verify that Repo.insert_all with returning: option works correctly, returning inserted rows with specified columns (:id, :name). Closes el-xih: RETURNING Enhancement for Batch Operations
- Create EctoLibSql.JSON module with helpers for JSON/JSONB functions - Implement json_extract, json_type, json_is_valid, json_array, json_object - Implement json_each, json_tree for recursive iteration - Implement convert() for JSONB binary format support - Add arrow_fragment() helper for -> and ->> operators - Comprehensive test suite with 54 passing tests - Support for both text JSON and JSONB binary format - All functions handle error cases gracefully
- Add comprehensive JSON schema helpers section under Advanced Features - Document all EctoLibSql.JSON functions with examples - Include JSONB binary format support and usage - Add arrow operators (-> and ->>) documentation - Include real-world settings management example - Add API reference for all JSON helper functions - Include performance notes for JSON operations
JSON and JSONB schema helpers
Contributor
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the You can disable this status message by setting the ✨ Finishing touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.