diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md new file mode 100644 index 0000000000000..1d5935f981aeb --- /dev/null +++ b/.github/copilot-instructions.md @@ -0,0 +1,114 @@ +# MariaDB Server — Code Review Instructions + +You are reviewing a change to MariaDB Server, a large C/C++ codebase with +strict conventions. **Correctness** comes first, but style matters too: no +automated formatter runs on this repo, so review is the only gate that +enforces the coding style — flag style violations as well as bugs. + +## Priorities (in order) +1. Correctness bugs and crashes (see checklist below). +2. Backward/forward compatibility (on-disk formats, protocol, replication). +3. Performance and scalability — but only where it plausibly matters (a hot + path, or large/unbounded data), not speculative micro-optimization. Look + for: + - Wrong algorithmic complexity for the expected scale — e.g. an O(N²) loop + where a hash or sort gives O(N)/O(N log N) and N can be large. Conversely, + do NOT flag a simple O(N²) over small, bounded input where a heavier data + structure would be slower and less readable — call out the trade-off, not + just the asymptotics. + - Expensive or blocking work inside a critical section — disk/network I/O, + `fsync`, memory allocation, logging, or acquiring another lock while + holding a mutex/latch/rwlock. Shrink the critical section instead. + - Avoidable per-row cost in a hot loop — repeated allocation, redundant + recomputation of a loop-invariant, needless charset/string conversion, or + a full scan where an index lookup exists. + - Unnecessary copying of large records/buffers where a reference, move, or + in-place operation suffices. +4. Missing or inadequate test coverage. +5. Style deviations from CODING_STANDARDS.md. These are NOT enforced by any + automated formatter, so review is the only gate — do flag them, as nits. + +## MariaDB-specific bug checklist +- **Memory lifetime.** Allocator must match lifetime: `MEM_ROOT` + (`new (thd->mem_root) T`, `alloc_root`) for query-scoped; `my_malloc`/plain + `new` for large or long-lived. Flag: pointers into a `MEM_ROOT`/`String`/`blob_heap` + buffer that survive `free_root` or a record re-read; mismatched alloc/free + families — handing `MEM_ROOT` memory to `free`/`my_free` (it is bulk-freed at + `free_root`), or `my_free`-ing something from `new`. Note `delete obj` on a + `MEM_ROOT` object is fine and often required — classes like `Item` make + `operator delete` a no-op, so `delete` runs the destructor without freeing. +- **Prepared-statement arena.** Objects that must survive re-execution belong + on `thd->stmt_arena` (`Query_arena_stmt`), not the runtime mem_root. + Statements whose shape can change need `CF_REEXECUTION_FRAGILE` / + `needs_reprepare`, or PS reuse crashes or returns stale results. Item + tree changes are either permanent on the stmt_arena or temporary + and must be registered. +- **Error contract.** `bool`: false=success, true=error; int: 0=success. + Every fallible call checked; `my_error()` issued before returning true; + error path frees what it allocated (`goto err`). +- **Scoped error interception.** `thd->is_error()` checked after an operation, + or `thd->clear_error()`, is usually an anti-pattern: both act on the whole + THD, so they also see (or wipe) a legitimate error raised *before* this + operation. To detect or suppress errors from a specific bounded operation, + push an `Internal_error_handler` (`thd->push_internal_handler()` / + `pop_internal_handler()`) around exactly that scope. +- **NULL.** SQL NULL (`null_value`/`is_null()`) vs C NULL pointer; `maybe_null` + propagation; `val_str()`/`val_*` may return NULL. `item->null_value` / + `is_null()` is valid ONLY after the item has been evaluated by a `val_*()` + call in the current row — it is set as a side effect of evaluation, so + checking it before (or without) evaluating reads a stale/undefined flag. +- **Replication.** Non-deterministic constructs and their SBR vs RBR logging + safety; binlog side effects; cross-version compatibility. +- **Concurrency.** Mutex acquisition order and init order; `LOCK_*` globals; + init-once races. Respect `thd->killed`, `thd->check_killed()` is preferred. +- **Compatibility.** `.frm`, redo/undo, system-table schema, wire protocol, + sysvar defaults — a format change without upgrade handling is blocking. +- **Portability.** No `long`/`ulong` (use fixed-width or `size_t`); don't rely + on char being signed; alignment, endianness; integer overflow/truncation in + size arithmetic. +- **Assertions.** `DBUG_ASSERT` is compiled out in release — for invariants + only, never to validate external/untrusted input. Use them also as a + self-enforcing code documentation ("note, ptr is never NULL here"). + +## Testing +- Expect an `mysql-test/` `.test`; it must FAIL without the code change + (demonstrate the regression), not merely pass with it. +- **Check the test is adequate, not just present.** It must exercise the + logic the fix actually adds. For every new condition/branch, expect a case + that hits each side — a fix guarded by `if (value is NULL)` needs both a + NULL case (takes the new path) and a non-NULL case (doesn't). A new loop + bound, error path, or type case is untested until an input reaches it. Flag + new conditionals with no covering test. +- **A `.result`-only change is a red flag, not noise.** When a `.test` is + unchanged but its `.result` is, ask: (1) Is the new output actually + correct, or does it silently bless a regression? (2) Did the change gut the + test — e.g. an `EXPLAIN` that once exercised a specific optimizer path now + shows a different plan, so the test no longer covers what it was written to + cover? Flag both. + +## PR hygiene +- Subject line starts with `MDEV-NNNNN`; body wrapped at 72 cols. +- Bug fix targets the oldest maintained branch that reproduces (≤3y since GA); + new feature targets the main branch. +- Bug fix commit is minimal; cleanups (unrelated and related - prerequisite + cleanups) belong in separate commits, even if they are allowed to be in the + same PR + +## Do NOT +- Restate what the diff does. +- Report pure compiler/CI failures (they are already visible) — but style is + NOT auto-enforced here, so style review is expected, not redundant. +- Suggest STL/`std::` replacements — this codebase uses `List`, + `Dynamic_array`, `Hash_set`, `String`, `LEX_CSTRING`, etc. by policy. +- Assert about code outside the diff — ask a question instead. Do not invent + API/symbol names; verify against the code shown. + +## Output +- Cite `file:line`. Tag each finding **blocking / should-fix / nit**. +- Prefer a few high-confidence findings over many speculative ones. + +## Skip +Submodules and bundled third-party code (`zlib/`). +Note: `.result` files are NOT skipped — see Testing. + +See CODING_STANDARDS.md for full style rules. diff --git a/client/mysqldump.cc b/client/mysqldump.cc index 0928f2a2266fd..8782386e05239 100644 --- a/client/mysqldump.cc +++ b/client/mysqldump.cc @@ -1751,7 +1751,6 @@ static char *my_case_str(const char *str, static int switch_db_collation(FILE *sql_file, const char *db_name, - const char *delimiter, const char *current_db_cl_name, const char *required_db_cl_name, int *db_cl_altered) @@ -1767,11 +1766,10 @@ static int switch_db_collation(FILE *sql_file, return 1; fprintf(sql_file, - "ALTER DATABASE %s CHARACTER SET %s COLLATE %s %s\n", + "ALTER DATABASE %s CHARACTER SET %s COLLATE %s ;\n", (const char *) quoted_db_name, (const char *) db_cl->cs_name.str, - (const char *) db_cl->coll_name.str, - (const char *) delimiter); + (const char *) db_cl->coll_name.str); *db_cl_altered= 1; @@ -1786,7 +1784,6 @@ static int switch_db_collation(FILE *sql_file, static int restore_db_collation(FILE *sql_file, const char *db_name, - const char *delimiter, const char *db_cl_name) { char quoted_db_buf[NAME_LEN * 2 + 3]; @@ -1798,100 +1795,71 @@ static int restore_db_collation(FILE *sql_file, return 1; fprintf(sql_file, - "ALTER DATABASE %s CHARACTER SET %s COLLATE %s %s\n", + "ALTER DATABASE %s CHARACTER SET %s COLLATE %s ;\n", (const char *) quoted_db_name, (const char *) db_cl->cs_name.str, - (const char *) db_cl->coll_name.str, - (const char *) delimiter); + (const char *) db_cl->coll_name.str); return 0; } static void switch_cs_variables(FILE *sql_file, - const char *delimiter, const char *character_set_client, const char *character_set_results, const char *collation_connection) { fprintf(sql_file, - "/*!50003 SET @saved_cs_client = @@character_set_client */ %s\n" - "/*!50003 SET @saved_cs_results = @@character_set_results */ %s\n" - "/*!50003 SET @saved_col_connection = @@collation_connection */ %s\n" - "/*!50003 SET character_set_client = %s */ %s\n" - "/*!50003 SET character_set_results = %s */ %s\n" - "/*!50003 SET collation_connection = %s */ %s\n", - (const char *) delimiter, - (const char *) delimiter, - (const char *) delimiter, - + "/*!50003 SET @saved_cs_client = @@character_set_client */ ;\n" + "/*!50003 SET @saved_cs_results = @@character_set_results */ ;\n" + "/*!50003 SET @saved_col_connection = @@collation_connection */ ;\n" + "/*!50003 SET character_set_client = %s */ ;\n" + "/*!50003 SET character_set_results = %s */ ;\n" + "/*!50003 SET collation_connection = %s */ ;\n", (const char *) character_set_client, - (const char *) delimiter, - (const char *) character_set_results, - (const char *) delimiter, - - (const char *) collation_connection, - (const char *) delimiter); + (const char *) collation_connection); } -static void restore_cs_variables(FILE *sql_file, - const char *delimiter) +static void restore_cs_variables(FILE *sql_file) { fprintf(sql_file, - "/*!50003 SET character_set_client = @saved_cs_client */ %s\n" - "/*!50003 SET character_set_results = @saved_cs_results */ %s\n" - "/*!50003 SET collation_connection = @saved_col_connection */ %s\n", - (const char *) delimiter, - (const char *) delimiter, - (const char *) delimiter); + "/*!50003 SET character_set_client = @saved_cs_client */ ;\n" + "/*!50003 SET character_set_results = @saved_cs_results */ ;\n" + "/*!50003 SET collation_connection = @saved_col_connection */ ;\n"); } -static void switch_sql_mode(FILE *sql_file, - const char *delimiter, - const char *sql_mode) +static void switch_sql_mode(FILE *sql_file, const char *sql_mode) { fprintf(sql_file, - "/*!50003 SET @saved_sql_mode = @@sql_mode */ %s\n" - "/*!50003 SET sql_mode = '%s' */ %s\n", - (const char *) delimiter, - - (const char *) sql_mode, - (const char *) delimiter); + "/*!50003 SET @saved_sql_mode = @@sql_mode */ ;\n" + "/*!50003 SET sql_mode = '%s' */ ;\n", + (const char *) sql_mode); } -static void restore_sql_mode(FILE *sql_file, - const char *delimiter) +static void restore_sql_mode(FILE *sql_file) { fprintf(sql_file, - "/*!50003 SET sql_mode = @saved_sql_mode */ %s\n", - (const char *) delimiter); + "/*!50003 SET sql_mode = @saved_sql_mode */ ;\n"); } -static void switch_time_zone(FILE *sql_file, - const char *delimiter, - const char *time_zone) +static void switch_time_zone(FILE *sql_file, const char *time_zone) { fprintf(sql_file, - "/*!50003 SET @saved_time_zone = @@time_zone */ %s\n" - "/*!50003 SET time_zone = '%s' */ %s\n", - (const char *) delimiter, - - (const char *) time_zone, - (const char *) delimiter); + "/*!50003 SET @saved_time_zone = @@time_zone */ ;\n" + "/*!50003 SET time_zone = '%s' */ ;\n", + (const char *) time_zone); } -static void restore_time_zone(FILE *sql_file, - const char *delimiter) +static void restore_time_zone(FILE *sql_file) { fprintf(sql_file, - "/*!50003 SET time_zone = @saved_time_zone */ %s\n", - (const char *) delimiter); + "/*!50003 SET time_zone = @saved_time_zone */ ;\n"); } @@ -2684,44 +2652,6 @@ static void print_comment(FILE *sql_file, my_bool is_error, const char *format, print_xml_comment(sql_file, strlen(comment_buff), comment_buff); } -/* - create_delimiter - Generate a new (null-terminated) string that does not exist in query - and is therefore suitable for use as a query delimiter. Store this - delimiter in delimiter_buff . - - This is quite simple in that it doesn't even try to parse statements as an - interpreter would. It merely returns a string that is not in the query, which - is much more than adequate for constructing a delimiter. - - RETURN - ptr to the delimiter on Success - NULL on Failure -*/ -static char *create_delimiter(char *query, char *delimiter_buff, - int delimiter_max_size) -{ - int proposed_length; - char *presence; - - delimiter_buff[0]= ';'; /* start with one semicolon, and */ - - for (proposed_length= 2; proposed_length < delimiter_max_size; - delimiter_max_size++) { - - delimiter_buff[proposed_length-1]= ';'; /* add semicolons, until */ - delimiter_buff[proposed_length]= '\0'; - - presence = strstr(query, delimiter_buff); - if (presence == NULL) { /* the proposed delimiter is not in the query. */ - return delimiter_buff; - } - - } - return NULL; /* but if we run out of space, return nothing at all. */ -} - - /* dump_events_for_db -- retrieves list of events for a given db, and prints out @@ -2736,7 +2666,6 @@ static uint dump_events_for_db(char *db) char query_buff[QUERY_LENGTH]; char db_name_buff[NAME_LEN*2+3], name_buff[NAME_LEN*2+3]; char *event_name; - char delimiter[QUERY_LENGTH]; FILE *sql_file= md_result_file; MYSQL_RES *event_res= NULL, *event_list_res= NULL; MYSQL_ROW row, event_list_row; @@ -2764,7 +2693,6 @@ static uint dump_events_for_db(char *db) if (mysql_query_with_error_report(mysql, &event_list_res, "show events")) DBUG_RETURN(0); - safe_strcpy(delimiter, sizeof(delimiter), ";"); if (mysql_num_rows(event_list_res) > 0) { if (opt_xml) @@ -2810,25 +2738,16 @@ static uint dump_events_for_db(char *db) char *query_str; if (opt_drop) - fprintf(sql_file, "/*!50106 DROP EVENT IF EXISTS %s */%s\n", - event_name, delimiter); - - if (create_delimiter(row[3], delimiter, sizeof(delimiter)) == NULL) - { - fprintf(stderr, "%s: Warning: Can't create delimiter for event '%s'\n", - my_progname_short, event_name); - goto err; - } - - fprintf(sql_file, "DELIMITER %s\n", delimiter); + fprintf(sql_file, "/*!50106 DROP EVENT IF EXISTS %s */;\n", + event_name); if (mysql_num_fields(event_res) >= 7) { - if (switch_db_collation(sql_file, db_name_buff, delimiter, + if (switch_db_collation(sql_file, db_name_buff, db_cl_name, row[6], &db_cl_altered)) goto err; - switch_cs_variables(sql_file, delimiter, + switch_cs_variables(sql_file, row[4], /* character_set_client */ row[4], /* character_set_results */ row[5]); /* collation_connection */ @@ -2851,32 +2770,31 @@ static uint dump_events_for_db(char *db) "--\n"); } - switch_sql_mode(sql_file, delimiter, row[1]); + switch_sql_mode(sql_file, row[1]); - switch_time_zone(sql_file, delimiter, row[2]); + switch_time_zone(sql_file, row[2]); query_str= cover_definer_clause(row[3], strlen(row[3]), C_STRING_WITH_LEN("50117"), C_STRING_WITH_LEN("50106"), C_STRING_WITH_LEN(" EVENT")); - fprintf(sql_file, - "/*!50106 %s \n*/ %s\n", - (const char *) (query_str != NULL ? query_str : row[3]), - (const char *) delimiter); + fprintf(sql_file, "DELIMITER ;;\n" + "/*!50106 %s \n*/ ;;\n" + "DELIMITER ;\n", + (const char *) (query_str != NULL ? query_str : row[3])); my_free(query_str); - restore_time_zone(sql_file, delimiter); - restore_sql_mode(sql_file, delimiter); + restore_time_zone(sql_file); + restore_sql_mode(sql_file); if (mysql_num_fields(event_res) >= 7) { - restore_cs_variables(sql_file, delimiter); + restore_cs_variables(sql_file); if (db_cl_altered) { - if (restore_db_collation(sql_file, db_name_buff, delimiter, - db_cl_name)) + if (restore_db_collation(sql_file, db_name_buff, db_cl_name)) goto err; } } @@ -2893,7 +2811,6 @@ static uint dump_events_for_db(char *db) } else { - fprintf(sql_file, "DELIMITER ;\n"); fprintf(sql_file, "/*!50106 SET TIME_ZONE= @save_time_zone */ ;\n"); } @@ -3053,7 +2970,7 @@ static uint dump_routines_for_db(char *db) continue; } - switch_sql_mode(sql_file, ";", row[1]); + switch_sql_mode(sql_file, row[1]); if (opt_drop) fprintf(sql_file, "/*!50003 DROP %s IF EXISTS %s */;\n", @@ -3061,7 +2978,7 @@ static uint dump_routines_for_db(char *db) if (mysql_num_fields(routine_res) >= 6) { - if (switch_db_collation(sql_file, db, ";", + if (switch_db_collation(sql_file, db, db_cl_name, row[5], &db_cl_altered)) { mysql_free_result(routine_res); @@ -3070,7 +2987,7 @@ static uint dump_routines_for_db(char *db) DBUG_RETURN(1); } - switch_cs_variables(sql_file, ";", + switch_cs_variables(sql_file, row[3], /* character_set_client */ row[3], /* character_set_results */ row[4]); /* collation_connection */ @@ -3101,15 +3018,15 @@ static uint dump_routines_for_db(char *db) "DELIMITER ;\n", (const char *) row[2]); - restore_sql_mode(sql_file, ";"); + restore_sql_mode(sql_file); if (mysql_num_fields(routine_res) >= 6) { - restore_cs_variables(sql_file, ";"); + restore_cs_variables(sql_file); if (db_cl_altered) { - if (restore_db_collation(sql_file, db, ";", db_cl_name)) + if (restore_db_collation(sql_file, db, db_cl_name)) { mysql_free_result(routine_res); mysql_free_result(routine_list_res); @@ -4015,16 +3932,16 @@ static int dump_trigger(FILE *sql_file, MYSQL_RES *show_create_trigger_rs, continue; } - if (switch_db_collation(sql_file, db_name, ";", + if (switch_db_collation(sql_file, db_name, db_cl_name, row[5], &db_cl_altered)) DBUG_RETURN(TRUE); - switch_cs_variables(sql_file, ";", + switch_cs_variables(sql_file, row[3], /* character_set_client */ row[3], /* character_set_results */ row[4]); /* collation_connection */ - switch_sql_mode(sql_file, ";", row[1]); + switch_sql_mode(sql_file, row[1]); if (opt_drop_trigger) fprintf(sql_file, "/*!50032 DROP TRIGGER IF EXISTS %s */;\n", @@ -4042,12 +3959,12 @@ static int dump_trigger(FILE *sql_file, MYSQL_RES *show_create_trigger_rs, my_free(query_str); - restore_sql_mode(sql_file, ";"); - restore_cs_variables(sql_file, ";"); + restore_sql_mode(sql_file); + restore_cs_variables(sql_file); if (db_cl_altered) { - if (restore_db_collation(sql_file, db_name, ";", db_cl_name)) + if (restore_db_collation(sql_file, db_name, db_cl_name)) DBUG_RETURN(TRUE); } } diff --git a/client/mysqlimport.cc b/client/mysqlimport.cc index 8bae9c6c68c1f..99be2b07a550c 100644 --- a/client/mysqlimport.cc +++ b/client/mysqlimport.cc @@ -854,6 +854,7 @@ static void lock_table(MYSQL *mysql, int tablecount, char **raw_tablename) } if (mysql_real_query(mysql, query.str, (ulong)query.length-1)) db_error(mysql); /* We shall continue here, if --force was given */ + dynstr_free(&query); } @@ -1315,9 +1316,9 @@ int main(int argc, char **argv) } else { - for (; *argv != NULL; argv++) + for (char **t=argv; *t != NULL; t++) { - table_load_params p(*argv, "", current_db, 0); + table_load_params p(*t, "", current_db, 0); files_to_load.push_back(p); } } diff --git a/client/mysqltest.cc b/client/mysqltest.cc index e510ddecaeaca..095c9693dd680 100644 --- a/client/mysqltest.cc +++ b/client/mysqltest.cc @@ -207,7 +207,7 @@ static char TMPDIR[FN_REFLEN]; static char global_subst_from[200]; static char global_subst_to[200]; static char *global_subst= NULL; -static char *read_command_buf= NULL; +static char *read_command_buf= NULL, *read_command_buf_end; static MEM_ROOT require_file_root; static const my_bool my_true= 1; static const my_bool my_false= 0; @@ -3887,9 +3887,24 @@ void do_exec(struct st_command *command) #endif if (error) { - uint status= WEXITSTATUS(error); + uint status; int i; +#ifdef _WIN32 + status= WEXITSTATUS(error); +#else + /* WEXITSTATUS() is only valid for a normal exit; a process killed by an + uncaught signal must be translated using the shell's 128+signal + convention, or the real error is silently lost as status 0. */ + if (WIFEXITED(error)) + status= WEXITSTATUS(error); + else if (WIFSIGNALED(error)) + status= 128 + WTERMSIG(error); + else + status= error; + +#endif + if (command->abort_on_error) { report_or_die("exec of '%s' failed, error: %d, status: %d, errno: %d\n" @@ -5699,7 +5714,7 @@ static void primary(Expression_value *result, const char **s) enum func_type func_type= get_expr_function_type(start, end - start); if (func_type == FUNC_UNKNOWN) - die("Syntax error: Unknown function"); + die("Syntax error: Unknown function '%.*s'", (int) (end - start), start); *s= end + 1; // skip '(' handle_expr_function_call(func_type, result, s); @@ -6146,47 +6161,48 @@ static void expr(Expression_value *result, const char **s) static struct { const char *name; + size_t length; enum func_type type; } function_table[]= { // Numeric functions - {"abs", FUNC_ABS}, - {"bin", FUNC_BIN}, - {"conv", FUNC_CONV}, - {"hex", FUNC_HEX}, - {"oct", FUNC_OCT}, + {STRING_WITH_LEN("abs"), FUNC_ABS}, + {STRING_WITH_LEN("bin"), FUNC_BIN}, + {STRING_WITH_LEN("conv"), FUNC_CONV}, + {STRING_WITH_LEN("hex"), FUNC_HEX}, + {STRING_WITH_LEN("oct"), FUNC_OCT}, // String functions - {"concat", FUNC_CONCAT}, - {"concat_ws", FUNC_CONCAT_WS}, - {"greatest", FUNC_GREATEST}, - {"insert", FUNC_INSERT}, - {"instr", FUNC_INSTR}, - {"lcase", FUNC_LOWER}, - {"least", FUNC_LEAST}, - {"length", FUNC_LENGTH}, - {"locate", FUNC_LOCATE}, - {"lower", FUNC_LOWER}, - {"lpad", FUNC_LPAD}, - {"ltrim", FUNC_LTRIM}, - {"repeat", FUNC_REPEAT}, - {"replace", FUNC_REPLACE}, - {"reverse", FUNC_REVERSE}, - {"rpad", FUNC_RPAD}, - {"rtrim", FUNC_RTRIM}, - {"substr", FUNC_SUBSTR}, - {"substring", FUNC_SUBSTR}, - {"substring_index", FUNC_SUBSTR_IDX}, - {"trim", FUNC_TRIM}, - {"ucase", FUNC_UPPER}, - {"upper", FUNC_UPPER}, + {STRING_WITH_LEN("concat"), FUNC_CONCAT}, + {STRING_WITH_LEN("concat_ws"), FUNC_CONCAT_WS}, + {STRING_WITH_LEN("greatest"), FUNC_GREATEST}, + {STRING_WITH_LEN("insert"), FUNC_INSERT}, + {STRING_WITH_LEN("instr"), FUNC_INSTR}, + {STRING_WITH_LEN("lcase"), FUNC_LOWER}, + {STRING_WITH_LEN("least"), FUNC_LEAST}, + {STRING_WITH_LEN("length"), FUNC_LENGTH}, + {STRING_WITH_LEN("locate"), FUNC_LOCATE}, + {STRING_WITH_LEN("lower"), FUNC_LOWER}, + {STRING_WITH_LEN("lpad"), FUNC_LPAD}, + {STRING_WITH_LEN("ltrim"), FUNC_LTRIM}, + {STRING_WITH_LEN("repeat"), FUNC_REPEAT}, + {STRING_WITH_LEN("replace"), FUNC_REPLACE}, + {STRING_WITH_LEN("reverse"), FUNC_REVERSE}, + {STRING_WITH_LEN("rpad"), FUNC_RPAD}, + {STRING_WITH_LEN("rtrim"), FUNC_RTRIM}, + {STRING_WITH_LEN("substr"), FUNC_SUBSTR}, + {STRING_WITH_LEN("substring"), FUNC_SUBSTR}, + {STRING_WITH_LEN("substring_index"), FUNC_SUBSTR_IDX}, + {STRING_WITH_LEN("trim"), FUNC_TRIM}, + {STRING_WITH_LEN("ucase"), FUNC_UPPER}, + {STRING_WITH_LEN("upper"), FUNC_UPPER}, // Regexp functions - {"regexp_instr", FUNC_REGEXP_INSTR}, - {"regexp_replace", FUNC_REGEXP_REPLACE}, - {"regexp_substr", FUNC_REGEXP_SUBSTR}, + {STRING_WITH_LEN("regexp_instr"), FUNC_REGEXP_INSTR}, + {STRING_WITH_LEN("regexp_replace"), FUNC_REGEXP_REPLACE}, + {STRING_WITH_LEN("regexp_substr"), FUNC_REGEXP_SUBSTR}, // Null functions - {"coalesce", FUNC_COALESCE}, - {"ifnull", FUNC_IFNULL}, - {"nullif", FUNC_NULLIF}, - {NULL, FUNC_UNKNOWN} + {STRING_WITH_LEN("coalesce"), FUNC_COALESCE}, + {STRING_WITH_LEN("ifnull"), FUNC_IFNULL}, + {STRING_WITH_LEN("nullif"), FUNC_NULLIF}, + {NULL, 0, FUNC_UNKNOWN} }; @@ -6426,9 +6442,9 @@ void func_oct(Expression_value args[], int count, Expression_value *result) @param[out] result Expression_value to store result @details - Converts a number to hexadecimal representation. HEX(N) returns a string representation of the hexadecimal value of N. This is equivalent to CONV(N, 10, 16). + HEX(str) returns a hexadecimal representation of the bytes of str. @note Dies if argument count != 1 */ @@ -6438,7 +6454,15 @@ void func_hex(Expression_value args[], int count, Expression_value *result) if (count != 1) die("hex() expects 1 argument, got %d", count); - convert_base_helper(args[0].to_string(), 10, 16, result); + if (args[0].is_numeric) + convert_base_helper(args[0].to_string(), 10, 16, result); + else + { + My_string str= args[0].to_string(); + My_string hex_str; + hex_str.set_hex(str.ptr(), str.length()); + result->set_string(hex_str.ptr(), hex_str.length()); + } } @@ -7694,7 +7718,8 @@ enum func_type get_expr_function_type(const char *name, size_t len) { for (int i= 0; function_table[i].name; ++i) { - if (!strncasecmp(function_table[i].name, name, len)) + if (function_table[i].length == len && + !strncasecmp(function_table[i].name, name, len)) return function_table[i].type; } return FUNC_UNKNOWN; @@ -9800,7 +9825,7 @@ int read_line() *p= 0; DBUG_PRINT("exit", ("Found delimiter '%s' at line %d", delimiter, cur_file->lineno)); - DBUG_RETURN(0); + goto ret; } else if ((c == '{' && (!my_strnncoll_simple(charset_info, (const uchar*) "while", 5, @@ -9813,7 +9838,7 @@ int read_line() *p= 0; DBUG_PRINT("exit", ("Found '{' indicating start of block at line %d", cur_file->lineno)); - DBUG_RETURN(0); + goto ret; } else if (c == '\'' || c == '"' || c == '`') { @@ -9860,7 +9885,7 @@ int read_line() *p= 0; DBUG_PRINT("exit", ("Found newline in comment at line: %d", cur_file->lineno)); - DBUG_RETURN(0); + goto ret; } break; @@ -9880,7 +9905,7 @@ int read_line() DBUG_PRINT("info", ("Found two new lines in a row")); *p++= c; *p= 0; - DBUG_RETURN(0); + goto ret; } /* Query hasn't started yet */ @@ -9897,7 +9922,7 @@ int read_line() *p= 0; DBUG_PRINT("exit", ("Found delimiter '%s' at line: %d", delimiter, cur_file->lineno)); - DBUG_RETURN(0); + goto ret; } else if (c == '}') { @@ -9906,7 +9931,7 @@ int read_line() *p= 0; DBUG_PRINT("exit", ("Found '}' in beginning of a line at line: %d", cur_file->lineno)); - DBUG_RETURN(0); + goto ret; } else if (c == '\'' || c == '"' || c == '`') { @@ -9973,6 +9998,8 @@ int read_line() } } } +ret: + read_command_buf_end= p; DBUG_RETURN(0); } @@ -9988,14 +10015,13 @@ int read_line() */ -void convert_to_format_v1(char* query) +void convert_to_format_v1(char* query, char **end) { int last_c_was_quote= 0; char *p= query, *to= query; - char *end= strend(query); char last_c; - while (p <= end) + while (p <= *end) { if (*p == '\n' && !last_c_was_quote) { @@ -10026,6 +10052,7 @@ void convert_to_format_v1(char* query) last_c_was_quote= 0; } } + *end= to; } @@ -10142,9 +10169,9 @@ int read_command(struct st_command** command_ptr) } if (opt_result_format_version == 1) - convert_to_format_v1(read_command_buf); + convert_to_format_v1(read_command_buf, &read_command_buf_end); - char *p= read_command_buf; + char *p= read_command_buf, *end= read_command_buf_end; DBUG_PRINT("info", ("query: '%s'", read_command_buf)); if (*p == '#') { @@ -10161,29 +10188,32 @@ int read_command(struct st_command** command_ptr) } /* Skip leading spaces */ - while (*p && my_isspace(charset_info, *p)) + while (p < end && my_isspace(charset_info, *p)) p++; - if (!(command->query_buf= command->query= my_strdup(PSI_NOT_INSTRUMENTED, p, MYF(MY_WME)))) + if (!(command->query_buf= command->query= + (char*)my_memdup(PSI_NOT_INSTRUMENTED, p, end - p, MYF(MY_WME)))) die("Out of memory"); + command->query_len= (int)(end - p - 1); + command->end= command->query + command->query_len; + p= command->query; + end= command->end; /* Calculate first word length(the command), terminated - by 'space' , '(' or 'delimiter' */ - p= command->query; - while (*p && !my_isspace(charset_info, *p) && *p != '(' && !is_delimiter(p)) + by 'space' , '(' or 'delimiter' + */ + while (p < end && !my_isspace(charset_info, *p) && *p != '(' && !is_delimiter(p)) p++; command->first_word_len= (uint) (p - command->query); DBUG_PRINT("info", ("first_word: %.*s", command->first_word_len, command->query)); /* Skip spaces between command and first argument */ - while (*p && my_isspace(charset_info, *p)) + while (p < end && my_isspace(charset_info, *p)) p++; command->first_argument= p; - command->end= strend(command->query); - command->query_len= (int)(command->end - command->query); parser.read_lines++; DBUG_RETURN(0); } @@ -12480,7 +12510,7 @@ void run_query(struct st_connection *cn, struct st_command *command, int flags) else { query = command->query; - query_len = strlen(query); + query_len = command->query_len; } /* @@ -12870,6 +12900,7 @@ void get_command_type(struct st_command* command) if (type == Q_QUERY) { /* Skip the "query" part */ + command->query_len-= (int)(command->first_argument - command->query); command->query= command->first_argument; } } @@ -13430,7 +13461,8 @@ int main(int argc, char **argv) if (command->query == command->query_buf) { /* Skip the first part of command, i.e query_xxx */ - command->query= command->first_argument; + command->query_len-= (int)(command->first_argument - command->query); + command->query= command->first_argument; command->first_word_len= 0; } /* fall through */ @@ -13492,7 +13524,10 @@ int main(int argc, char **argv) /* Remove "send" if this is first iteration */ if (command->query == command->query_buf) + { + command->query_len-= (int)(command->first_argument - command->query); command->query= command->first_argument; + } /* run_query() can execute a query partially, depending on the flags. diff --git a/cmake/cpack_rpm.cmake b/cmake/cpack_rpm.cmake index c23c48d6e8ce5..c46f2f210c7ee 100644 --- a/cmake/cpack_rpm.cmake +++ b/cmake/cpack_rpm.cmake @@ -273,7 +273,7 @@ IF(WITH_WSREP) "MariaDB-server >= ${SERVER_VERSION}" "galera-4" "rsync" "grep" "gawk" "iproute" "coreutils" "findutils" "tar") - SETA(CPACK_RPM_server-galera_PACKAGE_RECOMMENDS "lsof" "socat" "pv") + SETA(CPACK_RPM_server-galera_PACKAGE_RECOMMENDS "lsof" "socat" "pv" "stunnel") SETA(CPACK_RPM_server-galera_PACKAGE_CONFLICTS "MariaDB-server <= 12.3.2") ENDIF() diff --git a/cmake/install_layout.cmake b/cmake/install_layout.cmake index 0911fc94ea99b..bffe0e8dee5a6 100644 --- a/cmake/install_layout.cmake +++ b/cmake/install_layout.cmake @@ -64,6 +64,9 @@ # # - INSTALL_UNIX_ADDRDIR (path to mysql.sock) # +# - INSTALL_RUNDIR (runtime state files that mariadbd must not write, +# e.g. what systemd reads as EnvironmentFile) +# IF(NOT INSTALL_LAYOUT) IF(DEB) @@ -163,7 +166,7 @@ SET(INSTALL_UNIX_ADDRDIR_RPM "${INSTALL_MYSQLDATADIR_RPM}/mysql.sock" SET(INSTALL_SYSTEMD_UNITDIR_RPM "/usr/lib/systemd/system") SET(INSTALL_SYSTEMD_SYSUSERSDIR_RPM "/usr/lib/sysusers.d") SET(INSTALL_SYSTEMD_TMPFILESDIR_RPM "/usr/lib/tmpfiles.d") -SET(INSTALL_RUNDATADIR_RPM "/run/mariadb") +SET(INSTALL_RUNDIR_RPM "/run") SET(INSTALL_PAMDIR_RPM "/usr/${INSTALL_LIBDIR_RPM}/security") IF(RPM MATCHES "^(opensuse|sles)([0-9]+)$") IF(CMAKE_MATCH_2 LESS_EQUAL 1507) @@ -199,8 +202,8 @@ SET(INSTALL_SUPPORTFILESDIR_DEB "share/mariadb") # SET(INSTALL_MYSQLDATADIR_DEB "/var/lib/mysql") -SET(INSTALL_RUNDATADIR_DEB "/run/mysqld") -SET(INSTALL_UNIX_ADDRDIR_DEB "${INSTALL_RUNDATADIR_DEB}/mysqld.sock") +SET(INSTALL_RUNDIR_DEB "/run") +SET(INSTALL_UNIX_ADDRDIR_DEB "${INSTALL_RUNDIR_DEB}/mysqld/mysqld.sock") SET(INSTALL_SYSTEMD_UNITDIR_DEB "/usr/lib/systemd/system") SET(INSTALL_SYSTEMD_SYSUSERSDIR_DEB "/usr/lib/sysusers.d") SET(INSTALL_SYSTEMD_TMPFILESDIR_DEB "/usr/lib/tmpfiles.d") @@ -264,7 +267,10 @@ IF(NOT MYSQL_UNIX_ADDR) SET(MYSQL_UNIX_ADDR ${INSTALL_UNIX_ADDRDIR}) ENDIF() -IF(NOT INSTALL_RUNDATADIR) +IF(NOT INSTALL_RUNDIR) + # No /run in a tarball installation - fall back to the socket directory. + # Note that mariadbd can write there, so the isolation that /run gives to + # packages (see EnvironmentFile= in mariadb.service) is not achieved. get_filename_component(MYSQL_UNIX_DIR ${MYSQL_UNIX_ADDR} DIRECTORY) - SET(INSTALL_RUNDATADIR "${MYSQL_UNIX_DIR}" CACHE FILEPATH "Rundata installation directory" ${FORCE}) + SET(INSTALL_RUNDIR "${MYSQL_UNIX_DIR}" CACHE FILEPATH "Runtime state directory" ${FORCE}) ENDIF() diff --git a/config.h.cmake b/config.h.cmake index b378b464c9be8..1879928b27ec7 100644 --- a/config.h.cmake +++ b/config.h.cmake @@ -1,5 +1,6 @@ /* Copyright (c) 2009, 2013, Oracle and/or its affiliates. All rights reserved. - + Copyright (c) 2026, MariaDB plc. + This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. @@ -184,6 +185,7 @@ #cmakedefine HAVE_MMAP 1 #cmakedefine HAVE_MMAP64 1 #cmakedefine HAVE_MPROTECT 1 +#cmakedefine HAVE_RO_AFTER_INIT 1 #cmakedefine HAVE_PERROR 1 #cmakedefine HAVE_POLL 1 #cmakedefine HAVE_POSIX_FALLOCATE 1 diff --git a/configure.cmake b/configure.cmake index 1a0743bb7c4b5..9bd1c6b1ce525 100644 --- a/configure.cmake +++ b/configure.cmake @@ -1,5 +1,6 @@ # Copyright (c) 2009, 2013, Oracle and/or its affiliates. All rights reserved. -# +# Copyright (c) 2026, MariaDB plc. +# # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; version 2 of the License. @@ -370,6 +371,24 @@ CHECK_FUNCTION_EXISTS (mmap HAVE_MMAP) CHECK_FUNCTION_EXISTS (mmap64 HAVE_MMAP64) CHECK_FUNCTION_EXISTS (mprotect HAVE_MPROTECT) CHECK_FUNCTION_EXISTS (perror HAVE_PERROR) + +IF(HAVE_SYS_MMAN_H AND HAVE_MPROTECT) + SET(SAVE_CMAKE_REQUIRED_FLAGS "${CMAKE_REQUIRED_FLAGS}") + SET(CMAKE_REQUIRED_FLAGS + "${CMAKE_REQUIRED_FLAGS} -Wl,-T,${CMAKE_SOURCE_DIR}/sql/mysqld_ro.lds") + CHECK_C_SOURCE_COMPILES(" + #include + __attribute__((section(\"ro_after_init\"))) int hardened= 1; + extern char __ro_after_init_start[] __attribute__((weak)); + extern char __ro_after_init_end[] __attribute__((weak)); + int main(void) { + size_t size= __ro_after_init_end - __ro_after_init_start; + mprotect(__ro_after_init_start, size, PROT_READ | PROT_WRITE); + return (int)size + hardened; + }" HAVE_RO_AFTER_INIT) + SET(CMAKE_REQUIRED_FLAGS "${SAVE_CMAKE_REQUIRED_FLAGS}") +ENDIF() + CHECK_FUNCTION_EXISTS (poll HAVE_POLL) CHECK_FUNCTION_EXISTS (posix_fallocate HAVE_POSIX_FALLOCATE) CHECK_FUNCTION_EXISTS (pread HAVE_PREAD) diff --git a/debian/autobake-deb.sh b/debian/autobake-deb.sh index 5181a4be01649..1b7c71ec8fe5c 100755 --- a/debian/autobake-deb.sh +++ b/debian/autobake-deb.sh @@ -179,9 +179,12 @@ then fi # Enable DuckDB storage engine plugin packaging -if grep -q "$architecture" storage/duckdb/debian/control +if grep -q "$architecture" storage/duckdb/debian/control && \ + [[ ! -f debian/mariadb-plugin-duckdb.install ]] then - cat storage/duckdb/debian/mariadb-plugin-duckdb.install >> debian/mariadb-server.install + echo >> debian/control + cat storage/duckdb/debian/control >> debian/control + cp -v storage/duckdb/debian/mariadb-plugin-duckdb.install debian/ fi if [ -n "${AUTOBAKE_PREP_CONTROL_RULES_ONLY:-}" ] diff --git a/debian/control b/debian/control index df33235d82380..819fbee5cdddf 100644 --- a/debian/control +++ b/debian/control @@ -983,6 +983,7 @@ Depends: ${shlibs:Depends}, Recommends: pv, + stunnel, Conflicts: mariadb-galera-server-10.0, mariadb-galera-server-5.5, diff --git a/extra/mariabackup/backup_mysql.cc b/extra/mariabackup/backup_mysql.cc index d853772485bcf..4095e10bf768f 100644 --- a/extra/mariabackup/backup_mysql.cc +++ b/extra/mariabackup/backup_mysql.cc @@ -88,6 +88,25 @@ static mysql_cond_t kill_query_thread_stop; bool sql_thread_started = false; char *mysql_slave_position = NULL; char *mysql_binlog_position = NULL; +/* + MDEV-38147: the exact binary log file name that + write_current_binlog_file() rotated to and shipped into the backup under + --galera-info. Remembered here so that write_binlog_info() records the very + same file name in xtrabackup_binlog_info, i.e. the file the SST joiner looks + for is guaranteed to be the file that was actually sent (no rotation race). +*/ +char *mysql_binlog_file = NULL; +/* + Position and GTID of that shipped binary log, captured together with + mysql_binlog_file at flush time (see write_current_binlog_file()). Used by + write_binlog_info() so that, if a rotation intervenes, the whole coordinate + tuple it records stays consistent with the file that was actually shipped - + which matters for a manual --galera-info backup used as a replication seed, + not only for the SST joiner (which reads just the file name). +*/ +static char *mysql_binlog_file_pos = NULL; +static char *mysql_binlog_file_gtid_executed = NULL; +static char *mysql_binlog_file_gtid_current_pos = NULL; char *buffer_pool_filename = NULL; /* History on server */ @@ -348,6 +367,30 @@ read_mysql_one_value(MYSQL *mysql, const char *query) return read_mysql_one_value(mysql, query, 0/*offset*/, 1/*total columns*/); } +/** Poll InnoDB's durably flushed redo LSN from the running server. +Log copier uses this as its parse limit; it never accepts a redo log +record whose end exceeds this LSN +@param connection MariaDB client connection +@retval 0 on failure, lsn on success */ +uint64_t get_log_flushed_lsn(MYSQL *connection) noexcept +{ + if (!connection) + return 0; + + uint64_t lsn= 0; + MYSQL_RES *res= xb_mysql_query(connection, + "SHOW STATUS LIKE 'Innodb_lsn_flushed'", + true, false); + if (!res) + return 0; + + if (MYSQL_ROW row= mysql_fetch_row(res)) + if (row[1]) + lsn= strtoull(row[1], nullptr, 10); + mysql_free_result(res); + return lsn; +} + static bool @@ -1515,6 +1558,23 @@ write_galera_info(ds_ctxt *datasink, MYSQL *connection) domain_id ? domain_id : domain_id55); } + /* + MDEV-38147: Flush and copy the donor's current binary log into + the backup so that it is shipped to the SST joiner. + + A new joiner discards this file and starts a fresh binary log seeded from + the storage-engine checkpoint (see wsrep_seed_binlog_gtid_state() in + sql/log.cc and the joiner code in scripts/wsrep_sst_mariabackup.sh), which + avoids error 1950 with gtid_strict_mode=ON. The file is still shipped for + backward compatibility with an old joiner that expects it, and so that a + new joiner can deterministically identify and remove exactly the file that + was sent instead of colliding with it. + + write_current_binlog_file() remembers the rotated file name in + mysql_binlog_file so that write_binlog_info() records the same file in + xtrabackup_binlog_info - closing the old race where a concurrent rotation + could make the shipped file and the recorded file diverge. + */ if (result) write_current_binlog_file(datasink, connection); @@ -1535,13 +1595,25 @@ write_galera_info(ds_ctxt *datasink, MYSQL *connection) /*********************************************************************//** Flush and copy the current binary log file into the backup, -if GTID is enabled */ +if GTID is enabled. + +MDEV-38147: the file name that FLUSH BINARY LOGS rotates to is +remembered in the global mysql_binlog_file. write_binlog_info() then records +that exact name in xtrabackup_binlog_info, so the file the SST joiner looks +for is guaranteed to be the file that was shipped. Previously the shipped file +(determined here) and the recorded file (determined independently later by +write_binlog_info()) were read by two separate SHOW MASTER STATUS calls; a +binary log rotation happening in between made them diverge and the wrong file +was sent. */ bool write_current_binlog_file(ds_ctxt *datasink, MYSQL *connection) { char *executed_gtid_set = NULL; char *gtid_binlog_state = NULL; + char *gtid_current_pos = NULL; char *log_bin_file = NULL; + char *log_bin_pos = NULL; + char *log_bin_gtid_executed = NULL; char *log_bin_dir = NULL; bool gtid_exists; bool result = true; @@ -1552,13 +1624,18 @@ write_current_binlog_file(ds_ctxt *datasink, MYSQL *connection) {NULL, NULL} }; + /* Capture the shipped file's coordinates together, right after the flush, + so they describe exactly the file that is copied below. */ mysql_variable status_after_flush[] = { {"File", &log_bin_file}, + {"Position", &log_bin_pos}, + {"Executed_Gtid_Set", &log_bin_gtid_executed}, {NULL, NULL} }; mysql_variable vars[] = { {"gtid_binlog_state", >id_binlog_state}, + {"gtid_current_pos", >id_current_pos}, {"log_bin_basename", &log_bin_dir}, {NULL, NULL} }; @@ -1588,6 +1665,32 @@ write_current_binlog_file(ds_ctxt *datasink, MYSQL *connection) log_bin_dir = strdup("./"); } + if (log_bin_dir == NULL || log_bin_file == NULL) { + msg("Failed to get master binlog coordinates from " + "SHOW MASTER STATUS"); + result = false; + goto cleanup; + } + + /* + Remember the file we just rotated to (before any further + rotation can happen), together with its position and GTID, so + that write_binlog_info() records this very file - and a matching + position/GTID - in xtrabackup_binlog_info, and the joiner looks + for exactly the file that is shipped below. + */ + free(mysql_binlog_file); + mysql_binlog_file = strdup(log_bin_file); + free(mysql_binlog_file_pos); + mysql_binlog_file_pos = + log_bin_pos ? strdup(log_bin_pos) : NULL; + free(mysql_binlog_file_gtid_executed); + mysql_binlog_file_gtid_executed = + log_bin_gtid_executed ? strdup(log_bin_gtid_executed) : NULL; + free(mysql_binlog_file_gtid_current_pos); + mysql_binlog_file_gtid_current_pos = + gtid_current_pos ? strdup(gtid_current_pos) : NULL; + dirname_part(log_bin_dir, log_bin_dir, &log_bin_dir_length); /* strip final slash if it is not the only path component */ @@ -1596,13 +1699,6 @@ write_current_binlog_file(ds_ctxt *datasink, MYSQL *connection) log_bin_dir[log_bin_dir_length - 1] = 0; } - if (log_bin_dir == NULL || log_bin_file == NULL) { - msg("Failed to get master binlog coordinates from " - "SHOW MASTER STATUS"); - result = false; - goto cleanup; - } - snprintf(filepath, sizeof(filepath), "%s%c%s", log_bin_dir, FN_LIBCHAR, log_bin_file); result = datasink->copy_file(filepath, log_bin_file, 0); @@ -1624,11 +1720,15 @@ bool write_binlog_info(ds_ctxt *datasink, MYSQL *connection) { char *filename = NULL; + const char *out_filename; + const char *out_position; + const char *out_gtid_executed; + const char *out_gtid_current_pos; char *position = NULL; char *gtid_mode = NULL; char *gtid_current_pos = NULL; char *gtid_executed = NULL; - char *gtid = NULL; + const char *gtid = NULL; bool result; bool mysql_gtid; bool mariadb_gtid; @@ -1656,25 +1756,53 @@ write_binlog_info(ds_ctxt *datasink, MYSQL *connection) goto cleanup; } + /* + MDEV-38147: if write_current_binlog_file() already rotated + and shipped a binary log under --galera-info, record that exact file + name here rather than whatever SHOW MASTER STATUS reports now. The two + are normally identical, but a binary log rotation between the two + SHOW MASTER STATUS calls would otherwise make mariadb_backup_binlog_info + name a file different from the one that was shipped, so the SST joiner + would look for a file that is not there. Use a separate pointer so the + string owned by the status[] array is still freed at cleanup. + */ + out_filename = filename; + out_position = position; + out_gtid_executed = gtid_executed; + out_gtid_current_pos = gtid_current_pos; + if (mysql_binlog_file != NULL && strcmp(filename, mysql_binlog_file)) { + msg("Binary log rotated to '%s' after '%s' was shipped; recording " + "the shipped file and its coordinates in " + MB_BINLOG_INFO, filename, mysql_binlog_file); + /* Record the whole tuple from the shipped file, not just its name, + so file/position/GTID stay consistent (the position and GTID from + the SHOW MASTER STATUS above belong to the current, newer file). */ + out_filename = mysql_binlog_file; + out_position = mysql_binlog_file_pos; + out_gtid_executed = mysql_binlog_file_gtid_executed; + out_gtid_current_pos = mysql_binlog_file_gtid_current_pos; + } + mysql_gtid = ((gtid_mode != NULL) && (strcmp(gtid_mode, "ON") == 0)); - mariadb_gtid = (gtid_current_pos != NULL); + mariadb_gtid = (out_gtid_current_pos != NULL); - gtid = (gtid_executed != NULL ? gtid_executed : gtid_current_pos); + gtid = (out_gtid_executed != NULL ? out_gtid_executed + : out_gtid_current_pos); if (mariadb_gtid || mysql_gtid) { ut_a(asprintf(&mysql_binlog_position, "filename '%s', position '%s', " "GTID of the last change '%s'", - filename, position, gtid) != -1); + out_filename, out_position, gtid) != -1); result = datasink->backup_file_printf(MB_BINLOG_INFO, - "%s\t%s\t%s\n", filename, position, + "%s\t%s\t%s\n", out_filename, out_position, gtid); } else { ut_a(asprintf(&mysql_binlog_position, "filename '%s', position '%s'", - filename, position) != -1); + out_filename, out_position) != -1); result = datasink->backup_file_printf(MB_BINLOG_INFO, - "%s\t%s\n", filename, position); + "%s\t%s\n", out_filename, out_position); } cleanup: @@ -2008,6 +2136,10 @@ backup_cleanup() { free(mysql_slave_position); free(mysql_binlog_position); + free(mysql_binlog_file); + free(mysql_binlog_file_pos); + free(mysql_binlog_file_gtid_executed); + free(mysql_binlog_file_gtid_current_pos); free(buffer_pool_filename); if (mysql_connection) { diff --git a/extra/mariabackup/backup_mysql.h b/extra/mariabackup/backup_mysql.h index 55700dddf6d67..5bdd34c71602c 100644 --- a/extra/mariabackup/backup_mysql.h +++ b/extra/mariabackup/backup_mysql.h @@ -2,6 +2,7 @@ #define XTRABACKUP_BACKUP_MYSQL_H #include +#include #include #include #include "datasink.h" @@ -26,6 +27,7 @@ extern time_t history_lock_time; extern bool sql_thread_started; extern char *mysql_slave_position; extern char *mysql_binlog_position; +extern char *mysql_binlog_file; extern char *buffer_pool_filename; /** connection to mysql server */ @@ -97,4 +99,6 @@ bool write_slave_info(ds_ctxt *datasink, MYSQL *connection); ulonglong get_current_lsn(MYSQL *connection); + +uint64_t get_log_flushed_lsn(MYSQL *connection) noexcept; #endif diff --git a/extra/mariabackup/xtrabackup.cc b/extra/mariabackup/xtrabackup.cc index a428783c405d1..7688c579044a3 100644 --- a/extra/mariabackup/xtrabackup.cc +++ b/extra/mariabackup/xtrabackup.cc @@ -205,7 +205,7 @@ struct xb_filter_entry_t{ /** whether log_copying_thread() is active; protected by recv_sys.mutex */ static bool log_copying_running; /** the log parsing function for --backup */ -static recv_sys_t::parser backup_log_parse; +static recv_sys_t::parser backup_log_parse_low; /** for --backup, target LSN to copy the log to; protected by recv_sys.mutex */ lsn_t metadata_to_lsn; @@ -251,6 +251,7 @@ const char *defaults_group = "mysqld"; #define HA_INNOBASE_ROWS_IN_TABLE 10000 /* to get optimization right */ #define HA_INNOBASE_RANGE_COUNT 100 +#define METADATA_LSN_ERROR 1 /* The default values for the following, type long or longlong, start-up parameters are declared in mysqld.cc: */ @@ -3499,6 +3500,41 @@ static my_bool xtrabackup_copy_datafile(ds_ctxt *ds_data, return(FALSE); } +/* Maximum LSN the copier may parse up to. +Usually derived from the server's durably-flushed redo LSN. +When a final target LSN is set, this limit is raised to that +target so the last partial block can be copied in full. +The copier refuses to accept a mini-transaction whose end +exceeds this, so it never parses the volatile tail block +the server is still rewriting +(which mmap can read torn, leading to a mis-computed mtr +length and permanent mid-mtr drift). A pread() of the same +still-being-written tail block can likewise return a torn read +which can be observed on ext4, though not on XFS). +Read/written only under recv_sys.mutex. */ +static lsn_t max_parse_lsn; + +/* Set by backup_log_parse() when the last parse was +rejected because the mtr crossed max_parse_lsn. It means +"caught up, wait" condition, not a stall. */ +static bool reached_parse_limit; + +static recv_sys_t::parse_mtr_result backup_log_parse() +{ + const lsn_t prev_lsn= recv_sys.lsn; + const size_t prev_offset= recv_sys.offset; + reached_parse_limit= false; + recv_sys_t::parse_mtr_result r= backup_log_parse_low(false); + if (r == recv_sys_t::OK && max_parse_lsn && recv_sys.lsn > max_parse_lsn) + { + recv_sys.lsn= prev_lsn; + recv_sys.offset= prev_offset; + reached_parse_limit= true; + return recv_sys_t::GOT_EOF; + } + return r; +} + static int xtrabackup_copy_mmap_snippet(ds_file_t *ds, const byte *start, const byte *end) { @@ -3524,7 +3560,7 @@ static bool xtrabackup_copy_mmap_logfile() const byte *start= &log_sys.buf[recv_sys.offset]; ut_d(recv_sys_t::parse_mtr_result r); - if ((ut_d(r=) backup_log_parse(false)) == recv_sys_t::OK) + if ((ut_d(r=) backup_log_parse()) == recv_sys_t::OK) { do { @@ -3542,7 +3578,7 @@ static bool xtrabackup_copy_mmap_logfile() start = seq + 1; } } - while ((ut_d(r=) backup_log_parse(false)) == recv_sys_t::OK); + while ((ut_d(r=) backup_log_parse()) == recv_sys_t::OK); if (xtrabackup_copy_mmap_snippet(dst_log_file, start, &log_sys.buf[recv_sys.offset])) @@ -3615,7 +3651,7 @@ static bool xtrabackup_copy_logfile(bool early_exit) if (log_sys.buf[recv_sys.offset] <= 1) break; - if (backup_log_parse(false) == recv_sys_t::OK) + if (backup_log_parse() == recv_sys_t::OK) { do { @@ -3625,7 +3661,7 @@ static bool xtrabackup_copy_logfile(bool early_exit) sequence_offset)); *seq= 1; } - while ((r= backup_log_parse(false)) == recv_sys_t::OK); + while ((r= backup_log_parse()) == recv_sys_t::OK); if (ds_write(dst_log_file, log_sys.buf + start_offset, recv_sys.offset - start_offset)) @@ -3655,6 +3691,9 @@ static bool xtrabackup_copy_logfile(bool early_exit) if (retry_count == 100) break; + if (reached_parse_limit) + break; + mysql_mutex_unlock(&recv_sys.mutex); if (!retry_count++) msg("Retrying read of log at LSN=" LSN_PF, recv_sys.lsn); @@ -3669,13 +3708,32 @@ static bool xtrabackup_copy_logfile(bool early_exit) return false; } +/** Handles backup log copy timeout or incomplete LSN copy. +Checks if the target LSN was reached. If not, +logs diagnostic messages advising whether to increase +innodb_log_file_size (log wrapped around) or to check +server/backup configuration mismatches. +@param lsn Target LSN expected by the backup. +@param last_lsn Actual maximum LSN copied so far. +@return true if target LSN was reached, false otherwise. */ static bool backup_wait_timeout(lsn_t lsn, lsn_t last_lsn) { if (last_lsn >= lsn) return true; + + const lsn_t checkpoint_lsn= log_sys.last_checkpoint_lsn.load(); + const lsn_t capacity= log_sys.file_size - log_sys.START_OFFSET; + const lsn_t needed= lsn - checkpoint_lsn; + msg("Was only able to copy log from " LSN_PF " to " LSN_PF - ", not " LSN_PF "; try increasing innodb_log_file_size", - log_sys.last_checkpoint_lsn.load(), last_lsn, lsn); + ", not " LSN_PF, checkpoint_lsn, last_lsn, lsn); + + if (needed <= capacity) + msg("mariabackup: The required redo still fits within the log " + "capacity, so it has not been overwritten; check whether the " + "server and backup configuration are the same."); + else + msg("mariabackup: Try increasing the innodb_log_file_size."); return false; } @@ -3730,16 +3788,93 @@ static bool backup_wait_for_lsn(lsn_t lsn) static void log_copying_thread() { my_thread_init(); + MYSQL *limit_con= xb_mysql_connect(); + if (!limit_con) + { + /* Without this connection we cannot poll the durably-flushed + LSN, so max_parse_lsn could never advance and the copier + would either stall or parse the volatile tail block. + Fail the copy gracefully and let the main thread report + it via backup_wait_timeout(). */ + msg("mariabackup: Error: cannot open a server connection for " + "the log copying thread; the backup will fail."); + mysql_mutex_lock(&recv_sys.mutex); + log_copying_running= false; + pthread_cond_broadcast(&scanned_lsn_cond); + mysql_mutex_unlock(&recv_sys.mutex); + my_thread_end(); + return; + } + + /* + This thread polls Innodb_lsn_flushed via SHOW STATUS on its own connection. + On a Galera donor wsrep_sync_wait may include SHOW, which would make that + poll wait until the node has applied the latest cluster transactions. During + a backup the donor's commit position can legitimately lag (e.g. a transaction + sitting between its binary log write and engine commit), so the poll could + block indefinitely and stall the redo log copier - failing the backup with a + misleading "Was only able to copy log ..." error. The main backup connection + already disables wsrep_sync_wait for the same reason, so do the same here. + */ + if (have_galera_enabled) + xb_mysql_query(limit_con, "SET SESSION wsrep_sync_wait=0", false); + mysql_mutex_lock(&recv_sys.mutex); - while (!xtrabackup_copy_logfile(false) && - (!metadata_last_lsn || metadata_last_lsn > recv_sys.lsn)) + for (;;) { + /* metadata_last_lsn is set to 1 when error was encountered. + Abort the log copier if an error was signaled */ + if (metadata_last_lsn == METADATA_LSN_ERROR) + break; + /* Refresh the max_parse_lsn before each copy pass */ + const lsn_t final_target= metadata_last_lsn > metadata_to_lsn + ? metadata_last_lsn : metadata_to_lsn; + if (final_target) + /* Final phase (BLOCK_DDL/BLOCK_COMMIT): fence at the exact target + instead of the polled flushed LSN. This is only safe because the + target was derived from get_current_lsn(), which runs + FLUSH ENGINE LOGS and thus makes the redo durable up to at least + final_target; so final_target <= Innodb_lsn_flushed holds here and + the bytes up to it are stable. Do not feed this branch an LSN that + has not been flushed by the server, or the copier will parse the + volatile tail block again (MDEV-39468). */ + max_parse_lsn= final_target; + else + { + mysql_mutex_unlock(&recv_sys.mutex); + lsn_t flushed= get_log_flushed_lsn(limit_con); + mysql_mutex_lock(&recv_sys.mutex); + if (flushed > log_sys.get_first_lsn()) + max_parse_lsn= flushed; + else if (!max_parse_lsn) + { + /* We could not obtain a flushed LSN and none was seeded before, + so the parse limit is unset and the copier would fall back to + parsing the volatile tail block (the MDEV-39468 failure). Warn + once so this is diagnosable rather than a silent stall. */ + static bool warned; + if (!warned) + { + warned= true; + msg("mariabackup: Warning: could not read Innodb_lsn_flushed; " + "the redo log copier has no parse limit."); + } + } + } + + if (xtrabackup_copy_logfile(false)) + break; + if (final_target && final_target <= recv_sys.lsn) + break; + timespec abstime; set_timespec_nsec(abstime, 1000000ULL * xtrabackup_log_copy_interval); mysql_cond_timedwait(&log_copying_stop, &recv_sys.mutex, &abstime); } log_copying_running= false; mysql_mutex_unlock(&recv_sys.mutex); + if (limit_con) + mysql_close(limit_con); my_thread_end(); } @@ -4944,7 +5079,7 @@ static bool backup_wait_for_commit_lsn() else { msg("Error: recv_sys.find_checkpoint() failed."); - metadata_last_lsn= 1; + metadata_last_lsn= METADATA_LSN_ERROR; stop_backup_threads(); mysql_mutex_unlock(&recv_sys.mutex); return false; @@ -5571,7 +5706,7 @@ static bool xtrabackup_backup_func() fail: if (log_copying_running) { mysql_mutex_lock(&recv_sys.mutex); - metadata_last_lsn = 1; + metadata_last_lsn = METADATA_LSN_ERROR; stop_backup_threads(); mysql_mutex_unlock(&recv_sys.mutex); } @@ -5699,9 +5834,15 @@ static bool xtrabackup_backup_func() /* copy log file by current position */ mysql_mutex_lock(&recv_sys.mutex); - backup_log_parse = recv_sys.get_backup_parser(); + backup_log_parse_low = recv_sys.get_backup_parser(); recv_sys.lsn = log_sys.last_checkpoint_lsn; + if (lsn_t flushed= get_log_flushed_lsn(mysql_connection)) + { + if (flushed > log_sys.get_first_lsn()) + max_parse_lsn= flushed; + } + const bool log_copy_failed = xtrabackup_copy_logfile(true); mysql_mutex_unlock(&recv_sys.mutex); diff --git a/include/heap.h b/include/heap.h index 3fac752abd028..4df384d73d8f3 100644 --- a/include/heap.h +++ b/include/heap.h @@ -30,7 +30,6 @@ extern "C" { #include #include -#include "my_compare.h" #include "my_tree.h" /* defines used by heap-functions */ @@ -109,6 +108,7 @@ typedef struct st_heap_block } HP_BLOCK; struct st_heap_info; /* For reference */ +struct st_HA_KEYSEG; typedef struct st_hp_keydef /* Key definition with open */ { @@ -116,7 +116,7 @@ typedef struct st_hp_keydef /* Key definition with open */ uint keysegs; /* Number of key-segment */ uint length; /* Length of key (automatic) */ uint8 algorithm; /* HASH / BTREE */ - HA_KEYSEG *seg; + struct st_HA_KEYSEG *seg; HP_BLOCK block; /* Where keys are saved */ /* Number of buckets used in hash table. Used only to provide diff --git a/include/json_lib.h b/include/json_lib.h index fab12f9eeb03b..d51b1b59a4595 100644 --- a/include/json_lib.h +++ b/include/json_lib.h @@ -133,6 +133,7 @@ typedef struct st_json_path_t } json_path_t; +__attribute__((nonnull, warn_unused_result)) int json_path_setup(json_path_t *p, CHARSET_INFO *i_cs, const uchar *str, const uchar *end); diff --git a/include/my_global.h b/include/my_global.h index 2cc8c8479084b..031e79ac1c20c 100644 --- a/include/my_global.h +++ b/include/my_global.h @@ -1,6 +1,6 @@ /* Copyright (c) 2001, 2013, Oracle and/or its affiliates. - Copyright (c) 2009, 2022, MariaDB Corporation. + Copyright (c) 2009, 2026, MariaDB plc. This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by @@ -815,6 +815,7 @@ typedef long long my_ptrdiff_t; #define MY_ALIGN(A,L) (((A) + (L) - 1) & ~((L) - 1)) #define MY_ALIGN_DOWN(A,L) ((A) & ~((L) - 1)) + #define ALIGN_SIZE(A) MY_ALIGN((A),sizeof(double)) #define ALIGN_MAX_UNIT (sizeof(double)) /* Size to make addressable obj. */ @@ -1016,6 +1017,20 @@ typedef ulong myf; /* Type of MyFlags in my_funcs */ #endif #endif +/* + Tag a global variable as "read-only after init". It'll be made read-only + just before the server starts accepting connection. All READ_ONLY sysvars + must be tagged this way. +*/ +#if defined(HAVE_RO_AFTER_INIT) && !defined(EMBEDDED_LIBRARY) +#define READ_ONLY_SYSVAR __attribute__((section("ro_after_init"))) +#elif defined(_MSC_VER) && !defined(EMBEDDED_LIBRARY) +#pragma section("ro_after_init$m", read, write) +#define READ_ONLY_SYSVAR __declspec(allocate("ro_after_init$m")) +#else +#define READ_ONLY_SYSVAR +#endif + #include /* Some helper macros */ diff --git a/include/my_sys.h b/include/my_sys.h index db766777de358..a8c6546a77fa2 100644 --- a/include/my_sys.h +++ b/include/my_sys.h @@ -93,7 +93,7 @@ C_MODE_START #define MY_SYNC 4096U /* my_copy(): sync dst file */ #define MY_SYNC_DIR 32768U /* my_create/delete/rename: sync directory */ #define MY_THREAD_SPECIFIC 0x10000U /* my_malloc(): thread specific */ -#define MY_ROOT_USE_MPROTECT 0x20000U /* init_alloc_root: read only segments */ +#define MY_ROOT_USE_VMEM 0x20000U /* init_alloc_root: use my_virtual_mem_commit */ /* Tree that should delete things automatically */ #define MY_TREE_WITH_DELETE 0x40000U #define MY_TRACK 0x80000U /* Track tmp usage */ diff --git a/include/my_virtual_mem.h b/include/my_virtual_mem.h index b4f26ca979ca7..689c75d5258e3 100644 --- a/include/my_virtual_mem.h +++ b/include/my_virtual_mem.h @@ -1,4 +1,4 @@ -/* Copyright (c) 2025, MariaDB +/* Copyright (c) 2025, 2026, MariaDB plc. This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by @@ -24,12 +24,13 @@ extern "C" { #endif -#ifdef _WIN32 +enum my_vmem_prot { MY_VMEM_READONLY= 0, MY_VMEM_READWRITE }; + char *my_virtual_mem_reserve(size_t *size); -#endif char *my_virtual_mem_commit(char *ptr, size_t size); void my_virtual_mem_decommit(char *ptr, size_t size); void my_virtual_mem_release(char *ptr, size_t size); +void my_virtual_mem_protect(void *ptr, size_t size, enum my_vmem_prot prot); #ifdef __cplusplus } diff --git a/libmariadb b/libmariadb index 64e9fe09fea6a..81d7f0fd17bd2 160000 --- a/libmariadb +++ b/libmariadb @@ -1 +1 @@ -Subproject commit 64e9fe09fea6a017bab92e19a08d2301e3da5b56 +Subproject commit 81d7f0fd17bd287e6e4a13830e6a132dadc95ab2 diff --git a/mysql-test/main/alter_table_combinations.result b/mysql-test/main/alter_table_combinations.result index d14aaaac8f73d..a9e778c5a6b95 100644 --- a/mysql-test/main/alter_table_combinations.result +++ b/mysql-test/main/alter_table_combinations.result @@ -321,4 +321,13 @@ drop table t1; # # End of 10.5 tests # +# +# MDEV-31808 Server crash upon altering table with NEXTVAL for default under exclusive lock +# +CREATE SEQUENCE s ENGINE=Aria; +CREATE TABLE t (a INT DEFAULT(NEXTVAL(s)), b INT); +ALTER TABLE t FORCE, ALGORITHM=COPY, LOCK=EXCLUSIVE; +DROP TABLE t; +DROP SEQUENCE s; +# End of 10.6 tests set @@default_storage_engine= @save_default_engine; diff --git a/mysql-test/main/alter_table_combinations.test b/mysql-test/main/alter_table_combinations.test index 7c8d7f4209689..b208f0e4416f7 100644 --- a/mysql-test/main/alter_table_combinations.test +++ b/mysql-test/main/alter_table_combinations.test @@ -260,4 +260,18 @@ drop table t1; --echo # End of 10.5 tests --echo # +--echo # +--echo # MDEV-31808 Server crash upon altering table with NEXTVAL for default under exclusive lock +--echo # + +CREATE SEQUENCE s ENGINE=Aria; +CREATE TABLE t (a INT DEFAULT(NEXTVAL(s)), b INT); +ALTER TABLE t FORCE, ALGORITHM=COPY, LOCK=EXCLUSIVE; + +# Cleanup +DROP TABLE t; +DROP SEQUENCE s; + +--echo # End of 10.6 tests + set @@default_storage_engine= @save_default_engine; diff --git a/mysql-test/main/binlog_invalid_row_v2_tag.result b/mysql-test/main/binlog_invalid_row_v2_tag.result new file mode 100644 index 0000000000000..63f13f71399a1 --- /dev/null +++ b/mysql-test/main/binlog_invalid_row_v2_tag.result @@ -0,0 +1 @@ +FOUND 3 /[Uu]nknown [Ee]vent/ in invalid_row_v2_tag.sql diff --git a/mysql-test/main/binlog_invalid_row_v2_tag.test b/mysql-test/main/binlog_invalid_row_v2_tag.test new file mode 100644 index 0000000000000..283e27ff66a14 --- /dev/null +++ b/mysql-test/main/binlog_invalid_row_v2_tag.test @@ -0,0 +1,14 @@ +# MDEV-39485 Heap-buffer-overflow upon read in `Rows_log_event` constructor +# +# This binlog file contains a normal Format Description +# Event followed by 3 malformed v2 Write Rows Events: +# 1. A tag followed by no data +# 2. An undersized tagged data followed by an unrecognized tag +# 3. A tag followed by a length longer than the entire event + +--source include/not_embedded.inc + +--let SEARCH_PATTERN= [Uu]nknown [Ee]vent +--let SEARCH_FILE= $MYSQLTEST_VARDIR/tmp/invalid_row_v2_tag.sql +--exec $MYSQL_BINLOG --force-read --verbose std_data/binlog_invalid_row_v2_tag.001 > $SEARCH_FILE +--source include/search_pattern_in_file.inc diff --git a/mysql-test/main/ddl_i18n_koi8r.result b/mysql-test/main/ddl_i18n_koi8r.result index deae5eda0cdcb..68c275323d22b 100644 --- a/mysql-test/main/ddl_i18n_koi8r.result +++ b/mysql-test/main/ddl_i18n_koi8r.result @@ -2509,18 +2509,18 @@ CREATE DATABASE /*!32312 IF NOT EXISTS*/ `mysqltest1` /*!40100 DEFAULT CHARACTER USE `mysqltest1`; /*!50106 SET @save_time_zone= @@TIME_ZONE */ ; +ALTER DATABASE `mysqltest1` CHARACTER SET utf8mb3 COLLATE utf8mb3_unicode_ci ; +/*!50003 SET @saved_cs_client = @@character_set_client */ ; +/*!50003 SET @saved_cs_results = @@character_set_results */ ; +/*!50003 SET @saved_col_connection = @@collation_connection */ ; +/*!50003 SET character_set_client = koi8r */ ; +/*!50003 SET character_set_results = koi8r */ ; +/*!50003 SET collation_connection = koi8r_general_ci */ ; +/*!50003 SET @saved_sql_mode = @@sql_mode */ ; +/*!50003 SET sql_mode = '' */ ; +/*!50003 SET @saved_time_zone = @@time_zone */ ; +/*!50003 SET time_zone = 'SYSTEM' */ ; DELIMITER ;; -ALTER DATABASE `mysqltest1` CHARACTER SET utf8mb3 COLLATE utf8mb3_unicode_ci ;; -/*!50003 SET @saved_cs_client = @@character_set_client */ ;; -/*!50003 SET @saved_cs_results = @@character_set_results */ ;; -/*!50003 SET @saved_col_connection = @@collation_connection */ ;; -/*!50003 SET character_set_client = koi8r */ ;; -/*!50003 SET character_set_results = koi8r */ ;; -/*!50003 SET collation_connection = koi8r_general_ci */ ;; -/*!50003 SET @saved_sql_mode = @@sql_mode */ ;; -/*!50003 SET sql_mode = '' */ ;; -/*!50003 SET @saved_time_zone = @@time_zone */ ;; -/*!50003 SET time_zone = 'SYSTEM' */ ;; /*!50106 CREATE*/ /*!50117 DEFINER=`root`@`localhost`*/ /*!50106 EVENT `ev1` ON SCHEDULE AT '1970-01-02 00:00:00' ON COMPLETION PRESERVE DISABLE DO BEGIN DECLARE 1 CHAR(10); SELECT @@ -2532,24 +2532,25 @@ COLLATION(_utf8 'текст') AS c4, @@character_set_client AS c6; END */ ;; -/*!50003 SET time_zone = @saved_time_zone */ ;; -/*!50003 SET sql_mode = @saved_sql_mode */ ;; -/*!50003 SET character_set_client = @saved_cs_client */ ;; -/*!50003 SET character_set_results = @saved_cs_results */ ;; -/*!50003 SET collation_connection = @saved_col_connection */ ;; -ALTER DATABASE `mysqltest1` CHARACTER SET cp866 COLLATE cp866_general_ci ;; +DELIMITER ; +/*!50003 SET time_zone = @saved_time_zone */ ; +/*!50003 SET sql_mode = @saved_sql_mode */ ; +/*!50003 SET character_set_client = @saved_cs_client */ ; +/*!50003 SET character_set_results = @saved_cs_results */ ; +/*!50003 SET collation_connection = @saved_col_connection */ ; +ALTER DATABASE `mysqltest1` CHARACTER SET cp866 COLLATE cp866_general_ci ; +ALTER DATABASE `mysqltest1` CHARACTER SET utf8mb3 COLLATE utf8mb3_unicode_ci ; +/*!50003 SET @saved_cs_client = @@character_set_client */ ; +/*!50003 SET @saved_cs_results = @@character_set_results */ ; +/*!50003 SET @saved_col_connection = @@collation_connection */ ; +/*!50003 SET character_set_client = koi8r */ ; +/*!50003 SET character_set_results = koi8r */ ; +/*!50003 SET collation_connection = koi8r_general_ci */ ; +/*!50003 SET @saved_sql_mode = @@sql_mode */ ; +/*!50003 SET sql_mode = '' */ ; +/*!50003 SET @saved_time_zone = @@time_zone */ ; +/*!50003 SET time_zone = 'SYSTEM' */ ; DELIMITER ;; -ALTER DATABASE `mysqltest1` CHARACTER SET utf8mb3 COLLATE utf8mb3_unicode_ci ;; -/*!50003 SET @saved_cs_client = @@character_set_client */ ;; -/*!50003 SET @saved_cs_results = @@character_set_results */ ;; -/*!50003 SET @saved_col_connection = @@collation_connection */ ;; -/*!50003 SET character_set_client = koi8r */ ;; -/*!50003 SET character_set_results = koi8r */ ;; -/*!50003 SET collation_connection = koi8r_general_ci */ ;; -/*!50003 SET @saved_sql_mode = @@sql_mode */ ;; -/*!50003 SET sql_mode = '' */ ;; -/*!50003 SET @saved_time_zone = @@time_zone */ ;; -/*!50003 SET time_zone = 'SYSTEM' */ ;; /*!50106 CREATE*/ /*!50117 DEFINER=`root`@`localhost`*/ /*!50106 EVENT `ev2` ON SCHEDULE AT '1970-01-02 00:00:00' ON COMPLETION PRESERVE DISABLE DO BEGIN DECLARE 1 CHAR(10) CHARACTER SET utf8; SELECT @@ -2561,13 +2562,13 @@ COLLATION(_utf8 'текст') AS c4, @@character_set_client AS c6; END */ ;; -/*!50003 SET time_zone = @saved_time_zone */ ;; -/*!50003 SET sql_mode = @saved_sql_mode */ ;; -/*!50003 SET character_set_client = @saved_cs_client */ ;; -/*!50003 SET character_set_results = @saved_cs_results */ ;; -/*!50003 SET collation_connection = @saved_col_connection */ ;; -ALTER DATABASE `mysqltest1` CHARACTER SET cp866 COLLATE cp866_general_ci ;; DELIMITER ; +/*!50003 SET time_zone = @saved_time_zone */ ; +/*!50003 SET sql_mode = @saved_sql_mode */ ; +/*!50003 SET character_set_client = @saved_cs_client */ ; +/*!50003 SET character_set_results = @saved_cs_results */ ; +/*!50003 SET collation_connection = @saved_col_connection */ ; +ALTER DATABASE `mysqltest1` CHARACTER SET cp866 COLLATE cp866_general_ci ; /*!50106 SET TIME_ZONE= @save_time_zone */ ; ---> Dumping mysqltest1 to ddl_i18n_koi8r.events.mysqltest1.sql @@ -2579,18 +2580,18 @@ CREATE DATABASE /*!32312 IF NOT EXISTS*/ `mysqltest2` /*!40100 DEFAULT CHARACTER USE `mysqltest2`; /*!50106 SET @save_time_zone= @@TIME_ZONE */ ; +ALTER DATABASE `mysqltest2` CHARACTER SET utf8mb3 COLLATE utf8mb3_unicode_ci ; +/*!50003 SET @saved_cs_client = @@character_set_client */ ; +/*!50003 SET @saved_cs_results = @@character_set_results */ ; +/*!50003 SET @saved_col_connection = @@collation_connection */ ; +/*!50003 SET character_set_client = koi8r */ ; +/*!50003 SET character_set_results = koi8r */ ; +/*!50003 SET collation_connection = koi8r_general_ci */ ; +/*!50003 SET @saved_sql_mode = @@sql_mode */ ; +/*!50003 SET sql_mode = '' */ ; +/*!50003 SET @saved_time_zone = @@time_zone */ ; +/*!50003 SET time_zone = 'SYSTEM' */ ; DELIMITER ;; -ALTER DATABASE `mysqltest2` CHARACTER SET utf8mb3 COLLATE utf8mb3_unicode_ci ;; -/*!50003 SET @saved_cs_client = @@character_set_client */ ;; -/*!50003 SET @saved_cs_results = @@character_set_results */ ;; -/*!50003 SET @saved_col_connection = @@collation_connection */ ;; -/*!50003 SET character_set_client = koi8r */ ;; -/*!50003 SET character_set_results = koi8r */ ;; -/*!50003 SET collation_connection = koi8r_general_ci */ ;; -/*!50003 SET @saved_sql_mode = @@sql_mode */ ;; -/*!50003 SET sql_mode = '' */ ;; -/*!50003 SET @saved_time_zone = @@time_zone */ ;; -/*!50003 SET time_zone = 'SYSTEM' */ ;; /*!50106 CREATE*/ /*!50117 DEFINER=`root`@`localhost`*/ /*!50106 EVENT `ev3` ON SCHEDULE AT '1970-01-02 00:00:00' ON COMPLETION PRESERVE DISABLE DO BEGIN DECLARE 1 CHAR(10) CHARACTER SET utf8; SELECT @@ -2602,24 +2603,25 @@ COLLATION(_utf8 'текст') AS c4, @@character_set_client AS c6; END */ ;; -/*!50003 SET time_zone = @saved_time_zone */ ;; -/*!50003 SET sql_mode = @saved_sql_mode */ ;; -/*!50003 SET character_set_client = @saved_cs_client */ ;; -/*!50003 SET character_set_results = @saved_cs_results */ ;; -/*!50003 SET collation_connection = @saved_col_connection */ ;; -ALTER DATABASE `mysqltest2` CHARACTER SET cp866 COLLATE cp866_general_ci ;; +DELIMITER ; +/*!50003 SET time_zone = @saved_time_zone */ ; +/*!50003 SET sql_mode = @saved_sql_mode */ ; +/*!50003 SET character_set_client = @saved_cs_client */ ; +/*!50003 SET character_set_results = @saved_cs_results */ ; +/*!50003 SET collation_connection = @saved_col_connection */ ; +ALTER DATABASE `mysqltest2` CHARACTER SET cp866 COLLATE cp866_general_ci ; +ALTER DATABASE `mysqltest2` CHARACTER SET utf8mb3 COLLATE utf8mb3_unicode_ci ; +/*!50003 SET @saved_cs_client = @@character_set_client */ ; +/*!50003 SET @saved_cs_results = @@character_set_results */ ; +/*!50003 SET @saved_col_connection = @@collation_connection */ ; +/*!50003 SET character_set_client = koi8r */ ; +/*!50003 SET character_set_results = koi8r */ ; +/*!50003 SET collation_connection = koi8r_general_ci */ ; +/*!50003 SET @saved_sql_mode = @@sql_mode */ ; +/*!50003 SET sql_mode = '' */ ; +/*!50003 SET @saved_time_zone = @@time_zone */ ; +/*!50003 SET time_zone = 'SYSTEM' */ ; DELIMITER ;; -ALTER DATABASE `mysqltest2` CHARACTER SET utf8mb3 COLLATE utf8mb3_unicode_ci ;; -/*!50003 SET @saved_cs_client = @@character_set_client */ ;; -/*!50003 SET @saved_cs_results = @@character_set_results */ ;; -/*!50003 SET @saved_col_connection = @@collation_connection */ ;; -/*!50003 SET character_set_client = koi8r */ ;; -/*!50003 SET character_set_results = koi8r */ ;; -/*!50003 SET collation_connection = koi8r_general_ci */ ;; -/*!50003 SET @saved_sql_mode = @@sql_mode */ ;; -/*!50003 SET sql_mode = '' */ ;; -/*!50003 SET @saved_time_zone = @@time_zone */ ;; -/*!50003 SET time_zone = 'SYSTEM' */ ;; /*!50106 CREATE*/ /*!50117 DEFINER=`root`@`localhost`*/ /*!50106 EVENT `ev4` ON SCHEDULE AT '1970-01-02 00:00:00' ON COMPLETION PRESERVE DISABLE DO BEGIN DECLARE 1 CHAR(10) CHARACTER SET utf8; SELECT @@ -2631,13 +2633,13 @@ COLLATION(_utf8 'текст') AS c4, @@character_set_client AS c6; END */ ;; -/*!50003 SET time_zone = @saved_time_zone */ ;; -/*!50003 SET sql_mode = @saved_sql_mode */ ;; -/*!50003 SET character_set_client = @saved_cs_client */ ;; -/*!50003 SET character_set_results = @saved_cs_results */ ;; -/*!50003 SET collation_connection = @saved_col_connection */ ;; -ALTER DATABASE `mysqltest2` CHARACTER SET cp866 COLLATE cp866_general_ci ;; DELIMITER ; +/*!50003 SET time_zone = @saved_time_zone */ ; +/*!50003 SET sql_mode = @saved_sql_mode */ ; +/*!50003 SET character_set_client = @saved_cs_client */ ; +/*!50003 SET character_set_results = @saved_cs_results */ ; +/*!50003 SET collation_connection = @saved_col_connection */ ; +ALTER DATABASE `mysqltest2` CHARACTER SET cp866 COLLATE cp866_general_ci ; /*!50106 SET TIME_ZONE= @save_time_zone */ ; ---> Dumping mysqltest2 to ddl_i18n_koi8r.events.mysqltest2.sql diff --git a/mysql-test/main/ddl_i18n_utf8.result b/mysql-test/main/ddl_i18n_utf8.result index 0922471f7ea08..000ed997393d7 100644 --- a/mysql-test/main/ddl_i18n_utf8.result +++ b/mysql-test/main/ddl_i18n_utf8.result @@ -2509,18 +2509,18 @@ CREATE DATABASE /*!32312 IF NOT EXISTS*/ `mysqltest1` /*!40100 DEFAULT CHARACTER USE `mysqltest1`; /*!50106 SET @save_time_zone= @@TIME_ZONE */ ; +ALTER DATABASE `mysqltest1` CHARACTER SET utf8mb3 COLLATE utf8mb3_unicode_ci ; +/*!50003 SET @saved_cs_client = @@character_set_client */ ; +/*!50003 SET @saved_cs_results = @@character_set_results */ ; +/*!50003 SET @saved_col_connection = @@collation_connection */ ; +/*!50003 SET character_set_client = utf8mb3 */ ; +/*!50003 SET character_set_results = utf8mb3 */ ; +/*!50003 SET collation_connection = utf8mb3_uca1400_ai_ci */ ; +/*!50003 SET @saved_sql_mode = @@sql_mode */ ; +/*!50003 SET sql_mode = '' */ ; +/*!50003 SET @saved_time_zone = @@time_zone */ ; +/*!50003 SET time_zone = 'SYSTEM' */ ; DELIMITER ;; -ALTER DATABASE `mysqltest1` CHARACTER SET utf8mb3 COLLATE utf8mb3_unicode_ci ;; -/*!50003 SET @saved_cs_client = @@character_set_client */ ;; -/*!50003 SET @saved_cs_results = @@character_set_results */ ;; -/*!50003 SET @saved_col_connection = @@collation_connection */ ;; -/*!50003 SET character_set_client = utf8mb3 */ ;; -/*!50003 SET character_set_results = utf8mb3 */ ;; -/*!50003 SET collation_connection = utf8mb3_uca1400_ai_ci */ ;; -/*!50003 SET @saved_sql_mode = @@sql_mode */ ;; -/*!50003 SET sql_mode = '' */ ;; -/*!50003 SET @saved_time_zone = @@time_zone */ ;; -/*!50003 SET time_zone = 'SYSTEM' */ ;; /*!50106 CREATE*/ /*!50117 DEFINER=`root`@`localhost`*/ /*!50106 EVENT `ev1` ON SCHEDULE AT '1970-01-02 00:00:00' ON COMPLETION PRESERVE DISABLE DO BEGIN DECLARE перем1 CHAR(10); SELECT @@ -2532,24 +2532,25 @@ COLLATION(_koi8r ' @@character_set_client AS c6; END */ ;; -/*!50003 SET time_zone = @saved_time_zone */ ;; -/*!50003 SET sql_mode = @saved_sql_mode */ ;; -/*!50003 SET character_set_client = @saved_cs_client */ ;; -/*!50003 SET character_set_results = @saved_cs_results */ ;; -/*!50003 SET collation_connection = @saved_col_connection */ ;; -ALTER DATABASE `mysqltest1` CHARACTER SET cp866 COLLATE cp866_general_ci ;; +DELIMITER ; +/*!50003 SET time_zone = @saved_time_zone */ ; +/*!50003 SET sql_mode = @saved_sql_mode */ ; +/*!50003 SET character_set_client = @saved_cs_client */ ; +/*!50003 SET character_set_results = @saved_cs_results */ ; +/*!50003 SET collation_connection = @saved_col_connection */ ; +ALTER DATABASE `mysqltest1` CHARACTER SET cp866 COLLATE cp866_general_ci ; +ALTER DATABASE `mysqltest1` CHARACTER SET utf8mb3 COLLATE utf8mb3_unicode_ci ; +/*!50003 SET @saved_cs_client = @@character_set_client */ ; +/*!50003 SET @saved_cs_results = @@character_set_results */ ; +/*!50003 SET @saved_col_connection = @@collation_connection */ ; +/*!50003 SET character_set_client = utf8mb3 */ ; +/*!50003 SET character_set_results = utf8mb3 */ ; +/*!50003 SET collation_connection = utf8mb3_uca1400_ai_ci */ ; +/*!50003 SET @saved_sql_mode = @@sql_mode */ ; +/*!50003 SET sql_mode = '' */ ; +/*!50003 SET @saved_time_zone = @@time_zone */ ; +/*!50003 SET time_zone = 'SYSTEM' */ ; DELIMITER ;; -ALTER DATABASE `mysqltest1` CHARACTER SET utf8mb3 COLLATE utf8mb3_unicode_ci ;; -/*!50003 SET @saved_cs_client = @@character_set_client */ ;; -/*!50003 SET @saved_cs_results = @@character_set_results */ ;; -/*!50003 SET @saved_col_connection = @@collation_connection */ ;; -/*!50003 SET character_set_client = utf8mb3 */ ;; -/*!50003 SET character_set_results = utf8mb3 */ ;; -/*!50003 SET collation_connection = utf8mb3_uca1400_ai_ci */ ;; -/*!50003 SET @saved_sql_mode = @@sql_mode */ ;; -/*!50003 SET sql_mode = '' */ ;; -/*!50003 SET @saved_time_zone = @@time_zone */ ;; -/*!50003 SET time_zone = 'SYSTEM' */ ;; /*!50106 CREATE*/ /*!50117 DEFINER=`root`@`localhost`*/ /*!50106 EVENT `ev2` ON SCHEDULE AT '1970-01-02 00:00:00' ON COMPLETION PRESERVE DISABLE DO BEGIN DECLARE перем1 CHAR(10) CHARACTER SET utf8; SELECT @@ -2561,13 +2562,13 @@ COLLATION(_koi8r ' @@character_set_client AS c6; END */ ;; -/*!50003 SET time_zone = @saved_time_zone */ ;; -/*!50003 SET sql_mode = @saved_sql_mode */ ;; -/*!50003 SET character_set_client = @saved_cs_client */ ;; -/*!50003 SET character_set_results = @saved_cs_results */ ;; -/*!50003 SET collation_connection = @saved_col_connection */ ;; -ALTER DATABASE `mysqltest1` CHARACTER SET cp866 COLLATE cp866_general_ci ;; DELIMITER ; +/*!50003 SET time_zone = @saved_time_zone */ ; +/*!50003 SET sql_mode = @saved_sql_mode */ ; +/*!50003 SET character_set_client = @saved_cs_client */ ; +/*!50003 SET character_set_results = @saved_cs_results */ ; +/*!50003 SET collation_connection = @saved_col_connection */ ; +ALTER DATABASE `mysqltest1` CHARACTER SET cp866 COLLATE cp866_general_ci ; /*!50106 SET TIME_ZONE= @save_time_zone */ ; ---> Dumping mysqltest1 to ddl_i18n_utf8events.mysqltest1.sql @@ -2579,18 +2580,18 @@ CREATE DATABASE /*!32312 IF NOT EXISTS*/ `mysqltest2` /*!40100 DEFAULT CHARACTER USE `mysqltest2`; /*!50106 SET @save_time_zone= @@TIME_ZONE */ ; +ALTER DATABASE `mysqltest2` CHARACTER SET utf8mb3 COLLATE utf8mb3_unicode_ci ; +/*!50003 SET @saved_cs_client = @@character_set_client */ ; +/*!50003 SET @saved_cs_results = @@character_set_results */ ; +/*!50003 SET @saved_col_connection = @@collation_connection */ ; +/*!50003 SET character_set_client = utf8mb3 */ ; +/*!50003 SET character_set_results = utf8mb3 */ ; +/*!50003 SET collation_connection = utf8mb3_uca1400_ai_ci */ ; +/*!50003 SET @saved_sql_mode = @@sql_mode */ ; +/*!50003 SET sql_mode = '' */ ; +/*!50003 SET @saved_time_zone = @@time_zone */ ; +/*!50003 SET time_zone = 'SYSTEM' */ ; DELIMITER ;; -ALTER DATABASE `mysqltest2` CHARACTER SET utf8mb3 COLLATE utf8mb3_unicode_ci ;; -/*!50003 SET @saved_cs_client = @@character_set_client */ ;; -/*!50003 SET @saved_cs_results = @@character_set_results */ ;; -/*!50003 SET @saved_col_connection = @@collation_connection */ ;; -/*!50003 SET character_set_client = utf8mb3 */ ;; -/*!50003 SET character_set_results = utf8mb3 */ ;; -/*!50003 SET collation_connection = utf8mb3_uca1400_ai_ci */ ;; -/*!50003 SET @saved_sql_mode = @@sql_mode */ ;; -/*!50003 SET sql_mode = '' */ ;; -/*!50003 SET @saved_time_zone = @@time_zone */ ;; -/*!50003 SET time_zone = 'SYSTEM' */ ;; /*!50106 CREATE*/ /*!50117 DEFINER=`root`@`localhost`*/ /*!50106 EVENT `ev3` ON SCHEDULE AT '1970-01-02 00:00:00' ON COMPLETION PRESERVE DISABLE DO BEGIN DECLARE перем1 CHAR(10) CHARACTER SET utf8; SELECT @@ -2602,24 +2603,25 @@ COLLATION(_koi8r ' @@character_set_client AS c6; END */ ;; -/*!50003 SET time_zone = @saved_time_zone */ ;; -/*!50003 SET sql_mode = @saved_sql_mode */ ;; -/*!50003 SET character_set_client = @saved_cs_client */ ;; -/*!50003 SET character_set_results = @saved_cs_results */ ;; -/*!50003 SET collation_connection = @saved_col_connection */ ;; -ALTER DATABASE `mysqltest2` CHARACTER SET cp866 COLLATE cp866_general_ci ;; +DELIMITER ; +/*!50003 SET time_zone = @saved_time_zone */ ; +/*!50003 SET sql_mode = @saved_sql_mode */ ; +/*!50003 SET character_set_client = @saved_cs_client */ ; +/*!50003 SET character_set_results = @saved_cs_results */ ; +/*!50003 SET collation_connection = @saved_col_connection */ ; +ALTER DATABASE `mysqltest2` CHARACTER SET cp866 COLLATE cp866_general_ci ; +ALTER DATABASE `mysqltest2` CHARACTER SET utf8mb3 COLLATE utf8mb3_unicode_ci ; +/*!50003 SET @saved_cs_client = @@character_set_client */ ; +/*!50003 SET @saved_cs_results = @@character_set_results */ ; +/*!50003 SET @saved_col_connection = @@collation_connection */ ; +/*!50003 SET character_set_client = utf8mb3 */ ; +/*!50003 SET character_set_results = utf8mb3 */ ; +/*!50003 SET collation_connection = utf8mb3_uca1400_ai_ci */ ; +/*!50003 SET @saved_sql_mode = @@sql_mode */ ; +/*!50003 SET sql_mode = '' */ ; +/*!50003 SET @saved_time_zone = @@time_zone */ ; +/*!50003 SET time_zone = 'SYSTEM' */ ; DELIMITER ;; -ALTER DATABASE `mysqltest2` CHARACTER SET utf8mb3 COLLATE utf8mb3_unicode_ci ;; -/*!50003 SET @saved_cs_client = @@character_set_client */ ;; -/*!50003 SET @saved_cs_results = @@character_set_results */ ;; -/*!50003 SET @saved_col_connection = @@collation_connection */ ;; -/*!50003 SET character_set_client = utf8mb3 */ ;; -/*!50003 SET character_set_results = utf8mb3 */ ;; -/*!50003 SET collation_connection = utf8mb3_uca1400_ai_ci */ ;; -/*!50003 SET @saved_sql_mode = @@sql_mode */ ;; -/*!50003 SET sql_mode = '' */ ;; -/*!50003 SET @saved_time_zone = @@time_zone */ ;; -/*!50003 SET time_zone = 'SYSTEM' */ ;; /*!50106 CREATE*/ /*!50117 DEFINER=`root`@`localhost`*/ /*!50106 EVENT `ev4` ON SCHEDULE AT '1970-01-02 00:00:00' ON COMPLETION PRESERVE DISABLE DO BEGIN DECLARE перем1 CHAR(10) CHARACTER SET utf8; SELECT @@ -2631,13 +2633,13 @@ COLLATION(_koi8r ' @@character_set_client AS c6; END */ ;; -/*!50003 SET time_zone = @saved_time_zone */ ;; -/*!50003 SET sql_mode = @saved_sql_mode */ ;; -/*!50003 SET character_set_client = @saved_cs_client */ ;; -/*!50003 SET character_set_results = @saved_cs_results */ ;; -/*!50003 SET collation_connection = @saved_col_connection */ ;; -ALTER DATABASE `mysqltest2` CHARACTER SET cp866 COLLATE cp866_general_ci ;; DELIMITER ; +/*!50003 SET time_zone = @saved_time_zone */ ; +/*!50003 SET sql_mode = @saved_sql_mode */ ; +/*!50003 SET character_set_client = @saved_cs_client */ ; +/*!50003 SET character_set_results = @saved_cs_results */ ; +/*!50003 SET collation_connection = @saved_col_connection */ ; +ALTER DATABASE `mysqltest2` CHARACTER SET cp866 COLLATE cp866_general_ci ; /*!50106 SET TIME_ZONE= @save_time_zone */ ; ---> Dumping mysqltest2 to ddl_i18n_utf8events.mysqltest2.sql diff --git a/mysql-test/main/func_json.result b/mysql-test/main/func_json.result index 5d7c3268bf9d4..540e47fefbd5b 100644 --- a/mysql-test/main/func_json.result +++ b/mysql-test/main/func_json.result @@ -2980,6 +2980,133 @@ NULL SELECT JSON_OVERLAPS(JSON_ARRAY_INSERT(NULL,'$[0]',1), '[]') AS x; x NULL +# +# MDEV-32331 JSON path functions with no charset on path crash server +# +SELECT 1 FROM +(SELECT CASE WHEN x * x THEN x END as a +FROM ( SELECT json_array_append ( 'x' , ( 'x' % 'x' ) , 1 , 'x' , 1 ) as x ) dt2 +) dt +WHERE a; +1 +Warnings: +Warning 1292 Truncated incorrect DOUBLE value: 'x' +Warning 1292 Truncated incorrect DOUBLE value: 'x' +Warning 1292 Truncated incorrect DOUBLE value: 'x' +Warning 1292 Truncated incorrect DOUBLE value: 'x' +SELECT 1 FROM (SELECT CASE WHEN x * x THEN x END as a FROM ( SELECT json_array_append ( 'x' , ( 'x' % 'x' ) , 1 , 'x' , 1 ) as x ) dt2 ) dt WHERE a; +1 +Warnings: +Warning 1292 Truncated incorrect DOUBLE value: 'x' +Warning 1292 Truncated incorrect DOUBLE value: 'x' +Warning 1292 Truncated incorrect DOUBLE value: 'x' +Warning 1292 Truncated incorrect DOUBLE value: 'x' +SELECT 1 FROM (SELECT CASE WHEN x * x THEN x END as a FROM ( SELECT json_array_insert ( '[1]' , ( 'x' % 'x' ) , 1 , '$[0]' , 9 ) as x ) dt2 ) dt WHERE a; +1 +Warnings: +Warning 1292 Truncated incorrect DOUBLE value: 'x' +Warning 1292 Truncated incorrect DOUBLE value: 'x' +Warning 1292 Truncated incorrect DOUBLE value: 'x' +Warning 1292 Truncated incorrect DOUBLE value: 'x' +SELECT 1 FROM (SELECT CASE WHEN x * x THEN x END as a FROM ( SELECT json_insert ( '1' , ( 'x' % 'x' ) , 1 , '$.a' , 2 ) as x ) dt2 ) dt WHERE a; +1 +Warnings: +Warning 1292 Truncated incorrect DOUBLE value: 'x' +Warning 1292 Truncated incorrect DOUBLE value: 'x' +Warning 1292 Truncated incorrect DOUBLE value: 'x' +Warning 1292 Truncated incorrect DOUBLE value: 'x' +SELECT 1 FROM (SELECT CASE WHEN x * x THEN x END as a FROM ( SELECT json_remove ( '[1,2]' , ( 'x' % 'x' ) , '$[0]' ) as x ) dt2 ) dt WHERE a; +1 +Warnings: +Warning 1292 Truncated incorrect DOUBLE value: 'x' +Warning 1292 Truncated incorrect DOUBLE value: 'x' +Warning 1292 Truncated incorrect DOUBLE value: 'x' +Warning 1292 Truncated incorrect DOUBLE value: 'x' +SELECT 1 FROM (SELECT CASE WHEN x * x THEN x END as a FROM ( SELECT json_replace ( '1' , ( 'x' % 'x' ) , 1 , '$.a' , 2 ) as x ) dt2 ) dt WHERE a; +1 +Warnings: +Warning 1292 Truncated incorrect DOUBLE value: 'x' +Warning 1292 Truncated incorrect DOUBLE value: 'x' +Warning 1292 Truncated incorrect DOUBLE value: 'x' +Warning 1292 Truncated incorrect DOUBLE value: 'x' +SELECT 1 FROM (SELECT CASE WHEN x * x THEN x END as a FROM ( SELECT json_set ( '1' , ( 'x' % 'x' ) , 1 , '$.a' , 2 ) as x ) dt2 ) dt WHERE a; +1 +Warnings: +Warning 1292 Truncated incorrect DOUBLE value: 'x' +Warning 1292 Truncated incorrect DOUBLE value: 'x' +Warning 1292 Truncated incorrect DOUBLE value: 'x' +Warning 1292 Truncated incorrect DOUBLE value: 'x' +SELECT x FROM (SELECT 1 AS x UNION SELECT 2) AS t WHERE x IN ( SELECT JSON_REPLACE('1', UPPER(CAST(NULL AS CHAR)), 100)); +x +SELECT ( WITH RECURSIVE x ( x ) AS (WITH RECURSIVE x ( x ) AS ( SELECT 1 UNION SELECT x + 1 FROM x ) SELECT json_array_append ( '[[], [], []]' , NOT ( NULL LIKE 'ABC%' ) , 315 )) SELECT x FROM x WHERE x > 10 AND x < 1) AS x; +x +NULL +SELECT 1 FROM (SELECT JSON_REMOVE(46, DATE(NULL)) AS x EXCEPT SELECT 1) AS d WHERE NOT(NOT(x > 10)) AND (x < 1 OR x > 1); +1 +# +# MDEV-40572 SIGSEGV in JSON_OVERLAPS() on a truncated nested JSON object with different character sets/collations +# +SET @col=@@collation_connection; +SET @char=@@character_set_connection; +SET collation_connection=ucs2_general_ci; +SELECT JSON_OVERLAPS ('{"B":{}}', '{"B":{') AS exp_ucs2; +exp_ucs2 +0 +Warnings: +Warning 4037 Unexpected end of JSON text in argument 2 to function 'json_overlaps' +SET character_set_connection=utf16; +SELECT JSON_OVERLAPS ('{"B":{}}', '{"B":{') AS exp_utf16; +exp_utf16 +0 +Warnings: +Warning 4037 Unexpected end of JSON text in argument 2 to function 'json_overlaps' +SET @@collation_connection=@col; +SET @@character_set_connection=@char; +# +# MDEV-40165 JSON_EQUAL/JSON_NORMALIZE/JSON_CONTAINS error handling +# +CREATE TABLE t(c1 JSON, c2 JSON); +INSERT INTO t VALUES ('{"a":1,"b":2}', '{"b":2E0, "a":1E0}'), +(NULL, '["a", "b"]'), +('{"r": 1}', NULL), +('[{"a": 3}, {"d": [1, 2, 3]}]', '[{"d": [1E0, 2E0, 3E0]}, {"a":3E0}]'); +SELECT c1, JSON_EQUALS(c1, '{"r": 1E0}') AS je_c1_const FROM t; +c1 je_c1_const +{"a":1,"b":2} 0 +NULL NULL +{"r": 1} 1 +[{"a": 3}, {"d": [1, 2, 3]}] 0 +SELECT c1, c2, JSON_EQUALS(c1, c2) AS je_c1_c2 FROM t; +c1 c2 je_c1_c2 +{"a":1,"b":2} {"b":2E0, "a":1E0} 1 +NULL ["a", "b"] NULL +{"r": 1} NULL NULL +[{"a": 3}, {"d": [1, 2, 3]}] [{"d": [1E0, 2E0, 3E0]}, {"a":3E0}] 0 +SELECT c1, JSON_CONTAINS(c1, '{"r": 1E0}') AS jc_c1_const FROM t; +c1 jc_c1_const +{"a":1,"b":2} 0 +NULL NULL +{"r": 1} 1 +[{"a": 3}, {"d": [1, 2, 3]}] 0 +SELECT c1, c2, JSON_CONTAINS(c1, c2) AS jc_c1_c2 FROM t; +c1 c2 jc_c1_c2 +{"a":1,"b":2} {"b":2E0, "a":1E0} 1 +NULL ["a", "b"] NULL +{"r": 1} NULL NULL +[{"a": 3}, {"d": [1, 2, 3]}] [{"d": [1E0, 2E0, 3E0]}, {"a":3E0}] 1 +SELECT c1, JSON_OVERLAPS(c1, '{"r": 1E0}') AS jo_c1_const FROM t; +c1 jo_c1_const +{"a":1,"b":2} 0 +NULL NULL +{"r": 1} 1 +[{"a": 3}, {"d": [1, 2, 3]}] 0 +SELECT c1, c2, JSON_OVERLAPS(c1, c2) AS jo_c1_c2 FROM t; +c1 c2 jo_c1_c2 +{"a":1,"b":2} {"b":2E0, "a":1E0} 1 +NULL ["a", "b"] NULL +{"r": 1} NULL NULL +[{"a": 3}, {"d": [1, 2, 3]}] [{"d": [1E0, 2E0, 3E0]}, {"a":3E0}] 1 +DROP TABLE t; # End of 10.11 Test # # MDEV-32007: JSON_VALUE and JSON_EXTRACT doesn't handle dash (-) diff --git a/mysql-test/main/func_json.test b/mysql-test/main/func_json.test index cc679cc81c021..bdcd22c0a9f55 100644 --- a/mysql-test/main/func_json.test +++ b/mysql-test/main/func_json.test @@ -2114,6 +2114,61 @@ SELECT JSON_ARRAY_INSERT (0,NULL,1) as j; SELECT JSON_EQUALS('[]', JSON_ARRAY_INSERT(NULL,'$[0]',1)) AS x; SELECT JSON_OVERLAPS(JSON_ARRAY_INSERT(NULL,'$[0]',1), '[]') AS x; +--echo # +--echo # MDEV-32331 JSON path functions with no charset on path crash server +--echo # + +SELECT 1 FROM + (SELECT CASE WHEN x * x THEN x END as a + FROM ( SELECT json_array_append ( 'x' , ( 'x' % 'x' ) , 1 , 'x' , 1 ) as x ) dt2 + ) dt + WHERE a; +SELECT 1 FROM (SELECT CASE WHEN x * x THEN x END as a FROM ( SELECT json_array_append ( 'x' , ( 'x' % 'x' ) , 1 , 'x' , 1 ) as x ) dt2 ) dt WHERE a; +SELECT 1 FROM (SELECT CASE WHEN x * x THEN x END as a FROM ( SELECT json_array_insert ( '[1]' , ( 'x' % 'x' ) , 1 , '$[0]' , 9 ) as x ) dt2 ) dt WHERE a; +SELECT 1 FROM (SELECT CASE WHEN x * x THEN x END as a FROM ( SELECT json_insert ( '1' , ( 'x' % 'x' ) , 1 , '$.a' , 2 ) as x ) dt2 ) dt WHERE a; +SELECT 1 FROM (SELECT CASE WHEN x * x THEN x END as a FROM ( SELECT json_remove ( '[1,2]' , ( 'x' % 'x' ) , '$[0]' ) as x ) dt2 ) dt WHERE a; +SELECT 1 FROM (SELECT CASE WHEN x * x THEN x END as a FROM ( SELECT json_replace ( '1' , ( 'x' % 'x' ) , 1 , '$.a' , 2 ) as x ) dt2 ) dt WHERE a; +SELECT 1 FROM (SELECT CASE WHEN x * x THEN x END as a FROM ( SELECT json_set ( '1' , ( 'x' % 'x' ) , 1 , '$.a' , 2 ) as x ) dt2 ) dt WHERE a; + +SELECT x FROM (SELECT 1 AS x UNION SELECT 2) AS t WHERE x IN ( SELECT JSON_REPLACE('1', UPPER(CAST(NULL AS CHAR)), 100)); +SELECT ( WITH RECURSIVE x ( x ) AS (WITH RECURSIVE x ( x ) AS ( SELECT 1 UNION SELECT x + 1 FROM x ) SELECT json_array_append ( '[[], [], []]' , NOT ( NULL LIKE 'ABC%' ) , 315 )) SELECT x FROM x WHERE x > 10 AND x < 1) AS x; +SELECT 1 FROM (SELECT JSON_REMOVE(46, DATE(NULL)) AS x EXCEPT SELECT 1) AS d WHERE NOT(NOT(x > 10)) AND (x < 1 OR x > 1); + +--echo # +--echo # MDEV-40572 SIGSEGV in JSON_OVERLAPS() on a truncated nested JSON object with different character sets/collations +--echo # + +SET @col=@@collation_connection; +SET @char=@@character_set_connection; + +SET collation_connection=ucs2_general_ci; +SELECT JSON_OVERLAPS ('{"B":{}}', '{"B":{') AS exp_ucs2; +SET character_set_connection=utf16; +SELECT JSON_OVERLAPS ('{"B":{}}', '{"B":{') AS exp_utf16; + +SET @@collation_connection=@col; +SET @@character_set_connection=@char; + +--echo # +--echo # MDEV-40165 JSON_EQUAL/JSON_NORMALIZE/JSON_CONTAINS error handling +--echo # + +CREATE TABLE t(c1 JSON, c2 JSON); +INSERT INTO t VALUES ('{"a":1,"b":2}', '{"b":2E0, "a":1E0}'), + (NULL, '["a", "b"]'), + ('{"r": 1}', NULL), + ('[{"a": 3}, {"d": [1, 2, 3]}]', '[{"d": [1E0, 2E0, 3E0]}, {"a":3E0}]'); + +SELECT c1, JSON_EQUALS(c1, '{"r": 1E0}') AS je_c1_const FROM t; +SELECT c1, c2, JSON_EQUALS(c1, c2) AS je_c1_c2 FROM t; + +SELECT c1, JSON_CONTAINS(c1, '{"r": 1E0}') AS jc_c1_const FROM t; +SELECT c1, c2, JSON_CONTAINS(c1, c2) AS jc_c1_c2 FROM t; + +SELECT c1, JSON_OVERLAPS(c1, '{"r": 1E0}') AS jo_c1_const FROM t; +SELECT c1, c2, JSON_OVERLAPS(c1, c2) AS jo_c1_c2 FROM t; +DROP TABLE t; + --echo # End of 10.11 Test --echo # diff --git a/mysql-test/main/func_sformat.result b/mysql-test/main/func_sformat.result index aaae3e71b75ed..8f789ee8cdfc7 100644 --- a/mysql-test/main/func_sformat.result +++ b/mysql-test/main/func_sformat.result @@ -459,13 +459,8 @@ select sformat('{:f}', @a); sformat('{:f}', @a) 3.140000 set names latin1; -# # End of 10.7 tests # -# -# Start of 10.8 tests -# -# # MDEV-29646 sformat('Num [{:20}]', 42) gives incorrect result in view # create view v1 as select sformat('Num [{:20}]', 42); @@ -498,4 +493,36 @@ bar 2 foo 1 drop view v; drop table t; +# +# MDEV-40411 SFORMAT ignores max_allowed_packet +# +set @old_max_allowed_packet=@@global.max_allowed_packet; +set @@global.max_allowed_packet=512*1024; +connect u,localhost,root; +select length(sformat('{:2147483647}', 1)); +length(sformat('{:2147483647}', 1)) +NULL +select length(sformat('{:524288}', 1)); +length(sformat('{:524288}', 1)) +524288 +select length(sformat('{:524289}', 1)); +length(sformat('{:524289}', 1)) +NULL +select length(sformat('{:524287}.', 1)); +length(sformat('{:524287}.', 1)) +524288 +select length(sformat('{:524287}..', 1)); +length(sformat('{:524287}..', 1)) +NULL +disconnect u; +connection default; +set @@global.max_allowed_packet= @old_max_allowed_packet; +set max_session_mem_used=512*1024; +select length(sformat('{:524288}', 1)); +length(sformat('{:524288}', 1)) +524288 +select length(sformat('{:524289}', 1)); +length(sformat('{:524289}', 1)) +NULL +set max_session_mem_used=default; # End of 10.11 tests diff --git a/mysql-test/main/func_sformat.test b/mysql-test/main/func_sformat.test index 9e86615d268b8..332d5a67027fd 100644 --- a/mysql-test/main/func_sformat.test +++ b/mysql-test/main/func_sformat.test @@ -236,13 +236,7 @@ set @a=3.14; select sformat('{:f}', @a); set names latin1; -echo #; echo # End of 10.7 tests; -echo #; - -echo #; -echo # Start of 10.8 tests; -echo #; echo #; echo # MDEV-29646 sformat('Num [{:20}]', 42) gives incorrect result in view; @@ -274,4 +268,30 @@ select * from v where a != sformat('{}', 'qux'); drop view v; drop table t; +echo #; +echo # MDEV-40411 SFORMAT ignores max_allowed_packet; +echo #; +set @old_max_allowed_packet=@@global.max_allowed_packet; +set @@global.max_allowed_packet=512*1024; +connect u,localhost,root; +# these three case: fmt allocates the requested size at once +select length(sformat('{:2147483647}', 1)); +select length(sformat('{:524288}', 1)); +select length(sformat('{:524289}', 1)); +# here it allocates 524287 and then grows by *1.5 +select length(sformat('{:524287}.', 1)); +select length(sformat('{:524287}..', 1)); +disconnect u; +connection default; +set @@global.max_allowed_packet= @old_max_allowed_packet; + +disable_cursor_protocol; +disable_view_protocol; +set max_session_mem_used=512*1024; +select length(sformat('{:524288}', 1)); +select length(sformat('{:524289}', 1)); +set max_session_mem_used=default; +enable_view_protocol; +enable_cursor_protocol; + echo # End of 10.11 tests; diff --git a/mysql-test/main/gis-json.result b/mysql-test/main/gis-json.result index 9cb7cb774a2ec..ecc66f9a068b4 100644 --- a/mysql-test/main/gis-json.result +++ b/mysql-test/main/gis-json.result @@ -110,6 +110,8 @@ Warning 4076 Incorrect GeoJSON format - empty 'coordinates' array. SELECT ST_GEOMFROMGEOJSON("{ \"type\": \"Feature\", \"geometry\": [10, 20] }") as exp; exp NULL +Warnings: +Warning 4048 Incorrect GeoJSON format specified for st_geomfromgeojson function. SELECT ST_ASTEXT (ST_GEOMFROMGEOJSON ('{ "type": "GEOMETRYCOLLECTION", "coordinates": [102.0, 0.0]}')) as exp; exp NULL @@ -123,6 +125,125 @@ Warning 4048 Incorrect GeoJSON format specified for st_geomfromgeojson function. # # End of 10.2 tests # +# Boundary condition: foreign-member nesting at depth 32 +SET @geom := '{"type":"Point","coordinates":[1,2]}'; +SET @doc := CONCAT( +REPEAT('{"x":', 29), +@geom, +REPEAT('}', 29) +); +SELECT ST_ASTEXT( ST_GeomFromGeoJSON(@doc) ) AS expr; +expr +NULL +Warnings: +Warning 4048 Incorrect GeoJSON format specified for st_geomfromgeojson function. +# Foreign-member nesting exceeds depth 32 +SET @geom := '{"type":"Point","coordinates":[1,2]}'; +SET @doc := CONCAT( +REPEAT('{"x":', 30), +@geom, +REPEAT('}', 30) +); +SELECT ST_GeomFromGeoJSON(@doc) AS expr; +expr +NULL +Warnings: +Warning 4048 Incorrect GeoJSON format specified for st_geomfromgeojson function. +# Deep foreign member attached to otherwise valid GeoJSON +SET @foreign := CONCAT( +REPEAT('{"x":', 28), +'{"leaf":1}', +REPEAT('}', 28) +); +SET @doc := CONCAT( +'{"type":"Point","coordinates":[10,20],"foreign":', +@foreign, +'}' +); +SELECT ST_ASTEXT(ST_GeomFromGeoJSON(@doc)) AS expr; +expr +POINT(10 20) +# Foreign member exceeds depth limit +SET @foreign := CONCAT( +REPEAT('{"x":', 29), +'{"leaf":1}', +REPEAT('}', 29) +); +SET @doc := CONCAT( +'{"type":"Point","coordinates":[10,20],"foreign":', +@foreign, +'}' +); +SELECT ST_ASTEXT(ST_GeomFromGeoJSON(@doc)) AS expr; +expr +POINT(10 20) +# GeometryCollection containing 33 valid GeoJSON Point objects +SET @point := '{"type":"Point","coordinates":[1,2]}'; +SET @geoms := CONCAT( +@point, ',', @point, ',', @point, ',', @point, ',', @point, ',', +@point, ',', @point, ',', @point, ',', @point, ',', @point, ',', +@point, ',', @point, ',', @point, ',', @point, ',', @point, ',', +@point, ',', @point, ',', @point, ',', @point, ',', @point, ',', +@point, ',', @point, ',', @point, ',', @point, ',', @point, ',', +@point, ',', @point, ',', @point, ',', @point, ',', @point, ',', +@point, ',', @point, ',', @point, ',', @point, ',', @point, ',', +@point, ',', @point, ',', @point +); +SET @doc := CONCAT( +'{"type":"GeometryCollection","geometries":[', +@geoms, +']}' +); +SELECT ST_AsText(ST_GeomFromGeoJSON(@doc)) AS expr; +expr +GEOMETRYCOLLECTION(POINT(1 2),POINT(1 2),POINT(1 2),POINT(1 2),POINT(1 2),POINT(1 2),POINT(1 2),POINT(1 2),POINT(1 2),POINT(1 2),POINT(1 2),POINT(1 2),POINT(1 2),POINT(1 2),POINT(1 2),POINT(1 2),POINT(1 2),POINT(1 2),POINT(1 2),POINT(1 2),POINT(1 2),POINT(1 2),POINT(1 2),POINT(1 2),POINT(1 2),POINT(1 2),POINT(1 2),POINT(1 2),POINT(1 2),POINT(1 2),POINT(1 2),POINT(1 2),POINT(1 2),POINT(1 2),POINT(1 2),POINT(1 2),POINT(1 2),POINT(1 2)) +# +# MDEV-39981: ST_GEOMFROMGEOJSON returns wrong result with reversed key order +# +# GeometryCollection: type before geometries (baseline) +SELECT ST_ASTEXT(ST_GEOMFROMGEOJSON('{"type":"GeometryCollection","geometries":[]}')) as expr; +expr +GEOMETRYCOLLECTION EMPTY +# GeometryCollection: geometries before type (empty) +SELECT ST_ASTEXT(ST_GEOMFROMGEOJSON('{"geometries":[],"type":"GeometryCollection"}')) as expr; +expr +GEOMETRYCOLLECTION EMPTY +# GeometryCollection: geometries before type (non-empty) +SELECT ST_ASTEXT(ST_GEOMFROMGEOJSON('{"geometries":[{"type":"Point","coordinates":[0,1]}],"type":"GeometryCollection"}')) as expr; +expr +GEOMETRYCOLLECTION(POINT(0 1)) +# GeometryCollection: multiple geometries before type +SELECT ST_ASTEXT(ST_GEOMFROMGEOJSON('{"geometries":[{"type":"Point","coordinates":[1,2]},{"type":"Point","coordinates":[3,4]}],"type":"GeometryCollection"}')) as expr; +expr +GEOMETRYCOLLECTION(POINT(1 2),POINT(3 4)) +# Point: coordinates before type +SELECT ST_ASTEXT(ST_GEOMFROMGEOJSON('{"coordinates":[0,1],"type":"Point"}')) as expr; +expr +POINT(0 1) +# LineString: coordinates before type +SELECT ST_ASTEXT(ST_GEOMFROMGEOJSON('{"coordinates":[[0,0],[1,1],[2,2]],"type":"LineString"}')) as expr; +expr +LINESTRING(0 0,1 1,2 2) +# Polygon: coordinates before type +SELECT ST_ASTEXT(ST_GEOMFROMGEOJSON('{"coordinates":[[[0,0],[1,0],[1,1],[0,1],[0,0]]],"type":"Polygon"}')) as expr; +expr +POLYGON((0 0,1 0,1 1,0 1,0 0)) +# FeatureCollection: features before type +SELECT ST_ASTEXT(ST_GEOMFROMGEOJSON('{"features":[{"geometry":{"type":"Point","coordinates":[5,6]},"type":"Feature","properties":{}}],"type":"FeatureCollection"}')) as expr; +expr +GEOMETRYCOLLECTION(POINT(5 6)) +# Feature: geometry before type +SELECT ST_ASTEXT(ST_GEOMFROMGEOJSON('{"geometry":{"coordinates":[7,8],"type":"Point"},"type":"Feature","properties":{}}')) as expr; +expr +POINT(7 8) +# Unknown keys interspersed (should be ignored) +SELECT ST_ASTEXT(ST_GEOMFROMGEOJSON('{"foo":"bar","geometries":[],"type":"GeometryCollection","extra":123}')) as expr; +expr +GEOMETRYCOLLECTION EMPTY +# Deeply nested geometries before type +SELECT ST_ASTEXT(ST_GEOMFROMGEOJSON('{"geometries":[{"geometries":[{"type":"Point","coordinates":[9,10]}],"type":"GeometryCollection"}],"type":"GeometryCollection"}')) as expr; +expr +GEOMETRYCOLLECTION(GEOMETRYCOLLECTION(POINT(9 10))) # End of 10.6 tests # # MDEV-34079: ST_AsGeoJSON returns incorrect value for empty geometry diff --git a/mysql-test/main/gis-json.test b/mysql-test/main/gis-json.test index 022672ab9b669..ed3a4578ee7da 100644 --- a/mysql-test/main/gis-json.test +++ b/mysql-test/main/gis-json.test @@ -60,6 +60,120 @@ SELECT ST_ASTEXT(ST_GEOMFROMGEOJSON('{"type": ["POINT"], "coINates": [0,0] }')) --echo # +--echo # Boundary condition: foreign-member nesting at depth 32 +SET @geom := '{"type":"Point","coordinates":[1,2]}'; + +SET @doc := CONCAT( + REPEAT('{"x":', 29), + @geom, + REPEAT('}', 29) +); + +SELECT ST_ASTEXT( ST_GeomFromGeoJSON(@doc) ) AS expr; + + +--echo # Foreign-member nesting exceeds depth 32 +SET @geom := '{"type":"Point","coordinates":[1,2]}'; + +SET @doc := CONCAT( + REPEAT('{"x":', 30), + @geom, + REPEAT('}', 30) +); + +SELECT ST_GeomFromGeoJSON(@doc) AS expr; + + +--echo # Deep foreign member attached to otherwise valid GeoJSON +SET @foreign := CONCAT( + REPEAT('{"x":', 28), + '{"leaf":1}', + REPEAT('}', 28) +); + +SET @doc := CONCAT( + '{"type":"Point","coordinates":[10,20],"foreign":', + @foreign, + '}' +); + +SELECT ST_ASTEXT(ST_GeomFromGeoJSON(@doc)) AS expr; + + +--echo # Foreign member exceeds depth limit +SET @foreign := CONCAT( + REPEAT('{"x":', 29), + '{"leaf":1}', + REPEAT('}', 29) +); + +SET @doc := CONCAT( + '{"type":"Point","coordinates":[10,20],"foreign":', + @foreign, + '}' +); + +SELECT ST_ASTEXT(ST_GeomFromGeoJSON(@doc)) AS expr; + +--echo # GeometryCollection containing 33 valid GeoJSON Point objects +SET @point := '{"type":"Point","coordinates":[1,2]}'; + +SET @geoms := CONCAT( + @point, ',', @point, ',', @point, ',', @point, ',', @point, ',', + @point, ',', @point, ',', @point, ',', @point, ',', @point, ',', + @point, ',', @point, ',', @point, ',', @point, ',', @point, ',', + @point, ',', @point, ',', @point, ',', @point, ',', @point, ',', + @point, ',', @point, ',', @point, ',', @point, ',', @point, ',', + @point, ',', @point, ',', @point, ',', @point, ',', @point, ',', + @point, ',', @point, ',', @point, ',', @point, ',', @point, ',', + @point, ',', @point, ',', @point +); + +SET @doc := CONCAT( + '{"type":"GeometryCollection","geometries":[', + @geoms, + ']}' +); + +SELECT ST_AsText(ST_GeomFromGeoJSON(@doc)) AS expr; + +--echo # +--echo # MDEV-39981: ST_GEOMFROMGEOJSON returns wrong result with reversed key order +--echo # + +--echo # GeometryCollection: type before geometries (baseline) +SELECT ST_ASTEXT(ST_GEOMFROMGEOJSON('{"type":"GeometryCollection","geometries":[]}')) as expr; + +--echo # GeometryCollection: geometries before type (empty) +SELECT ST_ASTEXT(ST_GEOMFROMGEOJSON('{"geometries":[],"type":"GeometryCollection"}')) as expr; + +--echo # GeometryCollection: geometries before type (non-empty) +SELECT ST_ASTEXT(ST_GEOMFROMGEOJSON('{"geometries":[{"type":"Point","coordinates":[0,1]}],"type":"GeometryCollection"}')) as expr; + +--echo # GeometryCollection: multiple geometries before type +SELECT ST_ASTEXT(ST_GEOMFROMGEOJSON('{"geometries":[{"type":"Point","coordinates":[1,2]},{"type":"Point","coordinates":[3,4]}],"type":"GeometryCollection"}')) as expr; + +--echo # Point: coordinates before type +SELECT ST_ASTEXT(ST_GEOMFROMGEOJSON('{"coordinates":[0,1],"type":"Point"}')) as expr; + +--echo # LineString: coordinates before type +SELECT ST_ASTEXT(ST_GEOMFROMGEOJSON('{"coordinates":[[0,0],[1,1],[2,2]],"type":"LineString"}')) as expr; + +--echo # Polygon: coordinates before type +SELECT ST_ASTEXT(ST_GEOMFROMGEOJSON('{"coordinates":[[[0,0],[1,0],[1,1],[0,1],[0,0]]],"type":"Polygon"}')) as expr; + +--echo # FeatureCollection: features before type +SELECT ST_ASTEXT(ST_GEOMFROMGEOJSON('{"features":[{"geometry":{"type":"Point","coordinates":[5,6]},"type":"Feature","properties":{}}],"type":"FeatureCollection"}')) as expr; + +--echo # Feature: geometry before type +SELECT ST_ASTEXT(ST_GEOMFROMGEOJSON('{"geometry":{"coordinates":[7,8],"type":"Point"},"type":"Feature","properties":{}}')) as expr; + +--echo # Unknown keys interspersed (should be ignored) +SELECT ST_ASTEXT(ST_GEOMFROMGEOJSON('{"foo":"bar","geometries":[],"type":"GeometryCollection","extra":123}')) as expr; + +--echo # Deeply nested geometries before type +SELECT ST_ASTEXT(ST_GEOMFROMGEOJSON('{"geometries":[{"geometries":[{"type":"Point","coordinates":[9,10]}],"type":"GeometryCollection"}],"type":"GeometryCollection"}')) as expr; + --echo # End of 10.6 tests --echo # diff --git a/mysql-test/main/gis-precise.result b/mysql-test/main/gis-precise.result index 5ff69b901a489..8752eb9ce046a 100644 --- a/mysql-test/main/gis-precise.result +++ b/mysql-test/main/gis-precise.result @@ -881,6 +881,66 @@ SELECT ST_CROSSES( ST_GEOMFROMTEXT('POLYGON ((59 18,67 18,67 13,59 13,59 18)) ') a NULL # +# MDEV-40584 ST_CROSSES always returns 0 for geometries of different dimensions +# +SELECT ST_Crosses(ST_GeomFromText('MULTIPOINT(1 3, 2 2, 3 2)'), +ST_GeomFromText('LINESTRING(0 3, 1 1, 2 2, 2 0)')) a; +a +1 +SELECT ST_Crosses(ST_GeomFromText('MULTIPOINT(1 1, 1 3, 2 3)'), +ST_GeomFromText('POLYGON((0 0, 0 3, 2 0, 0 0))')) a; +a +1 +SELECT ST_Crosses(ST_GeomFromText('LINESTRING(0 1, 3 2, 4 1)'), +ST_GeomFromText('POLYGON((1 0, 1 2, 2 3, 2 1, 1 0))')) a; +a +1 +# line x line, crossing (dims 1,1) -> 1 +SELECT ST_Crosses(ST_GeomFromText('LINESTRING(0 0, 2 2)'), +ST_GeomFromText('LINESTRING(0 2, 2 0)')) a; +a +1 +# line x line, not crossing (dims 1,1) -> 0 +SELECT ST_Crosses(ST_GeomFromText('LINESTRING(0 0, 2 0)'), +ST_GeomFromText('LINESTRING(0 1, 2 1)')) a; +a +0 +# point x line, disjoint (dims 0,1) -> 0 +SELECT ST_Crosses(ST_GeomFromText('MULTIPOINT(10 10)'), +ST_GeomFromText('LINESTRING(0 0, 1 1)')) a; +a +0 +# point x polygon, disjoint (dims 0,2) -> 0 +SELECT ST_Crosses(ST_GeomFromText('MULTIPOINT(100 100)'), +ST_GeomFromText('POLYGON((0 0, 0 3, 2 0, 0 0))')) a; +a +0 +# line x point (g2 is a point) -> NULL +SELECT ST_Crosses(ST_GeomFromText('LINESTRING(0 0, 1 1)'), +ST_GeomFromText('POINT(5 5)')) a; +a +NULL +# line x multipoint (g2 is a multipoint) -> NULL +SELECT ST_Crosses(ST_GeomFromText('LINESTRING(0 0, 5 5)'), +ST_GeomFromText('MULTIPOINT(1 1, 2 2)')) a; +a +NULL +# multipolygon x line (g1 is a multipolygon) -> NULL +SELECT ST_Crosses(ST_GeomFromText('MULTIPOLYGON(((0 0, 0 3, 3 3, 3 0, 0 0)))'), +ST_GeomFromText('LINESTRING(1 1, 2 2)')) a; +a +NULL +# polygon x line, reversed order of a valid crossing case -> NULL +SELECT ST_Crosses(ST_GeomFromText('POLYGON((1 0, 1 2, 2 3, 2 1, 1 0))'), +ST_GeomFromText('LINESTRING(0 1, 3 2, 4 1)')) a; +a +NULL +# line x same line, equal (not lower-dim intersection) -> 0 +SELECT ST_Crosses(ST_GeomFromText('LINESTRING(0 0, 2 2)'), +ST_GeomFromText('LINESTRING(0 0, 2 2)')) a; +a +0 +# # MDEV-39449: Memory corruption (heap-buffer-overflow) in uint4korr and # Gcalc_function::count_internal, apparent partial stack looping in # Gcalc_function::count_internal and Assertion `(0)' failed in diff --git a/mysql-test/main/gis-precise.test b/mysql-test/main/gis-precise.test index 1266db8039067..d2595c44f827e 100644 --- a/mysql-test/main/gis-precise.test +++ b/mysql-test/main/gis-precise.test @@ -494,6 +494,45 @@ SELECT ST_CROSSES( ST_GEOMFROMTEXT('point(1 1)'), st_geomfromtext('point(1 1)') SELECT ST_CROSSES( ST_GEOMFROMTEXT('POLYGON ((59 18,67 18,67 13,59 13,59 18)) '), st_geomfromtext('polygon((2 2,2 4, 4 2,2 2))') ) a; SELECT ST_CROSSES( ST_GEOMFROMTEXT('POLYGON ((59 18,67 18,67 13,59 13,59 18)) '), st_geomfromtext('point(1 1)') ) a; +--echo # +--echo # MDEV-40584 ST_CROSSES always returns 0 for geometries of different dimensions +--echo # + +SELECT ST_Crosses(ST_GeomFromText('MULTIPOINT(1 3, 2 2, 3 2)'), + ST_GeomFromText('LINESTRING(0 3, 1 1, 2 2, 2 0)')) a; +SELECT ST_Crosses(ST_GeomFromText('MULTIPOINT(1 1, 1 3, 2 3)'), + ST_GeomFromText('POLYGON((0 0, 0 3, 2 0, 0 0))')) a; +SELECT ST_Crosses(ST_GeomFromText('LINESTRING(0 1, 3 2, 4 1)'), + ST_GeomFromText('POLYGON((1 0, 1 2, 2 3, 2 1, 1 0))')) a; + +--echo # line x line, crossing (dims 1,1) -> 1 +SELECT ST_Crosses(ST_GeomFromText('LINESTRING(0 0, 2 2)'), + ST_GeomFromText('LINESTRING(0 2, 2 0)')) a; +--echo # line x line, not crossing (dims 1,1) -> 0 +SELECT ST_Crosses(ST_GeomFromText('LINESTRING(0 0, 2 0)'), + ST_GeomFromText('LINESTRING(0 1, 2 1)')) a; +--echo # point x line, disjoint (dims 0,1) -> 0 +SELECT ST_Crosses(ST_GeomFromText('MULTIPOINT(10 10)'), + ST_GeomFromText('LINESTRING(0 0, 1 1)')) a; +--echo # point x polygon, disjoint (dims 0,2) -> 0 +SELECT ST_Crosses(ST_GeomFromText('MULTIPOINT(100 100)'), + ST_GeomFromText('POLYGON((0 0, 0 3, 2 0, 0 0))')) a; +--echo # line x point (g2 is a point) -> NULL +SELECT ST_Crosses(ST_GeomFromText('LINESTRING(0 0, 1 1)'), + ST_GeomFromText('POINT(5 5)')) a; +--echo # line x multipoint (g2 is a multipoint) -> NULL +SELECT ST_Crosses(ST_GeomFromText('LINESTRING(0 0, 5 5)'), + ST_GeomFromText('MULTIPOINT(1 1, 2 2)')) a; +--echo # multipolygon x line (g1 is a multipolygon) -> NULL +SELECT ST_Crosses(ST_GeomFromText('MULTIPOLYGON(((0 0, 0 3, 3 3, 3 0, 0 0)))'), + ST_GeomFromText('LINESTRING(1 1, 2 2)')) a; +--echo # polygon x line, reversed order of a valid crossing case -> NULL +SELECT ST_Crosses(ST_GeomFromText('POLYGON((1 0, 1 2, 2 3, 2 1, 1 0))'), + ST_GeomFromText('LINESTRING(0 1, 3 2, 4 1)')) a; +--echo # line x same line, equal (not lower-dim intersection) -> 0 +SELECT ST_Crosses(ST_GeomFromText('LINESTRING(0 0, 2 2)'), + ST_GeomFromText('LINESTRING(0 0, 2 2)')) a; + --echo # --echo # MDEV-39449: Memory corruption (heap-buffer-overflow) in uint4korr and --echo # Gcalc_function::count_internal, apparent partial stack looping in diff --git a/mysql-test/main/gis.result b/mysql-test/main/gis.result index 4dee31109a7b5..02a0bdf6c5f0f 100644 --- a/mysql-test/main/gis.result +++ b/mysql-test/main/gis.result @@ -5641,6 +5641,15 @@ NULL SELECT ST_GEOMFROMWKB (0x01050000000100000082040000000100000000000000000000000000000000000000) as g; g NULL +# +# MDEV-40552 UBSAN : load of value X, which is not a valid value for type 'wkbByteOrder' Gis_multi_point::init_from_wkb +# +SELECT ST_GEOMFROMWKB (0x0104000000010000000201000000000000000000000000000000000000000000000000) as multipoint_bad_inner_bo; +multipoint_bad_inner_bo +NULL +SELECT ST_GEOMFROMWKB (0x0104000000010000008201000000000000000000000000000000000000000000000000) as multipoint_bad_inner_bo; +multipoint_bad_inner_bo +NULL # End of 10.11 tests # # Start of 11.5 tests diff --git a/mysql-test/main/gis.test b/mysql-test/main/gis.test index e22bf32eb2658..dc3f326f5040a 100644 --- a/mysql-test/main/gis.test +++ b/mysql-test/main/gis.test @@ -3601,6 +3601,13 @@ SELECT ST_GEOMFROMWKB (0x01070000000100000082010000000000000000000000) as g; SELECT ST_GEOMFROMWKB (0x01050000000100000002040000000100000000000000000000000000000000000000) as g; SELECT ST_GEOMFROMWKB (0x01050000000100000082040000000100000000000000000000000000000000000000) as g; +--echo # +--echo # MDEV-40552 UBSAN : load of value X, which is not a valid value for type 'wkbByteOrder' Gis_multi_point::init_from_wkb +--echo # + +SELECT ST_GEOMFROMWKB (0x0104000000010000000201000000000000000000000000000000000000000000000000) as multipoint_bad_inner_bo; +SELECT ST_GEOMFROMWKB (0x0104000000010000008201000000000000000000000000000000000000000000000000) as multipoint_bad_inner_bo; + --echo # End of 10.11 tests --echo # diff --git a/mysql-test/main/grant5.result b/mysql-test/main/grant5.result index 5702af36941cd..c7151d03a68b2 100644 --- a/mysql-test/main/grant5.result +++ b/mysql-test/main/grant5.result @@ -500,6 +500,36 @@ drop user u1@localhost; Warnings: Note 4227 Dropped users 'u1'@'localhost' have active connections. Use KILL CONNECTION if they should not be used anymore. drop user u2@localhost; +# +# MDEV-40541 role vs user@localhost acl_cache key confusion +# +create database db; +create table db.t1(a varchar(40)); +insert into db.t1 values ('role-cache-collision-row'); +create role r1; +create user r1@localhost identified by 'r1pass'; +create user u1@'127.0.0.1' identified by 'u1pass'; +grant select on db.* to r1@localhost; +grant r1 to u1@'127.0.0.1'; +connect r1,localhost,r1,r1pass; +select a as precache from db.t1; +precache +role-cache-collision-row +connect u1,127.0.0.1,u1,u1pass; +set role r1; +show grants; +Grants for u1@127.0.0.1 +GRANT `r1` TO `u1`@`127.0.0.1` +GRANT USAGE ON *.* TO `u1`@`127.0.0.1` IDENTIFIED BY PASSWORD '*4C5C0A2C80D7567C7CFBDFC09409901AADC4B390' +GRANT USAGE ON *.* TO `r1` +select a from db.t1; +ERROR 42000: SELECT command denied to user 'u1'@'localhost' for table `db`.`t1` +connection default; +disconnect r1; +disconnect u1; +drop database db; +drop role r1; +drop user r1@localhost, u1@'127.0.0.1'; # End of 10.6 tests # # MDEV-37256: The permission check of LOAD INDEX INTO CACHE and CACHE INDEX is broken @@ -522,4 +552,50 @@ disconnect foo; connection default; drop table t1; drop user foo@localhost; +# +# MDEV-23086 Error codes/messages reveal information about table structure +# +create database db; +create table db.t1 (a int, b int, c int); +create user foo@localhost; +grant select (a,b) on db.t1 to foo@localhost; +alter table db.t1 drop column b; +create table db.t2 (x int, y int); +grant select on db.t2 to foo@localhost; +create table t1 (k int, l int, m int); +grant select on test.t1 to foo@localhost; +connect con1,localhost,foo,,db; +select a from t1; +a +select b from t1; +ERROR 42S22: Unknown column 'b' in 'SELECT' +select c from t1; +ERROR 42000: SELECT command denied to user 'foo'@'localhost' for column 'c' in table 't1' +select d from t1; +ERROR 42000: SELECT command denied to user 'foo'@'localhost' for column 'd' in table 't1' +select a from t0; +ERROR 42000: SELECT command denied to user 'foo'@'localhost' for table `db`.`t0` +use db2; +ERROR 42000: Access denied for user 'foo'@'localhost' to database 'db2' +select a from t2; +ERROR 42S22: Unknown column 'a' in 'SELECT' +select c from t2, t1; +ERROR 42000: SELECT command denied to user 'foo'@'localhost' for column 'c' in table 't1' +select c from t1, t2; +ERROR 42000: SELECT command denied to user 'foo'@'localhost' for column 'c' in table 't1' +select d from t2, t1; +ERROR 42000: SELECT command denied to user 'foo'@'localhost' for column 'd' in table 't1' +select d from t1, t2; +ERROR 42000: SELECT command denied to user 'foo'@'localhost' for column 'd' in table 't1' +select t2.c from t2,t1; +ERROR 42S22: Unknown column 't2.c' in 'SELECT' +select * from test.t1; +k l m +select test.t1.c from t1,test.t1; +ERROR 42S22: Unknown column 'test.t1.c' in 'SELECT' +connection default; +disconnect con1; +drop user foo@localhost; +drop database db; +drop table t1; # End of 10.11 tests diff --git a/mysql-test/main/grant5.test b/mysql-test/main/grant5.test index c95a71d70b042..d68a2456249a4 100644 --- a/mysql-test/main/grant5.test +++ b/mysql-test/main/grant5.test @@ -457,6 +457,37 @@ show create user u2@localhost; drop user u1@localhost; drop user u2@localhost; +--echo # +--echo # MDEV-40541 role vs user@localhost acl_cache key confusion +--echo # +create database db; +create table db.t1(a varchar(40)); +insert into db.t1 values ('role-cache-collision-row'); + +create role r1; +create user r1@localhost identified by 'r1pass'; +create user u1@'127.0.0.1' identified by 'u1pass'; + +grant select on db.* to r1@localhost; +grant r1 to u1@'127.0.0.1'; +#flush privileges; + +connect r1,localhost,r1,r1pass; +select a as precache from db.t1; + +connect u1,127.0.0.1,u1,u1pass; +set role r1; +show grants; +--error ER_TABLEACCESS_DENIED_ERROR +select a from db.t1; + +connection default; +disconnect r1; +disconnect u1; +drop database db; +drop role r1; +drop user r1@localhost, u1@'127.0.0.1'; + --echo # End of 10.6 tests --echo # @@ -483,4 +514,45 @@ load index into cache test.t2 key (i2); drop table t1; drop user foo@localhost; +--echo # +--echo # MDEV-23086 Error codes/messages reveal information about table structure +--echo # + +create database db; +create table db.t1 (a int, b int, c int); +create user foo@localhost; +grant select (a,b) on db.t1 to foo@localhost; +alter table db.t1 drop column b; + +create table db.t2 (x int, y int); +grant select on db.t2 to foo@localhost; + +create table t1 (k int, l int, m int); +grant select on test.t1 to foo@localhost; + +connect con1,localhost,foo,,db; + +disable_abort_on_error; +select a from t1; # can access, column exist +select b from t1; # can access, column doesn't exist +select c from t1; # no access, column exists +select d from t1; # no access, column doesn't exists +select a from t0; # no access, table doesn't exists +use db2; # no access, db doesn't exists +select a from t2; # table level grant, column doesn't exists +select c from t2, t1; # mix tbl/col grant, no access, column exist +select c from t1, t2; # mix tbl/col grant, no access, column exist +select d from t2, t1; # mix tbl/col grant, no access, column doesn't exists +select d from t1, t2; # mix tbl/col grant, no access, column doesn't exist +select t2.c from t2,t1; # mix tbl/col grant, explicit tbl, column doesn't exist +select * from test.t1; # this should work +select test.t1.c from t1,test.t1; # mix, explicit db name, column doesn't exist +enable_abort_on_error; + +connection default; +disconnect con1; +drop user foo@localhost; +drop database db; +drop table t1; + --echo # End of 10.11 tests diff --git a/mysql-test/main/grant_server.result b/mysql-test/main/grant_server.result index c4e6cd19f5383..71c47d7de8ea2 100644 --- a/mysql-test/main/grant_server.result +++ b/mysql-test/main/grant_server.result @@ -86,7 +86,7 @@ GRANT FEDERATED ADMIN ON *.* TO user1@localhost; connect con1,localhost,user1,,; SHOW CREATE SERVER srv; Server Create Server -srv CREATE SERVER `srv` FOREIGN DATA WRAPPER `mysql` OPTIONS (USER 'remote_user', HOST 'localhost', PASSWORD 'secret', DATABASE 'test2'); +srv CREATE SERVER `srv` FOREIGN DATA WRAPPER `mysql` OPTIONS (`USER` 'remote_user', `HOST` 'localhost', `PASSWORD` 'secret', `DATABASE` 'test2'); disconnect con1; connection default; DROP SERVER srv; diff --git a/mysql-test/main/group_min_max_innodb.result b/mysql-test/main/group_min_max_innodb.result index 87b718f53ed38..086877267aab3 100644 --- a/mysql-test/main/group_min_max_innodb.result +++ b/mysql-test/main/group_min_max_innodb.result @@ -478,3 +478,57 @@ drop table t1; set global innodb_stats_persistent= @innodb_stats_persistent_save; set global innodb_stats_persistent_sample_pages= @innodb_stats_persistent_sample_pages_save; +# +# Begin 10.11 tests +# +CREATE TABLE t1 (pk INT, a INT, b INT, PRIMARY KEY (pk), KEY (a,b)) ENGINE=InnoDB; +INSERT INTO t1 (pk,a) VALUES (1,11),(2,12),(3,13),(4,14); +connect con1,localhost,root,,; +DELETE FROM t1 WHERE pk <= 3; +connection default; +INSERT INTO t1 (pk) VALUES (5); +connection con1; +START TRANSACTION; +INSERT INTO t1 (pk) VALUES (6); +connection default; +SET @save_innodb_lock_wait_timeout= @@innodb_lock_wait_timeout; +SET innodb_lock_wait_timeout= 1; +EXPLAIN SELECT MIN(b), MAX(b), a FROM t1 GROUP BY a FOR UPDATE; +id select_type table type possible_keys key key_len ref rows Extra +1 SIMPLE t1 range NULL a 10 NULL # Using index for group-by +SELECT MIN(b), MAX(b), a FROM t1 GROUP BY a FOR UPDATE; +ERROR HY000: Lock wait timeout exceeded; try restarting transaction +SET innodb_lock_wait_timeout= @save_innodb_lock_wait_timeout; +connection con1; +COMMIT; +DROP TABLE t1; +disconnect con1; +connection default; +CREATE TABLE t1 (pk INT PRIMARY KEY AUTO_INCREMENT, a INT, b INT, c INT, +KEY (a,b,c)) ENGINE=InnoDB; +INSERT INTO t1 (a,b,c) VALUES +(1,1,10),(1,1,11),(1,2,20),(1,2,21), +(2,1,10),(2,1,11),(2,2,20),(2,2,21); +INSERT INTO t1 (a,b,c) SELECT a,b,c FROM t1; +INSERT INTO t1 (a,b,c) SELECT a,b,c FROM t1; +INSERT INTO t1 (a,b,c) SELECT a,b,c FROM t1; +connect con1,localhost,root,,; +START TRANSACTION; +INSERT INTO t1 (a,b,c) VALUES (1,2,0); +connection default; +SET @save_innodb_lock_wait_timeout= @@innodb_lock_wait_timeout; +SET innodb_lock_wait_timeout= 1; +EXPLAIN SELECT MIN(c) FROM t1 WHERE b = 2 GROUP BY a FOR UPDATE; +id select_type table type possible_keys key key_len ref rows Extra +1 SIMPLE t1 range NULL a 15 NULL # Using where; Using index for group-by +SELECT MIN(c) FROM t1 WHERE b = 2 GROUP BY a FOR UPDATE; +ERROR HY000: Lock wait timeout exceeded; try restarting transaction +SET innodb_lock_wait_timeout= @save_innodb_lock_wait_timeout; +connection con1; +COMMIT; +DROP TABLE t1; +disconnect con1; +connection default; +# +# End 10.11 tests +# diff --git a/mysql-test/main/group_min_max_innodb.test b/mysql-test/main/group_min_max_innodb.test index 165580a7f8fe4..7140a5d627b3b 100644 --- a/mysql-test/main/group_min_max_innodb.test +++ b/mysql-test/main/group_min_max_innodb.test @@ -360,3 +360,93 @@ drop table t1; set global innodb_stats_persistent= @innodb_stats_persistent_save; set global innodb_stats_persistent_sample_pages= @innodb_stats_persistent_sample_pages_save; + +--echo # +--echo # Begin 10.11 tests +--echo # + +# Default client connection to the database. +CREATE TABLE t1 (pk INT, a INT, b INT, PRIMARY KEY (pk), KEY (a,b)) ENGINE=InnoDB; +INSERT INTO t1 (pk,a) VALUES (1,11),(2,12),(3,13),(4,14); + +# Start a new, separate client connection to the database called 'con1'. +--connect (con1,localhost,root,,) +--send +DELETE FROM t1 WHERE pk <= 3; + +# On the default connection, insert a value into the table. +--connection default +INSERT INTO t1 (pk) VALUES (5); + +# On 'con1', insert a value within a transaction and hold the transaction while +# on the default connection try to compute an aggregate. +--connection con1 +--reap + +START TRANSACTION; +INSERT INTO t1 (pk) VALUES (6); + +--connection default +SET @save_innodb_lock_wait_timeout= @@innodb_lock_wait_timeout; +SET innodb_lock_wait_timeout= 1; +# SELECT ... FOR UPDATE locks the rows for write operations and prevents other transactions +# from modifying or reading the selected rows until the transaction ends. +# This should lead to the error, not a crash. +--replace_column 9 # +EXPLAIN SELECT MIN(b), MAX(b), a FROM t1 GROUP BY a FOR UPDATE; +--error ER_LOCK_WAIT_TIMEOUT +SELECT MIN(b), MAX(b), a FROM t1 GROUP BY a FOR UPDATE; +SET innodb_lock_wait_timeout= @save_innodb_lock_wait_timeout; + +# Cleanup, commit the 'con1' transaction. +--connection con1 +COMMIT; +DROP TABLE t1; +--disconnect con1 + +# This scenario passes with or without the patch. The query requests only a +# MIN, so the max branch never runs. The scenario is kept for coverage, to +# execute the early return, which the patch adds in the min branch for symmetry +# with the max branch and to honor the convention that no further reads follow +# a fatal handler error. +--connection default +CREATE TABLE t1 (pk INT PRIMARY KEY AUTO_INCREMENT, a INT, b INT, c INT, + KEY (a,b,c)) ENGINE=InnoDB; + +# Two groups over key part a. Each group has rows with b=1 and rows with +# b=2, with enough rows overall that the optimizer chooses the loose +# index scan ("Using index for group-by") over a full index scan. +INSERT INTO t1 (a,b,c) VALUES + (1,1,10),(1,1,11),(1,2,20),(1,2,21), + (2,1,10),(2,1,11),(2,2,20),(2,2,21); +INSERT INTO t1 (a,b,c) SELECT a,b,c FROM t1; +INSERT INTO t1 (a,b,c) SELECT a,b,c FROM t1; +INSERT INTO t1 (a,b,c) SELECT a,b,c FROM t1; + +--connect (con1,localhost,root,,) +START TRANSACTION; +# This uncommitted row becomes the first row of the a=1, b=2 prefix because its +# c is smaller than every committed c in that subgroup. The MIN read seeks +# exactly this position, so it waits for con1. +INSERT INTO t1 (a,b,c) VALUES (1,2,0); + +--connection default +SET @save_innodb_lock_wait_timeout= @@innodb_lock_wait_timeout; +SET innodb_lock_wait_timeout= 1; +# The MIN read inside the loose index scan lands on the row locked by con1. +--replace_column 9 # +EXPLAIN SELECT MIN(c) FROM t1 WHERE b = 2 GROUP BY a FOR UPDATE; +--error ER_LOCK_WAIT_TIMEOUT +SELECT MIN(c) FROM t1 WHERE b = 2 GROUP BY a FOR UPDATE; +SET innodb_lock_wait_timeout= @save_innodb_lock_wait_timeout; + +# Cleanup, commit the 'con1' transaction. +--connection con1 +COMMIT; +DROP TABLE t1; +--disconnect con1 +--connection default + +--echo # +--echo # End 10.11 tests +--echo # diff --git a/mysql-test/main/intersect_all.result b/mysql-test/main/intersect_all.result index 1cb65215ef7d7..ef3c0552b5ec6 100644 --- a/mysql-test/main/intersect_all.result +++ b/mysql-test/main/intersect_all.result @@ -1427,3 +1427,94 @@ a b 1 2 drop tables t1, t2, t3; set sql_mode= default; +# +# MDEV-38722 server crash hp_rec_key_cmp +# +# original reproducer (used to crash), returns a single row +SELECT 1 INTERSECT SELECT 1 UNION ALL SELECT 1 EXCEPT ALL SELECT 1; +1 +1 +CREATE TABLE t1 (i INT); +INSERT INTO t1 VALUES (1),(1),(2),(3); +CREATE TABLE t2 (i INT); +INSERT INTO t2 VALUES (1),(2),(2); +CREATE TABLE t3 (i INT); +INSERT INTO t3 VALUES (1),(3); +SELECT i FROM t1 INTERSECT SELECT i FROM t3 +UNION ALL SELECT i FROM t2 EXCEPT ALL SELECT i FROM t3 ORDER BY i; +i +1 +2 +2 +(((SELECT i FROM t1) INTERSECT (SELECT i FROM t3)) UNION ALL (SELECT i FROM t2)) +EXCEPT ALL (SELECT i FROM t3) ORDER BY i; +i +1 +2 +2 +# trailing UNION ALL after the EXCEPT ALL +SELECT i FROM t1 INTERSECT SELECT i FROM t2 UNION ALL SELECT i FROM t2 +EXCEPT ALL SELECT i FROM t3 UNION ALL SELECT i FROM t1 ORDER BY i; +i +1 +1 +1 +2 +2 +2 +2 +3 +((((SELECT i FROM t1) INTERSECT (SELECT i FROM t2)) UNION ALL (SELECT i FROM t2)) +EXCEPT ALL (SELECT i FROM t3)) UNION ALL (SELECT i FROM t1) ORDER BY i; +i +1 +1 +1 +2 +2 +2 +2 +3 +# a leading INTERSECT ALL keeps the operation extended while the last +# INTERSECT DISTINCT node is the index-release point +SELECT i FROM t1 INTERSECT ALL SELECT i FROM t2 INTERSECT SELECT i FROM t3 +UNION ALL SELECT i FROM t2 ORDER BY i; +i +1 +1 +2 +2 +(((SELECT i FROM t1) INTERSECT ALL (SELECT i FROM t2)) INTERSECT (SELECT i FROM t3)) +UNION ALL (SELECT i FROM t2) ORDER BY i; +i +1 +1 +2 +2 +DROP TABLE t1,t2,t3; +# End of 10.6 tests +# +# MDEV-38722 fix merge problem with view/derived +# +CREATE TABLE t1 (i INT); +INSERT INTO t1 VALUES (1),(1),(2),(3); +CREATE TABLE t2 (i INT); +INSERT INTO t2 VALUES (1),(2),(2); +CREATE TABLE t3 (i INT); +INSERT INTO t3 VALUES (1),(3); +SELECT * FROM (SELECT i FROM t1 INTERSECT SELECT i FROM t3 +UNION ALL SELECT i FROM t2 EXCEPT ALL SELECT i FROM t3) d ORDER BY i; +i +1 +2 +2 +CREATE VIEW v1 AS SELECT i FROM t1 INTERSECT SELECT i FROM t3 +UNION ALL SELECT i FROM t2 EXCEPT ALL SELECT i FROM t3; +SELECT * FROM v1 ORDER BY i; +i +1 +2 +2 +DROP VIEW v1; +DROP TABLE t1,t2,t3; +# End of 11.4 tests diff --git a/mysql-test/main/intersect_all.test b/mysql-test/main/intersect_all.test index 6ac8df4be14b4..45967a40e6d0d 100644 --- a/mysql-test/main/intersect_all.test +++ b/mysql-test/main/intersect_all.test @@ -499,3 +499,53 @@ select * from t1 except all select * from t2 intersect all select * from t1 unio select * from t1 except all select * from t2 intersect all select * from t1 union all select * from t2 order by a desc limit 3; drop tables t1, t2, t3; set sql_mode= default; + +--echo # +--echo # MDEV-38722 server crash hp_rec_key_cmp +--echo # +--echo # original reproducer (used to crash), returns a single row +SELECT 1 INTERSECT SELECT 1 UNION ALL SELECT 1 EXCEPT ALL SELECT 1; + +CREATE TABLE t1 (i INT); +INSERT INTO t1 VALUES (1),(1),(2),(3); +CREATE TABLE t2 (i INT); +INSERT INTO t2 VALUES (1),(2),(2); +CREATE TABLE t3 (i INT); +INSERT INTO t3 VALUES (1),(3); +SELECT i FROM t1 INTERSECT SELECT i FROM t3 + UNION ALL SELECT i FROM t2 EXCEPT ALL SELECT i FROM t3 ORDER BY i; +(((SELECT i FROM t1) INTERSECT (SELECT i FROM t3)) UNION ALL (SELECT i FROM t2)) + EXCEPT ALL (SELECT i FROM t3) ORDER BY i; +--echo # trailing UNION ALL after the EXCEPT ALL +SELECT i FROM t1 INTERSECT SELECT i FROM t2 UNION ALL SELECT i FROM t2 + EXCEPT ALL SELECT i FROM t3 UNION ALL SELECT i FROM t1 ORDER BY i; +((((SELECT i FROM t1) INTERSECT (SELECT i FROM t2)) UNION ALL (SELECT i FROM t2)) + EXCEPT ALL (SELECT i FROM t3)) UNION ALL (SELECT i FROM t1) ORDER BY i; +--echo # a leading INTERSECT ALL keeps the operation extended while the last +--echo # INTERSECT DISTINCT node is the index-release point +SELECT i FROM t1 INTERSECT ALL SELECT i FROM t2 INTERSECT SELECT i FROM t3 + UNION ALL SELECT i FROM t2 ORDER BY i; +(((SELECT i FROM t1) INTERSECT ALL (SELECT i FROM t2)) INTERSECT (SELECT i FROM t3)) + UNION ALL (SELECT i FROM t2) ORDER BY i; +DROP TABLE t1,t2,t3; + +--echo # End of 10.6 tests + +--echo # +--echo # MDEV-38722 fix merge problem with view/derived +--echo # +CREATE TABLE t1 (i INT); +INSERT INTO t1 VALUES (1),(1),(2),(3); +CREATE TABLE t2 (i INT); +INSERT INTO t2 VALUES (1),(2),(2); +CREATE TABLE t3 (i INT); +INSERT INTO t3 VALUES (1),(3); +SELECT * FROM (SELECT i FROM t1 INTERSECT SELECT i FROM t3 + UNION ALL SELECT i FROM t2 EXCEPT ALL SELECT i FROM t3) d ORDER BY i; +CREATE VIEW v1 AS SELECT i FROM t1 INTERSECT SELECT i FROM t3 + UNION ALL SELECT i FROM t2 EXCEPT ALL SELECT i FROM t3; +SELECT * FROM v1 ORDER BY i; +DROP VIEW v1; +DROP TABLE t1,t2,t3; + +--echo # End of 11.4 tests diff --git a/mysql-test/main/invisible_field_grant_completely.result b/mysql-test/main/invisible_field_grant_completely.result index ac24aee2fb8ef..dd5b7fea3be38 100644 --- a/mysql-test/main/invisible_field_grant_completely.result +++ b/mysql-test/main/invisible_field_grant_completely.result @@ -39,7 +39,7 @@ a 2 3 select invisible,a from t1; -ERROR 42S22: Unknown column 'invisible' in 'SELECT' +ERROR 42000: SELECT command denied to user 'user_1'@'localhost' for column 'invisible' in table 't1' delete from t1 where a =1; update t1 set a=1 where a=3; select * from t1; diff --git a/mysql-test/main/invisible_field_grant_completely.test b/mysql-test/main/invisible_field_grant_completely.test index df05af9a06923..e262837593d77 100644 --- a/mysql-test/main/invisible_field_grant_completely.test +++ b/mysql-test/main/invisible_field_grant_completely.test @@ -29,7 +29,7 @@ insert into t1 values(2); select * from t1; insert into t1(a) values(3); select * from t1; ---error ER_BAD_FIELD_ERROR +--error ER_COLUMNACCESS_DENIED_ERROR select invisible,a from t1; delete from t1 where a =1; update t1 set a=1 where a=3; diff --git a/mysql-test/main/kill-2.opt b/mysql-test/main/kill-2.opt index ab6ca1731f53a..94ed5e3778ba2 100644 --- a/mysql-test/main/kill-2.opt +++ b/mysql-test/main/kill-2.opt @@ -1 +1 @@ ---skip-name-resolve +--skip-name-resolve --bind-address=::1,127.0.0.1 diff --git a/mysql-test/main/kill-2.result b/mysql-test/main/kill-2.result index 543ad4bdb4a64..174b8a8b6b5a2 100644 --- a/mysql-test/main/kill-2.result +++ b/mysql-test/main/kill-2.result @@ -39,6 +39,83 @@ disconnect a; disconnect b; drop user a@'127.0.0.1'; drop user b@'127.0.0.1'; -# # End of 10.3 tests # +# MDEV-40554 KILL checks user (not priv_user) and doesn't verify hostname +# +create user u1@'::1' identified by 'pw1'; +create user ''@'127.0.0.1'; +connect u1_ipv6,::1,u1,pw1; +select current_user(); +current_user() +u1@::1 +# the reported case: same login name, but an anonymous account +connect u1_anon,127.0.0.1,u1,; +select user(), current_user(); +user() current_user() +u1@127.0.0.1 @127.0.0.1 +# anonymous users don't see the process list at all +show processlist; +ERROR 42000: Access denied; you need (at least one of) the PROCESS privilege(s) for this operation +select count(*) as `threads visible` from information_schema.processlist; +threads visible +1 +select count(*) as `u1@'::1' visible` + from information_schema.processlist where id=$cid_sock; +u1@'::1' visible +0 +kill $cid_sock; +ERROR HY000: You are not owner of thread CID +show explain for $cid_sock; +ERROR 42000: Access denied; you need (at least one of) the PROCESS privilege(s) for this operation +kill user u1@'::1'; +connection u1_ipv6; +select user(), current_user(); +user() current_user() +u1@::1 u1@::1 +connection u1_anon; +connection default; +disconnect u1_anon; +drop user ''@'127.0.0.1'; +# same login name, a real account, but a different host +create user u1@'127.0.0.1' identified by 'pw2'; +connect u1_ipv4,127.0.0.1,u1,pw2; +select current_user(); +current_user() +u1@127.0.0.1 +# u1@127.0.0.1 must not see u1@'::1' - same user name, other account +show processlist; +Id User Host db Command Time State Info Progress +# u1 # # Query # # # # +select count(*) as `u1@'::1' visible` + from information_schema.processlist where id=$cid_sock; +u1@'::1' visible +0 +kill $cid_sock; +ERROR HY000: You are not owner of thread CID +show explain for $cid_sock; +ERROR 42000: Access denied; you need (at least one of) the PROCESS privilege(s) for this operation +kill user u1@'::1'; +connection u1_ipv6; +select user(), current_user(); +user() current_user() +u1@::1 u1@::1 +connection u1_ipv4; +# but another u1@127.0.0.1 connection is the same account, so it may +connect u1_tcp2,127.0.0.1,u1,pw2; +select count(*) as `u1@127.0.0.1 visible` + from information_schema.processlist where id=$cid_tcp; +u1@127.0.0.1 visible +1 +kill $cid_tcp; +# u1@'::1' was not killed +connection u1_ipv6; +select 1; +1 +1 +connection default; +disconnect u1_ipv6; +disconnect u1_tcp2; +drop user u1@'::1'; +drop user u1@'127.0.0.1'; +# End of 10.11 tests diff --git a/mysql-test/main/kill-2.test b/mysql-test/main/kill-2.test index fee81a6390666..b5fd8fec18966 100644 --- a/mysql-test/main/kill-2.test +++ b/mysql-test/main/kill-2.test @@ -6,7 +6,9 @@ # connection does not read() from a socket, but returns control to the # application. 'mysqltest' does not handle the kill request. # --- source include/not_embedded.inc +--source include/not_embedded.inc +--source include/count_sessions.inc +--source include/check_ipv6.inc --echo # --echo # MDEV-6896 kill user command cause MariaDB crash! @@ -55,6 +57,87 @@ kill user b; drop user a@'127.0.0.1'; drop user b@'127.0.0.1'; ---echo # --echo # End of 10.3 tests + +--echo # +--echo # MDEV-40554 KILL checks user (not priv_user) and doesn't verify hostname --echo # + +create user u1@'::1' identified by 'pw1'; +create user ''@'127.0.0.1'; + +connect u1_ipv6,::1,u1,pw1; +select current_user(); +let $cid_sock=`select connection_id()`; + +--echo # the reported case: same login name, but an anonymous account +connect u1_anon,127.0.0.1,u1,; +select user(), current_user(); + +--echo # anonymous users don't see the process list at all +--error ER_SPECIFIC_ACCESS_DENIED_ERROR +show processlist; +select count(*) as `threads visible` from information_schema.processlist; +evalp select count(*) as `u1@'::1' visible` + from information_schema.processlist where id=$cid_sock; +--replace_regex /thread [0-9]+/thread CID/ +--error ER_KILL_DENIED_ERROR +evalp kill $cid_sock; +--error ER_SPECIFIC_ACCESS_DENIED_ERROR +evalp show explain for $cid_sock; + +# killing a non-matching user doesn't report an error to not disclose +# that the user exists. but doesn't kill either +kill user u1@'::1'; +connection u1_ipv6; +select user(), current_user(); +connection u1_anon; + +connection default; +disconnect u1_anon; +drop user ''@'127.0.0.1'; + +--echo # same login name, a real account, but a different host +create user u1@'127.0.0.1' identified by 'pw2'; +connect u1_ipv4,127.0.0.1,u1,pw2; +select current_user(); +let $cid_tcp=`select connection_id()`; + +--echo # u1@127.0.0.1 must not see u1@'::1' - same user name, other account +--replace_result Execute Query +--replace_column 1 # 3 # 4 # 6 # 7 # 8 # 9 # +show processlist; +evalp select count(*) as `u1@'::1' visible` + from information_schema.processlist where id=$cid_sock; +--replace_regex /thread [0-9]+/thread CID/ +--error ER_KILL_DENIED_ERROR +evalp kill $cid_sock; +--error ER_SPECIFIC_ACCESS_DENIED_ERROR +evalp show explain for $cid_sock; + +# doesn't report an error, doesn't kill +kill user u1@'::1'; +connection u1_ipv6; +select user(), current_user(); +connection u1_ipv4; + +--echo # but another u1@127.0.0.1 connection is the same account, so it may +connect u1_tcp2,127.0.0.1,u1,pw2; +evalp select count(*) as `u1@127.0.0.1 visible` + from information_schema.processlist where id=$cid_tcp; +evalp kill $cid_tcp; + +--echo # u1@'::1' was not killed +connection u1_ipv6; +select 1; + +connection default; +disconnect u1_ipv6; +#u1_ipv4 was killed, must not be disconnected +disconnect u1_tcp2; +--source include/wait_until_count_sessions.inc + +drop user u1@'::1'; +drop user u1@'127.0.0.1'; + +--echo # End of 10.11 tests diff --git a/mysql-test/main/lotofstack.result b/mysql-test/main/lotofstack.result index 1e17487cde820..5fb6e3bf52478 100644 --- a/mysql-test/main/lotofstack.result +++ b/mysql-test/main/lotofstack.result @@ -103,6 +103,18 @@ drop table t3| # MDEV-40409 ST_GeomFromText stack overrun on deeply nested GeometryCollection # SELECT ST_GeomFromText(CONCAT(REPEAT('GEOMETRYCOLLECTION(',5000),'POINT(1 1)',REPEAT(')',5000))); +# +# MDEV-40540 ST_GeomFromWKB stack overrun on deeply nested GeometryCollection +# +set @a=concat(0x0107000000, repeat(0x010000000107000000, 120000), 0x00000000); +SELECT ST_GeomFromWKB(@a); +# +# MDEV-39750 ExtractValue does not control recursion depth. +# +SELECT ExtractValue('', REPEAT('(', 100000)); +ERROR HY000: Thread stack overrun: 'used bytes' used of a 'available' byte stack, and 'X' bytes needed. Consider increasing the thread_stack system variable. +SELECT ExtractValue('', REPEAT('/n', 100000)); +ERROR HY000: Thread stack overrun: 'used bytes' used of a 'available' byte stack, and 'X' bytes needed. Consider increasing the thread_stack system variable. # End of 10.6 tests # # MDEV-39813 ST_GeomFromGeoJSON does not control recursion depth diff --git a/mysql-test/main/lotofstack.test b/mysql-test/main/lotofstack.test index 2c5d082168aaf..13e6056f5e878 100644 --- a/mysql-test/main/lotofstack.test +++ b/mysql-test/main/lotofstack.test @@ -147,6 +147,26 @@ delimiter ;| SELECT ST_GeomFromText(CONCAT(REPEAT('GEOMETRYCOLLECTION(',5000),'POINT(1 1)',REPEAT(')',5000))); --enable_result_log +--echo # +--echo # MDEV-40540 ST_GeomFromWKB stack overrun on deeply nested GeometryCollection +--echo # +set @a=concat(0x0107000000, repeat(0x010000000107000000, 120000), 0x00000000); +--disable_result_log +--error ER_STACK_OVERRUN_NEED_MORE +SELECT ST_GeomFromWKB(@a); +--enable_result_log + +--echo # +--echo # MDEV-39750 ExtractValue does not control recursion depth. +--echo # +--replace_regex /overrun: [0-9]* bytes used of a [0-9]* byte stack, and [0-9]* bytes needed/overrun: 'used bytes' used of a 'available' byte stack, and 'X' bytes needed/ +--error ER_STACK_OVERRUN_NEED_MORE +SELECT ExtractValue('', REPEAT('(', 100000)); + +--replace_regex /overrun: [0-9]* bytes used of a [0-9]* byte stack, and [0-9]* bytes needed/overrun: 'used bytes' used of a 'available' byte stack, and 'X' bytes needed/ +--error ER_STACK_OVERRUN_NEED_MORE +SELECT ExtractValue('', REPEAT('/n', 100000)); + --echo # End of 10.6 tests diff --git a/mysql-test/main/mdev-37000.result b/mysql-test/main/mdev-37000.result new file mode 100644 index 0000000000000..73e6812382a1d --- /dev/null +++ b/mysql-test/main/mdev-37000.result @@ -0,0 +1,8 @@ +# +# MDEV-37000 Deleting a row from Aria table results to 'Index is corrupt' +# +CREATE OR REPLACE TABLE t (a INT, b CHAR(8) NOT NULL, KEY(b)) ENGINE=Aria DEFAULT CHARSET=utf8; +INSERT INTO t VALUES (1,'20070101'), (2,'20070101'); +DELETE FROM t LIMIT 2; +DROP TABLE t; +# End of 10.6 tests diff --git a/mysql-test/main/mdev-37000.test b/mysql-test/main/mdev-37000.test new file mode 100644 index 0000000000000..631be073f8c1c --- /dev/null +++ b/mysql-test/main/mdev-37000.test @@ -0,0 +1,9 @@ +--echo # +--echo # MDEV-37000 Deleting a row from Aria table results to 'Index is corrupt' +--echo # +CREATE OR REPLACE TABLE t (a INT, b CHAR(8) NOT NULL, KEY(b)) ENGINE=Aria DEFAULT CHARSET=utf8; +INSERT INTO t VALUES (1,'20070101'), (2,'20070101'); +DELETE FROM t LIMIT 2; +DROP TABLE t; + +--echo # End of 10.6 tests diff --git a/mysql-test/main/mysql_json_table_recreate.result b/mysql-test/main/mysql_json_table_recreate.result index ffeee64b6ddeb..9cf9151f98019 100644 --- a/mysql-test/main/mysql_json_table_recreate.result +++ b/mysql-test/main/mysql_json_table_recreate.result @@ -245,6 +245,14 @@ DECLARE a mysql_json; END; $$ ERROR HY000: 'MYSQL_JSON' is not allowed in this context -# # End of 10.5 tests # +# MDEV-40678 mysql_json plugin OOB reads +# +alter table t1 force; +ERROR HY000: Error parsing MySQL JSON format, please dump this table from MySQL and then restore it to be able to use it in MariaDB. +flush tables; +alter table t1 force; +ERROR HY000: Error parsing MySQL JSON format, please dump this table from MySQL and then restore it to be able to use it in MariaDB. +drop table t1; +# End of 10.6 tests diff --git a/mysql-test/main/mysql_json_table_recreate.test b/mysql-test/main/mysql_json_table_recreate.test index 94477d4f328f5..26cc7a6cc6b4a 100644 --- a/mysql-test/main/mysql_json_table_recreate.test +++ b/mysql-test/main/mysql_json_table_recreate.test @@ -17,7 +17,7 @@ call mtr.add_suppression("Table rebuild required"); call mtr.add_suppression("is marked as crashed"); call mtr.add_suppression("Checking"); -let $MYSQLD_DATADIR= `select @@datadir`; +let MYSQLD_DATADIR= `select @@datadir`; SET NAMES utf8; @@ -161,6 +161,40 @@ END; $$ DELIMITER ;$$ ---echo # --echo # End of 10.5 tests + +--echo # +--echo # MDEV-40678 mysql_json plugin OOB reads --echo # + +--copy_file std_data/mysql_json/mysql_json_test.frm $MYSQLD_DATADIR/test/t1.frm +--copy_file std_data/mysql_json/mysql_json_test.MYI $MYSQLD_DATADIR/test/t1.MYI +--copy_file std_data/mysql_json/mysql_json_test.MYD $MYSQLD_DATADIR/test/t1.MYD + +perl; + my $f="$ENV{MYSQLD_DATADIR}/test/t1.MYD"; + open F, '+<', $f or die "open($f): $!"; + sysseek F, 0x3c5, 0 or die "sysseek: $!"; + syswrite F, '(' or die "syswrite: $!"; +EOF + +--error ER_UNKNOWN_ERROR +alter table t1 force; +flush tables; + +--remove_file $MYSQLD_DATADIR/test/t1.MYD +--copy_file std_data/mysql_json/mysql_json_test.MYD $MYSQLD_DATADIR/test/t1.MYD + +perl; + my $f="$ENV{MYSQLD_DATADIR}/test/t1.MYD"; + open F, '+<', $f or die "open($f): $!"; + sysseek F, 0x3bc, 0 or die "sysseek: $!"; + syswrite F, '(' or die "syswrite: $!"; +EOF + +--error ER_UNKNOWN_ERROR +alter table t1 force; + +drop table t1; + +--echo # End of 10.6 tests diff --git a/mysql-test/main/mysqld--help,win.rdiff b/mysql-test/main/mysqld--help,win.rdiff index 5a05d81656273..2edbb4e646760 100644 --- a/mysql-test/main/mysqld--help,win.rdiff +++ b/mysql-test/main/mysqld--help,win.rdiff @@ -16,7 +16,17 @@ --net-buffer-length=# Buffer length for TCP/IP and socket communication --net-read-timeout=# -@@ -1568,6 +1570,10 @@ +@@ -1399,8 +1401,7 @@ + (Defaults to on; use --skip-secure-auth to disable.) + --secure-file-priv=name + Limit LOAD DATA, SELECT ... OUTFILE, and LOAD_FILE() to +- files within specified directory. Empty value means no +- limits except /proc ++ files within specified directory. + --secure-timestamp=name + Restricts direct setting of a session timestamp. Possible + levels are: YES - timestamp cannot deviate from the +@@ -1569,6 +1570,10 @@ Alias for log_slow_query_file. Log slow queries to given log file. Defaults logging to 'hostname'-slow.log. Must be enabled to activate other slow log options @@ -27,7 +37,7 @@ --socket=name Socket file to use for connection --sort-buffer-size=# Each thread that needs to do a sort allocates a buffer of -@@ -1587,6 +1593,7 @@ +@@ -1588,6 +1593,7 @@ aborted. Prevents the common mistake of accidentally deleting or updating every row in a table --stack-trace Print a symbolic stack trace on failure @@ -35,7 +45,7 @@ --standard-compliant-cte Allow only CTEs compliant to SQL standard (Defaults to on; use --skip-standard-compliant-cte to disable.) -@@ -1666,6 +1673,12 @@ +@@ -1667,6 +1673,12 @@ --thread-pool-max-threads=# Maximum allowed number of worker threads in the thread pool @@ -48,7 +58,7 @@ --thread-pool-oversubscribe=# How many additional active worker threads in a group are allowed -@@ -1707,8 +1720,8 @@ +@@ -1708,8 +1720,8 @@ background for binlogging by user threads are placed in a separate location (see `binlog_large_commit_threshold` option). Several paths may be specified, separated by a @@ -59,7 +69,7 @@ --transaction-alloc-block-size=# Allocation block size for transactions to be stored in binary log -@@ -1958,6 +1971,7 @@ +@@ -1959,6 +1971,7 @@ myisam-stats-method NULLS_UNEQUAL myisam-use-mmap FALSE mysql56-temporal-format TRUE @@ -67,7 +77,7 @@ net-buffer-length 16384 net-read-timeout 30 net-retry-count 10 -@@ -2137,6 +2151,7 @@ +@@ -2138,6 +2151,7 @@ slave-type-conversions slow-launch-time 2 slow-query-log FALSE @@ -75,7 +85,7 @@ sort-buffer-size 2097152 sql-mode STRICT_TRANS_TABLES,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION sql-safe-updates FALSE -@@ -2163,6 +2178,8 @@ +@@ -2164,6 +2178,8 @@ thread-pool-exact-stats FALSE thread-pool-idle-timeout 60 thread-pool-max-threads 65536 diff --git a/mysql-test/main/mysqld--help.result b/mysql-test/main/mysqld--help.result index 2d8b5c1b5aea8..e61280cf979c6 100644 --- a/mysql-test/main/mysqld--help.result +++ b/mysql-test/main/mysqld--help.result @@ -1399,7 +1399,8 @@ The following specify which files/extra groups are read (specified before remain (Defaults to on; use --skip-secure-auth to disable.) --secure-file-priv=name Limit LOAD DATA, SELECT ... OUTFILE, and LOAD_FILE() to - files within specified directory + files within specified directory. Empty value means no + limits except /proc --secure-timestamp=name Restricts direct setting of a session timestamp. Possible levels are: YES - timestamp cannot deviate from the diff --git a/mysql-test/main/mysqldump-system,win.rdiff b/mysql-test/main/mysqldump-system,win.rdiff index d9435bd1a639c..d75c16dca9c24 100644 --- a/mysql-test/main/mysqldump-system,win.rdiff +++ b/mysql-test/main/mysqldump-system,win.rdiff @@ -6,7 +6,7 @@ mysql.time_zone_transition 3719776009 -mysql.plugin 1587119305 +mysql.plugin 2184891911 - mysql.servers 1109359608 + mysql.servers 3351805004 -mysql.func 2140302938 +mysql.func 2154697260 mysql.innodb_table_stats 1972297402 @@ -18,7 +18,7 @@ mysql.time_zone_transition 3719776009 -mysql.plugin 1587119305 +mysql.plugin 2184891911 - mysql.servers 1109359608 + mysql.servers 3351805004 -mysql.func 2140302938 +mysql.func 2154697260 mysql.innodb_table_stats 1972297402 diff --git a/mysql-test/main/mysqldump-system.result b/mysql-test/main/mysqldump-system.result index 23078fae5125d..3a160b38d4fab 100644 --- a/mysql-test/main/mysqldump-system.result +++ b/mysql-test/main/mysqldump-system.result @@ -40,6 +40,7 @@ CREATE SERVER s1 FOREIGN DATA WRAPPER mysql OPTIONS (HOST 'localhost', DATABASE 'd''b', USER 'o''brien', PASSWORD 'pw''d'); CREATE SERVER `s'2` FOREIGN DATA WRAPPER 'my''sql' OPTIONS (HOST 'h2', DATABASE 'db2', USER 'u2'); +create server `s``3` foreign data wrapper mysql options (host 'value'); CREATE FUNCTION metaphon RETURNS STRING SONAME "UDF_EXAMPLE_LIB"; CREATE AGGREGATE FUNCTION avgcost RETURNS REAL SONAME "UDF_EXAMPLE_LIB"; # @@ -95,8 +96,9 @@ DROP ROLE mariadb_dump_import_role; /*M!100203 EXECUTE IMMEDIATE CONCAT('SET ROLE ', @current_role) */; CREATE FUNCTION `metaphon` RETURNS STRING SONAME 'UDF_EXAMPLE_LIB'; CREATE AGGREGATE FUNCTION `avgcost` RETURNS REAL SONAME 'UDF_EXAMPLE_LIB'; -CREATE SERVER `s'2` FOREIGN DATA WRAPPER `my'sql` OPTIONS (HOST 'h2', DATABASE 'db2', USER 'u2'); -CREATE SERVER `s1` FOREIGN DATA WRAPPER `mysql` OPTIONS (HOST 'localhost', DATABASE 'd''b', USER 'o''brien', PASSWORD 'pw''d'); +CREATE SERVER `s'2` FOREIGN DATA WRAPPER `my'sql` OPTIONS (`HOST` 'h2', `DATABASE` 'db2', `USER` 'u2'); +CREATE SERVER `s1` FOREIGN DATA WRAPPER `mysql` OPTIONS (`HOST` 'localhost', `DATABASE` 'd''b', `USER` 'o''brien', `PASSWORD` 'pw''d'); +CREATE SERVER `s``3` FOREIGN DATA WRAPPER `mysql` OPTIONS (`host` 'value'); USE mysql; @@ -718,8 +720,9 @@ DROP ROLE mariadb_dump_import_role; CREATE /*M!100103 OR REPLACE */ FUNCTION `metaphon` RETURNS STRING SONAME 'UDF_EXAMPLE_LIB'; /*!50701 DROP FUNCTION IF EXISTS `avgcost` */; CREATE /*M!100103 OR REPLACE */ AGGREGATE FUNCTION `avgcost` RETURNS REAL SONAME 'UDF_EXAMPLE_LIB'; -CREATE /*M!100103 OR REPLACE */ SERVER `s'2` FOREIGN DATA WRAPPER `my'sql` OPTIONS (HOST 'h2', DATABASE 'db2', USER 'u2'); -CREATE /*M!100103 OR REPLACE */ SERVER `s1` FOREIGN DATA WRAPPER `mysql` OPTIONS (HOST 'localhost', DATABASE 'd''b', USER 'o''brien', PASSWORD 'pw''d'); +CREATE /*M!100103 OR REPLACE */ SERVER `s'2` FOREIGN DATA WRAPPER `my'sql` OPTIONS (`HOST` 'h2', `DATABASE` 'db2', `USER` 'u2'); +CREATE /*M!100103 OR REPLACE */ SERVER `s1` FOREIGN DATA WRAPPER `mysql` OPTIONS (`HOST` 'localhost', `DATABASE` 'd''b', `USER` 'o''brien', `PASSWORD` 'pw''d'); +CREATE /*M!100103 OR REPLACE */ SERVER `s``3` FOREIGN DATA WRAPPER `mysql` OPTIONS (`host` 'value'); USE mysql; @@ -1317,8 +1320,9 @@ DROP ROLE mariadb_dump_import_role; /*M!100203 EXECUTE IMMEDIATE CONCAT('SET ROLE ', @current_role) */; CREATE FUNCTION IF NOT EXISTS `metaphon` RETURNS STRING SONAME 'UDF_EXAMPLE_LIB'; CREATE AGGREGATE FUNCTION IF NOT EXISTS `avgcost` RETURNS REAL SONAME 'UDF_EXAMPLE_LIB'; -CREATE SERVER /*M!100103 IF NOT EXISTS */ `s'2` FOREIGN DATA WRAPPER `my'sql` OPTIONS (HOST 'h2', DATABASE 'db2', USER 'u2'); -CREATE SERVER /*M!100103 IF NOT EXISTS */ `s1` FOREIGN DATA WRAPPER `mysql` OPTIONS (HOST 'localhost', DATABASE 'd''b', USER 'o''brien', PASSWORD 'pw''d'); +CREATE SERVER /*M!100103 IF NOT EXISTS */ `s'2` FOREIGN DATA WRAPPER `my'sql` OPTIONS (`HOST` 'h2', `DATABASE` 'db2', `USER` 'u2'); +CREATE SERVER /*M!100103 IF NOT EXISTS */ `s1` FOREIGN DATA WRAPPER `mysql` OPTIONS (`HOST` 'localhost', `DATABASE` 'd''b', `USER` 'o''brien', `PASSWORD` 'pw''d'); +CREATE SERVER /*M!100103 IF NOT EXISTS */ `s``3` FOREIGN DATA WRAPPER `mysql` OPTIONS (`host` 'value'); USE mysql; @@ -1876,7 +1880,7 @@ Table Checksum mysql.roles_mapping 2510045525 mysql.time_zone_transition 3719776009 mysql.plugin 1587119305 -mysql.servers 1109359608 +mysql.servers 3351805004 mysql.func 2140302938 mysql.innodb_table_stats 1972297402 mysql.table_stats 1911089388 @@ -1897,6 +1901,7 @@ DROP FUNCTION IF EXISTS metaphon; DROP FUNCTION IF EXISTS avgcost; DROP SERVER s1; DROP SERVER `s'2`; +DROP SERVER `s``3`; set time_zone= @@global.time_zone; # Restore from mysqldump DROP USER mariadb_test_restore; @@ -1913,7 +1918,7 @@ Table Checksum mysql.roles_mapping 2510045525 mysql.time_zone_transition 3719776009 mysql.plugin 1587119305 -mysql.servers 1109359608 +mysql.servers 3351805004 mysql.func 2140302938 mysql.innodb_table_stats 1972297402 mysql.table_stats 1911089388 @@ -1921,6 +1926,7 @@ DROP FUNCTION IF EXISTS metaphon; DROP FUNCTION IF EXISTS avgcost; DROP SERVER s1; DROP SERVER `s'2`; +DROP SERVER `s``3`; DELETE FROM mysql.column_stats WHERE db_name='mysql'; DELETE FROM mysql.index_stats WHERE db_name='mysql'; DELETE FROM mysql.table_stats WHERE db_name='mysql'; diff --git a/mysql-test/main/mysqldump-system.test b/mysql-test/main/mysqldump-system.test index 08448603900e9..f6e4455dacfdc 100644 --- a/mysql-test/main/mysqldump-system.test +++ b/mysql-test/main/mysqldump-system.test @@ -62,6 +62,7 @@ CREATE SERVER s1 FOREIGN DATA WRAPPER mysql OPTIONS (HOST 'localhost', DATABASE 'd''b', USER 'o''brien', PASSWORD 'pw''d'); CREATE SERVER `s'2` FOREIGN DATA WRAPPER 'my''sql' OPTIONS (HOST 'h2', DATABASE 'db2', USER 'u2'); +create server `s``3` foreign data wrapper mysql options (host 'value'); --replace_result $UDF_EXAMPLE_SO UDF_EXAMPLE_LIB eval CREATE FUNCTION metaphon RETURNS STRING SONAME "$UDF_EXAMPLE_SO"; @@ -131,6 +132,7 @@ DROP FUNCTION IF EXISTS metaphon; DROP FUNCTION IF EXISTS avgcost; DROP SERVER s1; DROP SERVER `s'2`; +DROP SERVER `s``3`; set time_zone= @@global.time_zone; --echo # Restore from mysqldump @@ -157,6 +159,7 @@ DROP FUNCTION IF EXISTS avgcost; DROP SERVER s1; DROP SERVER `s'2`; +DROP SERVER `s``3`; # EITS && innodb stats DELETE FROM mysql.column_stats WHERE db_name='mysql'; diff --git a/mysql-test/main/mysqldump.result b/mysql-test/main/mysqldump.result index c62829ad4d27c..d792199dde75e 100644 --- a/mysql-test/main/mysqldump.result +++ b/mysql-test/main/mysqldump.result @@ -6974,25 +6974,25 @@ mariadb-dump: Couldn't execute 'SHOW CREATE FUNCTION `f1`': Undeclared variable: COMMIT; SET AUTOCOMMIT=@OLD_AUTOCOMMIT; /*!50106 SET @save_time_zone= @@TIME_ZONE */ ; +/*!50003 SET @saved_cs_client = @@character_set_client */ ; +/*!50003 SET @saved_cs_results = @@character_set_results */ ; +/*!50003 SET @saved_col_connection = @@collation_connection */ ; +/*!50003 SET character_set_client = utf8mb3 */ ; +/*!50003 SET character_set_results = utf8mb3 */ ; +/*!50003 SET collation_connection = utf8mb3_uca1400_ai_ci */ ; +/*!50003 SET @saved_sql_mode = @@sql_mode */ ; +/*!50003 SET sql_mode = '' */ ; +/*!50003 SET @saved_time_zone = @@time_zone */ ; +/*!50003 SET time_zone = 'SYSTEM' */ ; DELIMITER ;; -/*!50003 SET @saved_cs_client = @@character_set_client */ ;; -/*!50003 SET @saved_cs_results = @@character_set_results */ ;; -/*!50003 SET @saved_col_connection = @@collation_connection */ ;; -/*!50003 SET character_set_client = utf8mb3 */ ;; -/*!50003 SET character_set_results = utf8mb3 */ ;; -/*!50003 SET collation_connection = utf8mb3_uca1400_ai_ci */ ;; -/*!50003 SET @saved_sql_mode = @@sql_mode */ ;; -/*!50003 SET sql_mode = '' */ ;; -/*!50003 SET @saved_time_zone = @@time_zone */ ;; -/*!50003 SET time_zone = 'SYSTEM' */ ;; /*!50106 CREATE*/ /*!50117 DEFINER=`root`@`localhost`*/ /*!50106 EVENT `e1` ON SCHEDULE EVERY 1 YEAR STARTS '2030-01-01 00:00:00' ON COMPLETION NOT PRESERVE ENABLE DO select not_a_value */ ;; -/*!50003 SET time_zone = @saved_time_zone */ ;; -/*!50003 SET sql_mode = @saved_sql_mode */ ;; -/*!50003 SET character_set_client = @saved_cs_client */ ;; -/*!50003 SET character_set_results = @saved_cs_results */ ;; -/*!50003 SET collation_connection = @saved_col_connection */ ;; DELIMITER ; +/*!50003 SET time_zone = @saved_time_zone */ ; +/*!50003 SET sql_mode = @saved_sql_mode */ ; +/*!50003 SET character_set_client = @saved_cs_client */ ; +/*!50003 SET character_set_results = @saved_cs_results */ ; +/*!50003 SET collation_connection = @saved_col_connection */ ; /*!50106 SET TIME_ZONE= @save_time_zone */ ; /*!50003 SET @saved_sql_mode = @@sql_mode */ ; /*!50003 SET sql_mode = '' */ ; diff --git a/mysql-test/main/mysqldump.test b/mysql-test/main/mysqldump.test index 35bf5fe02cc01..04c8324a7a5e3 100644 --- a/mysql-test/main/mysqldump.test +++ b/mysql-test/main/mysqldump.test @@ -3221,7 +3221,7 @@ drop table db2.t1; --exec $MYSQL db1 < $MYSQLTEST_VARDIR/tmp/dump/db1/t1.sql --exec $MYSQL db2 < $MYSQLTEST_VARDIR/tmp/dump/db2/t1.sql --exec $MYSQL_IMPORT --silent db1 $MYSQLTEST_VARDIR/tmp/dump/db1/t1.txt ---exec $MYSQL_IMPORT --silent db2 $MYSQLTEST_VARDIR/tmp/dump/db2/t1.txt +--exec $MYSQL_IMPORT --lock-tables --silent db2 $MYSQLTEST_VARDIR/tmp/dump/db2/t1.txt select * from db1.t1; select * from db2.t1; drop database db1; diff --git a/mysql-test/main/mysqltest.result b/mysql-test/main/mysqltest.result index 1bc89212c5fe9..2b0b2ac309d15 100644 --- a/mysql-test/main/mysqltest.result +++ b/mysql-test/main/mysqltest.result @@ -1031,6 +1031,12 @@ the false block has written nothing mysqltest: At line 1: End of file encountered before 'eval' delimiter was found # Only 'eval' is allowed as the third argument mysqltest: At line 1: Invalid argument 'junk' to 'write_file', only 'eval' is allowed +# +# \0 in a query +# +select hex("12"); +hex("1\x002") +310032 # End of 10.6 tests # # MDEV-29344: engines/iuds.insert_time cannot run with PS protocol (syntax error) diff --git a/mysql-test/main/mysqltest.test b/mysql-test/main/mysqltest.test index 5d6543e4f725c..17772a405aeab 100644 --- a/mysql-test/main/mysqltest.test +++ b/mysql-test/main/mysqltest.test @@ -3057,6 +3057,11 @@ EOF --error 1 --exec echo "--write_file $file EOF junk" | $MYSQL_TEST 2>&1 +--echo # +--echo # \0 in a query +--echo # +select hex("12"); + --echo # End of 10.6 tests --echo # @@ -3080,12 +3085,11 @@ SELECT 1 /* doesn't throw error */; --rmdir $srcdir +--echo # End of 10.11 tests + # Wait till we reached the initial number of concurrent sessions --source include/wait_until_count_sessions.inc --disable_query_log --eval SET GLOBAL max_connections = $saved_max_connections --enable_query_log - ---echo # End of 10.11 tests - diff --git a/mysql-test/main/mysqltest_expression_evaluation.result b/mysql-test/main/mysqltest_expression_evaluation.result index d9fa0ccb3b835..b74fc767d4797 100644 --- a/mysql-test/main/mysqltest_expression_evaluation.result +++ b/mysql-test/main/mysqltest_expression_evaluation.result @@ -505,4 +505,11 @@ mysqltest: At line 1: Evaluation error: Modulo by zero # Test case: Overflow mysqltest: At line 1: Range error: 18446744073709551616 value out of range for Integer type mysqltest: At line 1: Range error: 18446744073709551616 value out of range for Integer type +# +# MDEV-37865: IF() function is returning incorrect error +# +# Test case: unknown function name that is a prefix of a known one +mysqltest: At line 1: Syntax error: Unknown function 'if' +mysqltest: At line 1: Syntax error: Unknown function 'l' # Expression evaluation tests completed successfully +# End of 10.6 tests diff --git a/mysql-test/main/mysqltest_expression_evaluation.test b/mysql-test/main/mysqltest_expression_evaluation.test index 7a108f2fe172f..b76648284fc76 100644 --- a/mysql-test/main/mysqltest_expression_evaluation.test +++ b/mysql-test/main/mysqltest_expression_evaluation.test @@ -894,4 +894,17 @@ let $large_xor = $(0x7FFFFFFFFFFFFFFF ^ 0x7FFFFFFFFFFFFFFE); --error 1 --exec echo "let \$a = \$(-18446744073709551616);" | $MYSQL_TEST 2>&1 +--echo # +--echo # MDEV-37865: IF() function is returning incorrect error +--echo # + +--echo # Test case: unknown function name that is a prefix of a known one +--error 1 +--exec echo "let \$a = \$(if(null, 'fallback', 'actual'));" | $MYSQL_TEST 2>&1 + +--error 1 +--exec echo "let \$a = \$(l('AB'));" | $MYSQL_TEST 2>&1 + --echo # Expression evaluation tests completed successfully + +--echo # End of 10.6 tests diff --git a/mysql-test/main/mysqltest_string_functions.result b/mysql-test/main/mysqltest_string_functions.result index eb3d56e655408..2f4eb0c94b0da 100644 --- a/mysql-test/main/mysqltest_string_functions.result +++ b/mysql-test/main/mysqltest_string_functions.result @@ -1116,4 +1116,11 @@ insert(upper('hello world'), 7, 5, lower('MARIADB')) -> HELLO mariadb concat_ws(' ', upper('hello'), lower('WORLD')) -> HELLO world ifnull(nullif('same', 'same'), 'default') -> default coalesce(nullif('test', 'test'), 'backup') -> backup +# +# MDEV-37859: Mysqltest hex() not working for string values +# +hex("abc") -> 616263 +hex("255") -> 323535 +hex('') -> # Expression evaluation string functions tests completed successfully +# End of 10.6 tests diff --git a/mysql-test/main/mysqltest_string_functions.test b/mysql-test/main/mysqltest_string_functions.test index 10e0d546e157c..4ccdfec59c071 100644 --- a/mysql-test/main/mysqltest_string_functions.test +++ b/mysql-test/main/mysqltest_string_functions.test @@ -2128,4 +2128,19 @@ let $result5 = $(ifnull(nullif('same', 'same'), 'default')); let $result6 = $(coalesce(nullif('test', 'test'), 'backup')); --echo coalesce(nullif('test', 'test'), 'backup') -> $result6 +--echo # +--echo # MDEV-37859: Mysqltest hex() not working for string values +--echo # + +let $hex_str1 = $(hex("abc")); +--echo hex("abc") -> $hex_str1 + +let $hex_str2 = $(hex("255")); +--echo hex("255") -> $hex_str2 + +let $hex_str3 = $(hex('')); +--echo hex('') -> $hex_str3 + --echo # Expression evaluation string functions tests completed successfully + +--echo # End of 10.6 tests diff --git a/mysql-test/main/rli_run_lock.result b/mysql-test/main/rli_run_lock.result new file mode 100644 index 0000000000000..e2776a356d0d2 --- /dev/null +++ b/mysql-test/main/rli_run_lock.result @@ -0,0 +1,18 @@ +CHANGE MASTER TO master_host='127.0.0.1', master_user='root'; +START SLAVE SQL_THREAD; +include/wait_for_slave_sql_to_start.inc +SET @@SESSION.debug_sync= +'hold_sss_with_run_lock SIGNAL sss_got_run_lock WAIT_FOR sss_continue'; +SHOW SLAVE STATUS; +connect stopper, 127.0.0.1, root, , , $SERVER_MYPORT_1; +SET @@SESSION.debug_sync= 'now WAIT_FOR sss_got_run_lock'; +STOP SLAVE SQL_THREAD; +connect continuer, 127.0.0.1, root, , , $SERVER_MYPORT_1; +SET @@SESSION.debug_sync= 'now SIGNAL sss_continue'; +connection stopper; +disconnect continuer; +connection default; +disconnect stopper; +SET @@SESSION.debug_sync= RESET; +include/wait_for_slave_sql_to_stop.inc +RESET SLAVE ALL; diff --git a/mysql-test/main/rli_run_lock.test b/mysql-test/main/rli_run_lock.test new file mode 100644 index 0000000000000..ca8eaf4d92943 --- /dev/null +++ b/mysql-test/main/rli_run_lock.test @@ -0,0 +1,43 @@ +# MDEV-40298 +# Use-After-Free when SQL Thread stops in the middle of SHOW SLAVE STATUS +# +# Test that the correct synchronization is performed + +--source include/have_debug_sync.inc +--source include/have_binlog_format_mixed.inc # No actual replication required +CHANGE MASTER TO master_host='127.0.0.1', master_user='root'; # basic setup + +START SLAVE SQL_THREAD; +--source include/wait_for_slave_sql_to_start.inc + + +SET @@SESSION.debug_sync= + 'hold_sss_with_run_lock SIGNAL sss_got_run_lock WAIT_FOR sss_continue'; +--send SHOW SLAVE STATUS + +--connect (stopper, 127.0.0.1, root, , , $SERVER_MYPORT_1) + SET @@SESSION.debug_sync= 'now WAIT_FOR sss_got_run_lock'; + --send STOP SLAVE SQL_THREAD + # Wait a bit for the thread to "shut down" + # (It should not shut down, but block waiting for the ongoing SSS; + # the tested bug instead pulled this carpet under the SSS's feet.) + --sleep 1 + + --connect (continuer, 127.0.0.1, root, , , $SERVER_MYPORT_1) + SET @@SESSION.debug_sync= 'now SIGNAL sss_continue'; + --connection stopper + --disconnect continuer + + --reap +--connection default +--disconnect stopper + + +--disable_result_log + --reap +--enable_result_log + +# Clean-up +SET @@SESSION.debug_sync= RESET; +--source include/wait_for_slave_sql_to_stop.inc +RESET SLAVE ALL; diff --git a/mysql-test/main/servers.result b/mysql-test/main/servers.result index 10ca5f953ed74..ea7567ecfef1a 100644 --- a/mysql-test/main/servers.result +++ b/mysql-test/main/servers.result @@ -59,28 +59,28 @@ create server srv foreign data wrapper mysql options (host "localhost", port 12345, wait_what "it's all good"); show create server srv; Server Create Server -srv CREATE SERVER `srv` FOREIGN DATA WRAPPER `mysql` OPTIONS (host 'localhost', port '12345', wait_what 'it''s all good'); +srv CREATE SERVER `srv` FOREIGN DATA WRAPPER `mysql` OPTIONS (`host` 'localhost', `port` '12345', `wait_what` 'it''s all good'); create or replace server srv foreign data wrapper Foo options (host "somewhere.else", port 54321, wait_what "it's all good", foo 'bar'); show create server srv; Server Create Server -srv CREATE SERVER `srv` FOREIGN DATA WRAPPER `Foo` OPTIONS (host 'somewhere.else', port '54321', wait_what 'it''s all good', foo 'bar'); +srv CREATE SERVER `srv` FOREIGN DATA WRAPPER `Foo` OPTIONS (`host` 'somewhere.else', `port` '54321', `wait_what` 'it''s all good', `foo` 'bar'); alter server srv options (socket "sock", port 123, foo "", bar ')"{'); show create server srv; Server Create Server -srv CREATE SERVER `srv` FOREIGN DATA WRAPPER `Foo` OPTIONS (host 'somewhere.else', wait_what 'it''s all good', socket 'sock', port '123', foo '', bar ')"{'); +srv CREATE SERVER `srv` FOREIGN DATA WRAPPER `Foo` OPTIONS (`host` 'somewhere.else', `wait_what` 'it''s all good', `socket` 'sock', `port` '123', `foo` '', `bar` ')"{'); alter server srv options (socket "sock", port 123, bar "quux"); show create server srv; Server Create Server -srv CREATE SERVER `srv` FOREIGN DATA WRAPPER `Foo` OPTIONS (host 'somewhere.else', wait_what 'it''s all good', foo '', socket 'sock', port '123', bar 'quux'); +srv CREATE SERVER `srv` FOREIGN DATA WRAPPER `Foo` OPTIONS (`host` 'somewhere.else', `wait_what` 'it''s all good', `foo` '', `socket` 'sock', `port` '123', `bar` 'quux'); create or replace server srv foreign data wrapper foo options (host "localhost", port "12345"); create or replace server srv foreign data wrapper mysql options (host "localhost", port "bar321"); -ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near '"bar321")' at line 2 +ERROR HY000: Incorrect value 'bar321' for option 'PORT' create or replace server srv foreign data wrapper mysql options (host "localhost", port "123bar"); -ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near '"123bar")' at line 2 +ERROR HY000: Incorrect value '123bar' for option 'PORT' create or replace server srv foreign data wrapper mysql options (host "localhost", port "0"); show create server nonexist; @@ -153,21 +153,51 @@ rename table mysql.plugin_save to mysql.plugin; # ## Error code depends on length of long CREATE SERVER s FOREIGN DATA WRAPPER mysql OPTIONS (HOST'',PORT 4294967295); -ERROR 22003: port value is out of range in 'INT' +ERROR HY000: Incorrect value '4294967295' for option 'PORT' +show warnings; +Level Code Message +Error 1912 Incorrect value '4294967295' for option 'PORT' CREATE SERVER s FOREIGN DATA WRAPPER mysql OPTIONS (HOST'',PORT 2147483647); select * from mysql.servers; Server_name Host Db Username Password Port Socket Wrapper Owner Options s 2147483647 mysql {"HOST": "", "PORT": "2147483647"} -ALTER SERVER s OPTIONS (PORT 4294967295); -ERROR 22003: port value is out of range in 'INT' +ALTER SERVER s OPTIONS (PORT '4294967295'); +ERROR HY000: Incorrect value '4294967295' for option 'PORT' select * from mysql.servers; Server_name Host Db Username Password Port Socket Wrapper Owner Options s 2147483647 mysql {"HOST": "", "PORT": "2147483647"} -drop server s; +CREATE OR REPLACE SERVER s FOREIGN DATA WRAPPER mysql OPTIONS (HOST'',PORT '4294967295'); +ERROR HY000: Incorrect value '4294967295' for option 'PORT' +select * from mysql.servers; +Server_name Host Db Username Password Port Socket Wrapper Owner Options +s 2147483647 mysql {"HOST": "", "PORT": "2147483647"} +DROP SERVER s; CREATE SERVER s FOREIGN DATA WRAPPER mysql OPTIONS (HOST'',PORT 2147483648); -ERROR 22003: port value is out of range in 'INT' -CREATE SERVER s FOREIGN DATA WRAPPER mysql OPTIONS (HOST'',PORT 18446744073709551615); -ERROR 22003: port value is out of range in 'INT' +ERROR HY000: Incorrect value '2147483648' for option 'PORT' CREATE SERVER s FOREIGN DATA WRAPPER mysql OPTIONS (HOST'',PORT -5); ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near '-5)' at line 1 +CREATE SERVER s FOREIGN DATA WRAPPER mysqL OPTIONS (HOST '',PORT '1', PORT '2'); +SHOW CREATE SERVER s; +Server Create Server +s CREATE SERVER `s` FOREIGN DATA WRAPPER `mysqL` OPTIONS (`HOST` '', `PORT` '2'); +select * from mysql.servers; +Server_name Host Db Username Password Port Socket Wrapper Owner Options +s 2 mysqL {"HOST": "", "PORT": "2"} +ALTER SERVER s options (PORT '7', PORT '8'); +SHOW CREATE SERVER s; +Server Create Server +s CREATE SERVER `s` FOREIGN DATA WRAPPER `mysqL` OPTIONS (`HOST` '', `PORT` '8'); +select * from mysql.servers; +Server_name Host Db Username Password Port Socket Wrapper Owner Options +s 8 mysqL {"HOST": "", "PORT": "8"} +DROP SERVER s; # End of 10.11 tests +# +# MDEV-40312 SHOW CREATE SERVER incorrect quoting +# +create server s1 foreign data wrapper 'my''sql' options (`ho``st` 'va''lue'); +show create server s1; +Server Create Server +s1 CREATE SERVER `s1` FOREIGN DATA WRAPPER `my'sql` OPTIONS (`ho``st` 'va''lue'); +drop server s1; +# End of 11.8 tests diff --git a/mysql-test/main/servers.test b/mysql-test/main/servers.test index 13c75e8e7bc9f..2e58fc2a4e9dc 100644 --- a/mysql-test/main/servers.test +++ b/mysql-test/main/servers.test @@ -71,11 +71,11 @@ show create server srv; create or replace server srv foreign data wrapper foo options (host "localhost", port "12345"); ---error ER_PARSE_ERROR +--error ER_BAD_OPTION_VALUE create or replace server srv foreign data wrapper mysql options (host "localhost", port "bar321"); ---error ER_PARSE_ERROR +--error ER_BAD_OPTION_VALUE create or replace server srv foreign data wrapper mysql options (host "localhost", port "123bar"); @@ -166,19 +166,38 @@ rename table mysql.plugin_save to mysql.plugin; --echo # MDEV-36230 SIGSEGV in store_server_fields on CREATE SERVER --echo # --echo ## Error code depends on length of long ---error ER_DATA_OUT_OF_RANGE +--error ER_BAD_OPTION_VALUE CREATE SERVER s FOREIGN DATA WRAPPER mysql OPTIONS (HOST'',PORT 4294967295); +show warnings; CREATE SERVER s FOREIGN DATA WRAPPER mysql OPTIONS (HOST'',PORT 2147483647); select * from mysql.servers; ---error ER_DATA_OUT_OF_RANGE -ALTER SERVER s OPTIONS (PORT 4294967295); +--error ER_BAD_OPTION_VALUE +ALTER SERVER s OPTIONS (PORT '4294967295'); +select * from mysql.servers; +--error ER_BAD_OPTION_VALUE +CREATE OR REPLACE SERVER s FOREIGN DATA WRAPPER mysql OPTIONS (HOST'',PORT '4294967295'); select * from mysql.servers; -drop server s; ---error ER_DATA_OUT_OF_RANGE +DROP SERVER s; +--error ER_BAD_OPTION_VALUE CREATE SERVER s FOREIGN DATA WRAPPER mysql OPTIONS (HOST'',PORT 2147483648); ---error ER_DATA_OUT_OF_RANGE -CREATE SERVER s FOREIGN DATA WRAPPER mysql OPTIONS (HOST'',PORT 18446744073709551615); --error ER_PARSE_ERROR CREATE SERVER s FOREIGN DATA WRAPPER mysql OPTIONS (HOST'',PORT -5); +CREATE SERVER s FOREIGN DATA WRAPPER mysqL OPTIONS (HOST '',PORT '1', PORT '2'); +SHOW CREATE SERVER s; +select * from mysql.servers; +ALTER SERVER s options (PORT '7', PORT '8'); +SHOW CREATE SERVER s; +select * from mysql.servers; +DROP SERVER s; --echo # End of 10.11 tests + +--echo # +--echo # MDEV-40312 SHOW CREATE SERVER incorrect quoting +--echo # + +create server s1 foreign data wrapper 'my''sql' options (`ho``st` 'va''lue'); +show create server s1; +drop server s1; + +--echo # End of 11.8 tests diff --git a/mysql-test/main/servers_upgrade.result b/mysql-test/main/servers_upgrade.result index 5fe4cc7355601..7290af58a6382 100644 --- a/mysql-test/main/servers_upgrade.result +++ b/mysql-test/main/servers_upgrade.result @@ -5,11 +5,11 @@ call mtr.add_suppression('$SEARCH_PATTERN'); create server s foreign data wrapper mysql options (host "127.0.0.1", database "test", user "root", port 3306); show create server s; Server Create Server -s CREATE SERVER `s` FOREIGN DATA WRAPPER `mysql` OPTIONS (host '127.0.0.1', database 'test', user 'root', port '3306'); +s CREATE SERVER `s` FOREIGN DATA WRAPPER `mysql` OPTIONS (`host` '127.0.0.1', `database` 'test', `user` 'root', `port` '3306'); alter table mysql.servers drop column Options; show create server s; Server Create Server -s CREATE SERVER `s` FOREIGN DATA WRAPPER `mysql` OPTIONS (host '127.0.0.1', database 'test', user 'root', port '3306'); +s CREATE SERVER `s` FOREIGN DATA WRAPPER `mysql` OPTIONS (`host` '127.0.0.1', `database` 'test', `user` 'root', `port` '3306'); # restart show create server s; Server Create Server diff --git a/mysql-test/main/table_elim.result b/mysql-test/main/table_elim.result index 2677146d9bba8..bc49e173bd2c9 100644 --- a/mysql-test/main/table_elim.result +++ b/mysql-test/main/table_elim.result @@ -1110,5 +1110,81 @@ id select_type table type possible_keys key key_len ref rows Extra 1 SIMPLE t1 ALL NULL NULL NULL NULL 2 DROP TABLE t1, t2; # +# MDEV-36610: Subquery wrongly marked as eliminated by table elimination +# when equality propagation shares a WHERE-clause subquery +# into an eliminated outer join's ON expression +# +CREATE TABLE t0 (k INT); +INSERT INTO t0 VALUES (20),(9); +CREATE TABLE t1 (a BIGINT); +INSERT INTO t1 VALUES (1),(2); +CREATE TABLE t2 (b INT PRIMARY KEY); +INSERT INTO t2 VALUES (3),(4); +INSERT INTO t2 VALUES (5),(6); +CREATE TABLE t3 (c INT); +INSERT INTO t3 VALUES (2); +CREATE TABLE t4 (a BIGINT, x INT); +INSERT INTO t4 VALUES (2,20),(3,30); +# t2 is eliminated, but the subquery must still be shown and executed: +explain SELECT t1.* FROM t1 LEFT JOIN t2 ON (t2.b = t1.a) +WHERE t1.a = (SELECT c FROM t3); +id select_type table type possible_keys key key_len ref rows Extra +1 PRIMARY t1 ALL NULL NULL NULL NULL 2 Using where +2 SUBQUERY t3 system NULL NULL NULL NULL 1 +SELECT t1.* FROM t1 LEFT JOIN t2 ON (t2.b = t1.a) +WHERE t1.a = (SELECT c FROM t3); +a +2 +# t2 is eliminated; the subquery must still be shown and executed: +explain SELECT t0.k, t4.x FROM t0 +LEFT JOIN (t4 LEFT JOIN t2 ON t2.b = t4.a) +ON (t0.k = t4.x AND t4.a = (SELECT c FROM t3)); +id select_type table type possible_keys key key_len ref rows Extra +1 PRIMARY t0 ALL NULL NULL NULL NULL 2 +1 PRIMARY t4 ALL NULL NULL NULL NULL 2 Using where; Using join buffer (flat, BNL join) +2 SUBQUERY t3 system NULL NULL NULL NULL 1 +SELECT t0.k, t4.x FROM t0 +LEFT JOIN (t4 LEFT JOIN t2 ON t2.b = t4.a) +ON (t0.k = t4.x AND t4.a = (SELECT c FROM t3)); +k x +20 20 +9 NULL +truncate t2; +DROP TABLE t4; +CREATE TABLE t4 (d INT); +INSERT INTO t4 VALUES (1),(2),(7); +# The subquery is shared with the WHERE clause, so it must still be +# shown and executed: +explain SELECT t1.* FROM t1 +LEFT JOIN t2 ON (t2.b = t1.a) +LEFT JOIN t4 ON (t4.d = t1.a) +WHERE t1.a = (SELECT c FROM t3); +id select_type table type possible_keys key key_len ref rows Extra +1 PRIMARY t2 system PRIMARY NULL NULL NULL 0 Const row not found +1 PRIMARY t1 ALL NULL NULL NULL NULL 2 Using where +1 PRIMARY t4 ALL NULL NULL NULL NULL 3 Using where; Using join buffer (flat, BNL join) +2 SUBQUERY t3 system NULL NULL NULL NULL 1 +SELECT t1.* FROM t1 +LEFT JOIN t2 ON (t2.b = t1.a) +LEFT JOIN t4 ON (t4.d = t1.a) +WHERE t1.a = (SELECT c FROM t3); +a +2 +explain SELECT t1.a FROM t1 +LEFT JOIN t2 ON (t2.b = t1.a AND t1.a <> (SELECT c FROM t3)) +LEFT JOIN t4 ON (t4.d = t1.a); +id select_type table type possible_keys key key_len ref rows Extra +1 PRIMARY t2 system PRIMARY NULL NULL NULL 0 Const row not found +1 PRIMARY t1 ALL NULL NULL NULL NULL 2 +1 PRIMARY t4 ALL NULL NULL NULL NULL 3 Using where; Using join buffer (flat, BNL join) +2 SUBQUERY t3 system NULL NULL NULL NULL 1 +SELECT t1.a FROM t1 +LEFT JOIN t2 ON (t2.b = t1.a AND t1.a <> (SELECT c FROM t3)) +LEFT JOIN t4 ON (t4.d = t1.a); +a +1 +2 +DROP TABLE t0, t1, t2, t3, t4; +# # End of 10.11 tests # diff --git a/mysql-test/main/table_elim.test b/mysql-test/main/table_elim.test index 4158d2ca5ac8c..43b921d8c53f8 100644 --- a/mysql-test/main/table_elim.test +++ b/mysql-test/main/table_elim.test @@ -841,6 +841,63 @@ select t1.null_col from t1 left join t2 on (t2.unique_col<=>t1.notnull_col); DROP TABLE t1, t2; +--echo # +--echo # MDEV-36610: Subquery wrongly marked as eliminated by table elimination +--echo # when equality propagation shares a WHERE-clause subquery +--echo # into an eliminated outer join's ON expression +--echo # +CREATE TABLE t0 (k INT); +INSERT INTO t0 VALUES (20),(9); +CREATE TABLE t1 (a BIGINT); +INSERT INTO t1 VALUES (1),(2); +CREATE TABLE t2 (b INT PRIMARY KEY); +INSERT INTO t2 VALUES (3),(4); +INSERT INTO t2 VALUES (5),(6); +CREATE TABLE t3 (c INT); +INSERT INTO t3 VALUES (2); +CREATE TABLE t4 (a BIGINT, x INT); +INSERT INTO t4 VALUES (2,20),(3,30); + +--echo # t2 is eliminated, but the subquery must still be shown and executed: + +let $q= +SELECT t1.* FROM t1 LEFT JOIN t2 ON (t2.b = t1.a) + WHERE t1.a = (SELECT c FROM t3); +eval explain $q; +eval $q; + +--echo # t2 is eliminated; the subquery must still be shown and executed: +let $q= +SELECT t0.k, t4.x FROM t0 +LEFT JOIN (t4 LEFT JOIN t2 ON t2.b = t4.a) +ON (t0.k = t4.x AND t4.a = (SELECT c FROM t3)); +eval explain $q; +eval $q; + +truncate t2; +DROP TABLE t4; +CREATE TABLE t4 (d INT); +INSERT INTO t4 VALUES (1),(2),(7); +--echo # The subquery is shared with the WHERE clause, so it must still be +--echo # shown and executed: +let $q= +SELECT t1.* FROM t1 + LEFT JOIN t2 ON (t2.b = t1.a) + LEFT JOIN t4 ON (t4.d = t1.a) + WHERE t1.a = (SELECT c FROM t3); +eval explain $q; +eval $q; + +let $q= +SELECT t1.a FROM t1 + LEFT JOIN t2 ON (t2.b = t1.a AND t1.a <> (SELECT c FROM t3)) + LEFT JOIN t4 ON (t4.d = t1.a); +eval explain $q; +eval $q; + + +DROP TABLE t0, t1, t2, t3, t4; + --echo # --echo # End of 10.11 tests --echo # diff --git a/mysql-test/main/view.result b/mysql-test/main/view.result index 09b4e12cb2185..24111441ee571 100644 --- a/mysql-test/main/view.result +++ b/mysql-test/main/view.result @@ -1914,8 +1914,6 @@ test.v5 check error Corrupt test.v6 check status OK drop view v1, v2, v3, v4, v5, v6; drop table t2; -drop function if exists f1; -drop function if exists f2; CREATE TABLE t1 (col1 time); CREATE TABLE t2 (col1 time); CREATE TABLE t3 (col1 time); @@ -2152,8 +2150,6 @@ CREATE VIEW v1 AS SELECT f1(); ERROR HY000: View's SELECT refers to a temporary table 't1' DROP FUNCTION f1; DROP TABLE t1; -DROP TABLE IF EXISTS t1; -DROP VIEW IF EXISTS v1; CREATE TABLE t1 (f4 CHAR(5)); CREATE VIEW v1 AS SELECT * FROM t1; DESCRIBE v1; @@ -2443,7 +2439,6 @@ f1 sum(f2) NULL 12 drop view v1; drop table t1; -drop procedure if exists p1; create procedure p1 () deterministic begin create view v1 as select 1; @@ -2456,7 +2451,6 @@ v1 CREATE ALGORITHM=UNDEFINED DEFINER=`root`@`localhost` SQL SECURITY DEFINER VI drop view v1; drop procedure p1; CREATE VIEW v1 AS SELECT 42 AS Meaning; -DROP FUNCTION IF EXISTS f1; CREATE FUNCTION f1() RETURNS INTEGER BEGIN DECLARE retn INTEGER; @@ -2874,7 +2868,6 @@ SHOW TABLES; Tables_in_test t1 DROP TABLE t1; -DROP VIEW IF EXISTS v1; set GLOBAL sql_mode=""; set LOCAL sql_mode=""; CREATE DATABASE bug21261DB; @@ -2915,9 +2908,6 @@ View Create View character_set_client collation_connection v1 CREATE ALGORITHM=UNDEFINED DEFINER=`root`@`localhost` SQL SECURITY DEFINER VIEW `v1` AS select `t1`.`f1` AS `f1` from `t1` where `t1`.`f1` between current_timestamp() and current_timestamp() + interval 1 minute latin1 latin1_swedish_ci drop view v1; drop table t1; -DROP TABLE IF EXISTS t1; -DROP VIEW IF EXISTS v1; -DROP VIEW IF EXISTS v2; CREATE TABLE t1(a INT, b INT); CREATE DEFINER=longer_than_80_456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789@localhost VIEW v1 AS SELECT a FROM t1; @@ -2926,10 +2916,6 @@ CREATE DEFINER=some_user_name@1234567890abcdefghij1234567890abcdefghij1234567890 VIEW v2 AS SELECT b FROM t1; ERROR HY000: String '1234567890abcdefghij1234567890abcdefghij1234567890abcdefghijQWERTYa...' is too long for host name (should be no longer than 255) DROP TABLE t1; -DROP FUNCTION IF EXISTS f1; -DROP FUNCTION IF EXISTS f2; -DROP VIEW IF EXISTS v1, v2; -DROP TABLE IF EXISTS t1; CREATE TABLE t1 (i INT); CREATE VIEW v1 AS SELECT * FROM t1; CREATE FUNCTION f1() RETURNS INT @@ -2992,9 +2978,6 @@ View Create View character_set_client collation_connection v1 CREATE ALGORITHM=MERGE DEFINER=`root`@`localhost` SQL SECURITY DEFINER VIEW `v1` AS select `t1`.`pk` AS `pk` from (`t1` join `t2` on(`t2`.`fk` = `t1`.`pk` and `t2`.`ver` = (select max(`t`.`ver`) from `t2` `t` where `t`.`org` = `t2`.`org`))) latin1 latin1_swedish_ci DROP VIEW v1; DROP TABLE t1, t2; -DROP FUNCTION IF EXISTS f1; -DROP VIEW IF EXISTS v1; -DROP TABLE IF EXISTS t1; CREATE TABLE t1 (i INT); INSERT INTO t1 VALUES (1); CREATE VIEW v1 AS SELECT MAX(i) FROM t1; @@ -3016,8 +2999,6 @@ UPDATE v1 SET val=6 WHERE id=2; ERROR 44000: CHECK OPTION failed `test`.`v1` DROP VIEW v1; DROP TABLE t1; -DROP VIEW IF EXISTS v1, v2; -DROP TABLE IF EXISTS t1; CREATE TABLE t1 (i INT AUTO_INCREMENT PRIMARY KEY, j INT); CREATE VIEW v1 AS SELECT j FROM t1; CREATE VIEW v2 AS SELECT * FROM t1; @@ -3061,7 +3042,6 @@ SELECT * FROM v; x 5 DROP VIEW v; -DROP VIEW IF EXISTS v1; CREATE VIEW v1 AS SELECT 'The\ZEnd'; SELECT * FROM v1; TheEnd @@ -3172,15 +3152,11 @@ code COUNT(DISTINCT country) 100 2 DROP VIEW v1; DROP TABLE t1; -DROP VIEW IF EXISTS v1; SELECT * FROM (SELECT 1) AS t into @w; CREATE VIEW v1 AS SELECT * FROM (SELECT 1) AS t into @w; ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'into @w' at line 1 # Previously the following would fail. SELECT * FROM (SELECT 1) AS t into @w; -drop view if exists view_24532_a; -drop view if exists view_24532_b; -drop table if exists table_24532; create table table_24532 ( a int, b bigint, @@ -3644,38 +3620,24 @@ DROP TABLE t1; # ----------------------------------------------------------------- # -- Bug#34337 Server crash when Altering a view using a table name. # ----------------------------------------------------------------- - -DROP TABLE IF EXISTS t1; - CREATE TABLE t1(c1 INT); - SELECT * FROM t1; c1 ALTER ALGORITHM=TEMPTABLE SQL SECURITY INVOKER VIEW t1 (c2) AS SELECT (1); ERROR HY000: 'test.t1' is not of type 'VIEW' - DROP TABLE t1; - -# -- End of test case for Bug#34337. - # ----------------------------------------------------------------- # -- Bug#35193 VIEW query is rewritten without "FROM DUAL", # -- causing syntax error # ----------------------------------------------------------------- - CREATE VIEW v1 AS SELECT 1 FROM DUAL WHERE 1; - SELECT * FROM v1; 1 1 SHOW CREATE TABLE v1; View Create View character_set_client collation_connection v1 CREATE ALGORITHM=UNDEFINED DEFINER=`root`@`localhost` SQL SECURITY DEFINER VIEW `v1` AS select 1 AS `1` from DUAL where 1 latin1 latin1_swedish_ci - DROP VIEW v1; - -# -- End of test case for Bug#35193. - CREATE VIEW v1 AS SELECT 1; DROP VIEW v1; CREATE TABLE t1 (c1 INT PRIMARY KEY, c2 INT, INDEX (c2)); @@ -3836,7 +3798,6 @@ DROP TABLE t1; # ----------------------------------------------------------------- # -- End of 5.0 tests. # ----------------------------------------------------------------- -DROP DATABASE IF EXISTS `d-1`; CREATE DATABASE `d-1`; USE `d-1`; CREATE TABLE `t-1` (c1 INT); @@ -3855,93 +3816,61 @@ DROP TABLE `t-2`; DROP VIEW `v-2`; DROP DATABASE `d-1`; USE test; - # # Bug#26676 VIEW using old table schema in a session. # - -DROP VIEW IF EXISTS v1; -DROP TABLE IF EXISTS t1; CREATE TABLE t1(c1 INT, c2 INT); INSERT INTO t1 VALUES (1, 2), (3, 4); - SELECT * FROM t1; c1 c2 1 2 3 4 - CREATE VIEW v1 AS SELECT * FROM t1; - SELECT * FROM v1; c1 c2 1 2 3 4 - ALTER TABLE t1 ADD COLUMN c3 INT AFTER c2; - SELECT * FROM t1; c1 c2 c3 1 2 NULL 3 4 NULL - SELECT * FROM v1; c1 c2 1 2 3 4 - SHOW CREATE VIEW v1; View Create View character_set_client collation_connection v1 CREATE ALGORITHM=UNDEFINED DEFINER=`root`@`localhost` SQL SECURITY DEFINER VIEW `v1` AS select `t1`.`c1` AS `c1`,`t1`.`c2` AS `c2` from `t1` latin1 latin1_swedish_ci - DROP VIEW v1; DROP TABLE t1; - -# End of test case for Bug#26676. - # ----------------------------------------------------------------- # -- Bug#32538 View definition picks up character set, but not collation # ----------------------------------------------------------------- - -DROP VIEW IF EXISTS v1; - SET collation_connection = latin1_general_ci; CREATE VIEW v1 AS SELECT _latin1 'text1' AS c1, 'text2' AS c2; - SELECT COLLATION(c1), COLLATION(c2) FROM v1; COLLATION(c1) COLLATION(c2) latin1_swedish_ci latin1_general_ci - SHOW CREATE VIEW v1; View Create View character_set_client collation_connection v1 CREATE ALGORITHM=UNDEFINED DEFINER=`root`@`localhost` SQL SECURITY DEFINER VIEW `v1` AS select _latin1'text1' AS `c1`,'text2' AS `c2` latin1 latin1_general_ci - SELECT * FROM v1 WHERE c1 = 'text1'; ERROR HY000: Illegal mix of collations (latin1_swedish_ci,COERCIBLE) and (latin1_general_ci,COERCIBLE) for operation '=' - SELECT * FROM v1 WHERE c2 = 'text2'; c1 c2 text1 text2 - use test; SET names latin1; - SELECT COLLATION(c1), COLLATION(c2) FROM v1; COLLATION(c1) COLLATION(c2) latin1_swedish_ci latin1_general_ci - SELECT * FROM v1 WHERE c1 = 'text1'; c1 c2 text1 text2 - SELECT * FROM v1 WHERE c2 = 'text2'; ERROR HY000: Illegal mix of collations (latin1_general_ci,COERCIBLE) and (latin1_swedish_ci,COERCIBLE) for operation '=' - DROP VIEW v1; - -# -- End of test case for Bug#32538. - -drop view if exists a; -drop procedure if exists p; create procedure p() begin declare continue handler for sqlexception begin end; @@ -3964,9 +3893,6 @@ DROP TABLE t1; # Bug#48449: hang on show create view after upgrading when # view contains function of view # -DROP VIEW IF EXISTS v1,v2; -DROP TABLE IF EXISTS t1,t2; -DROP FUNCTION IF EXISTS f1; CREATE TABLE t1 (a INT); CREATE TABLE t2 (a INT); CREATE FUNCTION f1() RETURNS INT @@ -4730,8 +4656,6 @@ c a b c 0 93 1 0 DROP VIEW v4; DROP TABLE t1,t2,t3,t4; -drop table if exists t_9801; -drop view if exists v_9801; create table t_9801 (s1 int); create view v_9801 as select sum(s1) from t_9801 with check option; @@ -4746,8 +4670,6 @@ drop table t_9801; # # Bug #47335 assert in get_table_share # -DROP TABLE IF EXISTS t1; -DROP VIEW IF EXISTS v1; CREATE TEMPORARY TABLE t1 (id INT); ALTER VIEW t1 AS SELECT 1 AS f1; ERROR 42S02: Table 'test.t1' doesn't exist @@ -4764,8 +4686,6 @@ DROP VIEW v1; # Bug #47635 assert in start_waiting_global_read_lock # during CREATE VIEW # -DROP TABLE IF EXISTS t1, t2; -DROP VIEW IF EXISTS t2; CREATE TABLE t1 (f1 integer); CREATE TEMPORARY TABLE IF NOT EXISTS t1 (f1 integer); CREATE TEMPORARY TABLE t2 (f1 integer); @@ -4779,8 +4699,6 @@ DROP TABLE t1, t2; # Bug#48315 Metadata lock is not taken for merged views that # use an INFORMATION_SCHEMA table # -DROP VIEW IF EXISTS v1; -DROP PROCEDURE IF EXISTS p1; connect con2, localhost, root; connect con3, localhost, root; connection default; @@ -4823,7 +4741,6 @@ disconnect con3; # Bug#12626844: WRONG ERROR MESSAGE WHILE CREATING A VIEW ON A # NON EXISTING DATABASE # -DROP DATABASE IF EXISTS nodb; CREATE VIEW nodb.a AS SELECT 1; ERROR 42000: Unknown database 'nodb' # @@ -6948,10 +6865,8 @@ SELECT * FROM v1; b'' DROP VIEW v1; -# # End of 10.3 tests # -# # MDEV-25206: view specification contains unknown column reference # CREATE TABLE t1 (a int); diff --git a/mysql-test/main/view.test b/mysql-test/main/view.test index 2aed3722bf711..50dbf1a862c04 100644 --- a/mysql-test/main/view.test +++ b/mysql-test/main/view.test @@ -70,7 +70,6 @@ create view v3 (c) as select a+1 from v1; -- error ER_BAD_FIELD_ERROR create view v3 (c) as select b+1 from v1; - # VIEW on VIEW test with mixing different algorithms on different order create view v3 (c) as select c+1 from v1; select c from v3; @@ -159,7 +158,6 @@ select * from v2 natural left join v1; drop view v1, v2; drop table t1; - # # DISTINCT option for VIEW # @@ -758,7 +756,6 @@ show create view v1; drop view v1; drop table t1; - # # VIEW with floating point (long number) as column # @@ -1680,10 +1677,6 @@ CHECK TABLE v1, v2, v3, v4, v5, v6; drop view v1, v2, v3, v4, v5, v6; drop table t2; ---disable_warnings -drop function if exists f1; -drop function if exists f2; ---enable_warnings CREATE TABLE t1 (col1 time); CREATE TABLE t2 (col1 time); CREATE TABLE t3 (col1 time); @@ -1737,7 +1730,6 @@ show create view v1; drop view v1; drop table t1; - # # Correct inserting data check (absence of default value) for view # underlying tables (Bug#6443) @@ -1756,7 +1748,6 @@ set sql_mode=default; drop view v2,v1; drop table t1; - # # Bug#11399 Use an alias in a select statement on a view # @@ -1767,7 +1758,6 @@ select f1 as alias from v1; drop view v1; drop table t1; - # # Test for Bug#6120 SP cache to be invalidated when altering a view # @@ -1788,7 +1778,6 @@ DROP PROCEDURE p1; DROP VIEW v1; DROP TABLE t1; - # # Test for Bug#11709 View was ordered by wrong column # @@ -1799,7 +1788,6 @@ select * from v1 order by f1; drop view v1; drop table t1; - # # Test for Bug#11771 wrong query_id in SELECT * FROM # @@ -1814,7 +1802,6 @@ SELECT * FROM t1; DROP VIEW v1; DROP TABLE t1; - # # opening table in correct locking mode (Bug#9597) # @@ -1833,7 +1820,6 @@ DROP PROCEDURE p1; DROP VIEW v1; DROP TABLE t1; - # # Bug#11760 Typo in Item_func_add_time::print() results in NULLs returned # subtime() in view @@ -1844,7 +1830,6 @@ select * from v1; drop view v1; drop table t1; - # # Test for Bug#11412 query over a multitable view with GROUP_CONCAT # @@ -1869,7 +1854,6 @@ SELECT pid,GROUP_CONCAT(CONCAT(fn,' ',ln) ORDER BY 1) FROM v1 GROUP BY pid; DROP VIEW v1; DROP TABLE t1,t2; - # # Test for Bug#12382 SELECT * FROM view after INSERT command # @@ -1887,7 +1871,6 @@ SELECT * FROM v1; DROP VIEW v1; DROP TABLE t1; - # # Test for Bug#12470 crash for a simple select from a view defined # as a join over 5 tables @@ -1907,7 +1890,6 @@ SELECT a FROM v1; DROP VIEW v1; DROP TABLE t1,t2,t3,t4,t5; - # # Bug#12298 Typo in function name results in erroneous view being created. # @@ -1927,7 +1909,6 @@ call p1(); call p1(); drop procedure p1; - # # Bug#10624 Views with multiple UNION and UNION ALL produce incorrect results # @@ -1940,7 +1921,6 @@ select * from v1; drop view v1; drop table t1,t2; - # # Test for Bug#10970 view referring a temporary table indirectly # @@ -1953,14 +1933,9 @@ CREATE VIEW v1 AS SELECT f1(); DROP FUNCTION f1; DROP TABLE t1; - # # Bug#12533 (crash on DESCRIBE after renaming base table column) # ---disable_warnings -DROP TABLE IF EXISTS t1; -DROP VIEW IF EXISTS v1; ---enable_warnings CREATE TABLE t1 (f4 CHAR(5)); CREATE VIEW v1 AS SELECT * FROM t1; @@ -1972,7 +1947,6 @@ DESCRIBE v1; DROP TABLE t1; DROP VIEW v1; - # # Bug#12489 wrongly printed strcmp() function results in creation of broken # view @@ -1982,7 +1956,6 @@ select * from v1; drop view v1; drop table t1; - # # Bug#12922 if(sum(),...) with group from view returns wrong results # @@ -1993,7 +1966,6 @@ select if(sum(f1)>1,f2,f3) from v1 group by f1; drop view v1; drop table t1; - # Bug#12941 # create table t1 ( @@ -2026,7 +1998,6 @@ order by users_names; drop view v1, v2; drop table t1, t2; - # # Bug#6808 Views: CREATE VIEW v ... FROM t AS v fails # @@ -2076,7 +2047,6 @@ create view v1 as SELECT CONVERT_TZ('2004-01-01 12:00:00','GMT','MET'); select * from v1; drop view v1; - # # Bugs#12963, #13000 wrong creation of VIEW with DAYNAME, DAYOFWEEK, and WEEKDAY # @@ -2108,7 +2078,6 @@ SELECT * FROM v3; DROP TABLE t1; DROP VIEW v1, v2, v3; - # # Bug#13411 crash when using non-qualified view column in HAVING clause # @@ -2122,7 +2091,6 @@ SELECT v1.a FROM v1 GROUP BY v1.a HAVING a > 1; DROP VIEW v1; DROP TABLE t1; - # # Bug#13410 failed name resolution for qualified view column in HAVING # @@ -2138,7 +2106,6 @@ SELECT v_1.a FROM v1 AS v_1 GROUP BY v_1.a HAVING v_1.a IN (1,2,3); DROP VIEW v1; DROP TABLE t1; - # # Bug#13327 view wasn't using index for const condition # @@ -2157,7 +2124,6 @@ EXPLAIN SELECT * FROM v2 WHERE a=1; DROP VIEW v1,v2; DROP TABLE t1,t2,t3; - # # Bug#13622 Wrong view .frm created if some field's alias contain \n # @@ -2168,7 +2134,6 @@ select * from v1; drop view v1; drop table t1; - # Bug#14466 lost sort order in GROUP_CONCAT() in a view # create table t1 (f1 int, f2 int); @@ -2180,7 +2145,6 @@ select * from v2; drop view v1,v2; drop table t1; - # # Bug#14026 Crash on second PS execution when using views # @@ -2205,7 +2169,6 @@ execute stmt1 using @parm1; drop view v1; drop table t1,t2,t3,t4; - # # Bug#14540 OPTIMIZE, ANALYZE, REPAIR applied to not a view # @@ -2224,7 +2187,6 @@ REPAIR TABLE v1; DROP VIEW v1; - # # Bug#14719 Views DEFINER grammar is incorrect # @@ -2237,7 +2199,6 @@ create definer = current_user sql security invoker view v1 as select 1; show create view v1; drop view v1; - # # Bug#14816 test_if_order_by_key() expected only Item_fields. # @@ -2248,7 +2209,6 @@ explain select id from v1 order by id; drop view v1; drop table t1; - # # Bug#14850 Item_ref's values wasn't updated # @@ -2260,14 +2220,10 @@ select f1, sum(f2) from v1 group by f1; drop view v1; drop table t1; - # # Bug#14885 incorrect SOURCE in view created in a procedure # TODO: here SOURCE string must be shown when it will be possible # ---disable_warnings -drop procedure if exists p1; ---enable_warnings delimiter //; create procedure p1 () deterministic begin @@ -2280,14 +2236,10 @@ show create view v1; drop view v1; drop procedure p1; - # # Bug#15096 using function with view for view creation # CREATE VIEW v1 AS SELECT 42 AS Meaning; ---disable_warnings -DROP FUNCTION IF EXISTS f1; ---enable_warnings DELIMITER //; --enable_prepare_warnings CREATE FUNCTION f1() RETURNS INTEGER @@ -2304,7 +2256,6 @@ select * from v2; drop view v2,v1; drop function f1; - # # Bug#14861 aliased column names are not preserved. # @@ -2324,7 +2275,6 @@ order by v2.receipt_id; drop view v2, v1; drop table t1; - # # Bug#16016 MIN/MAX optimization for views # @@ -2349,7 +2299,6 @@ EXPLAIN SELECT MIN(a) FROM v1; DROP VIEW v1; DROP TABLE t1; - # # Bug#16382 grouping name is resolved against a view column name # which coincides with a select column name @@ -2367,7 +2316,6 @@ SELECT IF(x IS NULL, 'blank', 'not blank') AS x FROM v1 GROUP BY x; DROP VIEW v1; DROP TABLE t1; - # # Bug#15943 mysql_next_result hangs on invalid SHOW CREATE VIEW # @@ -2385,7 +2333,6 @@ drop view v1; delimiter ;// --enable_ps_protocol - # # Bug#17726 Not checked empty list caused endless loop # @@ -2400,7 +2347,6 @@ select * from v2; drop view v2, v1; drop table t1; - # # Bug#18386 select from view over a table with ORDER BY view_col clause # given view_col is not an image of any column from the base table @@ -2415,7 +2361,6 @@ SELECT my_sqrt FROM v1 ORDER BY my_sqrt; DROP VIEW v1; DROP TABLE t1; - # # Bug#18237 invalid count optimization applied to an outer join with a view # @@ -2437,7 +2382,6 @@ DROP VIEW v2; DROP TABLE t1, t2; - # # Bug#16069 VIEW does return the same results as underlying SELECT # with WHERE condition containing BETWEEN over dates @@ -2459,7 +2403,6 @@ SELECT * FROM v1 WHERE td BETWEEN CAST('2005.01.02' AS DATE) AND CAST('2005.01.0 DROP VIEW v1; DROP TABLE t1; - # # Bug#14308 Recursive view definitions # @@ -2493,7 +2436,6 @@ select * from v1; drop function f1; drop view t1, v1; - # # Bug#15153 CONVERT_TZ() is not allowed in all places in VIEWs # @@ -2521,7 +2463,6 @@ select * from v2; drop view v1, v2; drop table t1; - # # Bug#19490 usage of view specified by a query with GROUP BY # an expression containing non-constant interval @@ -2538,7 +2479,6 @@ SELECT * FROM v1; DROP VIEW v1; DROP TABLE t1; - # # Bug#19077 A nested materialized view is used before being populated. # @@ -2550,7 +2490,6 @@ SELECT * FROM v2; DROP VIEW v2, v1; DROP TABLE t1; - # # Bug#19573 VIEW with HAVING that refers an alias name # @@ -2578,7 +2517,6 @@ SELECT * FROM v1; DROP VIEW v1; DROP TABLE t1; - # # Bug#19089 wrong inherited dafault values in temp table views # @@ -2610,7 +2548,6 @@ SELECT * FROM t2; DROP VIEW v1; DROP TABLE t1,t2; - # # Bug#16110 insert permitted into view col w/o default value # @@ -2629,7 +2566,6 @@ SELECT * FROM t1; DROP VIEW v1; DROP TABLE t1; - # # Bug#18243 expression over a view column that with the REVERSE function # @@ -2646,7 +2582,6 @@ SELECT CONCAT(LEFT(name,LENGTH(name)-INSTR(REVERSE(name)," ")), DROP VIEW v1; DROP TABLE t1; - # # Bug#19714 wrong type of a view column specified by an expressions over ints # @@ -2660,7 +2595,6 @@ DESCRIBE t2; DROP VIEW v1; DROP TABLE t1,t2; - # # Bug#17526 views with TRIM functions # @@ -2685,7 +2619,6 @@ DROP VIEW v1; DROP TABLE t1; - # # Bug#21080 ALTER VIEW makes user restate SQL SECURITY mode, and ALGORITHM # @@ -2699,7 +2632,6 @@ SHOW CREATE VIEW v1; DROP VIEW v1; DROP TABLE t1; - # Bug#21086 server crashes when VIEW defined with a SELECT with COLLATE # clause is called # @@ -2718,7 +2650,6 @@ SELECT s1 FROM t1; DROP VIEW v1, v2; DROP TABLE t1; - # # Bug#11551 Asymmetric + undocumented behaviour of DROP VIEW and DROP TABLE # @@ -2737,10 +2668,6 @@ show warnings; SHOW TABLES; DROP TABLE t1; ---disable_warnings -DROP VIEW IF EXISTS v1; ---enable_warnings - # # Bug#21261 Wrong access rights was required for an insert to a view @@ -2789,19 +2716,12 @@ show create view v1; drop view v1; drop table t1; - # # Test for Bug#16899 Possible buffer overflow in handling of DEFINER-clause. # # Prepare. ---disable_warnings -DROP TABLE IF EXISTS t1; -DROP VIEW IF EXISTS v1; -DROP VIEW IF EXISTS v2; ---enable_warnings - CREATE TABLE t1(a INT, b INT); --error ER_WRONG_STRING_LENGTH @@ -2816,19 +2736,12 @@ CREATE DEFINER=some_user_name@1234567890abcdefghij1234567890abcdefghij1234567890 DROP TABLE t1; - # # Bug#17591 Updatable view not possible with trigger or stored function # # During prelocking phase we didn't update lock type of view tables, # hence READ lock was always requested. # ---disable_warnings -DROP FUNCTION IF EXISTS f1; -DROP FUNCTION IF EXISTS f2; -DROP VIEW IF EXISTS v1, v2; -DROP TABLE IF EXISTS t1; ---enable_warnings CREATE TABLE t1 (i INT); @@ -2862,7 +2775,6 @@ DROP FUNCTION f2; DROP VIEW v1, v2; DROP TABLE t1; - # # Bug#5500 wrong select_type in EXPLAIN output for queries over views # @@ -2881,7 +2793,6 @@ EXPLAIN SELECT * FROM v1 t WHERE t.s1+1 < (SELECT MAX(t1.s1) FROM t1); DROP VIEW v1; DROP TABLE t1; - # # Bug#5505 Wrong error message on INSERT into a view # @@ -2893,7 +2804,6 @@ update v1 set a = 5; drop view v1; drop table t1; - # # Bug#21646 view qith a subquery in ON expression # @@ -2912,19 +2822,12 @@ SHOW CREATE VIEW v1; DROP VIEW v1; DROP TABLE t1, t2; - # # Bug#19111 TRIGGERs selecting from a VIEW on the firing base table fail # # Allow to select from a view on a table being modified in a trigger # and stored function, since plain select is allowed there. # ---disable_warnings -DROP FUNCTION IF EXISTS f1; -DROP VIEW IF EXISTS v1; -DROP TABLE IF EXISTS t1; ---enable_warnings - CREATE TABLE t1 (i INT); INSERT INTO t1 VALUES (1); @@ -2945,7 +2848,6 @@ DROP FUNCTION f1; DROP VIEW v1; DROP TABLE t1; - # # Bug#16813 (WITH CHECK OPTION doesn't work with UPDATE) # @@ -2960,7 +2862,6 @@ UPDATE v1 SET val=6 WHERE id=2; DROP VIEW v1; DROP TABLE t1; - # # Bug#22584 last_insert_id not updated after inserting a record # through a updatable view @@ -2969,11 +2870,6 @@ DROP TABLE t1; # not accessible through a view. However, we do not reset the value # of LAST_INSERT_ID, but keep it unchanged. # ---disable_warnings -DROP VIEW IF EXISTS v1, v2; -DROP TABLE IF EXISTS t1; ---enable_warnings - CREATE TABLE t1 (i INT AUTO_INCREMENT PRIMARY KEY, j INT); CREATE VIEW v1 AS SELECT j FROM t1; CREATE VIEW v2 AS SELECT * FROM t1; @@ -2998,7 +2894,6 @@ SELECT * FROM t1; DROP VIEW v1, v2; DROP TABLE t1; - # # Bug#25580 !0 as an operand in a select expression of a view # @@ -3011,15 +2906,9 @@ SELECT * FROM v; DROP VIEW v; - # # Bug#24293 '\Z' token is not handled correctly in views # - ---disable_warnings -DROP VIEW IF EXISTS v1; ---enable_warnings - CREATE VIEW v1 AS SELECT 'The\ZEnd'; SELECT * FROM v1; @@ -3027,7 +2916,6 @@ SHOW CREATE VIEW v1; DROP VIEW v1; - # # Bug#26124 BETWEEN over a view column of the DATETIME type # @@ -3044,7 +2932,6 @@ SELECT * FROM v1 WHERE mydate BETWEEN '2007-01-01' AND '2007-01-31'; DROP VIEW v1; DROP TABLE t1; - # # Bug#25931 update of a multi-table view with check option # @@ -3067,7 +2954,6 @@ SELECT * FROM t2; DROP VIEW v1; DROP TABLE t1,t2; - # # Bug#12122 Views with ORDER BY can't be resolved using MERGE algorithm. # @@ -3104,14 +2990,9 @@ SELECT code, COUNT(DISTINCT country) FROM v1 GROUP BY code ORDER BY MAX(id); DROP VIEW v1; DROP TABLE t1; - # # Bug#25897 Some queries are no longer possible after a CREATE VIEW fails # ---disable_warnings -DROP VIEW IF EXISTS v1; ---enable_warnings - let $query = SELECT * FROM (SELECT 1) AS t into @w; --disable_cursor_protocol @@ -3127,13 +3008,6 @@ eval $query; # # Bug#24532 The return data type of IS TRUE is different from similar operations # - ---disable_warnings -drop view if exists view_24532_a; -drop view if exists view_24532_b; -drop table if exists table_24532; ---enable_warnings - create table table_24532 ( a int, b bigint, @@ -3210,7 +3084,6 @@ drop view view_24532_a; drop view view_24532_b; drop table table_24532; - # # Bug#26560 view using subquery with a reference to an outer alias # @@ -3251,7 +3124,6 @@ SELECT * FROM v1; DROP VIEW v1; DROP table t1,t2; - # # Bug#27786 Inconsistent Operation Performing UNION On View With ORDER BY # @@ -3268,7 +3140,6 @@ EXPLAIN SELECT * FROM t1 UNION SELECT * FROM v1 ORDER BY a; DROP VIEW v1; DROP TABLE t1; - # # Bug#27921 View ignores precision for CAST() # @@ -3281,7 +3152,6 @@ CREATE VIEW v1 AS SELECT CAST(1.23456789 AS DECIMAL(8,0)) AS col; SHOW CREATE VIEW v1; DROP VIEW v1; - # # Bug#28716 CHECK OPTION expression is evaluated over expired record buffers # when VIEW is updated via temporary tables @@ -3298,7 +3168,6 @@ SELECT * FROM v1; DROP VIEW v1; DROP TABLE t1,t2; - # # Bug#28561 update on multi-table view with CHECK OPTION and a subquery # in WHERE condition @@ -3318,7 +3187,6 @@ UPDATE v1 SET c=1; DROP VIEW v1; DROP TABLE t1,t2; - # # Bug#27827 CHECK OPTION ignores ON conditions when updating # a multi-table view with CHECK OPTION. @@ -3371,7 +3239,6 @@ SELECT * FROM t1; DROP VIEW v1,v2; DROP TABLE t1,t2,t3,t4; - # # Bug#29104 assertion abort for a query with a view column reference # in the GROUP BY list and a condition requiring the value @@ -3383,7 +3250,6 @@ INSERT INTO t1 VALUES (1,2), (2,2), (1,3), (1,2); CREATE VIEW v1 AS SELECT a, b+1 as b FROM t1; - SELECT b, SUM(a) FROM v1 WHERE b=3 GROUP BY b; EXPLAIN SELECT b, SUM(a) FROM v1 WHERE b=3 GROUP BY b; @@ -3396,7 +3262,6 @@ EXPLAIN SELECT a, SUM(b) FROM v1 WHERE a=1 GROUP BY a; DROP VIEW v1; DROP TABLE t1; - # # Bug#29392 SELECT over a multi-table view with ORDER BY # selecting the same view column with two different aliases @@ -3450,7 +3315,6 @@ SELECT t.person_id AS a, t.person_id AS b FROM v1 t WHERE t.person_id=6; DROP VIEW v1; DROP TABLE t1,t2,t3; - # # Bug#30020 Insufficient check led to a wrong info provided by the # information schema table. @@ -3464,7 +3328,6 @@ select table_name, is_updatable from information_schema.views drop view v1; drop table t1; - # # Bug#28701 SELECTs from VIEWs completely ignore USE/FORCE KEY, allowing # invalid statements @@ -3483,7 +3346,6 @@ SELECT * FROM v1 IGNORE KEY(non_existant); DROP VIEW v1; DROP TABLE t1; - # # Bug#28702 VIEWs defined with USE/FORCE KEY ignore that request # @@ -3505,7 +3367,6 @@ DROP VIEW v2; DROP VIEW v3; DROP TABLE t1; - --echo # --echo # Bug#29477 Not all fields of the target table were checked to have --echo # a default value when inserting into a view. @@ -3521,7 +3382,6 @@ set @@sql_mode=@old_mode; drop view v1; drop table t1; - # # Bug#33389 Selecting from a view into a table from within SP or trigger # crashes server @@ -3543,7 +3403,6 @@ execute stmt; drop view v1, v2; drop table t1, t2; - # # Bug#33049 Assert while running test-as3ap test(mysql-bench suite) # @@ -3562,54 +3421,29 @@ DROP TABLE t1; --echo # ----------------------------------------------------------------- --echo # -- Bug#34337 Server crash when Altering a view using a table name. --echo # ----------------------------------------------------------------- ---echo - ---disable_warnings -DROP TABLE IF EXISTS t1; ---enable_warnings - ---echo CREATE TABLE t1(c1 INT); - ---echo - SELECT * FROM t1; --error ER_WRONG_OBJECT ALTER ALGORITHM=TEMPTABLE SQL SECURITY INVOKER VIEW t1 (c2) AS SELECT (1); ---echo - DROP TABLE t1; ---echo ---echo # -- End of test case for Bug#34337. ---echo - ########################################################################### --echo # ----------------------------------------------------------------- --echo # -- Bug#35193 VIEW query is rewritten without "FROM DUAL", --echo # -- causing syntax error --echo # ----------------------------------------------------------------- ---echo CREATE VIEW v1 AS SELECT 1 FROM DUAL WHERE 1; ---echo - SELECT * FROM v1; SHOW CREATE TABLE v1; ---echo - DROP VIEW v1; ---echo ---echo # -- End of test case for Bug#35193. ---echo - ########################################################################### # @@ -3620,7 +3454,6 @@ DROP VIEW v1; CREATE VIEW v1 AS SELECT 1; DROP VIEW v1; - # # Bug#33461 SELECT ... FROM USE INDEX (...) throws an error # @@ -3710,9 +3543,6 @@ DROP TABLE t1; # # Bug#21370 View renaming lacks tablename_to_filename encoding # ---disable_warnings -DROP DATABASE IF EXISTS `d-1`; ---enable_warnings CREATE DATABASE `d-1`; USE `d-1`; CREATE TABLE `t-1` (c1 INT); @@ -3726,125 +3556,64 @@ DROP VIEW `v-2`; DROP DATABASE `d-1`; USE test; ---echo --echo # --echo # Bug#26676 VIEW using old table schema in a session. --echo # ---echo - ---disable_warnings -DROP VIEW IF EXISTS v1; -DROP TABLE IF EXISTS t1; ---enable_warnings CREATE TABLE t1(c1 INT, c2 INT); INSERT INTO t1 VALUES (1, 2), (3, 4); ---echo - SELECT * FROM t1; ---echo - CREATE VIEW v1 AS SELECT * FROM t1; ---echo - SELECT * FROM v1; ---echo - ALTER TABLE t1 ADD COLUMN c3 INT AFTER c2; ---echo - SELECT * FROM t1; ---echo - SELECT * FROM v1; ---echo - SHOW CREATE VIEW v1; ---echo - DROP VIEW v1; DROP TABLE t1; ---echo ---echo # End of test case for Bug#26676. ---echo - ########################################################################### --echo # ----------------------------------------------------------------- --echo # -- Bug#32538 View definition picks up character set, but not collation --echo # ----------------------------------------------------------------- ---echo - ---disable_warnings -DROP VIEW IF EXISTS v1; ---enable_warnings - ---echo SET collation_connection = latin1_general_ci; CREATE VIEW v1 AS SELECT _latin1 'text1' AS c1, 'text2' AS c2; ---echo - SELECT COLLATION(c1), COLLATION(c2) FROM v1; ---echo - SHOW CREATE VIEW v1; ---echo - --error ER_CANT_AGGREGATE_2COLLATIONS SELECT * FROM v1 WHERE c1 = 'text1'; ---echo - SELECT * FROM v1 WHERE c2 = 'text2'; ---echo - use test; SET names latin1; ---echo - SELECT COLLATION(c1), COLLATION(c2) FROM v1; ---echo - SELECT * FROM v1 WHERE c1 = 'text1'; ---echo - --error ER_CANT_AGGREGATE_2COLLATIONS SELECT * FROM v1 WHERE c2 = 'text2'; ---echo - DROP VIEW v1; ---echo ---echo # -- End of test case for Bug#32538. ---echo - # # Bug#34587 Creating a view inside a stored procedure leads to a server crash # ---disable_warnings -drop view if exists a; -drop procedure if exists p; ---enable_warnings - delimiter |; create procedure p() begin @@ -3859,7 +3628,6 @@ drop procedure p; ########################################################################### - --echo # --echo # Bug #44860: ALTER TABLE on view crashes server --echo # @@ -3875,12 +3643,6 @@ DROP TABLE t1; --echo # view contains function of view --echo # ---disable_warnings -DROP VIEW IF EXISTS v1,v2; -DROP TABLE IF EXISTS t1,t2; -DROP FUNCTION IF EXISTS f1; ---enable_warnings - CREATE TABLE t1 (a INT); CREATE TABLE t2 (a INT); @@ -3908,7 +3670,6 @@ DROP VIEW v1,v2; DROP TABLE t1,t2; DROP FUNCTION f1; - # # Bug#48294 assertion when creating a view based on some row() construct in select query # @@ -4506,7 +4267,6 @@ SELECT * FROM t3 , t4 WHERE t4.c <= (SELECT t2.e FROM t2 LEFT JOIN t1 ON ( t1.a = t2.d ) WHERE t2.b > t4.b); - EXPLAIN EXTENDED SELECT * FROM t3, v4 WHERE v4.c <= (SELECT t2.e FROM t2 LEFT JOIN t1 ON ( t1.a = t2.d ) @@ -4523,11 +4283,6 @@ DROP TABLE t1,t2,t3,t4; # Bug#9801 (Views: imperfect error message) # ---disable_warnings -drop table if exists t_9801; -drop view if exists v_9801; ---enable_warnings - create table t_9801 (s1 int); --error ER_VIEW_NONUPD_CHECK @@ -4548,11 +4303,6 @@ drop table t_9801; --echo # Bug #47335 assert in get_table_share --echo # ---disable_warnings -DROP TABLE IF EXISTS t1; -DROP VIEW IF EXISTS v1; ---enable_warnings - CREATE TEMPORARY TABLE t1 (id INT); --error ER_NO_SUCH_TABLE ALTER VIEW t1 AS SELECT 1 AS f1; @@ -4565,17 +4315,11 @@ DROP TABLE v1; SELECT * FROM v1; DROP VIEW v1; - --echo # --echo # Bug #47635 assert in start_waiting_global_read_lock --echo # during CREATE VIEW --echo # ---disable_warnings -DROP TABLE IF EXISTS t1, t2; -DROP VIEW IF EXISTS t2; ---enable_warnings - CREATE TABLE t1 (f1 integer); CREATE TEMPORARY TABLE IF NOT EXISTS t1 (f1 integer); CREATE TEMPORARY TABLE t2 (f1 integer); @@ -4587,17 +4331,11 @@ CREATE VIEW t2 AS SELECT * FROM t1; UNLOCK TABLES; DROP TABLE t1, t2; - --echo # --echo # Bug#48315 Metadata lock is not taken for merged views that --echo # use an INFORMATION_SCHEMA table --echo # ---disable_warnings -DROP VIEW IF EXISTS v1; -DROP PROCEDURE IF EXISTS p1; ---enable_warnings - connect (con2, localhost, root); connect (con3, localhost, root); @@ -4662,19 +4400,14 @@ DROP PROCEDURE p1; disconnect con2; disconnect con3; - --echo # --echo # Bug#12626844: WRONG ERROR MESSAGE WHILE CREATING A VIEW ON A --echo # NON EXISTING DATABASE --echo # ---disable_warnings -DROP DATABASE IF EXISTS nodb; ---enable_warnings --error ER_BAD_DB_ERROR CREATE VIEW nodb.a AS SELECT 1; - --echo # --echo # BUG#14117018 - MYSQL SERVER CREATES INVALID VIEW DEFINITION --echo # BUG#18405221 - SHOW CREATE VIEW OUTPUT INCORRECT @@ -4715,7 +4448,6 @@ SHOW CREATE VIEW v4; DROP VIEW v1, v2, v3, v4, v5; - --echo # --echo # BUG#19886430: VIEW CREATION WITH NAMED COLUMNS, OVER UNION, --echo # IS REJECTED @@ -4953,7 +4685,6 @@ SELECT * FROM ( drop tables t1,t2; - --echo # --echo # MDEV-3876 Wrong result (extra rows) with ALL subquery --echo # from a MERGE view (duplicate of MDEV-3873) @@ -5062,7 +4793,6 @@ insert into t1 values (1),(2); create view v1 (a,r) as select a,rand() from t1; - create table t2 select a, r as r1, r as r2, r as r3 from v1; select a, r1 = r2, r2 = r3 from t2; @@ -5159,7 +4889,6 @@ INSERT INTO t2 VALUES (4),(6); CREATE TABLE t3 (c INT) ENGINE=MyISAM; INSERT INTO t3 VALUES (1),(2); - CREATE ALGORITHM=MERGE VIEW v1 AS SELECT ( SELECT a FROM t1 WHERE ( 1, 1 ) IN ( SELECT b, c FROM t2, t3 HAVING c > 2 ) ) AS field1, @@ -5226,7 +4955,6 @@ DECLARE lResult INTEGER UNSIGNED DEFAULT 0; END| --delimiter ; - SELECT `f1`(1); SELECT `f1`(1); SELECT `f1`(1); @@ -5236,7 +4964,6 @@ DROP FUNCTION f1; DROP VIEW v1; DROP TABLE t1, t2; - create view v1 as select 1; --let SEARCH_FILE= $MYSQLD_DATADIR/test/v1.frm @@ -5347,7 +5074,6 @@ alter table v1 check partition p1; drop view v1; drop table t1; - --echo # --echo # MDEV-10419: crash in mariadb 10.1.16-MariaDB-1~trusty --echo # @@ -5851,7 +5577,6 @@ drop table t1, t2; --error ER_BAD_FIELD_ERROR SELECT 1 FROM (SELECT 1 as a) AS b HAVING (SELECT `SOME_GARBAGE`.b.a)=1; - --echo # --echo # MDEV-10035: DBUG_ASSERT on CREATE VIEW v1 AS SELECT * FROM t1 --echo # FOR UPDATE @@ -6095,7 +5820,6 @@ DROP VIEW v1; DROP TABLE t1; - --echo # --echo # MDEV-9408 CREATE TABLE SELECT MAX(int_column) creates different columns for table vs view --echo # @@ -6338,7 +6062,6 @@ REPLACE INTO v (f1,f2) VALUES (1,1); drop view v; drop table t1,t2,t3; - --echo # --echo # MDEV-12379: Server crashes in TABLE_LIST::is_with_table on --echo # SHOW CREATE VIEW @@ -6538,7 +6261,6 @@ drop view v1; eval CREATE VIEW v1 AS $definition; - drop view v1; --echo # @@ -6604,7 +6326,6 @@ set global table_definition_cache= @tdc, table_open_cache= @tc; --echo # End of 10.2 tests --echo # - --echo # --echo # Start of 10.3 tests --echo # @@ -6673,10 +6394,7 @@ CREATE VIEW v1 as select b''; SELECT * FROM v1; DROP VIEW v1; - ---echo # --echo # End of 10.3 tests ---echo # --echo # --echo # MDEV-25206: view specification contains unknown column reference @@ -6753,7 +6471,6 @@ SHOW CREATE VIEW v1; DROP VIEW v1; DROP TABLE t1; - CREATE TABLE t1 (i INT); CREATE VIEW v1 AS SELECT 1 FROM t1 UNION diff --git a/mysql-test/main/xml.result b/mysql-test/main/xml.result index 11275f2e4b396..3a7273acc7777 100644 --- a/mysql-test/main/xml.result +++ b/mysql-test/main/xml.result @@ -1414,13 +1414,6 @@ ExtractValue(@xml, '/employee/dt3') NULL Warnings: Warning 1525 Incorrect XML value: 'parse error at line 10 pos 27: unexpected END-OF-INPUT' -# -# MDEV-39750 ExtractValue does not control recursion depth. -# -SELECT ExtractValue('', REPEAT('(', 100000)); -ERROR HY000: Thread stack overrun: 'used bytes' used of a 'available' byte stack, and 'X' bytes needed. Consider increasing the thread_stack system variable. -SELECT ExtractValue('', REPEAT('/n', 100000)); -ERROR HY000: Thread stack overrun: 'used bytes' used of a 'available' byte stack, and 'X' bytes needed. Consider increasing the thread_stack system variable. # End of 10.6 tests # Start of 11.4 tests # diff --git a/mysql-test/main/xml.test b/mysql-test/main/xml.test index f89b40fcc0e70..13af0885d072c 100644 --- a/mysql-test/main/xml.test +++ b/mysql-test/main/xml.test @@ -903,18 +903,6 @@ set @xml= ' SELECT ExtractValue(@xml, '/employee/dt3'); ---echo # ---echo # MDEV-39750 ExtractValue does not control recursion depth. ---echo # - ---replace_regex /overrun: [0-9]* bytes used of a [0-9]* byte stack, and [0-9]* bytes needed/overrun: 'used bytes' used of a 'available' byte stack, and 'X' bytes needed/ ---error ER_STACK_OVERRUN_NEED_MORE -SELECT ExtractValue('', REPEAT('(', 100000)); - ---replace_regex /overrun: [0-9]* bytes used of a [0-9]* byte stack, and [0-9]* bytes needed/overrun: 'used bytes' used of a 'available' byte stack, and 'X' bytes needed/ ---error ER_STACK_OVERRUN_NEED_MORE -SELECT ExtractValue('', REPEAT('/n', 100000)); - --echo # End of 10.6 tests --echo # Start of 11.4 tests diff --git a/mysql-test/std_data/binlog_invalid_row_v2_tag.001 b/mysql-test/std_data/binlog_invalid_row_v2_tag.001 new file mode 100644 index 0000000000000..4b558372ea550 Binary files /dev/null and b/mysql-test/std_data/binlog_invalid_row_v2_tag.001 differ diff --git a/mysql-test/suite/archive/repair_text.result b/mysql-test/suite/archive/repair_text.result new file mode 100644 index 0000000000000..67926d923cfa9 --- /dev/null +++ b/mysql-test/suite/archive/repair_text.result @@ -0,0 +1,12 @@ +CREATE TABLE t1( +t TEXT NULL DEFAULT NULL +) COLLATE='utf8_general_ci' ENGINE=archive; +INSERT INTO t1(t) VALUES ('Testtext'); +REPAIR TABLE t1; +Table Op Msg_type Msg_text +test.t1 repair status OK +SELECT t = 'Testtext' as Expect_1 from t1; +Expect_1 +1 +DROP TABLE t1; +# End of 10.6 tests diff --git a/mysql-test/suite/archive/repair_text.test b/mysql-test/suite/archive/repair_text.test new file mode 100644 index 0000000000000..842b1edb2016d --- /dev/null +++ b/mysql-test/suite/archive/repair_text.test @@ -0,0 +1,10 @@ +--source include/have_archive.inc +CREATE TABLE t1( + t TEXT NULL DEFAULT NULL +) COLLATE='utf8_general_ci' ENGINE=archive; +INSERT INTO t1(t) VALUES ('Testtext'); +REPAIR TABLE t1; +SELECT t = 'Testtext' as Expect_1 from t1; +DROP TABLE t1; + +--echo # End of 10.6 tests diff --git a/mysql-test/suite/binlog/r/fdle_overflow.result b/mysql-test/suite/binlog/r/fdle_overflow.result new file mode 100644 index 0000000000000..3fde805de82c2 --- /dev/null +++ b/mysql-test/suite/binlog/r/fdle_overflow.result @@ -0,0 +1,10 @@ +SET @saved_dbug= @@GLOBAL.debug_dbug; +SET @@GLOBAL.debug_dbug= +'+d,truncate_fde_post_header_len,truncate_fde_used_checksum_alg'; +FLUSH BINARY LOGS; +ERROR HY000: Error when executing command SHOW BINLOG EVENTS: Wrong offset or I/O error +SET @@GLOBAL.debug_dbug= '+d,truncate_fde_common_header_len'; +FLUSH BINARY LOGS; +ERROR HY000: Error when executing command SHOW BINLOG EVENTS: Wrong offset or I/O error +SET @@GLOBAL.debug_dbug= @saved_dbug; +RESET MASTER; diff --git a/mysql-test/suite/binlog/r/unknown_log_event.result b/mysql-test/suite/binlog/r/unknown_log_event.result new file mode 100644 index 0000000000000..b3bbc14a6de77 --- /dev/null +++ b/mysql-test/suite/binlog/r/unknown_log_event.result @@ -0,0 +1,8 @@ +CREATE TABLE t (a INT); +INSERT INTO t VALUES (0); +DROP TABLE t; +FLUSH BINARY LOGS; +#mariadb_binlog --debug=d,corrupt_table_map_colcnt_read --start-position=#binlog_start --stop-position=#binlog_stop #binlog_file 2>&1 +FOUND 1 /CRC32 0x[[:xdigit:]]{8}.*\n# [Uu]nknown [Ee]vent/ in unknown_log_event.sql +#mariadb_binlog --debug=d,simulate_checksum_test_failure --stop-position=5 #binlog_file 2>&1 +FOUND 1 /CRC32 0x[[:xdigit:]]{8}.*\n# [Uu]nknown [Ee]vent/ in unknown_log_event.sql diff --git a/mysql-test/suite/binlog/t/fdle_overflow.test b/mysql-test/suite/binlog/t/fdle_overflow.test new file mode 100644 index 0000000000000..abaf143d5156c --- /dev/null +++ b/mysql-test/suite/binlog/t/fdle_overflow.test @@ -0,0 +1,38 @@ +--source include/have_debug.inc +--source include/have_binlog_format_mixed.inc # format-agnostic + +# OOB read on malformed `Format_description_log_event` +# +# Verify that Format Description Events with truncated contents fails gracefully +# rather than underflow the post-header count and lead to buffer over-read. + +SET @saved_dbug= @@GLOBAL.debug_dbug; + + +# MDEV-40366: `used_checksum_alg` + +SET @@GLOBAL.debug_dbug= + '+d,truncate_fde_post_header_len,truncate_fde_used_checksum_alg'; +FLUSH BINARY LOGS; + +--let $binlog_file= query_get_value(SHOW BINLOG STATUS, File, 1) +--disable_query_log + --error ER_ERROR_WHEN_EXECUTING_COMMAND + --eval SHOW BINLOG EVENTS IN '$binlog_file' +--enable_query_log + + +# MDEV-40365: `common_header_len` & `post_header_len` + +SET @@GLOBAL.debug_dbug= '+d,truncate_fde_common_header_len'; +FLUSH BINARY LOGS; + +--let $binlog_file= query_get_value(SHOW BINLOG STATUS, File, 1) +--disable_query_log + --error ER_ERROR_WHEN_EXECUTING_COMMAND + --eval SHOW BINLOG EVENTS IN '$binlog_file' +--enable_query_log + +# Clean-up +SET @@GLOBAL.debug_dbug= @saved_dbug; +RESET MASTER; diff --git a/mysql-test/suite/binlog/t/unknown_log_event.test b/mysql-test/suite/binlog/t/unknown_log_event.test new file mode 100644 index 0000000000000..267d107816b45 --- /dev/null +++ b/mysql-test/suite/binlog/t/unknown_log_event.test @@ -0,0 +1,37 @@ +# MDEV-40674: Test `mariadb-binlog --force`'s Unknown event output +# +# Also: MDEV-40542 +# MSAN use-of-uninitialized-value on Unknown_log_event::read_checksum_alg + +--source include/have_debug.inc +--source include/have_binlog_format_row.inc # Testing with Table Map event + +# Setup +CREATE TABLE t (a INT); +--let $binlog_file= query_get_value(SHOW BINLOG STATUS, File, 1) +--let $binlog_start= query_get_value(SHOW BINLOG STATUS, Position, 1) + +# Generate an invalid Table Map event followed by an otherwise-valid Rows Event +INSERT INTO t VALUES (0); + +--let $binlog_stop= query_get_value(SHOW BINLOG STATUS, Position, 1) +DROP TABLE t; +FLUSH BINARY LOGS; + + +--let SEARCH_FILE= $MYSQLTEST_VARDIR/tmp/unknown_log_event.sql +--let SEARCH_PATTERN= CRC32 0x[[:xdigit:]]{8}.*\\n# [Uu]nknown [Ee]vent +--let $mariadb_binlog= $MYSQL_BINLOG --result-file=$SEARCH_FILE --verify-binlog-checksum --force-read +--let $binlog_file= `SELECT CONCAT(@@datadir, '/$binlog_file')` + +--echo #mariadb_binlog --debug=d,corrupt_table_map_colcnt_read --start-position=#binlog_start --stop-position=#binlog_stop #binlog_file 2>&1 +--exec $mariadb_binlog --debug=d,corrupt_table_map_colcnt_read --start-position=$binlog_start --stop-position=$binlog_stop $binlog_file 2>&1 +--source include/search_pattern_in_file.inc + +# Test checksum failure +# +# Only test the first event since the rest of the binlog is unreadable +# without a valid Format Description, which is typically the first event. +--echo #mariadb_binlog --debug=d,simulate_checksum_test_failure --stop-position=5 #binlog_file 2>&1 +--exec $mariadb_binlog --debug=d,simulate_checksum_test_failure --stop-position=5 $binlog_file 2>&1 +--source include/search_pattern_in_file.inc diff --git a/mysql-test/suite/compat/oracle/r/sp-package-security.result b/mysql-test/suite/compat/oracle/r/sp-package-security.result index fa4d4f0a7f680..1cd8771248dce 100644 --- a/mysql-test/suite/compat/oracle/r/sp-package-security.result +++ b/mysql-test/suite/compat/oracle/r/sp-package-security.result @@ -320,3 +320,26 @@ SESSION_USER() CURRENT_USER() msg root@localhost root@localhost p1.p1 DROP PACKAGE p1; DROP USER xxx@localhost; +# +# MDEV-40722 DROP PACKAGE leaves PACKAGE BODY grant in mysql.procs_priv +# +CREATE PACKAGE pkg1 AS +FUNCTION f RETURN VARCHAR2(64); +END +$$ +CREATE PACKAGE BODY pkg1 AS +FUNCTION f RETURN VARCHAR2(64) AS +BEGIN RETURN 'f'; +END; +END$$ +CREATE USER reader@localhost; +GRANT EXECUTE ON PACKAGE test.pkg1 TO reader@localhost; +GRANT EXECUTE ON PACKAGE BODY test.pkg1 TO reader@localhost; +SELECT host,db,user,routine_name,routine_type,proc_priv FROM mysql.procs_priv ORDER BY routine_type; +host db user routine_name routine_type proc_priv +localhost test reader pkg1 PACKAGE Execute +localhost test reader pkg1 PACKAGE BODY Execute +DROP PACKAGE pkg1; +SELECT host,db,user,routine_name,routine_type,proc_priv FROM mysql.procs_priv ORDER BY routine_type; +host db user routine_name routine_type proc_priv +DROP USER reader@localhost; diff --git a/mysql-test/suite/compat/oracle/t/sp-package-security.test b/mysql-test/suite/compat/oracle/t/sp-package-security.test index 583f70afe1470..20a4fddb5b437 100644 --- a/mysql-test/suite/compat/oracle/t/sp-package-security.test +++ b/mysql-test/suite/compat/oracle/t/sp-package-security.test @@ -330,3 +330,28 @@ DELIMITER ;$$ CALL p1.p1; DROP PACKAGE p1; DROP USER xxx@localhost; + + +--echo # +--echo # MDEV-40722 DROP PACKAGE leaves PACKAGE BODY grant in mysql.procs_priv +--echo # + +DELIMITER $$; +CREATE PACKAGE pkg1 AS + FUNCTION f RETURN VARCHAR2(64); +END +$$ +CREATE PACKAGE BODY pkg1 AS + FUNCTION f RETURN VARCHAR2(64) AS + BEGIN RETURN 'f'; + END; +END$$ +DELIMITER ;$$ + +CREATE USER reader@localhost; +GRANT EXECUTE ON PACKAGE test.pkg1 TO reader@localhost; +GRANT EXECUTE ON PACKAGE BODY test.pkg1 TO reader@localhost; +SELECT host,db,user,routine_name,routine_type,proc_priv FROM mysql.procs_priv ORDER BY routine_type; +DROP PACKAGE pkg1; +SELECT host,db,user,routine_name,routine_type,proc_priv FROM mysql.procs_priv ORDER BY routine_type; +DROP USER reader@localhost; diff --git a/mysql-test/suite/csv/delete.result b/mysql-test/suite/csv/delete.result new file mode 100644 index 0000000000000..73dc585fc2629 --- /dev/null +++ b/mysql-test/suite/csv/delete.result @@ -0,0 +1,11 @@ +# +# MDEV-40636 CSV crashes on DELETE +# +create table t1 (id int not null, pad char(10) not null) engine=csv; +insert t1 select seq,'AAAAAAAAAA' from seq_1_to_2048; +delete from t1 where id % 2 = 0; +select count(*), sum(id) from t1; +count(*) sum(id) +1024 1048576 +drop table t1; +# End of 10.6 tests diff --git a/mysql-test/suite/csv/delete.test b/mysql-test/suite/csv/delete.test new file mode 100644 index 0000000000000..2b4b1eea6c97f --- /dev/null +++ b/mysql-test/suite/csv/delete.test @@ -0,0 +1,13 @@ +--source include/have_csv.inc +--source include/have_sequence.inc + +--echo # +--echo # MDEV-40636 CSV crashes on DELETE +--echo # +create table t1 (id int not null, pad char(10) not null) engine=csv; +insert t1 select seq,'AAAAAAAAAA' from seq_1_to_2048; +delete from t1 where id % 2 = 0; +select count(*), sum(id) from t1; +drop table t1; + +--echo # End of 10.6 tests diff --git a/mysql-test/suite/encryption/r/filekeys_encfile_badfile.result b/mysql-test/suite/encryption/r/filekeys_encfile_badfile.result index 7e244c2c38121..c1a2a0a97e5d0 100644 --- a/mysql-test/suite/encryption/r/filekeys_encfile_badfile.result +++ b/mysql-test/suite/encryption/r/filekeys_encfile_badfile.result @@ -1,9 +1,46 @@ -call mtr.add_suppression("File 'bad' not found"); +call mtr.add_suppression("File 'bad.key' not found"); call mtr.add_suppression("Plugin 'file_key_management' init function returned error"); call mtr.add_suppression("Plugin 'file_key_management' registration.*failed"); -FOUND 1 /File 'bad' not found/ in mysqld.1.err +FOUND 1 /File 'bad.key' not found/ in mysqld.1.err create table t1(c1 bigint not null, b char(200)) engine=innodb encrypted=yes encryption_key_id=1; ERROR HY000: Can't create table `test`.`t1` (errno: 140 "Wrong create options") select plugin_status from information_schema.plugins where plugin_name = 'file_key_management'; plugin_status +install soname 'file_key_management'; +ERROR HY000: Cannot decrypt MTR_SUITE_DIR/t/filekeys-data.enc. Wrong key? +call mtr.add_suppression("Cannot decrypt .*filekeys-data.enc. Wrong key"); +call mtr.add_suppression("Plugin 'file_key_management' init function returned error"); +call mtr.add_suppression("Plugin 'file_key_management' registration.*failed"); +FOUND 1 /Cannot decrypt .*filekeys-data.enc. Wrong key/ in mysqld.1.err +create table t1(c1 bigint not null, b char(200)) engine=innodb encrypted=yes encryption_key_id=1; +ERROR HY000: Can't create table `test`.`t1` (errno: 140 "Wrong create options") +select plugin_status from information_schema.plugins +where plugin_name = 'file_key_management'; +plugin_status +install soname 'file_key_management'; +ERROR HY000: Cannot read bad.key, the filekey is too long, max secret size is 256 bytes +call mtr.add_suppression("the filekey is too long"); +call mtr.add_suppression("Plugin 'file_key_management' init function returned error"); +call mtr.add_suppression("Plugin 'file_key_management' registration.*failed"); +FOUND 1 /the filekey is too long/ in mysqld.1.err +create table t1(c1 bigint not null, b char(200)) engine=innodb encrypted=yes encryption_key_id=1; +ERROR HY000: Can't create table `test`.`t1` (errno: 140 "Wrong create options") +select plugin_status from information_schema.plugins +where plugin_name = 'file_key_management'; +plugin_status +# +# MDEV-40658 file_key_management crash on empty FILE: file +# +install soname 'file_key_management'; +ERROR HY000: Cannot decrypt MTR_SUITE_DIR/t/filekeys-data.enc. Wrong key? +call mtr.add_suppression("Cannot decrypt .*filekeys-data.enc. Wrong key"); +call mtr.add_suppression("Plugin 'file_key_management' init function returned error"); +call mtr.add_suppression("Plugin 'file_key_management' registration.*failed"); +FOUND 1 /Cannot decrypt .*filekeys-data.enc. Wrong key/ in mysqld.1.err +create table t1(c1 bigint not null, b char(200)) engine=innodb encrypted=yes encryption_key_id=1; +ERROR HY000: Can't create table `test`.`t1` (errno: 140 "Wrong create options") +select plugin_status from information_schema.plugins +where plugin_name = 'file_key_management'; +plugin_status +# End of 10.6 tests diff --git a/mysql-test/suite/encryption/r/filekeys_secret_too_long.result b/mysql-test/suite/encryption/r/filekeys_secret_too_long.result deleted file mode 100644 index bd11e8d925ed5..0000000000000 --- a/mysql-test/suite/encryption/r/filekeys_secret_too_long.result +++ /dev/null @@ -1,10 +0,0 @@ -call mtr.add_suppression("the filekey is too long"); -call mtr.add_suppression("Plugin 'file_key_management' init function returned error"); -call mtr.add_suppression("Plugin 'file_key_management' registration.*failed"); -FOUND 1 /the filekey is too long/ in mysqld.1.err -create table t1(c1 bigint not null, b char(200)) engine=innodb encrypted=yes encryption_key_id=1; -ERROR HY000: Can't create table `test`.`t1` (errno: 140 "Wrong create options") -select plugin_status from information_schema.plugins -where plugin_name = 'file_key_management'; -plugin_status -# Test checks if opening an too large secret does not crash the server. diff --git a/mysql-test/suite/encryption/t/filekeys-data-too-long.key b/mysql-test/suite/encryption/t/filekeys-data-too-long.key deleted file mode 100644 index ba1624fb32449..0000000000000 --- a/mysql-test/suite/encryption/t/filekeys-data-too-long.key +++ /dev/null @@ -1,4 +0,0 @@ -secretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecret -secretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecret -secretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecret - diff --git a/mysql-test/suite/encryption/t/filekeys_encfile_badfile.opt b/mysql-test/suite/encryption/t/filekeys_encfile_badfile.opt index 374a41395b8c4..713ce507eac8f 100644 --- a/mysql-test/suite/encryption/t/filekeys_encfile_badfile.opt +++ b/mysql-test/suite/encryption/t/filekeys_encfile_badfile.opt @@ -1,2 +1,2 @@ ---loose-file-key-management-filekey=FILE:bad +--loose-file-key-management-filekey=FILE:bad.key --loose-file-key-management-filename=$MTR_SUITE_DIR/t/filekeys-data.enc diff --git a/mysql-test/suite/encryption/t/filekeys_encfile_badfile.test b/mysql-test/suite/encryption/t/filekeys_encfile_badfile.test index e0b07ac62b181..6af2e1c2f013f 100644 --- a/mysql-test/suite/encryption/t/filekeys_encfile_badfile.test +++ b/mysql-test/suite/encryption/t/filekeys_encfile_badfile.test @@ -1,2 +1,51 @@ -let SEARCH_PATTERN=File 'bad' not found; -source filekeys_badtest.inc; +# +# first test: no file +# +--let SEARCH_PATTERN=File 'bad.key' not found +--source filekeys_badtest.inc + +# +# second test: wrong key +# +--let $datadir=`select @@datadir` +--write_file $datadir/bad.key +wrong key +EOF +--replace_result $MTR_SUITE_DIR MTR_SUITE_DIR +--error 2 +install soname 'file_key_management'; +--let SEARCH_PATTERN=Cannot decrypt .*filekeys-data.enc. Wrong key +--source filekeys_badtest.inc +--remove_file $datadir/bad.key + +# +# third test: too long key +# +--let $datadir=`select @@datadir` +--write_file $datadir/bad.key +secretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecret +secretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecret +secretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecretsecret +EOF +--error 2 +install soname 'file_key_management'; +--let SEARCH_PATTERN=the filekey is too long +--source filekeys_badtest.inc +--remove_file $datadir/bad.key + +--echo # +--echo # MDEV-40658 file_key_management crash on empty FILE: file +--echo # +--let $datadir=`select @@datadir` +--write_file $datadir/bad.key + + +EOF +--replace_result $MTR_SUITE_DIR MTR_SUITE_DIR +--error 2 +install soname 'file_key_management'; +--let SEARCH_PATTERN=Cannot decrypt .*filekeys-data.enc. Wrong key +--source filekeys_badtest.inc +--remove_file $datadir/bad.key + +--echo # End of 10.6 tests diff --git a/mysql-test/suite/encryption/t/filekeys_secret_too_long.opt b/mysql-test/suite/encryption/t/filekeys_secret_too_long.opt deleted file mode 100644 index c3f95019f2ab7..0000000000000 --- a/mysql-test/suite/encryption/t/filekeys_secret_too_long.opt +++ /dev/null @@ -1,3 +0,0 @@ ---loose-file-key-management-filekey=FILE:$MTR_SUITE_DIR/t/filekeys-data-too-long.key ---loose-file-key-management-filename=$MTR_SUITE_DIR/t/filekeys-data.enc - diff --git a/mysql-test/suite/encryption/t/filekeys_secret_too_long.test b/mysql-test/suite/encryption/t/filekeys_secret_too_long.test deleted file mode 100644 index 0032e94de3749..0000000000000 --- a/mysql-test/suite/encryption/t/filekeys_secret_too_long.test +++ /dev/null @@ -1,4 +0,0 @@ -let SEARCH_PATTERN=the filekey is too long; -source filekeys_badtest.inc; - ---echo # Test checks if opening an too large secret does not crash the server. diff --git a/mysql-test/suite/federated/federated_server.result b/mysql-test/suite/federated/federated_server.result index 009c4fd06e8d7..dac064b2ab235 100644 --- a/mysql-test/suite/federated/federated_server.result +++ b/mysql-test/suite/federated/federated_server.result @@ -44,13 +44,13 @@ DEFAULT CHARSET=latin1; connection master; drop server if exists 'server_one'; create server 'server_one' foreign data wrapper 'mysql' options -(HOST '127.0.0.1', -DATABASE 'first_db', -USER 'root', -PASSWORD '', -PORT SLAVE_PORT, -SOCKET '', -OWNER 'root'); +(`HOST` '127.0.0.1', +`DATABASE` 'first_db', +`USER` 'root', +`PASSWORD` '', +`PORT` 'SLAVE_PORT', +`SOCKET` '', +`OWNER` 'root'); drop server if exists 'server_two'; create server 'server_two' foreign data wrapper 'mysql' options (HOST '127.0.0.1', diff --git a/mysql-test/suite/federated/federated_server.test b/mysql-test/suite/federated/federated_server.test index efc7cc176a1a1..2997cd284ab9a 100644 --- a/mysql-test/suite/federated/federated_server.test +++ b/mysql-test/suite/federated/federated_server.test @@ -46,13 +46,13 @@ connection master; drop server if exists 'server_one'; --replace_result $SLAVE_MYPORT SLAVE_PORT eval create server 'server_one' foreign data wrapper 'mysql' options - (HOST '127.0.0.1', - DATABASE 'first_db', - USER 'root', - PASSWORD '', - PORT $SLAVE_MYPORT, - SOCKET '', - OWNER 'root'); + (`HOST` '127.0.0.1', + `DATABASE` 'first_db', + `USER` 'root', + `PASSWORD` '', + `PORT` '$SLAVE_MYPORT', + `SOCKET` '', + `OWNER` 'root'); drop server if exists 'server_two'; --replace_result $SLAVE_MYPORT SLAVE_PORT diff --git a/mysql-test/suite/galera/galera_2nodes.cnf b/mysql-test/suite/galera/galera_2nodes.cnf index 072664c4b3470..cec70c87fcfe0 100644 --- a/mysql-test/suite/galera/galera_2nodes.cnf +++ b/mysql-test/suite/galera/galera_2nodes.cnf @@ -34,6 +34,8 @@ wsrep_node_incoming_address=127.0.0.1:@mysqld.2.port wsrep_sst_receive_address='127.0.0.1:@mysqld.2.#sst_port' [sst] +# SSL mode is disabled for SST unless tests specifically enable it. +ssl-mode=DISABLED sst-log-archive-dir=@ENV.MYSQLTEST_VARDIR/log [ENV] diff --git a/mysql-test/suite/galera/galera_2nodes_as_master.cnf b/mysql-test/suite/galera/galera_2nodes_as_master.cnf index 0dd1182bb5660..26f9a5c43a863 100644 --- a/mysql-test/suite/galera/galera_2nodes_as_master.cnf +++ b/mysql-test/suite/galera/galera_2nodes_as_master.cnf @@ -48,6 +48,8 @@ wsrep-on=OFF server-id=3 [sst] +# SSL mode is disabled for SST unless tests specifically enable it. +ssl-mode=DISABLED sst-log-archive-dir=@ENV.MYSQLTEST_VARDIR/log [ENV] diff --git a/mysql-test/suite/galera/galera_2nodes_as_replica_2primary.cnf b/mysql-test/suite/galera/galera_2nodes_as_replica_2primary.cnf index 570eeda0bc697..32f02ad0876f9 100644 --- a/mysql-test/suite/galera/galera_2nodes_as_replica_2primary.cnf +++ b/mysql-test/suite/galera/galera_2nodes_as_replica_2primary.cnf @@ -54,6 +54,8 @@ gtid_domain_id=4 [sst] +# SSL mode is disabled for SST unless tests specifically enable it. +ssl-mode=DISABLED sst-log-archive-dir=@ENV.MYSQLTEST_VARDIR/log [ENV] diff --git a/mysql-test/suite/galera/galera_2nodes_as_slave.cnf b/mysql-test/suite/galera/galera_2nodes_as_slave.cnf index c4a91e4650dfb..2a10e60afe90c 100644 --- a/mysql-test/suite/galera/galera_2nodes_as_slave.cnf +++ b/mysql-test/suite/galera/galera_2nodes_as_slave.cnf @@ -47,6 +47,8 @@ wsrep-on=OFF server-id=3 [sst] +# SSL mode is disabled for SST unless tests specifically enable it. +ssl-mode=DISABLED sst-log-archive-dir=@ENV.MYSQLTEST_VARDIR/log [ENV] diff --git a/mysql-test/suite/galera/galera_2x2nodes.cnf b/mysql-test/suite/galera/galera_2x2nodes.cnf index ac6b08d9bb226..1814afed87118 100644 --- a/mysql-test/suite/galera/galera_2x2nodes.cnf +++ b/mysql-test/suite/galera/galera_2x2nodes.cnf @@ -61,6 +61,8 @@ wsrep_node_incoming_address=127.0.0.1:@mysqld.4.port wsrep_sst_receive_address='127.0.0.1:@mysqld.4.#sst_port' [sst] +# SSL mode is disabled for SST unless tests specifically enable it. +ssl-mode=DISABLED sst-log-archive-dir=@ENV.MYSQLTEST_VARDIR/log [ENV] diff --git a/mysql-test/suite/galera/galera_3nodes_as_slave.cnf b/mysql-test/suite/galera/galera_3nodes_as_slave.cnf index 00bb137f076fe..cd13c8a5fa021 100644 --- a/mysql-test/suite/galera/galera_3nodes_as_slave.cnf +++ b/mysql-test/suite/galera/galera_3nodes_as_slave.cnf @@ -60,6 +60,8 @@ wsrep-on=OFF server-id=4 [sst] +# SSL mode is disabled for SST unless tests specifically enable it. +ssl-mode=DISABLED sst-log-archive-dir=@ENV.MYSQLTEST_VARDIR/log [ENV] diff --git a/mysql-test/suite/galera/galera_4nodes.cnf b/mysql-test/suite/galera/galera_4nodes.cnf index 3d1278a24cfb1..e8bd79aa6f30d 100644 --- a/mysql-test/suite/galera/galera_4nodes.cnf +++ b/mysql-test/suite/galera/galera_4nodes.cnf @@ -61,6 +61,8 @@ wsrep_sst_receive_address='127.0.0.1:@mysqld.4.#sst_port' auto-increment-offset=4 [sst] +# SSL mode is disabled for SST unless tests specifically enable it. +ssl-mode=DISABLED sst-log-archive-dir=@ENV.MYSQLTEST_VARDIR/log [ENV] diff --git a/mysql-test/suite/galera/include/have_pkill.inc b/mysql-test/suite/galera/include/have_pkill.inc new file mode 100644 index 0000000000000..0dd693f2c6399 --- /dev/null +++ b/mysql-test/suite/galera/include/have_pkill.inc @@ -0,0 +1,4 @@ +# +# suite.pm will make sure that all tests including this file +# will be skipped as needed +# diff --git a/mysql-test/suite/galera/r/galera_bf_abort_orphan_lock.result b/mysql-test/suite/galera/r/galera_bf_abort_orphan_lock.result index b75ff658e3f26..994abfff28952 100644 --- a/mysql-test/suite/galera/r/galera_bf_abort_orphan_lock.result +++ b/mysql-test/suite/galera/r/galera_bf_abort_orphan_lock.result @@ -11,7 +11,7 @@ SELECT * FROM t_b WHERE id = 1 FOR UPDATE; id c 1 0 connect node_1_victim, 127.0.0.1, root, , test, $NODE_MYPORT_1; -SET SESSION tx_isolation = 'READ-COMMITTED'; +SET SESSION transaction_isolation = 'READ-COMMITTED'; BEGIN; UPDATE t_a SET c = c + 1 WHERE id > 0; connection node_1; diff --git a/mysql-test/suite/galera/r/galera_bf_kill,debug.rdiff b/mysql-test/suite/galera/r/galera_bf_kill,debug.rdiff deleted file mode 100644 index 098ce2b28a1e5..0000000000000 --- a/mysql-test/suite/galera/r/galera_bf_kill,debug.rdiff +++ /dev/null @@ -1,37 +0,0 @@ ---- r/galera_bf_kill.result -+++ r/galera_bf_kill,debug.reject -@@ -77,4 +77,34 @@ a b - 5 2 - disconnect node_2a; - connection node_1; -+connect node_2a, 127.0.0.1, root, , test, $NODE_MYPORT_2; -+connection node_2a; -+truncate t1; -+insert into t1 values (7,0); -+connection node_2; -+set wsrep_sync_wait=0; -+begin; -+update t1 set b=2 where a=7; -+connect node_2b, 127.0.0.1, root, , test, $NODE_MYPORT_2; -+set wsrep_sync_wait=0; -+SET GLOBAL debug_dbug = "d,sync.wsrep_apply_cb"; -+connection node_1; -+update t1 set b=1 where a=7; -+connection node_2b; -+SET SESSION DEBUG_SYNC = "now WAIT_FOR sync.wsrep_apply_cb_reached"; -+connection node_2; -+connection node_2b; -+SET DEBUG_SYNC = "now SIGNAL signal.wsrep_apply_cb"; -+connection node_2; -+ERROR 40001: Deadlock found when trying to get lock; try restarting transaction -+commit; -+select * from t1; -+a b -+7 1 -+connection node_2a; -+SET DEBUG_SYNC= 'RESET'; -+SET GLOBAL debug_dbug = ""; -+disconnect node_2a; -+disconnect node_2b; -+connection node_1; - drop table t1; diff --git a/mysql-test/suite/galera/r/galera_bf_kill.result b/mysql-test/suite/galera/r/galera_bf_kill.result index e6ddf3fc3c571..537a6d48835d2 100644 --- a/mysql-test/suite/galera/r/galera_bf_kill.result +++ b/mysql-test/suite/galera/r/galera_bf_kill.result @@ -61,9 +61,6 @@ ALTER TABLE t1 ADD UNIQUE KEY b3(b); connection node_2b; SET SESSION wsrep_sync_wait=0; connection node_2a; -select * from t1; -a b -5 2 commit; connection node_2; disconnect node_2a; @@ -77,6 +74,6 @@ connection node_2; select * from t1; a b 5 2 -disconnect node_2a; connection node_1; +disconnect node_2a; drop table t1; diff --git a/mysql-test/suite/galera/r/galera_log_bin_ext_mariabackup.result b/mysql-test/suite/galera/r/galera_log_bin_ext_mariabackup.result index 9d7ea47324165..5f63c971ec912 100644 --- a/mysql-test/suite/galera/r/galera_log_bin_ext_mariabackup.result +++ b/mysql-test/suite/galera/r/galera_log_bin_ext_mariabackup.result @@ -58,8 +58,6 @@ SELECT COUNT(*) = 2 FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_NAME = 't1'; COUNT(*) = 2 1 include/show_binlog_events.inc -Log_name Pos Event_type Server_id End_log_pos Info -hostname1-bin.000002 # Binlog_checkpoint # # hostname1-bin.000002 DROP TABLE t1; DROP TABLE t2; #cleanup diff --git a/mysql-test/suite/galera/r/galera_sst_mariabackup_missing_ssl.result b/mysql-test/suite/galera/r/galera_sst_mariabackup_missing_ssl.result new file mode 100644 index 0000000000000..0424988061ea7 --- /dev/null +++ b/mysql-test/suite/galera/r/galera_sst_mariabackup_missing_ssl.result @@ -0,0 +1,27 @@ +connection node_2; +connection node_1; +connection node_1; +connection node_2; +connection node_1; +call mtr.add_suppression("WSREP: .*State transfer to.* failed:"); +call mtr.add_suppression("WSREP: Will never receive state. Need to abort."); +connection node_2; +# Force a fresh SST on node_2 +# Start node_2 with SSL cert/key simulated as missing +# node_2 failed to start, as expected +# Joiner refused the SST instead of running unencrypted +FOUND 1 /ssl-mode is set to .REQUIRED., but no usable SSL/ in mysqld.2.err +# Restart node_2 normally so it rejoins +connection node_2; +call mtr.add_suppression("WSREP_SST:"); +call mtr.add_suppression("WSREP: Process completed with error:"); +call mtr.add_suppression("WSREP: Failed to read .ready ."); +call mtr.add_suppression("WSREP: Failed to read uuid:seqno and wsrep_gtid_domain_id from joiner script."); +call mtr.add_suppression("WSREP: Failed to read state from: .*"); +call mtr.add_suppression("WSREP: Failed to prepare for .*"); +call mtr.add_suppression("WSREP: SST failed:.*"); +call mtr.add_suppression("WSREP: SST request callback failed. This is unrecoverable, restart required."); +call mtr.add_suppression("WSREP: .*State transfer to.* failed:"); +call mtr.add_suppression("WSREP: Will never receive state. Need to abort."); +call mtr.add_suppression("WSREP: Requesting state transfer failed:.*"); +connection node_1; diff --git a/mysql-test/suite/galera/r/galera_sst_rsync_missing_stunnel.result b/mysql-test/suite/galera/r/galera_sst_rsync_missing_stunnel.result new file mode 100644 index 0000000000000..771d6e0fd7583 --- /dev/null +++ b/mysql-test/suite/galera/r/galera_sst_rsync_missing_stunnel.result @@ -0,0 +1,27 @@ +connection node_2; +connection node_1; +connection node_1; +connection node_2; +connection node_1; +call mtr.add_suppression("WSREP: .*State transfer to.* failed:"); +call mtr.add_suppression("WSREP: Will never receive state. Need to abort."); +connection node_2; +# Force a fresh SST on node_2 +# Start node_2 with stunnel simulated as missing +# node_2 failed to start, as expected +# Joiner refused the SST instead of running unencrypted +FOUND 1 /ssl-mode is set to .REQUIRED., but the .stunnel. binary was not found/ in mysqld.2.err +# Restart node_2 normally so it rejoins +connection node_2; +call mtr.add_suppression("WSREP_SST:"); +call mtr.add_suppression("WSREP: Process completed with error:"); +call mtr.add_suppression("WSREP: Failed to read .ready ."); +call mtr.add_suppression("WSREP: Failed to read uuid:seqno and wsrep_gtid_domain_id from joiner script."); +call mtr.add_suppression("WSREP: Failed to read state from: .*"); +call mtr.add_suppression("WSREP: Failed to prepare for .*"); +call mtr.add_suppression("WSREP: SST failed:.*"); +call mtr.add_suppression("WSREP: SST request callback failed. This is unrecoverable, restart required."); +call mtr.add_suppression("WSREP: .*State transfer to.* failed:"); +call mtr.add_suppression("WSREP: Will never receive state. Need to abort."); +call mtr.add_suppression("WSREP: Requesting state transfer failed:.*"); +connection node_1; diff --git a/mysql-test/suite/galera/suite.pm b/mysql-test/suite/galera/suite.pm index 612a8bfa5953c..b4fd4c62f065e 100644 --- a/mysql-test/suite/galera/suite.pm +++ b/mysql-test/suite/galera/suite.pm @@ -80,6 +80,10 @@ sub skip_combinations { unless which("lsof") || which("sockstat") || which("ss"); $skip{'include/have_stunnel.inc'} = "Need 'stunnel' utility" unless which("stunnel"); + # 'mariabackup' combination doesn't need stunnel: only 'rsync' SST goes + # through it. + $skip{'t/galera_sst_cn_injection.combinations'} = [ 'rsync' ] + unless which("stunnel"); $skip{'include/have_qpress.inc'} = "Need 'qpress' utility" unless which("qpress"); $skip{'../encryption/include/have_file_key_management_plugin.combinations'} = [ 'ctr' ] @@ -93,6 +97,9 @@ sub skip_combinations { $skip{'t/galera_sst_cn_injection.test'} = 'does not work with OpenSSL <= 1.1.1' unless $openssl_ver ge "3.0.0"; + $skip{'include/have_pkill.inc'} = "Need 'pkill' utility" + unless which("pkill"); + %skip; } diff --git a/mysql-test/suite/galera/t/galera_bf_abort_orphan_lock.test b/mysql-test/suite/galera/t/galera_bf_abort_orphan_lock.test index 303d272dcbc3a..65eb4c8bd3c3d 100644 --- a/mysql-test/suite/galera/t/galera_bf_abort_orphan_lock.test +++ b/mysql-test/suite/galera/t/galera_bf_abort_orphan_lock.test @@ -34,7 +34,7 @@ SELECT * FROM t_b WHERE id = 1 FOR UPDATE; # Victim: holds a granted lock on t_a (makes it a BF target), then locks t_b --connect node_1_victim, 127.0.0.1, root, , test, $NODE_MYPORT_1 -SET SESSION tx_isolation = 'READ-COMMITTED'; +SET SESSION transaction_isolation = 'READ-COMMITTED'; BEGIN; UPDATE t_a SET c = c + 1 WHERE id > 0; diff --git a/mysql-test/suite/galera/t/galera_bf_kill.test b/mysql-test/suite/galera/t/galera_bf_kill.test index a64445b53e1d1..5cf98e9fb7541 100644 --- a/mysql-test/suite/galera/t/galera_bf_kill.test +++ b/mysql-test/suite/galera/t/galera_bf_kill.test @@ -97,8 +97,8 @@ select * from t1; # # Test case 5: Start a transaction on node_2a with wsrep disabled. -# A conflicting DDL on other transaction can't BF abort -# transaction from node_2a (wsrep disabled). +# A conflicting DDL on other transaction can BF abort +# transaction from node_2a. # --connect node_2a, 127.0.0.1, root, , test, $NODE_MYPORT_2 @@ -117,10 +117,6 @@ SET SESSION wsrep_sync_wait=0; --source include/wait_condition.inc --connection node_2a -select * from t1; - -# We expect that ALTER should not be able to BF abort -# this transaction, it must wait for it to finish. # Expect commit to succeed. commit; @@ -153,80 +149,7 @@ update t1 set a =5, b=2; select * from t1; ---disconnect node_2a - ---connection node_1 - -source include/maybe_debug.inc; -if ($have_debug) { -# -# Test case 7: Start a transaction on node_2 and use KILL to abort -# a query in connection node_2a -# During the KILL execution replicate conflicting transaction from node_1 -# to BF abort the transaction executing the KILL -# - ---connect node_2a, 127.0.0.1, root, , test, $NODE_MYPORT_2 ---connection node_2a -truncate t1; -insert into t1 values (7,0); - ---connection node_2 -set wsrep_sync_wait=0; - -# get the ID of connection to be later killed ---let $wait_condition = SELECT COUNT(*) = 1 FROM INFORMATION_SCHEMA.PROCESSLIST WHERE USER = 'root' AND COMMAND = 'Sleep' LIMIT 1 ---source include/wait_condition.inc ---let $k_thread = `SELECT ID FROM INFORMATION_SCHEMA.PROCESSLIST WHERE USER = 'root' AND COMMAND = 'Sleep' LIMIT 1` - -# start a transaction -begin; -update t1 set b=2 where a=7; - -# set sync point for incoming applying ---connect node_2b, 127.0.0.1, root, , test, $NODE_MYPORT_2 -set wsrep_sync_wait=0; - -SET GLOBAL debug_dbug = "d,sync.wsrep_apply_cb"; - -# replicate conflicting transaction, should stop in the sync point --connection node_1 -update t1 set b=1 where a=7; - -# wait for the applier to reach the sync point ---connection node_2b -SET SESSION DEBUG_SYNC = "now WAIT_FOR sync.wsrep_apply_cb_reached"; - -# issue KILL inside the transaction, implicit commit is expected ---connection node_2 ---disable_query_log ---send_eval KILL QUERY $k_thread ---enable_query_log - -# wait for the KILL processing to be seen in processlist ---connection node_2b ---let $wait_condition = SELECT COUNT(*) = 1 FROM INFORMATION_SCHEMA.PROCESSLIST WHERE USER = 'root' AND INFO LIKE 'KILL QUERY%' ---source include/wait_condition.inc - -# resume applying, BF abort should follow -SET DEBUG_SYNC = "now SIGNAL signal.wsrep_apply_cb"; - ---connection node_2 ---error ER_LOCK_DEADLOCK ---reap - -commit; - -select * from t1; - ---connection node_2a -SET DEBUG_SYNC= 'RESET'; -SET GLOBAL debug_dbug = ""; - --disconnect node_2a ---disconnect node_2b - ---connection node_1 -} drop table t1; diff --git a/mysql-test/suite/galera/t/galera_ist_rsync_verify_ca.test b/mysql-test/suite/galera/t/galera_ist_rsync_verify_ca.test index d9f7bb152fcb4..fb16ba7f8d327 100644 --- a/mysql-test/suite/galera/t/galera_ist_rsync_verify_ca.test +++ b/mysql-test/suite/galera/t/galera_ist_rsync_verify_ca.test @@ -1,6 +1,7 @@ --source include/big_test.inc --source include/galera_cluster.inc --source include/have_innodb.inc +--source include/have_stunnel.inc --let $node_1=node_1 --let $node_2=node_2 diff --git a/mysql-test/suite/galera/t/galera_sst_cn_injection.test b/mysql-test/suite/galera/t/galera_sst_cn_injection.test index 9b8058267e857..b970950a0a52b 100644 --- a/mysql-test/suite/galera/t/galera_sst_cn_injection.test +++ b/mysql-test/suite/galera/t/galera_sst_cn_injection.test @@ -10,6 +10,7 @@ --source include/have_mariabackup.inc --source include/have_ssl_communication.inc --source include/have_openssl.inc +--source include/have_pkill.inc # This test asserts that the initial cluster-bootstrap SST was SSL-encrypted by # grepping the donor's error log. That is a one-time event, so the cluster must # be freshly bootstrapped on every run (e.g. with --repeat) for node_2 to SST @@ -56,7 +57,8 @@ call mtr.add_suppression("WSREP: Illegal character in variable:"); --echo # cleanup # we have to kill joiner's socat here, because the donor has aborted SST # and joiner's socat will timeout in 5 minutes -# pkill exit code varies by platform 0 or 1 +# pkill below might return 0 = success or 1 in error when it does not find socat +# anymore --error 0,1,15 --exec pkill -f 'socat.*[s]erver-new-cert' --exec echo ssl-cert=$MYSQL_TEST_DIR/std_data/server-cert.pem >> $MYSQLTEST_VARDIR/my.cnf diff --git a/mysql-test/suite/galera/t/galera_sst_mariabackup_missing_ssl.cnf b/mysql-test/suite/galera/t/galera_sst_mariabackup_missing_ssl.cnf new file mode 100644 index 0000000000000..5c9ea2149a3c7 --- /dev/null +++ b/mysql-test/suite/galera/t/galera_sst_mariabackup_missing_ssl.cnf @@ -0,0 +1,11 @@ +!include ../galera_2nodes.cnf + +[mysqld] +wsrep_sst_method=mariabackup +wsrep_sst_auth="root:" +ssl-cert=@ENV.MYSQL_TEST_DIR/std_data/server-cert.pem +ssl-key=@ENV.MYSQL_TEST_DIR/std_data/server-key.pem +ssl-ca=@ENV.MYSQL_TEST_DIR/std_data/cacert.pem + +[sst] +ssl-mode=REQUIRED diff --git a/mysql-test/suite/galera/t/galera_sst_mariabackup_missing_ssl.test b/mysql-test/suite/galera/t/galera_sst_mariabackup_missing_ssl.test new file mode 100644 index 0000000000000..a636634cd5025 --- /dev/null +++ b/mysql-test/suite/galera/t/galera_sst_mariabackup_missing_ssl.test @@ -0,0 +1,69 @@ +# MDEV-28233: with ssl-mode requiring encryption but no usable cert/key, +# wsrep_sst_mariabackup used to fall back to an unencrypted transfer +# WITHOUT any error or warning. Before the fix the joiner only logged the +# informational line: +# WSREP_SST: [INFO] SSL configuration: ... MODE='REQUIRED' ... encrypt='0' +# and then opened a cleartext socket. +# +# Steps: +# 1. Form a 2-node cluster with ssl-mode=REQUIRED and SSL cert/key. +# 2. Shut node_2 down and force a fresh SST. +# 3. Restart node_2 with cert/key simulated as missing +# (MTR_SST_SIMULATE_NO_SSL_CERT=1). +# 4. Check the joiner refuses the SST and logs the error. +# 5. Restart node_2 normally so it rejoins. +# +--source include/galera_cluster.inc +--source include/have_innodb.inc +--source include/have_mariabackup.inc + +# Save auto_increment_offset values. +--let $node_1=node_1 +--let $node_2=node_2 +--source include/auto_increment_offset_save.inc + +--connection node_1 +# Donor-side noise when the joiner aborts the SST. +call mtr.add_suppression("WSREP: .*State transfer to.* failed:"); +call mtr.add_suppression("WSREP: Will never receive state. Need to abort."); + +--connection node_2 +--source include/shutdown_mysqld.inc + +--echo # Force a fresh SST on node_2 +--remove_file $MYSQLTEST_VARDIR/mysqld.2/data/grastate.dat + +--echo # Start node_2 with SSL cert/key simulated as missing +# Abort exit code varies by platform. +--error 1,134 +--exec MTR_SST_SIMULATE_NO_SSL_CERT=1 $MYSQLD_LAST_CMD +--echo # node_2 failed to start, as expected + +--echo # Joiner refused the SST instead of running unencrypted +--let SEARCH_FILE = $MYSQLTEST_VARDIR/log/mysqld.2.err +--let SEARCH_PATTERN = ssl-mode is set to .REQUIRED., but no usable SSL +--source include/search_pattern_in_file.inc + +--echo # Restart node_2 normally so it rejoins +--source include/start_mysqld.inc +--source include/wait_until_connected_again.inc + +--connection node_2 +# Joiner-side noise from the failed SST attempt. +call mtr.add_suppression("WSREP_SST:"); +call mtr.add_suppression("WSREP: Process completed with error:"); +call mtr.add_suppression("WSREP: Failed to read .ready ."); +call mtr.add_suppression("WSREP: Failed to read uuid:seqno and wsrep_gtid_domain_id from joiner script."); +call mtr.add_suppression("WSREP: Failed to read state from: .*"); +call mtr.add_suppression("WSREP: Failed to prepare for .*"); +call mtr.add_suppression("WSREP: SST failed:.*"); +call mtr.add_suppression("WSREP: SST request callback failed. This is unrecoverable, restart required."); +call mtr.add_suppression("WSREP: .*State transfer to.* failed:"); +call mtr.add_suppression("WSREP: Will never receive state. Need to abort."); +call mtr.add_suppression("WSREP: Requesting state transfer failed:.*"); + +--connection node_1 +--let $wait_condition = SELECT VARIABLE_VALUE = 2 FROM INFORMATION_SCHEMA.GLOBAL_STATUS WHERE VARIABLE_NAME = 'wsrep_cluster_size' +--source include/wait_condition.inc + +--source include/auto_increment_offset_restore.inc diff --git a/mysql-test/suite/galera/t/galera_sst_rsync_missing_stunnel.cnf b/mysql-test/suite/galera/t/galera_sst_rsync_missing_stunnel.cnf new file mode 100644 index 0000000000000..39fbab6cb5cc4 --- /dev/null +++ b/mysql-test/suite/galera/t/galera_sst_rsync_missing_stunnel.cnf @@ -0,0 +1,10 @@ +!include ../galera_2nodes.cnf + +[mysqld] +wsrep_sst_method=rsync +ssl-cert=@ENV.MYSQL_TEST_DIR/std_data/server-cert.pem +ssl-key=@ENV.MYSQL_TEST_DIR/std_data/server-key.pem +ssl-ca=@ENV.MYSQL_TEST_DIR/std_data/cacert.pem + +[sst] +ssl-mode=REQUIRED diff --git a/mysql-test/suite/galera/t/galera_sst_rsync_missing_stunnel.test b/mysql-test/suite/galera/t/galera_sst_rsync_missing_stunnel.test new file mode 100644 index 0000000000000..1310c27c6db09 --- /dev/null +++ b/mysql-test/suite/galera/t/galera_sst_rsync_missing_stunnel.test @@ -0,0 +1,66 @@ +# MDEV-28233: the rsync SST script must NOT silently fall back to an +# unencrypted transfer when encryption is requested (ssl-mode is set to a +# value other than DISABLED) but the stunnel binary is not installed. +# +# Steps: +# 1. Form a 2-node cluster with ssl-mode=REQUIRED (stunnel present). +# 2. Shut node_2 down and force a fresh SST. +# 3. Restart node_2 with stunnel hidden (MTR_SST_SIMULATE_NO_STUNNEL=1). +# 4. Check the joiner refuses the SST and logs the error. +# 5. Restart node_2 normally so it rejoins. +# +--source include/galera_cluster.inc +--source include/have_innodb.inc +# Needed for the initial SST; only simulated as missing below. +--source include/have_stunnel.inc + +# Save auto_increment_offset values. +--let $node_1=node_1 +--let $node_2=node_2 +--source include/auto_increment_offset_save.inc + +--connection node_1 +# Donor-side noise when the joiner aborts the SST. +call mtr.add_suppression("WSREP: .*State transfer to.* failed:"); +call mtr.add_suppression("WSREP: Will never receive state. Need to abort."); + +--connection node_2 +--source include/shutdown_mysqld.inc + +--echo # Force a fresh SST on node_2 +--remove_file $MYSQLTEST_VARDIR/mysqld.2/data/grastate.dat + +--echo # Start node_2 with stunnel simulated as missing +# Abort exit code +--error 1,134 +--exec MTR_SST_SIMULATE_NO_STUNNEL=1 $MYSQLD_LAST_CMD +--echo # node_2 failed to start, as expected + +--echo # Joiner refused the SST instead of running unencrypted +--let SEARCH_FILE = $MYSQLTEST_VARDIR/log/mysqld.2.err +--let SEARCH_PATTERN = ssl-mode is set to .REQUIRED., but the .stunnel. binary was not found +--source include/search_pattern_in_file.inc + +--echo # Restart node_2 normally so it rejoins +--source include/start_mysqld.inc +--source include/wait_until_connected_again.inc + +--connection node_2 +# Joiner-side noise from the failed SST attempt. +call mtr.add_suppression("WSREP_SST:"); +call mtr.add_suppression("WSREP: Process completed with error:"); +call mtr.add_suppression("WSREP: Failed to read .ready ."); +call mtr.add_suppression("WSREP: Failed to read uuid:seqno and wsrep_gtid_domain_id from joiner script."); +call mtr.add_suppression("WSREP: Failed to read state from: .*"); +call mtr.add_suppression("WSREP: Failed to prepare for .*"); +call mtr.add_suppression("WSREP: SST failed:.*"); +call mtr.add_suppression("WSREP: SST request callback failed. This is unrecoverable, restart required."); +call mtr.add_suppression("WSREP: .*State transfer to.* failed:"); +call mtr.add_suppression("WSREP: Will never receive state. Need to abort."); +call mtr.add_suppression("WSREP: Requesting state transfer failed:.*"); + +--connection node_1 +--let $wait_condition = SELECT VARIABLE_VALUE = 2 FROM INFORMATION_SCHEMA.GLOBAL_STATUS WHERE VARIABLE_NAME = 'wsrep_cluster_size' +--source include/wait_condition.inc + +--source include/auto_increment_offset_restore.inc diff --git a/mysql-test/suite/galera_3nodes/galera_2x3nodes.cnf b/mysql-test/suite/galera_3nodes/galera_2x3nodes.cnf index e63035ac6bf72..1f638b2008d09 100644 --- a/mysql-test/suite/galera_3nodes/galera_2x3nodes.cnf +++ b/mysql-test/suite/galera_3nodes/galera_2x3nodes.cnf @@ -85,6 +85,8 @@ wsrep_node_incoming_address=127.0.0.1:@mysqld.6.port wsrep_sst_receive_address='127.0.0.1:@mysqld.6.#sst_port' [sst] +# SSL mode is disabled for SST unless tests specifically enable it. +ssl-mode=DISABLED sst-log-archive-dir=@ENV.MYSQLTEST_VARDIR/log [ENV] diff --git a/mysql-test/suite/galera_3nodes/galera_3nodes.cnf b/mysql-test/suite/galera_3nodes/galera_3nodes.cnf index fb6b2b6fc1c2d..21c36427c5497 100644 --- a/mysql-test/suite/galera_3nodes/galera_3nodes.cnf +++ b/mysql-test/suite/galera_3nodes/galera_3nodes.cnf @@ -50,6 +50,8 @@ wsrep_sst_receive_address='127.0.0.1:@mysqld.3.#sst_port' wsrep_node_name=node3 [sst] +# SSL mode is disabled for SST unless tests specifically enable it. +ssl-mode=DISABLED sst-log-archive-dir=@ENV.MYSQLTEST_VARDIR/log transferfmt=@ENV.MTR_GALERA_TFMT diff --git a/mysql-test/suite/galera_3nodes/r/MDEV-38147.result b/mysql-test/suite/galera_3nodes/r/MDEV-38147.result new file mode 100644 index 0000000000000..2150b39cdec7a --- /dev/null +++ b/mysql-test/suite/galera_3nodes/r/MDEV-38147.result @@ -0,0 +1,52 @@ +connection node_2; +connection node_1; +connection node_1; +connection node_2; +connection node_3; +# gtid_strict_mode must be enabled on all nodes +SELECT @@global.gtid_strict_mode AS gtid_strict_mode; +gtid_strict_mode +1 +connection node_1; +connection node_2; +connection node_3; +connection node_3; +connection node_1; +connection node_1; +SET SESSION wsrep_sync_wait = 0; +SET GLOBAL debug_dbug = '+d,sync.after_mdl_block_ddl'; +connection node_1; +SET DEBUG_SYNC = 'now WAIT_FOR sync.after_mdl_block_ddl_reached'; +connect node_1_freeze, 127.0.0.1, root, , test, $NODE_MYPORT_1; +connection node_1_freeze; +SET DEBUG_SYNC = 'commit_before_get_LOCK_commit_ordered SIGNAL t_frozen WAIT_FOR t_go'; +INSERT INTO t1 (val) VALUES (1); +connection node_1; +SET DEBUG_SYNC = 'now WAIT_FOR t_frozen'; +SET DEBUG_SYNC = 'now SIGNAL signal.after_mdl_block_ddl_continue'; +SET DEBUG_SYNC = 'now SIGNAL t_go'; +connection node_1_freeze; +connection node_1; +SET DEBUG_SYNC = 'RESET'; +SET GLOBAL debug_dbug = ''; +connection node_1; +connection node_3; +connection node_1; +connection node_2; +connection node_3; +connection node_1; +SET SESSION wsrep_sync_wait = 15; +SELECT VARIABLE_VALUE AS wsrep_cluster_size FROM INFORMATION_SCHEMA.GLOBAL_STATUS WHERE VARIABLE_NAME = 'wsrep_cluster_size'; +wsrep_cluster_size +3 +connection node_3; +SET SESSION wsrep_sync_wait = 15; +count_match checksum_match gtid_match +1 1 1 +connection node_2; +SET SESSION wsrep_sync_wait = 15; +count_match checksum_match gtid_match +1 1 1 +DROP TABLE t1; +disconnect node_2; +disconnect node_1; diff --git a/mysql-test/suite/galera_3nodes/r/MDEV-38147_binlogdir.result b/mysql-test/suite/galera_3nodes/r/MDEV-38147_binlogdir.result new file mode 100644 index 0000000000000..e62da7e59dba3 --- /dev/null +++ b/mysql-test/suite/galera_3nodes/r/MDEV-38147_binlogdir.result @@ -0,0 +1,20 @@ +connection node_2; +connection node_1; +connection node_1; +connection node_2; +connection node_3; +connection node_1; +CREATE TABLE t1 (pk INT PRIMARY KEY) ENGINE=InnoDB; +INSERT INTO t1 VALUES (1),(2),(3); +connection node_3; +connection node_3; +connection node_1; +connection node_1; +connection node_3; +SELECT COUNT(*) AS rows_on_joiner FROM t1; +rows_on_joiner +3 +connection node_1; +DROP TABLE t1; +disconnect node_2; +disconnect node_1; diff --git a/mysql-test/suite/galera_3nodes/r/MDEV-40179.result b/mysql-test/suite/galera_3nodes/r/MDEV-40179.result new file mode 100644 index 0000000000000..9d8d43bbd7345 --- /dev/null +++ b/mysql-test/suite/galera_3nodes/r/MDEV-40179.result @@ -0,0 +1,47 @@ +connection node_2; +connection node_1; +connection node_1; +connection node_2; +connection node_3; +connection node_1; +connection node_2; +connection node_3; +connection n1_load_1; +CALL p_load('t1_1'); +connection n2_load_1; +CALL p_load('t1_5'); +connection n1_load_2; +CALL p_load('t1_2'); +connection n2_load_2; +CALL p_load('t1_6'); +connection n1_load_3; +CALL p_load('t1_3'); +connection n2_load_3; +CALL p_load('t1_7'); +connection n1_load_4; +CALL p_load('t1_4'); +connection n2_load_4; +CALL p_load('t1_8'); +connection node_1; +connection node_2; +connection node_3; +connection node_1; +UPDATE ctrl SET stop = 1 WHERE id = 1; +connection node_1; +SET SESSION wsrep_sync_wait = 15; +SELECT VARIABLE_VALUE AS wsrep_cluster_size FROM INFORMATION_SCHEMA.GLOBAL_STATUS WHERE VARIABLE_NAME = 'wsrep_cluster_size'; +wsrep_cluster_size +3 +connection node_2; +SET SESSION wsrep_sync_wait = 15; +count_match checksum_match gtid_match +1 1 1 +connection node_3; +SET SESSION wsrep_sync_wait = 15; +count_match checksum_match gtid_match +1 1 1 +connection node_1; +connection node_2; +connection node_3; +disconnect node_2; +disconnect node_1; diff --git a/mysql-test/suite/galera_3nodes/r/MDEV-40179_nobinlog.result b/mysql-test/suite/galera_3nodes/r/MDEV-40179_nobinlog.result new file mode 100644 index 0000000000000..6ca4480ce03dd --- /dev/null +++ b/mysql-test/suite/galera_3nodes/r/MDEV-40179_nobinlog.result @@ -0,0 +1,47 @@ +connection node_2; +connection node_1; +connection node_1; +connection node_2; +connection node_3; +connection node_1; +connection node_2; +connection node_3; +connection n1_load_1; +CALL p_load('t1_1'); +connection n2_load_1; +CALL p_load('t1_5'); +connection n1_load_2; +CALL p_load('t1_2'); +connection n2_load_2; +CALL p_load('t1_6'); +connection n1_load_3; +CALL p_load('t1_3'); +connection n2_load_3; +CALL p_load('t1_7'); +connection n1_load_4; +CALL p_load('t1_4'); +connection n2_load_4; +CALL p_load('t1_8'); +connection node_1; +connection node_2; +connection node_3; +connection node_1; +UPDATE ctrl SET stop = 1 WHERE id = 1; +connection node_1; +SET SESSION wsrep_sync_wait = 15; +SELECT VARIABLE_VALUE AS wsrep_cluster_size FROM INFORMATION_SCHEMA.GLOBAL_STATUS WHERE VARIABLE_NAME = 'wsrep_cluster_size'; +wsrep_cluster_size +3 +connection node_2; +SET SESSION wsrep_sync_wait = 15; +count_match checksum_match +1 1 +connection node_3; +SET SESSION wsrep_sync_wait = 15; +count_match checksum_match +1 1 +connection node_1; +connection node_2; +connection node_3; +disconnect node_2; +disconnect node_1; diff --git a/mysql-test/suite/galera_3nodes/t/MDEV-38147.cnf b/mysql-test/suite/galera_3nodes/t/MDEV-38147.cnf new file mode 100644 index 0000000000000..3393252e28723 --- /dev/null +++ b/mysql-test/suite/galera_3nodes/t/MDEV-38147.cnf @@ -0,0 +1,31 @@ +!include ../galera_3nodes.cnf + +[mysqld] +wsrep_sst_method=mariabackup +wsrep_sst_auth="root:" +gtid_strict_mode=ON +wsrep_gtid_mode=ON +wsrep_gtid_domain_id=100 +gtid_domain_id=10 +log_bin +log_slave_updates=ON +innodb_flush_log_at_trx_commit=1 +sync_binlog=1 +wsrep_sync_wait=6 # allow SHOW to workaround MDEV-39468 and reproduce "error 1950" + +[mysqld.1] +server_id=11 + +[mysqld.2] +server_id=12 + +[mysqld.3] +server_id=13 +# Force node_3 to always SST from node_1 (the node on which we freeze a +# transaction between binary log write and engine commit), so the snapshot +# node_3 receives is the one whose binary log is ahead of its engine checkpoint. +wsrep_sst_donor=node1 + +[sst] +transferfmt=@ENV.MTR_GALERA_TFMT +streamfmt=mbstream diff --git a/mysql-test/suite/galera_3nodes/t/MDEV-38147.test b/mysql-test/suite/galera_3nodes/t/MDEV-38147.test new file mode 100644 index 0000000000000..fcfdb025a9403 --- /dev/null +++ b/mysql-test/suite/galera_3nodes/t/MDEV-38147.test @@ -0,0 +1,198 @@ +# +# MDEV-38147 - Mariadb error 1950 after SST +# +# Here a single transaction is frozen on the donor in the 2PC window between +# the binary log write (step 2) and the engine commit (step 3), while a +# mariabackup SST to a joiner is paused just before it fixes its redo-log +# copy point. That makes the copied binary log carry a Gtid_list ahead of +# the copied engine snapshot - exactly the condition the bug is about. +# +# The bug: +# +# BACKUP STAGE BLOCK_COMMIT fixes the InnoDB redo copy point but does not stop +# a wsrep transaction from having its GTID written to the binary log before it +# commits in the engine. So the copied binary log can carry a Gtid_list ahead +# of the copied engine snapshot. After the SST the joiner reports the +# (committed, behind) engine position, IST resends the missing transaction, and +# re-binlogging it under gtid_strict_mode=ON collides with the ahead Gtid_list +# -> ER_GTID_STRICT_OUT_OF_ORDER (error 1950), and the joiner never reaches +# synced state. (The same snapshot also captures the transaction in the InnoDB +# XA-prepared state, i.e. the MDEV-40179 condition.) +# +# The fix makes the joiner discard the received binary log and seed its GTID +# position from the engine checkpoint, so re-binlogging over IST stays in +# lockstep with the cluster and node_3 rejoins cleanly. +# + +--source include/galera_cluster.inc +--source include/have_innodb.inc +--source include/have_mariabackup.inc +--source include/have_debug_sync.inc + +--let $galera_connection_name = node_3 +--let $galera_server_number = 3 +--source include/galera_connect.inc + +# Save original auto_increment_offset values so that MTR's post-check is +# happy after node_3 has been restarted. +--let $node_1=node_1 +--let $node_2=node_2 +--let $node_3=node_3 +--source ../galera/include/auto_increment_offset_save.inc + +--echo # gtid_strict_mode must be enabled on all nodes +SELECT @@global.gtid_strict_mode AS gtid_strict_mode; + +--connection node_1 +--disable_query_log +CREATE TABLE t1 (pk BIGINT AUTO_INCREMENT PRIMARY KEY, val INT) ENGINE=InnoDB; +--enable_query_log + +# Make sure the schema reached all nodes before we purge node_3. +--connection node_2 +--let $wait_condition = SELECT COUNT(*) = 1 FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_SCHEMA = 'test' AND TABLE_NAME = 't1'; +--source include/wait_condition.inc +--connection node_3 +--let $wait_condition = SELECT COUNT(*) = 1 FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_SCHEMA = 'test' AND TABLE_NAME = 't1'; +--source include/wait_condition.inc + +# +# Stop node_3 and purge its data directory so that rejoining forces a full +# mariabackup SST. +# +--connection node_3 +--source include/shutdown_mysqld.inc + +--connection node_1 +--let $wait_condition = SELECT VARIABLE_VALUE = 2 FROM INFORMATION_SCHEMA.GLOBAL_STATUS WHERE VARIABLE_NAME = 'wsrep_cluster_size'; +--source include/wait_condition.inc + +--remove_files_wildcard $MYSQLTEST_VARDIR/mysqld.3/data/test +--remove_files_wildcard $MYSQLTEST_VARDIR/mysqld.3/data/mysql +--remove_files_wildcard $MYSQLTEST_VARDIR/mysqld.3/data/performance_schema +--remove_files_wildcard $MYSQLTEST_VARDIR/mysqld.3/data/mtr +--remove_files_wildcard $MYSQLTEST_VARDIR/mysqld.3/data + +# +# Arm the donor-side backup sync point: mariabackup will pause after +# BACKUP STAGE BLOCK_DDL, before BLOCK_COMMIT (which fixes the redo copy point). +# +--connection node_1 +SET SESSION wsrep_sync_wait = 0; +SET GLOBAL debug_dbug = '+d,sync.after_mdl_block_ddl'; + +# +# Start node_3 WITHOUT waiting for it to become ready: it rejoins via a +# mariabackup SST from node_1, whose donor backup pauses at the sync point armed +# above. We must not block on node_3 here, because releasing that pause (and +# everything that lets node_3 finish) happens below on the node_1 connection. +# So just trigger the restart via the expect file and continue. +# +--let $_expect_file_name= $MYSQLTEST_VARDIR/tmp/mysqld.3.expect +--write_line restart $_expect_file_name + +--connection node_1 +SET DEBUG_SYNC = 'now WAIT_FOR sync.after_mdl_block_ddl_reached'; + +# +# Freeze one transaction between binary log write and engine commit. +# commit_before_get_LOCK_commit_ordered is reached after the GTID/Xid events +# have been written and fsync'd to the binary log but before the engine commit. +# +--connect node_1_freeze, 127.0.0.1, root, , test, $NODE_MYPORT_1 +--connection node_1_freeze +SET DEBUG_SYNC = 'commit_before_get_LOCK_commit_ordered SIGNAL t_frozen WAIT_FOR t_go'; +--send INSERT INTO t1 (val) VALUES (1) + +--connection node_1 +SET DEBUG_SYNC = 'now WAIT_FOR t_frozen'; + +# +# Let mariabackup continue. It fixes the redo copy point at BLOCK_COMMIT with +# the frozen transaction still in-doubt (so it is excluded from the engine +# snapshot), then issues "FLUSH BINARY LOGS" while shipping the binary log, +# which blocks on the in-doubt transaction's binary log checkpoint. +# +SET DEBUG_SYNC = 'now SIGNAL signal.after_mdl_block_ddl_continue'; + +--let $wait_condition = SELECT COUNT(*) >= 1 FROM INFORMATION_SCHEMA.PROCESSLIST WHERE INFO LIKE 'FLUSH BINARY LOGS%' +--source include/wait_condition.inc + +# +# Release the frozen transaction. Its engine commit happens now - after the +# redo copy point was fixed - so it is absent from the copied engine snapshot +# while present in the shipped binary log's Gtid_list. +# +SET DEBUG_SYNC = 'now SIGNAL t_go'; + +--connection node_1_freeze +--reap +--connection node_1 +SET DEBUG_SYNC = 'RESET'; +SET GLOBAL debug_dbug = ''; + +# +# node_3 must rejoin and the cluster must reconverge to three nodes. +# +--connection node_1 +--let $wait_condition = SELECT VARIABLE_VALUE = 3 FROM INFORMATION_SCHEMA.GLOBAL_STATUS WHERE VARIABLE_NAME = 'wsrep_cluster_size'; +--source include/wait_condition.inc + +# Re-establish the node_3 client connection (it was started without waiting). +--connection node_3 +--enable_reconnect +--source include/wait_until_connected_again.inc +--disable_reconnect + +--connection node_1 +--source include/galera_wait_ready.inc +--connection node_2 +--source include/galera_wait_ready.inc +--connection node_3 +--source include/galera_wait_ready.inc + +# +# Verify data / GTID consistency across all nodes. node_1 is the origin of the +# highest GTID, so node_2 and node_3 are waited *up* to node_1's position. +# +--connection node_1 +SET SESSION wsrep_sync_wait = 15; +SELECT VARIABLE_VALUE AS wsrep_cluster_size FROM INFORMATION_SCHEMA.GLOBAL_STATUS WHERE VARIABLE_NAME = 'wsrep_cluster_size'; + +--let $expect_count = `SELECT COUNT(*) FROM t1` +--let $expect_sum = `SELECT COALESCE(SUM(pk), 0) + COALESCE(SUM(val), 0) FROM t1` +# Compare only the wsrep domain (wsrep_gtid_domain_id) of gtid_binlog_pos - that +# is the part the whole cluster shares; other domains are node-local. +--let $wsrep_dom = `SELECT @@global.wsrep_gtid_domain_id` +--let $expect_gtid = `SELECT REGEXP_SUBSTR(@@global.gtid_binlog_pos, '(?.index' not found (Errcode: 2)". +# +# Here node_3 rejoins via a full mariabackup SST with log_bin relocated to a +# directory that does not exist yet (outside its datadir). The SST script must +# (re)create it so the joiner comes up and rejoins the cluster. +# + +--source include/galera_cluster.inc +--source include/have_innodb.inc +--source include/have_mariabackup.inc + +--let $galera_connection_name = node_3 +--let $galera_server_number = 3 +--source include/galera_connect.inc + +# Save original auto_increment_offset values so that MTR's post-check is +# happy after node_3 has been restarted. +--let $node_1=node_1 +--let $node_2=node_2 +--let $node_3=node_3 +--source ../galera/include/auto_increment_offset_save.inc + +--connection node_1 +CREATE TABLE t1 (pk INT PRIMARY KEY) ENGINE=InnoDB; +INSERT INTO t1 VALUES (1),(2),(3); + +# Make sure the data reached node_3 before we purge it. +--connection node_3 +--let $wait_condition = SELECT COUNT(*) = 3 FROM t1 +--source include/wait_condition.inc + +# +# Stop node_3 and purge its data directory so that rejoining forces a full +# mariabackup SST. +# +--connection node_3 +--source include/shutdown_mysqld.inc + +--connection node_1 +--let $wait_condition = SELECT VARIABLE_VALUE = 2 FROM INFORMATION_SCHEMA.GLOBAL_STATUS WHERE VARIABLE_NAME = 'wsrep_cluster_size' +--source include/wait_condition.inc + +--remove_files_wildcard $MYSQLTEST_VARDIR/mysqld.3/data/test +--remove_files_wildcard $MYSQLTEST_VARDIR/mysqld.3/data/mysql +--remove_files_wildcard $MYSQLTEST_VARDIR/mysqld.3/data/performance_schema +--remove_files_wildcard $MYSQLTEST_VARDIR/mysqld.3/data/mtr +--remove_files_wildcard $MYSQLTEST_VARDIR/mysqld.3/data + +# A binary-log directory that does not exist yet, outside node_3's datadir. +--let $binlog_reloc = $MYSQLTEST_VARDIR/mysqld.3/reloc_binlog +--perl +use File::Path qw(rmtree); +rmtree("$ENV{MYSQLTEST_VARDIR}/mysqld.3/reloc_binlog"); +EOF + +# +# Restart node_3 with the binary log relocated to the (missing) directory. +# Do not block on it: it rejoins via a mariabackup SST which, with the fix, +# creates the directory before mysqld reopens the binary log. +# +--let $_expect_file_name= $MYSQLTEST_VARDIR/tmp/mysqld.3.expect +--write_line "restart:--log-bin=$binlog_reloc/mdev-bin --log-bin-index=$binlog_reloc/mdev-bin.index" $_expect_file_name + +--connection node_1 +--let $wait_timeout = 60 +--let $wait_condition = SELECT VARIABLE_VALUE = 3 FROM INFORMATION_SCHEMA.GLOBAL_STATUS WHERE VARIABLE_NAME = 'wsrep_cluster_size' +--source include/wait_condition.inc + +# Re-establish the node_3 client connection (it was started without waiting). +--connection node_3 +--enable_reconnect +--source include/wait_until_connected_again.inc +--disable_reconnect +--source include/galera_wait_ready.inc + +# The joiner (re)created the relocated binlog directory and opened a fresh +# binary log there. +--file_exists $binlog_reloc/mdev-bin.index +--file_exists $binlog_reloc/mdev-bin.000001 + +# The SST data made it across. +--let $wait_condition = SELECT COUNT(*) = 3 FROM t1 +--source include/wait_condition.inc +SELECT COUNT(*) AS rows_on_joiner FROM t1; + +# +# Cleanup. +# +--connection node_1 +DROP TABLE t1; + +# Restore original auto_increment_offset values. +--source ../galera/include/auto_increment_offset_restore.inc + +--source include/galera_end.inc diff --git a/mysql-test/suite/galera_3nodes/t/MDEV-38147_gtid_off.cnf b/mysql-test/suite/galera_3nodes/t/MDEV-38147_gtid_off.cnf new file mode 100644 index 0000000000000..1fd5b6057f65d --- /dev/null +++ b/mysql-test/suite/galera_3nodes/t/MDEV-38147_gtid_off.cnf @@ -0,0 +1,30 @@ +!include ../galera_3nodes.cnf + +[mysqld] +wsrep_sst_method=mariabackup +wsrep_sst_auth="root:" +log_bin +log_slave_updates=ON +# The case under test: binary logging ON, but Galera GTID mode OFF. Cluster +# writes are then binlogged under the node's own gtid_domain_id with a locally +# allocated seq_no, unrelated to the wsrep cluster seqno in the SE checkpoint. +wsrep_gtid_mode=OFF +gtid_domain_id=10 +# Distinct from gtid_domain_id so that any (incorrect) seeding from the wsrep +# checkpoint would be visible as a foreign domain/seqno. +wsrep_gtid_domain_id=100 + +[mysqld.1] +server_id=11 + +[mysqld.2] +server_id=12 + +[mysqld.3] +server_id=13 +# Force node_3 to always SST from node_1. +wsrep_sst_donor=node1 + +[sst] +transferfmt=@ENV.MTR_GALERA_TFMT +streamfmt=mbstream diff --git a/mysql-test/suite/galera_3nodes/t/MDEV-38147_gtid_off.test b/mysql-test/suite/galera_3nodes/t/MDEV-38147_gtid_off.test new file mode 100644 index 0000000000000..54cbe37754e19 --- /dev/null +++ b/mysql-test/suite/galera_3nodes/t/MDEV-38147_gtid_off.test @@ -0,0 +1,109 @@ +# +# MDEV-38147 follow-up: binary-log GTID seeding must apply only with +# wsrep_gtid_mode=ON. +# +# After MDEV-38147 a mariabackup-SST joiner starts a fresh binary log and, in +# wsrep_gtid_mode=ON, seeds @@gtid_binlog_pos from the storage-engine checkpoint +# (cluster writes are re-tagged to wsrep_gtid_domain_id and binlogged with the +# cluster seqno, so the checkpoint position maps directly onto the binlog GTID +# state - see MDEV-38147.test for the ON-mode case). +# +# With wsrep_gtid_mode=OFF that mapping does not hold: cluster writes keep the +# node's own gtid_domain_id and are binlogged with a locally allocated seq_no, +# while the cluster seqno lives only in thd->wsrep_current_gtid_seqno and is +# never written to the binlog GTID. Seeding gtid_domain_id from the checkpoint +# seqno would therefore inject a wrong, node-unrelated position. So in OFF mode +# the joiner must seed nothing; a fresh binary log simply resumes its own local +# counter. +# +# This test drives a full mariabackup SST of node_3 in wsrep_gtid_mode=OFF and +# asserts the seeding code path did NOT run on the joiner. +# + +--source include/galera_cluster.inc +--source include/have_innodb.inc +--source include/have_mariabackup.inc + +--let $galera_connection_name = node_3 +--let $galera_server_number = 3 +--source include/galera_connect.inc + +# Save original auto_increment_offset values so that MTR's post-check is +# happy after node_3 has been restarted. +--let $node_1=node_1 +--let $node_2=node_2 +--let $node_3=node_3 +--source ../galera/include/auto_increment_offset_save.inc + +--echo # Precondition: Galera GTID mode is OFF +SELECT @@global.wsrep_gtid_mode AS wsrep_gtid_mode; + +--connection node_1 +CREATE TABLE t1 (pk INT PRIMARY KEY) ENGINE=InnoDB; +INSERT INTO t1 VALUES (1),(2),(3),(4),(5); + +# Make sure the data reached node_3 before we purge it. +--connection node_3 +--let $wait_condition = SELECT COUNT(*) = 5 FROM t1 +--source include/wait_condition.inc + +# +# Stop node_3 and purge its data directory so that rejoining forces a full +# mariabackup SST (which restores the wsrep position into the SE checkpoint). +# +--connection node_3 +--source include/shutdown_mysqld.inc + +--connection node_1 +--let $wait_condition = SELECT VARIABLE_VALUE = 2 FROM INFORMATION_SCHEMA.GLOBAL_STATUS WHERE VARIABLE_NAME = 'wsrep_cluster_size' +--source include/wait_condition.inc + +--remove_files_wildcard $MYSQLTEST_VARDIR/mysqld.3/data/test +--remove_files_wildcard $MYSQLTEST_VARDIR/mysqld.3/data/mysql +--remove_files_wildcard $MYSQLTEST_VARDIR/mysqld.3/data/performance_schema +--remove_files_wildcard $MYSQLTEST_VARDIR/mysqld.3/data/mtr +--remove_files_wildcard $MYSQLTEST_VARDIR/mysqld.3/data + +# Restart node_3 (non-blocking): it rejoins via a mariabackup SST from node_1. +--let $_expect_file_name= $MYSQLTEST_VARDIR/tmp/mysqld.3.expect +--write_line restart $_expect_file_name + +--connection node_1 +--let $wait_timeout = 60 +--let $wait_condition = SELECT VARIABLE_VALUE = 3 FROM INFORMATION_SCHEMA.GLOBAL_STATUS WHERE VARIABLE_NAME = 'wsrep_cluster_size' +--source include/wait_condition.inc + +# Re-establish the node_3 client connection (it was started without waiting). +--connection node_3 +--enable_reconnect +--source include/wait_until_connected_again.inc +--disable_reconnect +--source include/galera_wait_ready.inc + +# The SST data made it across. +--let $wait_condition = SELECT COUNT(*) = 5 FROM t1 +--source include/wait_condition.inc +SELECT COUNT(*) AS rows_on_joiner FROM t1; + +# +# Regression assertion: in wsrep_gtid_mode=OFF the joiner must NOT seed its +# binary-log GTID state from the wsrep checkpoint. The seeding code path emits +# this exact line only when it runs (it does run in the ON-mode +# MDEV-38147.test); here it must be absent on the joiner. +# +--let $assert_text = joiner did not seed binlog GTID state in wsrep_gtid_mode=OFF +--let $assert_file = $MYSQLTEST_VARDIR/log/mysqld.3.err +--let $assert_select = seeding binlog GTID state +--let $assert_count = 0 +--source include/assert_grep.inc + +# +# Cleanup. +# +--connection node_1 +DROP TABLE t1; + +# Restore original auto_increment_offset values. +--source ../galera/include/auto_increment_offset_restore.inc + +--source include/galera_end.inc diff --git a/mysql-test/suite/galera_3nodes/t/MDEV-40179.cnf b/mysql-test/suite/galera_3nodes/t/MDEV-40179.cnf new file mode 100644 index 0000000000000..1ce02400920ae --- /dev/null +++ b/mysql-test/suite/galera_3nodes/t/MDEV-40179.cnf @@ -0,0 +1,37 @@ +!include ../galera_3nodes.cnf + +[mysqld] +wsrep_sst_method=mariabackup +wsrep_sst_auth="root:" +gtid_strict_mode=ON +wsrep_gtid_mode=ON +wsrep_gtid_domain_id=100 +gtid_domain_id=10 +log_bin +log_slave_updates=ON +# Parallel apply so that prepared transactions can be committed out of order, +# producing a non-contiguous prepared set on the donor. +wsrep_slave_threads=8 +# Slow, durable commits widen the window during which transactions sit in the +# prepared (XA) state of two-phase commit, so the backup's BLOCK_COMMIT +# snapshot is more likely to capture in-doubt transactions. sync_binlog adds an +# fsync inside the 2PC window without exploding the binlog file count, and a +# moderate max_binlog_size keeps the binlog rotating (further widening the +# window) while avoiding the tens of thousands of tiny files that a 4 KB limit +# would create under this load. +innodb_flush_log_at_trx_commit=1 +sync_binlog=1 +max_binlog_size=16384 + +[mysqld.1] +server_id=11 + +[mysqld.2] +server_id=12 + +[mysqld.3] +server_id=13 + +[sst] +transferfmt=@ENV.MTR_GALERA_TFMT +streamfmt=mbstream diff --git a/mysql-test/suite/galera_3nodes/t/MDEV-40179.inc b/mysql-test/suite/galera_3nodes/t/MDEV-40179.inc new file mode 100644 index 0000000000000..8e4bcc0b6777b --- /dev/null +++ b/mysql-test/suite/galera_3nodes/t/MDEV-40179.inc @@ -0,0 +1,347 @@ +# +# Shared body for the MDEV-40179 tests (sourced by MDEV-40179.test with +# log_bin=ON and MDEV-40179_nobinlog.test with log_bin=OFF). +# +# The bug (reproduced by the log_bin=ON variant): +# +# With log_bin=ON a transaction is committed via two-phase commit (the binary +# log is the second participant), so it passes through the InnoDB XA-prepare +# state. While a donor is held in BLOCK_COMMIT for a mariabackup backup, its +# parallel appliers (wsrep_slave_threads > 1) leave one or more such writesets +# prepared-but-not-yet-committed, and the snapshot captures them. On a freshly +# SST'd joiner nothing resolves these prepared transactions: binlog crash +# recovery does not run (the joiner has no in-use binlog to recover from), and +# the wsrep continuity-based commit is inactive because wsrep_emulate_bin_log +# is FALSE when log_bin is ON. The leftover prepared transactions then abort +# startup with "Found prepared transactions!". Note this does not depend on +# the prepared set being non-contiguous - even a contiguous run aborts, because +# nothing commits or rolls it back. +# +# The log_bin=OFF variant is coverage only: with a single (InnoDB) read-write +# engine and no binary log, commits use one-phase commit, so transactions never +# enter the XA-prepared state and the snapshot has nothing in doubt. It simply +# verifies that mariabackup SST and reconvergence keep working with log_bin=OFF. +# +# To maximize parallel apply on the donor (and thus the chance of catching +# prepared transactions in the snapshot) each client thread writes to its own +# table: there are no certification conflicts between writers, so all of them +# apply concurrently. $writers client threads load on each of node_1 and node_2 +# while node_3 is repeatedly stopped, has its data directory purged and is +# started again, forcing a full mariabackup SST on every rejoin. At the end the +# cluster must reconverge to three nodes and all three nodes must hold identical +# data (and, with log_bin, identical GTID positions). +# +# Parameters set by the including .test: +# $restarts - number of stop/purge/start cycles for node_3 +# $writers - number of concurrent loader threads per node (each gets its +# own table to avoid certification conflicts) +# $check_gtid - 1 to also compare @@global.gtid_binlog_pos across nodes +# (only meaningful with log_bin), 0 otherwise +# + +--source include/big_test.inc +--source include/galera_cluster.inc +--source include/have_innodb.inc +--source include/have_mariabackup.inc + +--let $galera_connection_name = node_3 +--let $galera_server_number = 3 +--source include/galera_connect.inc + +# Save original auto_increment_offset values so that MTR's post-check is +# happy after node_3 has been restarted multiple times. +--let $node_1=node_1 +--let $node_2=node_2 +--let $node_3=node_3 +--source ../galera/include/auto_increment_offset_save.inc + +# Total number of data tables: one per writer thread across both nodes. +--let $ntables = `SELECT 2 * $writers` + +# +# Schema: t1_1 .. t1_$ntables hold the load (one table per writer thread), +# ctrl carries the stop flag for the loaders. +# +--connection node_1 +--disable_query_log +CREATE TABLE ctrl (id INT PRIMARY KEY, stop INT) ENGINE=InnoDB; +INSERT INTO ctrl VALUES (1, 0); + +--let $t = 1 +while ($t <= $ntables) +{ + --eval CREATE TABLE t1_$t (pk BIGINT AUTO_INCREMENT PRIMARY KEY, val INT) ENGINE=InnoDB + --inc $t +} + +DELIMITER |; +CREATE PROCEDURE p_load(IN tname VARCHAR(64)) +BEGIN + DECLARE v_stop INT DEFAULT 0; + DECLARE v_i INT; + # Keep the loop alive across transient cluster errors (BF aborts, + # certification failures, donor desync timeouts, ...). + DECLARE CONTINUE HANDLER FOR SQLEXCEPTION + BEGIN + ROLLBACK; + END; + SET @ins_sql = CONCAT('INSERT INTO ', tname, ' (pk, val) VALUES (DEFAULT, 1)'); + PREPARE ins FROM @ins_sql; + WHILE v_stop = 0 DO + START TRANSACTION; + SET v_i = 0; + WHILE v_i < 16 DO + EXECUTE ins; + SET v_i = v_i + 1; + END WHILE; + COMMIT; + # Throttle slightly between transactions so that a freshly joined node can + # catch up its replication queue instead of being starved by the load. + DO SLEEP(0.01); + SELECT stop INTO v_stop FROM ctrl WHERE id = 1; + END WHILE; + DEALLOCATE PREPARE ins; +END| +DELIMITER ;| +--enable_query_log + +# Make sure the schema reached the other nodes before starting the load. +--connection node_2 +--let $wait_condition = SELECT COUNT(*) = $ntables FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_SCHEMA = 'test' AND TABLE_NAME LIKE 't1\_%'; +--source include/wait_condition.inc +--connection node_3 +--let $wait_condition = SELECT COUNT(*) = $ntables FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_SCHEMA = 'test' AND TABLE_NAME LIKE 't1\_%'; +--source include/wait_condition.inc + +# +# Start the continuous load: $writers threads on node_1 (tables t1_1..t1_W) +# and $writers threads on node_2 (tables t1_(W+1)..t1_2W). +# +--disable_query_log +--let $w = 1 +while ($w <= $writers) +{ + --connect (n1_load_$w, 127.0.0.1, root, , test, $NODE_MYPORT_1) + --connect (n2_load_$w, 127.0.0.1, root, , test, $NODE_MYPORT_2) + --inc $w +} +--enable_query_log + +--let $w = 1 +--let $n2 = $writers +while ($w <= $writers) +{ + --connection n1_load_$w + --send_eval CALL p_load('t1_$w') + --inc $n2 + --connection n2_load_$w + --send_eval CALL p_load('t1_$n2') + --inc $w +} + +# +# While the load is running, repeatedly stop node_3, purge its data +# directory and start it again. An empty data directory forces a full +# mariabackup SST on every rejoin. +# +--disable_query_log +--let $i = $restarts +while ($i) +{ + --connection node_3 + --source include/shutdown_mysqld.inc + --disable_query_log + + # Wait until node_3 has actually left the cluster. + # (shutdown_mysqld.inc / wait_condition.inc / start_mysqld.inc / + # galera_wait_ready.inc each re-enable the query log, so re-disable it after + # every such include to keep the loop output out of the result file.) + --connection node_1 + --let $wait_condition = SELECT VARIABLE_VALUE = 2 FROM INFORMATION_SCHEMA.GLOBAL_STATUS WHERE VARIABLE_NAME = 'wsrep_cluster_size'; + --source include/wait_condition.inc + --disable_query_log + + # Purge node_3's data directory. + --remove_files_wildcard $MYSQLTEST_VARDIR/mysqld.3/data/test + --remove_files_wildcard $MYSQLTEST_VARDIR/mysqld.3/data/mysql + --remove_files_wildcard $MYSQLTEST_VARDIR/mysqld.3/data/performance_schema + --remove_files_wildcard $MYSQLTEST_VARDIR/mysqld.3/data/mtr + --remove_files_wildcard $MYSQLTEST_VARDIR/mysqld.3/data + + # Start node_3 again (rejoins via mariabackup SST). + --connection node_3 + --let $restart_noprint = 2 + --source include/start_mysqld.inc + --disable_query_log + --source include/galera_wait_ready.inc + --disable_query_log + + # Wait until the cluster is back to three nodes before the next cycle. + --connection node_1 + --let $wait_condition = SELECT VARIABLE_VALUE = 3 FROM INFORMATION_SCHEMA.GLOBAL_STATUS WHERE VARIABLE_NAME = 'wsrep_cluster_size'; + --source include/wait_condition.inc + --disable_query_log + + --dec $i +} +--enable_query_log + +# +# Make sure the whole cluster is healthy before stopping the load, so that +# any donor that desynced during SST has resynced and the loaders can read +# the stop flag without blocking. +# +--connection node_1 +--source include/galera_wait_ready.inc +--connection node_2 +--source include/galera_wait_ready.inc +--connection node_3 +--source include/galera_wait_ready.inc + +# +# Signal the loaders to stop and collect them. +# +--connection node_1 +UPDATE ctrl SET stop = 1 WHERE id = 1; + +--disable_query_log +--let $w = 1 +while ($w <= $writers) +{ + --connection n1_load_$w + --reap + --connection n2_load_$w + --reap + --inc $w +} +--enable_query_log + +# +# Build the aggregate count / checksum expressions over all data tables. +# +--let $count_expr = 0 +--let $sum_expr = 0 +--let $t = 1 +while ($t <= $ntables) +{ + --let $count_expr = $count_expr + (SELECT COUNT(*) FROM t1_$t) + --let $sum_expr = $sum_expr + (SELECT COALESCE(SUM(pk),0)+COALESCE(SUM(val),0) FROM t1_$t) + --inc $t +} + +# +# Verify reconvergence and data / GTID consistency across all nodes. +# +--connection node_1 +SET SESSION wsrep_sync_wait = 15; +SELECT VARIABLE_VALUE AS wsrep_cluster_size FROM INFORMATION_SCHEMA.GLOBAL_STATUS WHERE VARIABLE_NAME = 'wsrep_cluster_size'; + +# The load has stopped; issue one final transaction from node_1 (sync_wait is +# on, so node_1 first applies everything else). This makes node_1 the origin of +# the cluster's highest GTID, so the checks below can wait for node_2/node_3 to +# converge *up* to node_1's position instead of comparing a single snapshot: +# @@gtid_binlog_pos is a system variable, so reading it is not covered by +# wsrep_sync_wait and a plain read can otherwise sample a position before the +# node has finished applying (a race that grows with accumulated load, e.g. +# under --repeat). +--disable_query_log +UPDATE ctrl SET stop = 2 WHERE id = 1; +--enable_query_log + +--let $expect_count = `SELECT $count_expr` +--let $expect_sum = `SELECT $sum_expr` +if ($check_gtid) +{ + # Compare only the wsrep domain (wsrep_gtid_domain_id) of gtid_binlog_pos. + # That is the part the whole cluster shares. Other domains in the position + # are node-local and legitimately differ: e.g. CALL mtr.add_suppression() + # below writes to the non-replicated 'mtr' database, which each node binlogs + # under its own gtid_domain_id/server_id - so those entries accumulate + # per-node across runs (visible under --repeat) and must not be compared. + --let $wsrep_dom = `SELECT @@global.wsrep_gtid_domain_id` + --let $expect_gtid = `SELECT REGEXP_SUBSTR(@@global.gtid_binlog_pos, '(? prepared transactions!". (gtid_strict_mode is enabled so any +# binlog/engine position inconsistency would also be caught.) +# +# See MDEV-40179.inc for the shared test body. +# + +--let $restarts = 8 +--let $writers = 4 +--let $check_gtid = 1 +--source MDEV-40179.inc diff --git a/mysql-test/suite/galera_3nodes/t/MDEV-40179_nobinlog.cnf b/mysql-test/suite/galera_3nodes/t/MDEV-40179_nobinlog.cnf new file mode 100644 index 0000000000000..38740c2ec8702 --- /dev/null +++ b/mysql-test/suite/galera_3nodes/t/MDEV-40179_nobinlog.cnf @@ -0,0 +1,27 @@ +!include ../galera_3nodes.cnf + +[mysqld] +wsrep_sst_method=mariabackup +wsrep_sst_auth="root:" +# No log_bin: Galera uses its emulated binlog (wsrep_emulate_bin_log), so the +# wsrep XID continuity check is what resolves prepared transactions on a joiner. +# Parallel apply so that prepared transactions can be committed out of order, +# producing a non-contiguous prepared set on the donor. +wsrep_slave_threads=8 +# Slow, durable commits widen the window during which transactions sit in the +# prepared state, so the backup's BLOCK_COMMIT snapshot is more likely to +# capture in-doubt transactions. +innodb_flush_log_at_trx_commit=1 + +[mysqld.1] +server_id=11 + +[mysqld.2] +server_id=12 + +[mysqld.3] +server_id=13 + +[sst] +transferfmt=@ENV.MTR_GALERA_TFMT +streamfmt=mbstream diff --git a/mysql-test/suite/galera_3nodes/t/MDEV-40179_nobinlog.test b/mysql-test/suite/galera_3nodes/t/MDEV-40179_nobinlog.test new file mode 100644 index 0000000000000..3eb96d3defd1c --- /dev/null +++ b/mysql-test/suite/galera_3nodes/t/MDEV-40179_nobinlog.test @@ -0,0 +1,16 @@ +# +# MDEV-40179 - prepared transactions left behind by a mariabackup SST. +# +# log_bin=OFF variant: coverage only. With a single InnoDB read-write engine +# and no binary log, commits use one-phase commit, so transactions never enter +# the XA-prepared state and a mariabackup snapshot has nothing in doubt - the +# bug cannot occur here. This variant just exercises the same load and repeated +# mariabackup SST with log_bin=OFF and checks the cluster reconverges. +# +# See MDEV-40179.inc for the shared test body. +# + +--let $restarts = 8 +--let $writers = 4 +--let $check_gtid = 0 +--source MDEV-40179.inc diff --git a/mysql-test/suite/gcol/r/gcol_bugfixes.result b/mysql-test/suite/gcol/r/gcol_bugfixes.result index c790658ccaaad..571084720e585 100644 --- a/mysql-test/suite/gcol/r/gcol_bugfixes.result +++ b/mysql-test/suite/gcol/r/gcol_bugfixes.result @@ -782,3 +782,22 @@ a c1 Warnings: Warning 1292 Incorrect datetime value: '0' DROP TABLE t1; +# +# MDEV-40480 LOAD DATA leaves a stale STORED generated column after a +# BEFORE INSERT trigger changes its base column +# +create table t ( +id int primary key, +v int not null, +g int generated always as (v * 2) stored, +note varchar(20) +) engine=innodb; +create trigger t_bi before insert on t for each row set new.v = 20; +select 2, 12, 'new' into outfile 'load_40480'; +load data infile 'load_40480' into table t (id, v, note); +# The trigger sets v=20, so the STORED column g must be v*2 = 40, not 24. +select * from t; +id v g note +2 20 40 new +drop table t; +# End of 10.11 tests diff --git a/mysql-test/suite/gcol/t/gcol_bugfixes.test b/mysql-test/suite/gcol/t/gcol_bugfixes.test index 4ff3f7247a77c..d053d659f42df 100644 --- a/mysql-test/suite/gcol/t/gcol_bugfixes.test +++ b/mysql-test/suite/gcol/t/gcol_bugfixes.test @@ -753,3 +753,30 @@ UPDATE t1 SET a=2; UPDATE t1 SET c1=1; SELECT * FROM t1; DROP TABLE t1; + + +--echo # +--echo # MDEV-40480 LOAD DATA leaves a stale STORED generated column after a +--echo # BEFORE INSERT trigger changes its base column +--echo # + +create table t ( + id int primary key, + v int not null, + g int generated always as (v * 2) stored, + note varchar(20) +) engine=innodb; + +create trigger t_bi before insert on t for each row set new.v = 20; + +select 2, 12, 'new' into outfile 'load_40480'; +load data infile 'load_40480' into table t (id, v, note); + +--echo # The trigger sets v=20, so the STORED column g must be v*2 = 40, not 24. +select * from t; + +drop table t; +--let $datadir= `select @@datadir` +--remove_file $datadir/test/load_40480 + +--echo # End of 10.11 tests diff --git a/mysql-test/suite/handler/heap.result b/mysql-test/suite/handler/heap.result index 5a47656fb51de..ee0c3f31a208e 100644 --- a/mysql-test/suite/handler/heap.result +++ b/mysql-test/suite/handler/heap.result @@ -1697,3 +1697,36 @@ HANDLER m READ `ba`= (30); ERROR HY000: HASH index `ba` does not support this operation HANDLER m CLOSE; DROP TABLE t1; +# +# MDEV-39821 heap-use-after-free in heap_rnext with tree indexes +# +CREATE TABLE t1 ( +k INT, +pad VARCHAR(200), +INDEX idx USING BTREE (k) +) ENGINE=MEMORY; +INSERT t1 VALUES (10,'a'),(20,'b'),(30,'c'),(40,'d'),(50,'e'), +(60,'f'),(70,'g'),(80,'h'),(90,'i'),(100,'j'), +(110,'k'),(120,'l'),(130,'m'),(140,'n'),(150,'o'), +(160,'p'),(170,'q'),(180,'r'),(190,'s'),(200,'t'); +HANDLER t1 OPEN; +HANDLER t1 READ idx FIRST; +k pad +10 a +HANDLER t1 READ idx NEXT; +k pad +20 b +HANDLER t1 READ idx NEXT; +k pad +30 c +HANDLER t1 READ idx NEXT; +k pad +40 d +HANDLER t1 READ idx NEXT; +k pad +50 e +UPDATE t1 SET k = k + 5000 WHERE k BETWEEN 20 AND 180; +HANDLER t1 READ idx NEXT; +ERROR HY000: Record has changed since last read in table 't1'; try restarting transaction +DROP TABLE t1; +# End of 10.6 tests diff --git a/mysql-test/suite/handler/heap.test b/mysql-test/suite/handler/heap.test index d60f92daa76b0..175044d392ec3 100644 --- a/mysql-test/suite/handler/heap.test +++ b/mysql-test/suite/handler/heap.test @@ -103,3 +103,29 @@ HANDLER m READ `ba`= (30); HANDLER m CLOSE; DROP TABLE t1; +--echo # +--echo # MDEV-39821 heap-use-after-free in heap_rnext with tree indexes +--echo # +CREATE TABLE t1 ( + k INT, + pad VARCHAR(200), + INDEX idx USING BTREE (k) +) ENGINE=MEMORY; + +INSERT t1 VALUES (10,'a'),(20,'b'),(30,'c'),(40,'d'),(50,'e'), + (60,'f'),(70,'g'),(80,'h'),(90,'i'),(100,'j'), + (110,'k'),(120,'l'),(130,'m'),(140,'n'),(150,'o'), + (160,'p'),(170,'q'),(180,'r'),(190,'s'),(200,'t'); + +HANDLER t1 OPEN; +HANDLER t1 READ idx FIRST; +HANDLER t1 READ idx NEXT; +HANDLER t1 READ idx NEXT; +HANDLER t1 READ idx NEXT; +HANDLER t1 READ idx NEXT; +UPDATE t1 SET k = k + 5000 WHERE k BETWEEN 20 AND 180; +--error ER_CHECKREAD +HANDLER t1 READ idx NEXT; +DROP TABLE t1; + +--echo # End of 10.6 tests diff --git a/mysql-test/suite/heap/heap_btree.result b/mysql-test/suite/heap/heap_btree.result index 96874cc29d0b9..c835ccaec6944 100644 --- a/mysql-test/suite/heap/heap_btree.result +++ b/mysql-test/suite/heap/heap_btree.result @@ -1,4 +1,3 @@ -drop table if exists t1; create table t1 (a int not null,b int not null, primary key using BTREE (a)) engine=heap comment="testing heaps" avg_row_length=100 min_rows=1 max_rows=100; insert into t1 values(1,1),(2,2),(3,3),(4,4); delete from t1 where a=1 or a=0; @@ -321,7 +320,7 @@ SELECT * FROM t1; a 1 DROP TABLE t1; -End of 4.1 tests +# End of 4.1 tests CREATE TABLE t1(val INT, KEY USING BTREE(val)) ENGINE=memory; INSERT INTO t1 VALUES(0); SELECT INDEX_LENGTH FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_SCHEMA=DATABASE() AND TABLE_NAME='t1'; @@ -350,7 +349,7 @@ CREATE TABLE t1(a INT, KEY USING BTREE (a)) ENGINE=MEMORY; INSERT INTO t1 VALUES(1),(1); DELETE a1 FROM t1 AS a1, t1 AS a2 WHERE a1.a=a2.a; DROP TABLE t1; -End of 5.0 tests +# End of 5.0 tests # bit index in heap tables create table t1 (a bit(63) not null) engine=heap; insert into t1 values (869751),(736494),(226312),(802616),(728912); @@ -386,8 +385,64 @@ explain select 0+a from t1 where a in (869751,736494,226312,802616); id select_type table type possible_keys key key_len ref rows Extra 1 SIMPLE t1 range uniq_id uniq_id 8 NULL 4 Using where drop table t1; -End of 5.3 tests +# End of 5.3 tests create table t1 (id int, a varchar(300) not null, key using btree(a)) engine=heap; insert t1 values (1, repeat('a', 300)); drop table t1; -End of 5.5 tests +# End of 5.5 tests +# +# MDEV-40186 MEMORY tables incorrectly restart index scan on DELETE +# +create table t1 (a int, b int, c int, key (a, b) using btree) engine=heap; +insert t1 select seq % 10, seq, seq from seq_1_to_80; +flush status; +delete from t1 where a=3 and c>60; +select * from information_schema.session_status where variable_name like 'handler_r%' and variable_value>0; +VARIABLE_NAME VARIABLE_VALUE +HANDLER_READ_KEY 1 +HANDLER_READ_NEXT 8 +select * from t1 where a=3; +a b c +3 3 3 +3 13 13 +3 23 23 +3 33 33 +3 43 43 +3 53 53 +delete from t1; +insert t1 select seq % 10, seq, seq from seq_1_to_80; +flush status; +delete from t1 where a=3; +select * from information_schema.session_status where variable_name like 'handler_r%' and variable_value>0; +VARIABLE_NAME VARIABLE_VALUE +HANDLER_READ_KEY 1 +HANDLER_READ_NEXT 8 +select * from t1 where a=3; +a b c +delete from t1; +insert t1 select seq % 10, seq, seq from seq_1_to_80; +flush status; +delete from t1 where a=3 and c<20 order by a desc, b desc limit 10; +select * from information_schema.session_status where variable_name like 'handler_r%' and variable_value>0; +VARIABLE_NAME VARIABLE_VALUE +HANDLER_READ_KEY 1 +HANDLER_READ_PREV 8 +select * from t1 where a=3; +a b c +3 23 23 +3 33 33 +3 43 43 +3 53 53 +3 63 63 +3 73 73 +delete from t1; +insert t1 select seq % 10, seq, seq from seq_1_to_80; +flush status; +delete from t1 where a=3 order by a desc, b desc limit 10; +select * from information_schema.session_status where variable_name like 'handler_r%' and variable_value>0; +VARIABLE_NAME VARIABLE_VALUE +HANDLER_READ_KEY 1 +HANDLER_READ_PREV 8 +select * from t1 where a=3; +a b c +drop table t1; diff --git a/mysql-test/suite/heap/heap_btree.test b/mysql-test/suite/heap/heap_btree.test index e8f7c02c6f3f1..a05f2c80336ba 100644 --- a/mysql-test/suite/heap/heap_btree.test +++ b/mysql-test/suite/heap/heap_btree.test @@ -4,10 +4,6 @@ # Test of heap tables. # ---disable_warnings -drop table if exists t1; ---enable_warnings - create table t1 (a int not null,b int not null, primary key using BTREE (a)) engine=heap comment="testing heaps" avg_row_length=100 min_rows=1 max_rows=100; insert into t1 values(1,1),(2,2),(3,3),(4,4); delete from t1 where a=1 or a=0; @@ -229,7 +225,7 @@ DELETE FROM t1 WHERE a=2; SELECT * FROM t1; DROP TABLE t1; ---echo End of 4.1 tests +--echo # End of 4.1 tests # # BUG#18160 - Memory-/HEAP Table endless growing indexes @@ -268,7 +264,7 @@ CREATE TABLE t1(a INT, KEY USING BTREE (a)) ENGINE=MEMORY; INSERT INTO t1 VALUES(1),(1); DELETE a1 FROM t1 AS a1, t1 AS a2 WHERE a1.a=a2.a; DROP TABLE t1; ---echo End of 5.0 tests +--echo # End of 5.0 tests -- echo # bit index in heap tables @@ -287,7 +283,7 @@ select 0+a from t1 where a in (869751,736494,226312,802616); explain select 0+a from t1 where a in (869751,736494,226312,802616); drop table t1; ---echo End of 5.3 tests +--echo # End of 5.3 tests # # Bug#27799513: POTENTIAL DOUBLE FREE OR CORRUPTION OF HEAP INFO (HP_INFO) @@ -296,4 +292,41 @@ create table t1 (id int, a varchar(300) not null, key using btree(a)) engine=hea insert t1 values (1, repeat('a', 300)); drop table t1; ---echo End of 5.5 tests +--echo # End of 5.5 tests + +--echo # +--echo # MDEV-40186 MEMORY tables incorrectly restart index scan on DELETE +--echo # + +--disable_ps2_protocol +create table t1 (a int, b int, c int, key (a, b) using btree) engine=heap; + +insert t1 select seq % 10, seq, seq from seq_1_to_80; +flush status; +delete from t1 where a=3 and c>60; +select * from information_schema.session_status where variable_name like 'handler_r%' and variable_value>0; +select * from t1 where a=3; + +delete from t1; +insert t1 select seq % 10, seq, seq from seq_1_to_80; +flush status; +delete from t1 where a=3; +select * from information_schema.session_status where variable_name like 'handler_r%' and variable_value>0; +select * from t1 where a=3; + +delete from t1; +insert t1 select seq % 10, seq, seq from seq_1_to_80; +flush status; +delete from t1 where a=3 and c<20 order by a desc, b desc limit 10; +select * from information_schema.session_status where variable_name like 'handler_r%' and variable_value>0; +select * from t1 where a=3; + +delete from t1; +insert t1 select seq % 10, seq, seq from seq_1_to_80; +flush status; +delete from t1 where a=3 order by a desc, b desc limit 10; +select * from information_schema.session_status where variable_name like 'handler_r%' and variable_value>0; +select * from t1 where a=3; + +drop table t1; +--enable_ps2_protocol diff --git a/mysql-test/suite/mariabackup/innodb_redo_overwrite.result b/mysql-test/suite/mariabackup/innodb_redo_overwrite.result index bab5d4331e07b..cbc7f44cab325 100644 --- a/mysql-test/suite/mariabackup/innodb_redo_overwrite.result +++ b/mysql-test/suite/mariabackup/innodb_redo_overwrite.result @@ -1,5 +1,5 @@ CREATE TABLE t ENGINE=INNODB SELECT seq%10 i FROM seq_0_to_204796; # xtrabackup backup -FOUND 1 /Was only able to copy log from \d+ to \d+, not \d+; try increasing innodb_log_file_size\b/ in backup.log +FOUND 1 /mariabackup: Try increasing the innodb_log_file_size./ in backup.log NOT FOUND /failed: redo log block checksum does not match/ in backup.log DROP TABLE t; diff --git a/mysql-test/suite/mariabackup/innodb_redo_overwrite.test b/mysql-test/suite/mariabackup/innodb_redo_overwrite.test index ab5993d232c4e..8c26acf4141ae 100644 --- a/mysql-test/suite/mariabackup/innodb_redo_overwrite.test +++ b/mysql-test/suite/mariabackup/innodb_redo_overwrite.test @@ -17,7 +17,7 @@ CREATE TABLE t ENGINE=INNODB SELECT seq%10 i FROM seq_0_to_204796; --exec $XTRABACKUP --defaults-file=$MYSQLTEST_VARDIR/my.cnf --backup --target-dir=$targetdir --dbug=+d,mariabackup_events > $backuplog --enable_result_log ---let SEARCH_PATTERN=Was only able to copy log from \\d+ to \\d+, not \\d+; try increasing innodb_log_file_size\\b +--let SEARCH_PATTERN=mariabackup: Try increasing the innodb_log_file_size. --let SEARCH_FILE=$backuplog --source include/search_pattern_in_file.inc --remove_file $backuplog diff --git a/mysql-test/suite/plugins/r/qc_info.result b/mysql-test/suite/plugins/r/qc_info.result index afab7671b5cc1..5ad4e13f1e1e1 100644 Binary files a/mysql-test/suite/plugins/r/qc_info.result and b/mysql-test/suite/plugins/r/qc_info.result differ diff --git a/mysql-test/suite/plugins/t/multiauth.test b/mysql-test/suite/plugins/t/multiauth.test index 18cad406e915e..5ce958bb4a182 100644 --- a/mysql-test/suite/plugins/t/multiauth.test +++ b/mysql-test/suite/plugins/t/multiauth.test @@ -1,6 +1,6 @@ ---source include/not_ubsan.inc --source include/count_sessions.inc + let $REGEX_VERSION_ID=/$mysql_get_server_version/VERSION_ID/; let $REGEX_PASSWORD_LAST_CHANGED=/password_last_changed": [0-9]*/password_last_changed": #/; let $REGEX_GLOBAL_PRIV=$REGEX_PASSWORD_LAST_CHANGED $REGEX_VERSION_ID; diff --git a/mysql-test/suite/plugins/t/qc_info.test b/mysql-test/suite/plugins/t/qc_info.test index 5345bac12ff0d..0bccd7f66723b 100644 Binary files a/mysql-test/suite/plugins/t/qc_info.test and b/mysql-test/suite/plugins/t/qc_info.test differ diff --git a/mysql-test/suite/roles/definer.result b/mysql-test/suite/roles/definer.result index 3dcc10a00c0d0..11b6417fa21b8 100644 --- a/mysql-test/suite/roles/definer.result +++ b/mysql-test/suite/roles/definer.result @@ -393,61 +393,63 @@ DELIMITER ; /*!50003 SET character_set_results = @saved_cs_results */ ; /*!50003 SET collation_connection = @saved_col_connection */ ; /*!50106 SET @save_time_zone= @@TIME_ZONE */ ; +/*!50003 SET @saved_cs_client = @@character_set_client */ ; +/*!50003 SET @saved_cs_results = @@character_set_results */ ; +/*!50003 SET @saved_col_connection = @@collation_connection */ ; +/*!50003 SET character_set_client = latin1 */ ; +/*!50003 SET character_set_results = latin1 */ ; +/*!50003 SET collation_connection = latin1_swedish_ci */ ; +/*!50003 SET @saved_sql_mode = @@sql_mode */ ; +/*!50003 SET sql_mode = 'STRICT_TRANS_TABLES,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION' */ ; +/*!50003 SET @saved_time_zone = @@time_zone */ ; +/*!50003 SET time_zone = 'SYSTEM' */ ; DELIMITER ;; -/*!50003 SET @saved_cs_client = @@character_set_client */ ;; -/*!50003 SET @saved_cs_results = @@character_set_results */ ;; -/*!50003 SET @saved_col_connection = @@collation_connection */ ;; -/*!50003 SET character_set_client = latin1 */ ;; -/*!50003 SET character_set_results = latin1 */ ;; -/*!50003 SET collation_connection = latin1_swedish_ci */ ;; -/*!50003 SET @saved_sql_mode = @@sql_mode */ ;; -/*!50003 SET sql_mode = 'STRICT_TRANS_TABLES,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION' */ ;; -/*!50003 SET @saved_time_zone = @@time_zone */ ;; -/*!50003 SET time_zone = 'SYSTEM' */ ;; /*!50106 CREATE*/ /*!50117 DEFINER=`role1`*/ /*!50106 EVENT `e1` ON SCHEDULE EVERY 1 SECOND STARTS '2000-01-01 00:00:00' ON COMPLETION NOT PRESERVE ENABLE DO insert t1 values (111, 2, 0) */ ;; -/*!50003 SET time_zone = @saved_time_zone */ ;; -/*!50003 SET sql_mode = @saved_sql_mode */ ;; -/*!50003 SET character_set_client = @saved_cs_client */ ;; -/*!50003 SET character_set_results = @saved_cs_results */ ;; -/*!50003 SET collation_connection = @saved_col_connection */ ;; +DELIMITER ; +/*!50003 SET time_zone = @saved_time_zone */ ; +/*!50003 SET sql_mode = @saved_sql_mode */ ; +/*!50003 SET character_set_client = @saved_cs_client */ ; +/*!50003 SET character_set_results = @saved_cs_results */ ; +/*!50003 SET collation_connection = @saved_col_connection */ ; +/*!50003 SET @saved_cs_client = @@character_set_client */ ; +/*!50003 SET @saved_cs_results = @@character_set_results */ ; +/*!50003 SET @saved_col_connection = @@collation_connection */ ; +/*!50003 SET character_set_client = latin1 */ ; +/*!50003 SET character_set_results = latin1 */ ; +/*!50003 SET collation_connection = latin1_swedish_ci */ ; +/*!50003 SET @saved_sql_mode = @@sql_mode */ ; +/*!50003 SET sql_mode = 'STRICT_TRANS_TABLES,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION' */ ; +/*!50003 SET @saved_time_zone = @@time_zone */ ; +/*!50003 SET time_zone = 'SYSTEM' */ ; DELIMITER ;; -/*!50003 SET @saved_cs_client = @@character_set_client */ ;; -/*!50003 SET @saved_cs_results = @@character_set_results */ ;; -/*!50003 SET @saved_col_connection = @@collation_connection */ ;; -/*!50003 SET character_set_client = latin1 */ ;; -/*!50003 SET character_set_results = latin1 */ ;; -/*!50003 SET collation_connection = latin1_swedish_ci */ ;; -/*!50003 SET @saved_sql_mode = @@sql_mode */ ;; -/*!50003 SET sql_mode = 'STRICT_TRANS_TABLES,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION' */ ;; -/*!50003 SET @saved_time_zone = @@time_zone */ ;; -/*!50003 SET time_zone = 'SYSTEM' */ ;; /*!50106 CREATE*/ /*!50117 DEFINER=`role2`*/ /*!50106 EVENT `e2` ON SCHEDULE EVERY 1 SECOND STARTS '2000-01-01 00:00:00' ON COMPLETION NOT PRESERVE ENABLE DO insert t1 values (111, 4, 0) */ ;; -/*!50003 SET time_zone = @saved_time_zone */ ;; -/*!50003 SET sql_mode = @saved_sql_mode */ ;; -/*!50003 SET character_set_client = @saved_cs_client */ ;; -/*!50003 SET character_set_results = @saved_cs_results */ ;; -/*!50003 SET collation_connection = @saved_col_connection */ ;; +DELIMITER ; +/*!50003 SET time_zone = @saved_time_zone */ ; +/*!50003 SET sql_mode = @saved_sql_mode */ ; +/*!50003 SET character_set_client = @saved_cs_client */ ; +/*!50003 SET character_set_results = @saved_cs_results */ ; +/*!50003 SET collation_connection = @saved_col_connection */ ; +/*!50003 SET @saved_cs_client = @@character_set_client */ ; +/*!50003 SET @saved_cs_results = @@character_set_results */ ; +/*!50003 SET @saved_col_connection = @@collation_connection */ ; +/*!50003 SET character_set_client = latin1 */ ; +/*!50003 SET character_set_results = latin1 */ ; +/*!50003 SET collation_connection = latin1_swedish_ci */ ; +/*!50003 SET @saved_sql_mode = @@sql_mode */ ; +/*!50003 SET sql_mode = 'STRICT_TRANS_TABLES,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION' */ ; +/*!50003 SET @saved_time_zone = @@time_zone */ ; +/*!50003 SET time_zone = 'SYSTEM' */ ; DELIMITER ;; -/*!50003 SET @saved_cs_client = @@character_set_client */ ;; -/*!50003 SET @saved_cs_results = @@character_set_results */ ;; -/*!50003 SET @saved_col_connection = @@collation_connection */ ;; -/*!50003 SET character_set_client = latin1 */ ;; -/*!50003 SET character_set_results = latin1 */ ;; -/*!50003 SET collation_connection = latin1_swedish_ci */ ;; -/*!50003 SET @saved_sql_mode = @@sql_mode */ ;; -/*!50003 SET sql_mode = 'STRICT_TRANS_TABLES,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION' */ ;; -/*!50003 SET @saved_time_zone = @@time_zone */ ;; -/*!50003 SET time_zone = 'SYSTEM' */ ;; /*!50106 CREATE*/ /*!50117 DEFINER=`role3`@`%`*/ /*!50106 EVENT `e3` ON SCHEDULE EVERY 1 SECOND STARTS '2000-01-01 00:00:00' ON COMPLETION NOT PRESERVE ENABLE DO insert t1 values (111, 3, 0) */ ;; -/*!50003 SET time_zone = @saved_time_zone */ ;; -/*!50003 SET sql_mode = @saved_sql_mode */ ;; -/*!50003 SET character_set_client = @saved_cs_client */ ;; -/*!50003 SET character_set_results = @saved_cs_results */ ;; -/*!50003 SET collation_connection = @saved_col_connection */ ;; DELIMITER ; +/*!50003 SET time_zone = @saved_time_zone */ ; +/*!50003 SET sql_mode = @saved_sql_mode */ ; +/*!50003 SET character_set_client = @saved_cs_client */ ; +/*!50003 SET character_set_results = @saved_cs_results */ ; +/*!50003 SET collation_connection = @saved_col_connection */ ; /*!50106 SET TIME_ZONE= @save_time_zone */ ; /*!50003 SET @saved_sql_mode = @@sql_mode */ ; /*!50003 SET sql_mode = 'STRICT_TRANS_TABLES,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION' */ ; diff --git a/mysql-test/suite/rpl/r/rpl_corrupted_decimal_precision.result b/mysql-test/suite/rpl/r/rpl_corrupted_decimal_precision.result new file mode 100644 index 0000000000000..b56808e112888 --- /dev/null +++ b/mysql-test/suite/rpl/r/rpl_corrupted_decimal_precision.result @@ -0,0 +1,70 @@ +include/master-slave.inc +[connection master] +connection slave; +call mtr.add_suppression("Slave SQL: Invalid variable length at User var event"); +connection master; +create table t1 (a decimal(10,5)); +insert into t1 values (0.5); +connection slave; +# +# Write a User_var_log_event whose DECIMAL precision and scale do not +# match the length of the packed value +# +connection master; +set @@session.debug_dbug= "+d,corrupt_user_var_decimal_precision"; +set @a= 1.5; +insert into t1 values (@a); +set @@session.debug_dbug= ""; +# +# SHOW BINLOG EVENTS must not crash in User_var_log_event::pack_info() +# +set @@session.debug_dbug= "+d,corrupt_user_var_decimal_precision"; +include/show_binlog_events.inc +Log_name Pos Event_type Server_id End_log_pos Info +master-bin.000001 # Gtid # # GTID #-#-# +master-bin.000001 # Query # # use `test`; create table t1 (a decimal(10,5)) +master-bin.000001 # Gtid # # BEGIN GTID #-#-# +master-bin.000001 # Query # # use `test`; insert into t1 values (0.5) +master-bin.000001 # Query # # COMMIT +master-bin.000001 # Gtid # # BEGIN GTID #-#-# +master-bin.000001 # User var # # NULL +master-bin.000001 # Query # # use `test`; insert into t1 values (@a) +master-bin.000001 # Query # # COMMIT +set @@session.debug_dbug= ""; +# +# mysqlbinlog must not crash in User_var_log_event::print() +# +# +# The slave rejects the event instead of crashing on it +# +connection slave; +include/wait_for_slave_sql_error.inc [errno=1593] +# +# Replication continues once the corrupted event group is skipped +# +set global sql_slave_skip_counter= 1; +include/start_slave.inc +connection master; +insert into t1 values (3.5); +connection slave; +# Only the skipped row (1.50000) is missing on the slave, every later +# event is applied normally +connection master; +select * from t1 order by a; +a +0.50000 +1.50000 +3.50000 +connection slave; +select * from t1 order by a; +a +0.50000 +3.50000 +# +# Cleanup +# +connection master; +drop table t1; +connection slave; +include/rpl_end.inc +# End of rpl_corrupted_decimal_precision.test diff --git a/mysql-test/suite/rpl/r/rpl_rotate_ev_overflow.result b/mysql-test/suite/rpl/r/rpl_rotate_ev_overflow.result new file mode 100644 index 0000000000000..0529ff7ca4f64 --- /dev/null +++ b/mysql-test/suite/rpl/r/rpl_rotate_ev_overflow.result @@ -0,0 +1,19 @@ +SET @saved_dbug= @@GLOBAL.debug_dbug; +SET @@GLOBAL.debug_dbug= '+d,truncate_fde_post_header_len'; +include/master-slave.inc +[connection master] +FLUSH BINARY LOGS; +CALL mtr.add_suppression('Found invalid event in binary log'); +ERROR HY000: Error when executing command SHOW BINLOG EVENTS: Wrong offset or I/O error +connection slave; +START SLAVE IO_THREAD; +CALL mtr.add_suppression('Slave I/O: Relay log write failure'); +include/wait_for_slave_io_error.inc [errno=1595] +connection master; +SET @@GLOBAL.debug_dbug= @saved_dbug; +include/reset_master.inc +connection slave; +CHANGE MASTER TO master_use_gtid=SLAVE_POS; +SET @@GLOBAL.gtid_slave_pos=''; +include/start_slave.inc +include/rpl_end.inc diff --git a/mysql-test/suite/rpl/t/rpl_corrupted_decimal_precision.test b/mysql-test/suite/rpl/t/rpl_corrupted_decimal_precision.test new file mode 100644 index 0000000000000..36058770ff39c --- /dev/null +++ b/mysql-test/suite/rpl/t/rpl_corrupted_decimal_precision.test @@ -0,0 +1,82 @@ +# +# MDEV-40645 Slave crash on malformed User_var_log_event +# +# A User_var_log_event of type DECIMAL_RESULT carries the precision and the +# scale of the value in the first two bytes of the packed value. Nothing ties +# them to the length of the event, so a crafted event makes bin2decimal() +# alloca() and copy a size taken entirely from those two bytes. +# +--source include/have_debug.inc +# A User_var_log_event is only written when the statement itself is binlogged, +# i.e. never for row based replication. +--source include/have_binlog_format_mixed_or_statement.inc +--source include/master-slave.inc + +# The master intentionally writes a corrupted event, the slave has to reject it +--connection slave +call mtr.add_suppression("Slave SQL: Invalid variable length at User var event"); + +--connection master +create table t1 (a decimal(10,5)); +insert into t1 values (0.5); +--sync_slave_with_master + +--echo # +--echo # Write a User_var_log_event whose DECIMAL precision and scale do not +--echo # match the length of the packed value +--echo # +--connection master +set @@session.debug_dbug= "+d,corrupt_user_var_decimal_precision"; +set @a= 1.5; +insert into t1 values (@a); +set @@session.debug_dbug= ""; +--let $corrupt_file= query_get_value(SHOW MASTER STATUS, File, 1) + +--echo # +--echo # SHOW BINLOG EVENTS must not crash in User_var_log_event::pack_info() +--echo # +set @@session.debug_dbug= "+d,corrupt_user_var_decimal_precision"; +--source include/show_binlog_events.inc +set @@session.debug_dbug= ""; + +--echo # +--echo # mysqlbinlog must not crash in User_var_log_event::print() +--echo # +--let $datadir= `select @@datadir` +# The corrupted event is rejected, only the exit code matters here +--error 1 +--exec $MYSQL_BINLOG $datadir/$corrupt_file >/dev/null 2>&1 + +--echo # +--echo # The slave rejects the event instead of crashing on it +--echo # +--connection slave +--let $slave_sql_errno= 1593 +--source include/wait_for_slave_sql_error.inc + +--echo # +--echo # Replication continues once the corrupted event group is skipped +--echo # +set global sql_slave_skip_counter= 1; +--source include/start_slave.inc + +--connection master +insert into t1 values (3.5); +--sync_slave_with_master + +--echo # Only the skipped row (1.50000) is missing on the slave, every later +--echo # event is applied normally +--connection master +select * from t1 order by a; +--connection slave +select * from t1 order by a; + +--echo # +--echo # Cleanup +--echo # +--connection master +drop table t1; +--sync_slave_with_master + +--source include/rpl_end.inc +--echo # End of rpl_corrupted_decimal_precision.test diff --git a/mysql-test/suite/rpl/t/rpl_rotate_ev_overflow.test b/mysql-test/suite/rpl/t/rpl_rotate_ev_overflow.test new file mode 100644 index 0000000000000..f7782a311d662 --- /dev/null +++ b/mysql-test/suite/rpl/t/rpl_rotate_ev_overflow.test @@ -0,0 +1,51 @@ +# MDEV-40647 OOB read in IO Thread if the FDEv does not support Rotate Events + +--source include/have_debug.inc +--source include/have_binlog_format_mixed.inc # should be format-agnostic + +SET @saved_dbug= @@GLOBAL.debug_dbug; +SET @@GLOBAL.debug_dbug= '+d,truncate_fde_post_header_len'; +--let $rpl_skip_start_slave= 1 +# This setup will also RESET MASTER ... +--source include/master-slave.inc +# ... which means the binlog file is consistent. +--let $binlog_file= master-bin.000001 + +# Use a specific binlog position to avoid the +# test failing early from other metadata events +--let $binlog_start= query_get_value(SHOW BINLOG STATUS, Position, 1) +# Generate the Rotate Event +FLUSH BINARY LOGS; + + +# Control: SHOW BINLOG EVENTS fails as expected. +CALL mtr.add_suppression('Found invalid event in binary log'); +--disable_query_log + --error ER_ERROR_WHEN_EXECUTING_COMMAND + --eval SHOW BINLOG EVENTS IN '$binlog_file' FROM $binlog_start +--enable_query_log + +--connection slave +--disable_query_log + eval CHANGE MASTER TO master_use_gtid=NO, + master_log_file='$binlog_file', master_log_pos=$binlog_start; +--enable_query_log +START SLAVE IO_THREAD; +CALL mtr.add_suppression('Slave I/O: Relay log write failure'); +# ER_SLAVE_RELAY_LOG_WRITE_FAILURE +--let $slave_io_errno= 1595 +# Experiment: The IO thread should fail. +--source include/wait_for_slave_io_error.inc + +# Clean-up +--connection master +SET @@GLOBAL.debug_dbug= @saved_dbug; +--source include/reset_master.inc + +--connection slave +CHANGE MASTER TO master_use_gtid=SLAVE_POS; # restore the default +--disable_warnings + SET @@GLOBAL.gtid_slave_pos=''; # for good measure +--enable_warnings +--source include/start_slave.inc +--source include/rpl_end.inc diff --git a/mysql-test/suite/s3/clone.result b/mysql-test/suite/s3/clone.result index a33c8c4259a99..308b0a809235b 100644 --- a/mysql-test/suite/s3/clone.result +++ b/mysql-test/suite/s3/clone.result @@ -1,12 +1,10 @@ # # SELECT using ror_merged scan fails with s3 tables # -DROP TABLE IF EXISTS t1; -Warnings: -Note 1051 Unknown table 'test.t1' CREATE TABLE t1 (a INT, b INT, KEY(a), KEY(b)) ENGINE=Aria; INSERT INTO t1 VALUES (0,0),(0,10),(3,10); ALTER TABLE t1 ENGINE=S3; SELECT * FROM t1 WHERE a = 99 OR b = 2; a b DROP TABLE t1; +End of 10.11 tests diff --git a/mysql-test/suite/s3/clone.test b/mysql-test/suite/s3/clone.test index 5695257c404e3..304671654e6ac 100644 --- a/mysql-test/suite/s3/clone.test +++ b/mysql-test/suite/s3/clone.test @@ -2,13 +2,21 @@ --source include/have_sequence.inc --source include/have_innodb.inc +# +# Create unique database for running the tests +# +--source create_database.inc + --echo # --echo # SELECT using ror_merged scan fails with s3 tables --echo # -DROP TABLE IF EXISTS t1; CREATE TABLE t1 (a INT, b INT, KEY(a), KEY(b)) ENGINE=Aria; INSERT INTO t1 VALUES (0,0),(0,10),(3,10); ALTER TABLE t1 ENGINE=S3; SELECT * FROM t1 WHERE a = 99 OR b = 2; DROP TABLE t1; + +--source drop_database.inc + +--echo End of 10.11 tests diff --git a/mysql-test/suite/sys_vars/r/secure_file_priv,win.rdiff b/mysql-test/suite/sys_vars/r/secure_file_priv,win.rdiff new file mode 100644 index 0000000000000..44674696c9cb1 --- /dev/null +++ b/mysql-test/suite/sys_vars/r/secure_file_priv,win.rdiff @@ -0,0 +1,10 @@ +--- secure_file_priv.result ++++ secure_file_priv.reject +@@ -30,6 +30,6 @@ + # + CREATE TABLE t1 (c1 VARCHAR(50)); + LOAD DATA INFILE '/proc/cpuinfo' INTO TABLE t1; +-ERROR HY000: The MariaDB server is running with the --secure-file-priv option so it cannot execute this statement ++ERROR HY000: File 'C:\proc\cpuinfo' not found (Errcode: 2 "No such file or directory") + DROP TABLE t1; + # End of 10.6 tests diff --git a/mysql-test/suite/sys_vars/r/secure_file_priv.result b/mysql-test/suite/sys_vars/r/secure_file_priv.result index eeeb9a58c0fa0..06968a185e773 100644 --- a/mysql-test/suite/sys_vars/r/secure_file_priv.result +++ b/mysql-test/suite/sys_vars/r/secure_file_priv.result @@ -6,12 +6,17 @@ INSERT INTO t1 VALUES ("one"),("two"),("three"),("four"),("five"); SHOW VARIABLES LIKE 'secure_file_priv'; Variable_name Value secure_file_priv +SELECT * INTO OUTFILE '$MYSQLTEST_VARDIR/../bug50373.txt' FROM t1; +DELETE FROM t1; +LOAD DATA INFILE '$MYSQLTEST_VARDIR/../bug50373.txt' INTO TABLE t1; +SELECT * FROM t1; c1 one two three four five +SELECT load_file('$MYSQLTEST_VARDIR/../bug50373.txt') AS loaded_file; loaded_file one two @@ -20,3 +25,11 @@ four five DROP TABLE t1; +# +# MDEV-40589 default exclude list for secure-file-priv +# +CREATE TABLE t1 (c1 VARCHAR(50)); +LOAD DATA INFILE '/proc/cpuinfo' INTO TABLE t1; +ERROR HY000: The MariaDB server is running with the --secure-file-priv option so it cannot execute this statement +DROP TABLE t1; +# End of 10.6 tests diff --git a/mysql-test/suite/sys_vars/r/sysvars_readonly_debug.result b/mysql-test/suite/sys_vars/r/sysvars_readonly_debug.result new file mode 100644 index 0000000000000..7badd1cb5f03d --- /dev/null +++ b/mysql-test/suite/sys_vars/r/sysvars_readonly_debug.result @@ -0,0 +1,18 @@ +# +# MDEV-40341 store read-only sysvars in a read-only root and a read-only segment +# +call mtr.add_suppression('mariadbd(.exe)? got (exception 0xc0000005|signal 11)'); +select @@global.skip_name_resolve; +@@global.skip_name_resolve +0 +set @@global.skip_name_resolve=0; +ERROR HY000: Variable 'skip_name_resolve' is a read only variable +set @@debug_dbug='d,set_skip_name_resolve'; +Got one of the listed errors +select @@global.version; +@@global.version +12.11.10-fake-version-for-the-test +set @@global.version=0; +ERROR HY000: Variable 'version' is a read only variable +set @@debug_dbug='d,set_version_buf'; +Got one of the listed errors diff --git a/mysql-test/suite/sys_vars/r/sysvars_server_embedded.result b/mysql-test/suite/sys_vars/r/sysvars_server_embedded.result index 44f299efb919f..c4ac449a9fe02 100644 --- a/mysql-test/suite/sys_vars/r/sysvars_server_embedded.result +++ b/mysql-test/suite/sys_vars/r/sysvars_server_embedded.result @@ -3485,7 +3485,7 @@ COMMAND_LINE_ARGUMENT OPTIONAL VARIABLE_NAME SECURE_FILE_PRIV VARIABLE_SCOPE GLOBAL VARIABLE_TYPE VARCHAR -VARIABLE_COMMENT Limit LOAD DATA, SELECT ... OUTFILE, and LOAD_FILE() to files within specified directory +VARIABLE_COMMENT Limit LOAD DATA, SELECT ... OUTFILE, and LOAD_FILE() to files within specified directory. Empty value means no limits except /proc NUMERIC_MIN_VALUE NULL NUMERIC_MAX_VALUE NULL NUMERIC_BLOCK_SIZE NULL diff --git a/mysql-test/suite/sys_vars/r/sysvars_server_notembedded,win.rdiff b/mysql-test/suite/sys_vars/r/sysvars_server_notembedded,win.rdiff index 3156c5d1a406e..45b0f592d5963 100644 --- a/mysql-test/suite/sys_vars/r/sysvars_server_notembedded,win.rdiff +++ b/mysql-test/suite/sys_vars/r/sysvars_server_notembedded,win.rdiff @@ -1261,6 +1261,15 @@ NUMERIC_BLOCK_SIZE 1 ENUM_VALUE_LIST NULL READ_ONLY NO +@@ -3885,7 +3885,7 @@ COMMAND_LINE_ARGUMENT OPTIONAL + VARIABLE_NAME SECURE_FILE_PRIV + VARIABLE_SCOPE GLOBAL + VARIABLE_TYPE VARCHAR +-VARIABLE_COMMENT Limit LOAD DATA, SELECT ... OUTFILE, and LOAD_FILE() to files within specified directory. Empty value means no limits except /proc ++VARIABLE_COMMENT Limit LOAD DATA, SELECT ... OUTFILE, and LOAD_FILE() to files within specified directory. + NUMERIC_MIN_VALUE NULL + NUMERIC_MAX_VALUE NULL + NUMERIC_BLOCK_SIZE NULL @@ -3904,7 +3904,7 @@ READ_ONLY YES COMMAND_LINE_ARGUMENT REQUIRED VARIABLE_NAME SERVER_ID @@ -1395,7 +1404,7 @@ +VARIABLE_NAME THREAD_POOL_MODE +VARIABLE_SCOPE GLOBAL +VARIABLE_TYPE ENUM -+VARIABLE_COMMENT Chose implementation of the threadpool ++VARIABLE_COMMENT Chose implementation of the threadpool. Use 'windows' unless you have a workload with a lot of concurrent connections and minimal contention +NUMERIC_MIN_VALUE NULL +NUMERIC_MAX_VALUE NULL +NUMERIC_BLOCK_SIZE NULL diff --git a/mysql-test/suite/sys_vars/r/sysvars_server_notembedded.result b/mysql-test/suite/sys_vars/r/sysvars_server_notembedded.result index e39f34bdca7e7..a5c73257b6541 100644 --- a/mysql-test/suite/sys_vars/r/sysvars_server_notembedded.result +++ b/mysql-test/suite/sys_vars/r/sysvars_server_notembedded.result @@ -4045,7 +4045,7 @@ COMMAND_LINE_ARGUMENT OPTIONAL VARIABLE_NAME SECURE_FILE_PRIV VARIABLE_SCOPE GLOBAL VARIABLE_TYPE VARCHAR -VARIABLE_COMMENT Limit LOAD DATA, SELECT ... OUTFILE, and LOAD_FILE() to files within specified directory +VARIABLE_COMMENT Limit LOAD DATA, SELECT ... OUTFILE, and LOAD_FILE() to files within specified directory. Empty value means no limits except /proc NUMERIC_MIN_VALUE NULL NUMERIC_MAX_VALUE NULL NUMERIC_BLOCK_SIZE NULL diff --git a/mysql-test/suite/sys_vars/t/secure_file_priv-master.opt b/mysql-test/suite/sys_vars/t/secure_file_priv.opt similarity index 100% rename from mysql-test/suite/sys_vars/t/secure_file_priv-master.opt rename to mysql-test/suite/sys_vars/t/secure_file_priv.opt diff --git a/mysql-test/suite/sys_vars/t/secure_file_priv.test b/mysql-test/suite/sys_vars/t/secure_file_priv.test index 5ba9bf15194cc..17f129028ab39 100644 --- a/mysql-test/suite/sys_vars/t/secure_file_priv.test +++ b/mysql-test/suite/sys_vars/t/secure_file_priv.test @@ -1,10 +1,10 @@ +--source include/platform.inc --echo # --echo # Bug50373 --secure-file-priv="" --echo # CREATE TABLE t1 (c1 VARCHAR(50)); INSERT INTO t1 VALUES ("one"),("two"),("three"),("four"),("five"); SHOW VARIABLES LIKE 'secure_file_priv'; ---disable_query_log # Attempt to create a file where we normally aren't allowed to create one. # @@ -17,35 +17,23 @@ SHOW VARIABLES LIKE 'secure_file_priv'; # If we run tests with --mem, it will be /dev/shm. # If we run tests with --parallel, it will be mysql-test/var # (because MYSQLTEST_VARDIR in this case is mysql-test/var/N). ---disable_cursor_protocol ---perl -use File::Basename; -my $protected_file= dirname($ENV{MYSQLTEST_VARDIR}).'/bug50373.txt'; -# Ensure bug50373.txt does not exist (e.g. leftover from previous -# test runs). -unlink $protected_file; -open(FILE, ">", "$ENV{MYSQL_TMP_DIR}/bug50373.inc") or die; -print FILE "SELECT * FROM t1 INTO OUTFILE '".$protected_file."';\n"; -print FILE "DELETE FROM t1;\n"; -print FILE "LOAD DATA INFILE '".$protected_file."' INTO TABLE t1;\n"; -print FILE "SELECT * FROM t1;\n"; -print FILE "SELECT load_file('",$protected_file,"') AS loaded_file;\n"; -close(FILE); -EOF - ---enable_prepare_warnings ---source $MYSQL_TMP_DIR/bug50373.inc ---disable_prepare_warnings ---remove_file $MYSQL_TMP_DIR/bug50373.inc ---enable_query_log +evalp SELECT * INTO OUTFILE '$MYSQLTEST_VARDIR/../bug50373.txt' FROM t1; +DELETE FROM t1; +evalp LOAD DATA INFILE '$MYSQLTEST_VARDIR/../bug50373.txt' INTO TABLE t1; +SELECT * FROM t1; +evalp SELECT load_file('$MYSQLTEST_VARDIR/../bug50373.txt') AS loaded_file; +DROP TABLE t1; ---enable_cursor_protocol +--remove_file $MYSQLTEST_VARDIR/../bug50373.txt +--echo # +--echo # MDEV-40589 default exclude list for secure-file-priv +--echo # +CREATE TABLE t1 (c1 VARCHAR(50)); +--disable_abort_on_error +LOAD DATA INFILE '/proc/cpuinfo' INTO TABLE t1; +--enable_abort_on_error DROP TABLE t1; ---perl -use File::Basename; -unlink dirname($ENV{MYSQLTEST_VARDIR}).'/bug50373.txt'; -EOF - +--echo # End of 10.6 tests diff --git a/mysql-test/suite/sys_vars/t/sysvars_readonly_debug.opt b/mysql-test/suite/sys_vars/t/sysvars_readonly_debug.opt new file mode 100644 index 0000000000000..a5812bb7aa0e2 --- /dev/null +++ b/mysql-test/suite/sys_vars/t/sysvars_readonly_debug.opt @@ -0,0 +1,2 @@ +--loose-skip-stack-trace --skip-core-file +--version=12.11.10-fake-version-for-the-test diff --git a/mysql-test/suite/sys_vars/t/sysvars_readonly_debug.test b/mysql-test/suite/sys_vars/t/sysvars_readonly_debug.test new file mode 100644 index 0000000000000..192c481b4cef3 --- /dev/null +++ b/mysql-test/suite/sys_vars/t/sysvars_readonly_debug.test @@ -0,0 +1,27 @@ +--source include/have_debug.inc +--source include/not_asan.inc +--source include/not_embedded.inc +--source include/not_valgrind.inc + +--echo # +--echo # MDEV-40341 store read-only sysvars in a read-only root and a read-only segment +--echo # +call mtr.add_suppression('mariadbd(.exe)? got (exception 0xc0000005|signal 11)'); + +select @@global.skip_name_resolve; +--error ER_INCORRECT_GLOBAL_LOCAL_VAR +set @@global.skip_name_resolve=0; +--write_line restart $MYSQLTEST_VARDIR/tmp/mysqld.1.expect +--error 2013,2026 +set @@debug_dbug='d,set_skip_name_resolve'; +--enable_reconnect +--source include/wait_until_connected_again.inc + +select @@global.version; +--error ER_INCORRECT_GLOBAL_LOCAL_VAR +set @@global.version=0; +--write_line restart $MYSQLTEST_VARDIR/tmp/mysqld.1.expect +--error 2013,2026 +set @@debug_dbug='d,set_version_buf'; +--enable_reconnect +--source include/wait_until_connected_again.inc diff --git a/mysql-test/suite/sys_vars/t/thread_stack_basic.test b/mysql-test/suite/sys_vars/t/thread_stack_basic.test index a1f52576d1526..5efc42d0a8327 100644 --- a/mysql-test/suite/sys_vars/t/thread_stack_basic.test +++ b/mysql-test/suite/sys_vars/t/thread_stack_basic.test @@ -1,20 +1,18 @@ # # only global # ---source include/not_asan.inc ---source include/not_ubsan.inc ---source include/not_msan.inc ---replace_result 392192 299008 +# sizes per DEFAULT_THREAD_STACK in include/my_pthread.h +--replace_result 11534336 299008 2097152 299008 select @@global.thread_stack; --error ER_INCORRECT_GLOBAL_LOCAL_VAR select @@session.thread_stack; ---replace_result 392192 299008 +--replace_result 11534336 299008 2097152 299008 show global variables like 'thread_stack'; ---replace_result 392192 299008 +--replace_result 11534336 299008 2097152 299008 show session variables like 'thread_stack'; ---replace_result 392192 299008 +--replace_result 11534336 299008 2097152 299008 select * from information_schema.global_variables where variable_name='thread_stack'; ---replace_result 392192 299008 +--replace_result 11534336 299008 2097152 299008 select * from information_schema.session_variables where variable_name='thread_stack'; # diff --git a/mysql-test/suite/wsrep/r/wsrep_provider_plugin.result b/mysql-test/suite/wsrep/r/wsrep_provider_plugin.result index abcc15b5c41d3..486b91e70cc4d 100644 --- a/mysql-test/suite/wsrep/r/wsrep_provider_plugin.result +++ b/mysql-test/suite/wsrep/r/wsrep_provider_plugin.result @@ -12,7 +12,7 @@ Variable_name Value wsrep_provider_repl_max_ws_size 2147483647 INSERT INTO t1 VALUES (1); SET GLOBAL wsrep_provider_options='repl.max_ws_size=1'; -ERROR HY000: Variable 'wsrep_provider_options' is a read only variable +ERROR HY000: wsrep_provider_options cannot be changed while the wsrep-provider plugin is loaded INSERT INTO t1 VALUES (2); SET GLOBAL wsrep_provider='none'; ERROR HY000: Variable 'wsrep_provider' is a read only variable @@ -21,4 +21,4 @@ CALL mtr.add_suppression("transaction size limit"); CALL mtr.add_suppression("rbr write fail"); SELECT VARIABLE_NAME,READ_ONLY FROM information_schema.system_variables where VARIABLE_NAME like '%wsrep_provider_options%'; VARIABLE_NAME READ_ONLY -WSREP_PROVIDER_OPTIONS YES +WSREP_PROVIDER_OPTIONS NO diff --git a/mysql-test/suite/wsrep/t/wsrep_provider_plugin.test b/mysql-test/suite/wsrep/t/wsrep_provider_plugin.test index de4533f879c8b..3c4fa621fff96 100644 --- a/mysql-test/suite/wsrep/t/wsrep_provider_plugin.test +++ b/mysql-test/suite/wsrep/t/wsrep_provider_plugin.test @@ -22,8 +22,8 @@ SHOW VARIABLES LIKE 'wsrep_provider_repl_max_ws_size'; INSERT INTO t1 VALUES (1); -# Variable should be read only, must not take effect ---error ER_INCORRECT_GLOBAL_LOCAL_VAR +# wsrep-provider plugin is loaded, must not take effect +--error ER_WRONG_ARGUMENTS SET GLOBAL wsrep_provider_options='repl.max_ws_size=1'; INSERT INTO t1 VALUES (2); @@ -36,6 +36,8 @@ CALL mtr.add_suppression("transaction size limit"); CALL mtr.add_suppression("rbr write fail"); # -# MDEV-30120 :Update the wsrep_provider_options read_only value in the system_variables table. +# MDEV-30120: wsrep_provider_options is not flagged READ_ONLY in +# system_variables, even though SET on it is rejected above while the +# wsrep-provider plugin is loaded. # SELECT VARIABLE_NAME,READ_ONLY FROM information_schema.system_variables where VARIABLE_NAME like '%wsrep_provider_options%'; diff --git a/mysys/charset.c b/mysys/charset.c index da522f257bf26..2bd977bf7349b 100644 --- a/mysys/charset.c +++ b/mysys/charset.c @@ -1,6 +1,6 @@ /* Copyright (c) 2000, 2011, Oracle and/or its affiliates - Copyright (c) 2009, 2020, MariaDB Corporation. + Copyright (c) 2009, 2026, MariaDB plc. This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by @@ -518,7 +518,7 @@ my_charset_loader_init_mysys(MY_CHARSET_LOADER *loader) #define MY_MAX_ALLOWED_BUF 1024*1024 #define MY_CHARSET_INDEX "Index.xml" -const char *charsets_dir= NULL; +READ_ONLY_SYSVAR const char *charsets_dir= NULL; static my_bool diff --git a/mysys/my_alloc.c b/mysys/my_alloc.c index 228ac97bb0593..70b8c732ae7fb 100644 --- a/mysys/my_alloc.c +++ b/mysys/my_alloc.c @@ -1,6 +1,6 @@ /* Copyright (c) 2000, 2010, Oracle and/or its affiliates - Copyright (c) 2010, 2020, MariaDB + Copyright (c) 2010, 2026, MariaDB plc. This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by @@ -21,15 +21,13 @@ #include #include #include -#ifdef HAVE_SYS_MMAN_H -#include -#endif +#include #undef EXTRA_DEBUG #define EXTRA_DEBUG #define ROOT_FLAG_THREAD_SPECIFIC 1 -#define ROOT_FLAG_MPROTECT 2 +#define ROOT_FLAG_VMEM 2 #define ROOT_FLAG_READ_ONLY 4 /* data packed in MEM_ROOT -> min_malloc */ @@ -45,37 +43,35 @@ #define ALIGN_SIZE(X) MY_ALIGN(X, 16) /* - Alloc memory through either my_malloc or mmap() + Alloc memory through either my_malloc() or my_virtual_mem_commit() */ static void *root_alloc(MEM_ROOT *root, size_t size, size_t *alloced_size, myf my_flags) { *alloced_size= size; -#if defined(HAVE_MMAP) && defined(HAVE_MPROTECT) && defined(MAP_ANONYMOUS) - if (root->flags & ROOT_FLAG_MPROTECT) + if (root->flags & ROOT_FLAG_VMEM) { - void *res; + void *ptr; *alloced_size= MY_ALIGN(size, my_system_page_size); - res= my_mmap(0, *alloced_size, PROT_READ | PROT_WRITE, - MAP_NORESERVE | MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); - if (res == MAP_FAILED) - res= 0; - return res; + if ((ptr= my_virtual_mem_commit(NULL, *alloced_size))) + update_malloc_size(*alloced_size, + MY_TEST(root->flags & ROOT_FLAG_THREAD_SPECIFIC)); + return ptr; } -#endif /* HAVE_MMAP */ - return my_malloc(root->psi_key, size, - my_flags | MALLOC_FLAG(root)); + return my_malloc(root->psi_key, size, my_flags | MALLOC_FLAG(root)); } static void root_free(MEM_ROOT *root, void *ptr, size_t size) { -#if defined(HAVE_MMAP) && defined(HAVE_MPROTECT) && defined(MAP_ANONYMOUS) - if (root->flags & ROOT_FLAG_MPROTECT) - my_munmap(ptr, size); + if (root->flags & ROOT_FLAG_VMEM) + { + update_malloc_size(-(longlong) size, + MY_TEST(root->flags & ROOT_FLAG_THREAD_SPECIFIC)); + my_virtual_mem_release(ptr, size); + } else -#endif my_free(ptr); } @@ -95,7 +91,7 @@ static void calculate_block_sizes(MEM_ROOT *mem_root, size_t block_size, { size_t pre_alloc= *pre_alloc_size; - if (mem_root->flags & ROOT_FLAG_MPROTECT) + if (mem_root->flags & ROOT_FLAG_VMEM) { mem_root->block_size= MY_ALIGN(block_size, my_system_page_size); if (pre_alloc) @@ -134,7 +130,7 @@ static void calculate_block_sizes(MEM_ROOT *mem_root, size_t block_size, pre_alloc_size - if non-0, then size of block that should be pre-allocated during memory root initialization. my_flags MY_THREAD_SPECIFIC flag for my_malloc - MY_RROOT_USE_MPROTECT for read only protected memory + MY_ROOT_USE_VMEM to use anonymos mmap instead of malloc DESCRIPTION This function prepares memory root for further use, sets initial size of @@ -159,11 +155,11 @@ void init_alloc_root(PSI_memory_key key, MEM_ROOT *mem_root, size_t block_size, mem_root->flags= 0; DBUG_ASSERT(!test_all_bits(mem_root->flags, - (MY_THREAD_SPECIFIC | MY_ROOT_USE_MPROTECT))); + (MY_THREAD_SPECIFIC | MY_ROOT_USE_VMEM))); if (my_flags & MY_THREAD_SPECIFIC) mem_root->flags|= ROOT_FLAG_THREAD_SPECIFIC; - if (my_flags & MY_ROOT_USE_MPROTECT) - mem_root->flags|= ROOT_FLAG_MPROTECT; + if (my_flags & MY_ROOT_USE_VMEM) + mem_root->flags|= ROOT_FLAG_VMEM; calculate_block_sizes(mem_root, block_size, &pre_alloc_size); @@ -288,7 +284,7 @@ void *alloc_root(MEM_ROOT *mem_root, size_t length) }); #if defined(HAVE_valgrind) && defined(EXTRA_DEBUG) - if (!(mem_root->flags & ROOT_FLAG_MPROTECT)) + if (!(mem_root->flags & ROOT_FLAG_VMEM)) { length+= ALIGN_SIZE(sizeof(USED_MEM)); if (!(next = (USED_MEM*) my_malloc(mem_root->psi_key, length, @@ -639,32 +635,20 @@ void root_free_to_savepoint(const MEM_ROOT_SAVEPOINT *sv) Change protection for all blocks in the mem root */ -#if defined(HAVE_MMAP) && defined(HAVE_MPROTECT) && defined(MAP_ANONYMOUS) void protect_root(MEM_ROOT *root, int prot) { - USED_MEM *next,*old; + USED_MEM *next; DBUG_ENTER("protect_root"); DBUG_PRINT("enter",("root: %p prot: %d", root, prot)); - DBUG_ASSERT(root->flags & ROOT_FLAG_MPROTECT); + DBUG_ASSERT(root->flags & ROOT_FLAG_VMEM); - for (next= root->used; next ;) - { - old= next; next= next->next ; - mprotect(old, old->size, prot); - } - for (next= root->free; next ;) - { - old= next; next= next->next ; - mprotect(old, old->size, prot); - } + for (next= root->used; next; next= next->next) + my_virtual_mem_protect(next, next->size, prot); + for (next= root->free; next; next= next->next) + my_virtual_mem_protect(next, next->size, prot); DBUG_VOID_RETURN; } -#else -void protect_root(MEM_ROOT *root, int prot) -{ -} -#endif /* defined(HAVE_MMAP) && ... */ char *strdup_root(MEM_ROOT *root, const char *str) diff --git a/mysys/my_init.c b/mysys/my_init.c index 3525365d1559c..172a97cfe9695 100644 --- a/mysys/my_init.c +++ b/mysys/my_init.c @@ -1,6 +1,7 @@ /* Copyright (c) 2000, 2012, Oracle and/or its affiliates Copyright (c) 2009, 2011, Monty Program Ab + Copyright (c) 2026, MariaDB plc. This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by @@ -58,7 +59,7 @@ my_bool my_init_done= 0; uint mysys_usage_id= 0; /* Incremented for each my_init() */ size_t my_system_page_size= 8192; /* Default if no sysconf() */ -ulonglong my_thread_stack_size= (sizeof(void*) <= 4)? 65536: ((256-16)*1024); +READ_ONLY_SYSVAR ulonglong my_thread_stack_size= (sizeof(void*) <= 4)? 65536: ((256-16)*1024); static mode_t atoi_octal(const char *str) { diff --git a/mysys/my_static.c b/mysys/my_static.c index bc90975f6329d..858e2f26605ab 100644 --- a/mysys/my_static.c +++ b/mysys/my_static.c @@ -1,5 +1,5 @@ /* Copyright (c) 2000, 2011, Oracle and/or its affiliates. - Copyright (c) 2009, 2019, MariaDB Corporation. + Copyright (c) 2009, 2026, MariaDB plc. This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by @@ -119,8 +119,8 @@ void (*proc_info_hook)(void *, const PSI_stage_info *, PSI_stage_info *, const char *, const char *, const unsigned int)= proc_info_dummy; void (*debug_sync_C_callback_ptr)(MYSQL_THD, const char *, size_t)= 0; - /* How to disable options */ -my_bool my_disable_locking=0; +/* How to disable options */ +READ_ONLY_SYSVAR my_bool my_disable_locking=0; my_bool my_disable_sync=0; my_bool my_disable_async_io=0; my_bool my_disable_flush_key_blocks=0; diff --git a/mysys/my_virtual_mem.c b/mysys/my_virtual_mem.c index 7157dd7ae0611..acd63a9b0a81c 100644 --- a/mysys/my_virtual_mem.c +++ b/mysys/my_virtual_mem.c @@ -1,4 +1,4 @@ -/* Copyright (c) 2025, MariaDB +/* Copyright (c) 2025, 2026, MariaDB plc. This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by @@ -24,19 +24,24 @@ /* Functionality for handling virtual memory - - reserve range, - - commit memory (within reserved range) - - decommit previously commited memory - - release range + - reserve range - memory is reserved, not allocated. + - commit memory (within reserved range) - memory is allocated + - decommit previously commited memory - memory is deallocated, stays reserved + - release range - reserved range is released to OS Not every OS has a "reserve" functionality, i.e it is not always possible to reserve memory larger than swap or RAM for example. - We try to respect use_large_pages setting, on Windows and Linux + The unfortunate terminology comes from WinAPI, see MEM_* flags below. + It's rather confusing in the context of a database (and specifically, + InnoDB, the only user of this functionality), but it's the established + terminology. + + We try to respect use_large_pages setting, both on Windows and Linux */ -#ifdef _WIN32 char *my_virtual_mem_reserve(size_t *size) { +#ifdef _WIN32 DWORD flags= my_use_large_pages ? MEM_LARGE_PAGES | MEM_RESERVE | MEM_COMMIT : MEM_RESERVE; @@ -49,8 +54,10 @@ char *my_virtual_mem_reserve(size_t *size) my_error(EE_OUTOFMEMORY, MYF(ME_BELL + ME_ERROR_LOG), *size); } return ptr; -} +#else + return my_large_virtual_alloc(size); #endif +} #if defined _WIN32 && !defined DBUG_OFF static my_bool is_memory_committed(char *ptr, size_t size) @@ -62,10 +69,18 @@ static my_bool is_memory_committed(char *ptr, size_t size) } #endif +/* + "commit" - in other words, allocate - memory. + ptr must lie within a previously reserved range or be NULL. + in the latter case new memory is reserved and committed. + + This is compatible with the mmap / VirtualAlloc semantics. +*/ char *my_virtual_mem_commit(char *ptr, size_t size) { - DBUG_ASSERT(ptr); #ifdef _WIN32 + if (!ptr) + return VirtualAlloc(NULL, size, MEM_RESERVE | MEM_COMMIT, PAGE_READWRITE); if (my_use_large_pages) { DBUG_ASSERT(is_memory_committed(ptr, size)); @@ -81,6 +96,12 @@ char *my_virtual_mem_commit(char *ptr, size_t size) } } #else + if (!ptr) + { + void *p= mmap(NULL, size, PROT_READ | PROT_WRITE, + MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0); + return p == MAP_FAILED ? NULL : p; + } if (my_use_large_pages) /* my_large_virtual_alloc() already created a read/write mapping. */; else @@ -165,7 +186,6 @@ void my_virtual_mem_decommit(char *ptr, size_t size) void my_virtual_mem_release(char *ptr, size_t size) { #ifdef _WIN32 - DBUG_ASSERT(my_use_large_pages || !is_memory_committed(ptr, size)); if (!VirtualFree(ptr, 0, MEM_RELEASE)) { my_error(EE_BADMEMORYRELEASE, MYF(ME_ERROR_LOG_ONLY), ptr, size, @@ -180,3 +200,16 @@ void my_virtual_mem_release(char *ptr, size_t size) } #endif } + + +void my_virtual_mem_protect(void *ptr, size_t size, enum my_vmem_prot prot) +{ + int res __attribute__((unused)); +#ifdef _WIN32 + DWORD old_prot; + res= !VirtualProtect(ptr, size, prot ? PAGE_READWRITE : PAGE_READONLY, &old_prot); +#else + res= mprotect(ptr, size, prot ? PROT_READ | PROT_WRITE : PROT_READ); +#endif + DBUG_ASSERT(res == 0); // bad ptr or size alignment, neither should be possible +} diff --git a/mysys/thr_lock.c b/mysys/thr_lock.c index de9828069efed..0fa20600a9cb3 100644 --- a/mysys/thr_lock.c +++ b/mysys/thr_lock.c @@ -1605,7 +1605,6 @@ int lock_counts[]= {sizeof(test_0)/sizeof(struct st_test), static mysql_cond_t COND_thread_count; static mysql_mutex_t LOCK_thread_count; static uint thread_count; -static ulong sum=0; #define MAX_LOCK_COUNT 8 #define TEST_TIMEOUT 100000 @@ -1632,6 +1631,7 @@ static my_bool test_check_status(void* param __attribute__((unused))) return 0; } +#include "my_cpu.h" static void *test_thread(void *arg) { @@ -1665,7 +1665,7 @@ static void *test_thread(void *arg) { ulong k; for (k=0 ; k < (ulong) (tmp-2)*100000L ; k++) - sum+=k; + MY_RELAX_CPU(); } } mysql_mutex_unlock(&LOCK_thread_count); diff --git a/mysys/thr_mutex.c b/mysys/thr_mutex.c index 622e44a40f1c2..11d1168a03e16 100644 --- a/mysys/thr_mutex.c +++ b/mysys/thr_mutex.c @@ -1,6 +1,7 @@ /* Copyright (c) 2000, 2011, Oracle and/or its affiliates. Copyright (c) 2010, 2011, Monty Program Ab + Copyright (c) 2026, MariaDB plc. This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by @@ -56,7 +57,7 @@ pthread_mutexattr_t my_errorcheck_mutexattr; static pthread_mutex_t THR_LOCK_mutex; static ulong safe_mutex_count= 0; /* Number of mutexes created */ static ulong safe_mutex_id= 0; -my_bool safe_mutex_deadlock_detector= 1; /* On by default */ +READ_ONLY_SYSVAR my_bool safe_mutex_deadlock_detector= 1; /* On by default */ #ifdef SAFE_MUTEX_DETECT_DESTROY static struct st_safe_mutex_create_info_t *safe_mutex_create_root= NULL; diff --git a/plugin/auth_mysql_sha2/mysql-test/mysql_sha2/ssl_auto.result b/plugin/auth_mysql_sha2/mysql-test/mysql_sha2/ssl_auto.result index 4d45bc89c5053..42964a7d8d2cb 100644 --- a/plugin/auth_mysql_sha2/mysql-test/mysql_sha2/ssl_auto.result +++ b/plugin/auth_mysql_sha2/mysql-test/mysql_sha2/ssl_auto.result @@ -23,7 +23,7 @@ test2@localhost test2@% 1 # mysql -utest2 -ppwd --disable-ssl-verify-server-cert -e "call test.checkme()" ERROR 1045 (28000): Access denied for user 'test2'@'localhost' (using password: YES) # mysql -utest1 -ppwd --ssl-verify-server-cert -e "call test.checkme()" -ERROR 2026 (HY000): TLS/SSL error: Certificate verification failure: The certificate is NOT trusted. +ERROR 2026 (HY000): TLS/SSL error: Failed to verify the server certificate drop procedure checkme; drop user test1@'%'; drop user test2@'%'; diff --git a/plugin/file_key_management/parser.cc b/plugin/file_key_management/parser.cc index e8aecd3500b82..51fc8348eec89 100644 --- a/plugin/file_key_management/parser.cc +++ b/plugin/file_key_management/parser.cc @@ -127,7 +127,7 @@ bool Parser::read_filekey(const char *filekey, char *secret) } close(f); - while (secret[len - 1] == '\r' || secret[len - 1] == '\n') len--; + while (len && (secret[len - 1] == '\r' || secret[len - 1] == '\n')) len--; if (len > MAX_SECRET_SIZE) { my_printf_error(EE_READ, diff --git a/plugin/qc_info/qc_info.cc b/plugin/qc_info/qc_info.cc index 52cfaa7a9cacd..6a275550a6019 100644 --- a/plugin/qc_info/qc_info.cc +++ b/plugin/qc_info/qc_info.cc @@ -233,6 +233,7 @@ static int qc_info_fill_table(THD *thd, TABLE_LIST *tables, compile_time_assert(QUERY_CACHE_DB_LENGTH_SIZE == 2); db= key + statement_text_length + 1 + QUERY_CACHE_DB_LENGTH_SIZE; db_length= uint2korr(db - QUERY_CACHE_DB_LENGTH_SIZE); + set_if_smaller(db_length, (size_t)(key + key_length - QUERY_CACHE_FLAGS_SIZE - db)); table->field[COLUMN_STATEMENT_SCHEMA]->store(db, db_length, scs); diff --git a/plugin/type_mysql_json/mysql_json.cc b/plugin/type_mysql_json/mysql_json.cc index c28a403a6031d..6b267d404bd5e 100644 --- a/plugin/type_mysql_json/mysql_json.cc +++ b/plugin/type_mysql_json/mysql_json.cc @@ -373,7 +373,8 @@ static bool parse_mysql_scalar_or_value(String *buffer, const uchar *data, if (type_is_stored_inline(value_type, large)) { const size_t value_start = value_type_offset + 1; - if (parse_mysql_scalar(buffer, value_type, data + value_start, + if (value_start >= len || + parse_mysql_scalar(buffer, value_type, data + value_start, len - value_start)) return true; } @@ -383,7 +384,8 @@ static bool parse_mysql_scalar_or_value(String *buffer, const uchar *data, of the Object / Array */ const size_t value_start= read_offset_or_size( data + value_type_offset + 1, large); - if (parse_mysql_json_value(buffer, value_type, data + value_start, + if (value_start >= len || + parse_mysql_json_value(buffer, value_type, data + value_start, len - value_start, depth)) return true; } @@ -453,7 +455,7 @@ static bool parse_array_or_object(String *buffer, const uchar *data, size_t len, i * value_size(large)); /* First print the key. */ - if (buffer->append('"') || + if (key_start + key_len > bytes || buffer->append('"') || append_string_json(buffer, data + key_start, key_len) || buffer->append(STRING_WITH_LEN("\": "))) { diff --git a/plugin/wsrep_info/mysql-test/wsrep_info/my.cnf b/plugin/wsrep_info/mysql-test/wsrep_info/my.cnf index 5adf631384354..988096e134a9f 100644 --- a/plugin/wsrep_info/mysql-test/wsrep_info/my.cnf +++ b/plugin/wsrep_info/mysql-test/wsrep_info/my.cnf @@ -26,6 +26,10 @@ wsrep_provider_options='base_port=@mysqld.2.#galera_port' wsrep_sst_receive_address='127.0.0.1:@mysqld.2.#sst_port' wsrep_node_name=test-node-2 +[sst] +# SSL mode is disabled for SST unless tests specifically enable it. +ssl-mode=DISABLED + [ENV] NODE_MYPORT_1= @mysqld.1.port NODE_MYSOCK_1= @mysqld.1.socket diff --git a/scripts/galera_new_cluster.sh b/scripts/galera_new_cluster.sh index 4d27ff94e2ad2..98d23dd82f4ed 100644 --- a/scripts/galera_new_cluster.sh +++ b/scripts/galera_new_cluster.sh @@ -21,11 +21,17 @@ EOF exit 0 fi -echo _WSREP_NEW_CLUSTER='--wsrep-new-cluster' > "@INSTALL_RUNDATADIR@/wsrep-new-cluster" && \ +# INSTALL_RUNDIR, not the socket directory - the latter is writable by +# mariadbd, which could then inject arbitrary variables into the service +# environment. Removing the file here and not in the service file, +# because that runs as mysql and cannot unlink in /run. +new_cluster=@INSTALL_RUNDIR@/mariadb-wsrep-new-cluster +trap 'rm -f "$new_cluster"' EXIT +trap 'exit 1' HUP INT TERM + +echo _WSREP_NEW_CLUSTER='--wsrep-new-cluster' > "$new_cluster" && \ systemctl restart "${1:-mariadb.service}" extcode=$? -rm -f "@INSTALL_RUNDATADIR@/wsrep-new-cluster" - exit $extcode diff --git a/scripts/mysqld_multi.sh b/scripts/mysqld_multi.sh index 316cc59536de5..fbcf47362d437 100644 --- a/scripts/mysqld_multi.sh +++ b/scripts/mysqld_multi.sh @@ -622,7 +622,8 @@ sub find_groups my ($raw_gids) = @_; my %gids; - my %groups; + my %seen_group; + my @groups; if (defined($raw_gids)) { @@ -661,7 +662,8 @@ sub find_groups # Use $2 + 0 to normalize numbers (002 + 0 -> 2) if (not defined($raw_gids) or $gids{$2 + 0}) { - $groups{"$1$2"} = 1; + push @groups, "$1$2" unless $seen_group{"$1$2"}; + $seen_group{"$1$2"}=1; } } elsif (/^\s*!include\s+(\S.*?)\s*$/) @@ -676,7 +678,7 @@ sub find_groups close CONF; } - return keys %groups; + return @groups; } #### diff --git a/scripts/wsrep_sst_mariabackup.sh b/scripts/wsrep_sst_mariabackup.sh index 8df3e22a1abf8..25f86b276f110 100644 --- a/scripts/wsrep_sst_mariabackup.sh +++ b/scripts/wsrep_sst_mariabackup.sh @@ -571,6 +571,11 @@ read_cnf() if [ "$tmode" != 'DISABLED' -o $encrypt -ge 2 ]; then check_server_ssl_config fi + # MTR test hook: simulate a missing SSL certificate and key. + if [ -n "${MTR_SST_SIMULATE_NO_SSL_CERT:-}" ]; then + tpem="" + tkey="" + fi if [ "$tmode" != 'DISABLED' ]; then if [ 0 -eq $encrypt -a -n "$tpem" -a -n "$tkey" ] then @@ -599,6 +604,16 @@ read_cnf() "CERT='$tpem', KEY='$tkey', MODE='$tmode'," \ "encrypt='$encrypt'" + # ssl-mode requires encryption but none could be set up (no usable + # cert/key): abort instead of silently transferring in cleartext. + if [ "$tmode" != 'DISABLED' -a $encrypt -eq 0 ]; then + wsrep_log_error "ssl-mode is set to '$tmode', but no usable SSL" \ + "certificate and key were found. Cannot perform an" \ + "encrypted transfer. Please configure ssl-cert and" \ + "ssl-key, or set ssl-mode to DISABLED." + exit 22 # EINVAL + fi + if [ $encrypt -ge 2 ]; then ssl_dhparams=$(parse_cnf "$encgroups" 'ssl-dhparams') fi @@ -1515,37 +1530,51 @@ else # joiner if [ -n "$WSREP_SST_OPT_BINLOG" ]; then cd "$DATA" - binlogs="" + # + # MDEV-38147: do NOT move the donor's binary log into + # place on the joiner. + # + # The donor still ships its (freshly rotated) current binary log so + # that an old joiner keeps working and a new joiner can identify + # exactly which file was sent. That file, however, only carries a + # Gtid_list, and its position can be ahead of the engine snapshot + # (BACKUP STAGE BLOCK_COMMIT does not pause commits if mariabackup + # is used for SST). With gtid_strict_mode=ON that ahead position + # makes the joiner raise error 1950 when it re-binlogs transactions + # during IST, and keeping the file could also collide with the + # joiner's own binary log numbering. + # + # Therefore the joiner removes the received binary log file(s) and + # starts a fresh binary log, seeding its GTID position from the + # storage-engine checkpoint during recovery (see + # wsrep_seed_binlog_gtid_state() in sql/log.cc) - the exact position + # from which IST resumes, which keeps the joiner's binary log in + # lockstep with the rest of the cluster. + # + binlogs=() if [ -f 'mariadb_backup_binlog_info' ]; then - NL=$'\n' while read bin_string || [ -n "$bin_string" ]; do bin_file=$(echo "$bin_string" | cut -f1) - if [ -f "$bin_file" ]; then - binlogs="$binlogs${binlogs:+$NL}$bin_file" + if [ -n "$bin_file" -a -f "$bin_file" ]; then + binlogs+=("$bin_file") fi done < 'mariadb_backup_binlog_info' else - binlogs=$(ls -d -1 "$binlog_base".[0-9]* 2>/dev/null || :) - fi - cd "$DATA_DIR" - if [ -n "$binlog_dir" -a "$binlog_dir" != '.' -a \ - "$binlog_dir" != "$DATA_DIR" ] - then - [ ! -d "$binlog_dir" ] && mkdir -p "$binlog_dir" - fi - index_dir=$(dirname "$binlog_index"); - if [ -n "$index_dir" -a "$index_dir" != '.' -a \ - "$index_dir" != "$DATA_DIR" ] - then - [ ! -d "$index_dir" ] && mkdir -p "$index_dir" + for bin_file in "$binlog_base".[0-9]*; do + [ -f "$bin_file" ] && binlogs+=("$bin_file") + done fi - if [ -n "$binlogs" ]; then - wsrep_log_info "Moving binary logs to $binlog_dir" - echo "$binlogs" | \ - while read bin_file || [ -n "$bin_file" ]; do - mv "$DATA/$bin_file" "$binlog_dir" - echo "$binlog_dir${binlog_dir:+/}$bin_file" >> "$binlog_index" + if [ ${#binlogs[@]} -ne 0 ]; then + wsrep_log_info "Removing received binary log(s) so the joiner" \ + "starts a fresh binary log seeded from the" \ + "storage-engine checkpoint" + for bin_file in "${binlogs[@]}"; do + rm -f "$DATA/$bin_file" done + else + wsrep_log_info "No binary log received from donor; the joiner" \ + "will start a fresh binary log seeded from the" \ + "storage-engine checkpoint" fi cd "$OLD_PWD" fi @@ -1565,6 +1594,38 @@ else # joiner exit 22 fi + # + # The joiner starts a fresh binary log after SST (see the MDEV-38147 + # note above), but the server does not create the log-bin directory + # itself - it must exist before mysqld reopens the binary log. When the + # binary log lives outside the datadir it may not exist yet on a freshly + # provisioned joiner, and when it lives in a subdirectory of the datadir + # the cleanup above removed it and the move stage did not restore it + # (binary logs are not part of the backup). Recreate the binary log and + # index directories here, after the move so they are not wiped again. + # Relative paths are resolved against the datadir, matching the server. + # + if [ -n "$WSREP_SST_OPT_BINLOG" ]; then + cd "$DATA_DIR" + if [ -n "$binlog_dir" -a "$binlog_dir" != '.' -a \ + "$binlog_dir" != "$DATA_DIR" -a ! -d "$binlog_dir" ] + then + wsrep_log_info "Creating the binlog directory '$binlog_dir'" + mkdir -p "$binlog_dir" + fi + if [ -n "$binlog_index" ]; then + index_dir=$(dirname "$binlog_index") + if [ -n "$index_dir" -a "$index_dir" != '.' -a \ + "$index_dir" != "$DATA_DIR" -a ! -d "$index_dir" ] + then + wsrep_log_info \ + "Creating the binlog index directory '$index_dir'" + mkdir -p "$index_dir" + fi + fi + cd "$OLD_PWD" + fi + else wsrep_log_info "'$IST_FILE' received from donor: Running IST" diff --git a/scripts/wsrep_sst_rsync.sh b/scripts/wsrep_sst_rsync.sh index a559c6749e901..faa8e592d65b0 100644 --- a/scripts/wsrep_sst_rsync.sh +++ b/scripts/wsrep_sst_rsync.sh @@ -197,14 +197,11 @@ SSTCAP="$tcap" SSLMODE=$(parse_cnf "$encgroups" 'ssl-mode' | tr '[[:lower:]]' '[[:upper:]]') if [ -z "$SSLMODE" ]; then - # Implicit verification if CA is set and the SSL mode - # is not specified by user: + # ssl-mode not set: derive it from the SSL config. Set it even when + # stunnel is absent, so the check below aborts instead of silently + # falling back to an unencrypted transfer. if [ -n "$SSTCA$SSTCAP" ]; then - STUNNEL_BIN=$(commandex 'stunnel') - if [ -n "$STUNNEL_BIN" ]; then - SSLMODE='VERIFY_CA' - fi - # Require SSL by default if SSL key and cert are present: + SSLMODE='VERIFY_CA' elif [ -n "$SSTKEY" -a -n "$SSTCERT" ]; then SSLMODE='REQUIRED' fi @@ -279,10 +276,22 @@ if [ -n "$SSLMODE" -a "$SSLMODE" != 'DISABLED' ]; then if [ -z "${STUNNEL_BIN+x}" ]; then STUNNEL_BIN=$(commandex 'stunnel') fi + # MTR test hook: simulate a missing stunnel binary. + if [ -n "${MTR_SST_SIMULATE_NO_STUNNEL:-}" ]; then + STUNNEL_BIN="" + fi if [ -n "$STUNNEL_BIN" ]; then wsrep_log_info "Using stunnel for SSL encryption: CA: '$SSTCA'," \ "CAPATH='$SSTCAP', ssl-mode='$SSLMODE'" STUNNEL="$STUNNEL_BIN $STUNNEL_CONF" + else + # Encryption required but stunnel missing: abort instead of + # silently falling back to an unencrypted transfer. + wsrep_log_error "ssl-mode is set to '$SSLMODE', but the 'stunnel'" \ + "binary was not found in the path. Cannot perform" \ + "an encrypted transfer. Please install stunnel or" \ + "set ssl-mode to DISABLED." + exit 2 # ENOENT fi fi diff --git a/sql-common/client.c b/sql-common/client.c index 72cb3efdc57ef..f401fc8d8e58f 100644 --- a/sql-common/client.c +++ b/sql-common/client.c @@ -1,5 +1,5 @@ /* Copyright (c) 2003, 2016, Oracle and/or its affiliates. - Copyright (c) 2009, 2020, MariaDB + Copyright (c) 2009, 2026, MariaDB plc. This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by @@ -1701,7 +1701,7 @@ mysql_set_character_set_with_default_collation(MYSQL *mysql) { const char *save= charsets_dir; if (mysql->options.charset_dir) - charsets_dir=mysql->options.charset_dir; + charsets_dir= mysql->options.charset_dir; if ((mysql->charset= get_charset_by_csname(mysql->options.charset_name, MY_CS_PRIMARY, MYF(MY_WME| @@ -1725,7 +1725,8 @@ mysql_set_character_set_with_default_collation(MYSQL *mysql) */ } } - charsets_dir= save; + if (mysql->options.charset_dir) + charsets_dir= save; } @@ -3837,10 +3838,12 @@ mysql_options(MYSQL *mysql,enum mysql_option option, const void *arg) my_free(mysql->options.my_cnf_group); mysql->options.my_cnf_group= opt_strdup(arg,MYF(MY_WME)); break; +#ifdef EMBEDDED_LIBRARY case MYSQL_SET_CHARSET_DIR: my_free(mysql->options.charset_dir); mysql->options.charset_dir= opt_strdup(arg,MYF(MY_WME)); break; +#endif case MYSQL_SET_CHARSET_NAME: my_free(mysql->options.charset_name); mysql->options.charset_name= opt_strdup(arg,MYF(MY_WME)); @@ -4156,7 +4159,8 @@ int STDCALL mysql_set_character_set(MYSQL *mysql, const char *cs_name) (cs= get_charset_by_csname(cs_name, MY_CS_PRIMARY, MYF(MY_UTF8_IS_UTF8MB3)))) { char buff[MY_CS_CHARACTER_SET_NAME_SIZE + 10]; - charsets_dir= save_csdir; + if (mysql->options.charset_dir) + charsets_dir= save_csdir; /* Skip execution of "SET NAMES" for pre-4.1 servers */ if (mysql_get_server_version(mysql) < 40100) return 0; @@ -4173,7 +4177,8 @@ int STDCALL mysql_set_character_set(MYSQL *mysql, const char *cs_name) set_mysql_extended_error(mysql, CR_CANT_READ_CHARSET, unknown_sqlstate, ER(CR_CANT_READ_CHARSET), cs_name, cs_dir_name); } - charsets_dir= save_csdir; + if (mysql->options.charset_dir) + charsets_dir= save_csdir; return mysql->net.last_errno; } diff --git a/sql/CMakeLists.txt b/sql/CMakeLists.txt index 32af809849be1..9ddf32cd10eab 100644 --- a/sql/CMakeLists.txt +++ b/sql/CMakeLists.txt @@ -1,5 +1,5 @@ # Copyright (c) 2006, 2014, Oracle and/or its affiliates. -# Copyright (c) 2010, 2022, MariaDB Corporation. +# Copyright (c) 2010, 2026, MariaDB plc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -369,6 +369,16 @@ ENDIF() DTRACE_INSTRUMENT_STATIC_LIBS(mariadbd "sql;mysys;mysys_ssl;${MYSQLD_STATIC_PLUGIN_LIBS}") +IF(HAVE_RO_AFTER_INIT) + GET_TARGET_PROPERTY(RO_AFTER_INIT_LDFLAGS mariadbd LINK_FLAGS) + IF(NOT RO_AFTER_INIT_LDFLAGS) + SET(RO_AFTER_INIT_LDFLAGS) + ENDIF() + SET_TARGET_PROPERTIES(mariadbd PROPERTIES LINK_FLAGS + "${RO_AFTER_INIT_LDFLAGS} -Wl,-T,${CMAKE_CURRENT_SOURCE_DIR}/mysqld_ro.lds") + SET_TARGET_PROPERTIES(mariadbd PROPERTIES + LINK_DEPENDS ${CMAKE_CURRENT_SOURCE_DIR}/mysqld_ro.lds) +ENDIF() SET(WITH_MYSQLD_LDFLAGS "" CACHE STRING "Additional linker flags for mysqld") MARK_AS_ADVANCED(WITH_MYSQLD_LDFLAGS) diff --git a/sql/handler.cc b/sql/handler.cc index 280e974e66a6f..98e9b82e76132 100644 --- a/sql/handler.cc +++ b/sql/handler.cc @@ -2614,7 +2614,7 @@ static my_xid wsrep_order_and_check_continuity(XID *list, int len) if (!wsrep_is_wsrep_xid(list + i) || wsrep_xid_seqno(list + i) != cur_seqno + 1) { - WSREP_WARN("Discovered discontinuity in recovered wsrep " + WSREP_INFO("Discovered discontinuity in recovered wsrep " "transaction XIDs. Truncating the recovery list to " "%d entries", i); break; @@ -3113,6 +3113,53 @@ static bool xarecover_handlerton(THD *, transaction_participant *hton, void *arg x <= wsrep_limit) && info->dry_run, info->dry_run)) { +#ifdef WITH_WSREP + /* + MDEV-40179: a wsrep transaction still in the prepared state at the + final recovery pass (the dry run, commit_list == 0) but they don't + have corresponding binlog events because they are no longer part of + SST. With log_bin=ON they can't be committed. + After recovering from SST without binlogs in place the joiner runs + no binlog XA recovery to commit or roll back such transactions, so + without binlog events they would abort startup with "Found N prepared + transactions!". Roll them back here; the node re-receives them + from the donor via IST. Non-wsrep (e.g. user XA) prepared transactions + are left untouched and still reported. + + Notice that in the wsrep_emulate_bin_log case below we don't need the + binlog events, so these prepared and wsrep-ordered transactions can be + safely committed. + + The guard is WSREP_PROVIDER_EXISTS ("a Galera provider is loaded"): + a node configured with a provider will rejoin and receive + these transactions; a standalone node (no provider) cannot, so + there we keep the conservative default and still report them. + */ + if (WSREP_PROVIDER_EXISTS && wsrep_is_wsrep_xid(info->list + i)) + { + int rc= hton->rollback_by_xid(info->list + i); + if (rc == 0) + { + sql_print_warning("Rolled back orphan prepared wsrep " + "transaction %lld", (longlong) x); + continue; + } + /* + A failed rollback is critical: the storage engine is left with + a transaction in the prepared state, which blocks purge and will + re-surface at the next recovery. We cannot safely continue, so + flag the error and abort startup (ha_recover() returns non-zero, + which makes the caller unireg_abort()). + */ + sql_print_error("Failed to roll back orphan prepared wsrep " + "transaction %lld during recovery (error %d). " + "The storage engine is left with a transaction in " + "the prepared state; aborting startup.", + (longlong) x, rc); + info->error= true; + break; + } +#endif /* WITH_WSREP */ info->found_my_xids++; continue; } @@ -3162,7 +3209,7 @@ static bool xarecover_handlerton(THD *, transaction_participant *hton, void *arg } } } - if (got < info->len) + if (got < info->len || info->error) break; } } diff --git a/sql/item.h b/sql/item.h index 9ca44f20cb4c3..7ae711c585f47 100644 --- a/sql/item.h +++ b/sql/item.h @@ -2305,6 +2305,7 @@ class Item :public Value_source, virtual bool get_context_for_vcol_processor(void *arg) { return 0; } virtual bool enumerate_field_refs_processor(void *arg) { return 0; } virtual bool mark_as_eliminated_processor(void *arg) { return 0; } + virtual bool unmark_as_eliminated_processor(void *arg) { return 0; } virtual bool eliminate_subselect_processor(void *arg) { return 0; } virtual bool view_used_tables_processor(void *arg) { return 0; } virtual bool eval_not_null_tables(void *arg) { return 0; } diff --git a/sql/item_cmpfunc.cc b/sql/item_cmpfunc.cc index 5538917d10eca..1c42127ffcf65 100644 --- a/sql/item_cmpfunc.cc +++ b/sql/item_cmpfunc.cc @@ -7613,6 +7613,21 @@ bool Item_equal::walk(Item_processor processor, } +/** + @brief + Subqueries that equality propagation converted to constants must not + be marked as eliminated, so unmark them. +*/ + +bool Item_equal::unmark_as_eliminated_processor(void *arg) +{ + Item *c= get_const(); + if (c) + c->walk(&Item::unmark_as_eliminated_processor, arg, 0); + return FALSE; +} + + Item *Item_equal::transform(THD *thd, Item_transformer transformer, uchar *arg) { DBUG_ASSERT(!thd->stmt_arena->is_stmt_prepare()); diff --git a/sql/item_cmpfunc.h b/sql/item_cmpfunc.h index 22a8cebca668f..cf3418401f091 100644 --- a/sql/item_cmpfunc.h +++ b/sql/item_cmpfunc.h @@ -3631,6 +3631,7 @@ class Item_equal: public Item_bool_func SEL_TREE *get_mm_tree(RANGE_OPT_PARAM *param, Item **cond_ptr) override; bool walk(Item_processor processor, void *arg, item_walk_flags flags) override; + bool unmark_as_eliminated_processor(void *arg) override; Item *transform(THD *thd, Item_transformer transformer, uchar *arg) override; void print(String *str, enum_query_type query_type) override; const Type_handler *compare_type_handler() const { return m_compare_handler; } diff --git a/sql/item_func.cc b/sql/item_func.cc index 1b91d47f3edff..07b811eba670b 100644 --- a/sql/item_func.cc +++ b/sql/item_func.cc @@ -1,5 +1,5 @@ /* Copyright (c) 2000, 2015, Oracle and/or its affiliates. - Copyright (c) 2009, 2022, MariaDB + Copyright (c) 2009, 2026, MariaDB plc. This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by @@ -4760,9 +4760,9 @@ user_var_entry *get_variable(HASH *hash, LEX_CSTRING *name, size_t size=ALIGN_SIZE(sizeof(user_var_entry))+name->length+1+extra_size; if (!my_hash_inited(hash)) return 0; - if (!(entry = (user_var_entry*) my_malloc(key_memory_user_var_entry, size, - MYF(MY_WME | ME_FATAL | - MY_THREAD_SPECIFIC)))) + + if (!(entry= (user_var_entry*) alloc_root(current_thd->user_vars_root(), + size))) return 0; entry->name.str=(char*) entry+ ALIGN_SIZE(sizeof(user_var_entry))+ extra_size; @@ -4785,10 +4785,7 @@ user_var_entry *get_variable(HASH *hash, LEX_CSTRING *name, entry->set_handler(&type_handler_long_blob); memcpy((char*) entry->name.str, name->str, name->length+1); if (my_hash_insert(hash,(uchar*) entry)) - { - my_free(entry); return 0; - } } return entry; } diff --git a/sql/item_geofunc.cc b/sql/item_geofunc.cc index 7f39c0a51d8a0..0a54368245e7e 100644 --- a/sql/item_geofunc.cc +++ b/sql/item_geofunc.cc @@ -173,7 +173,8 @@ String *Item_func_geometry_from_json::val_str(String *str) (const uchar *) js->end()); je.killed_ptr= (uint32_t *) ¤t_thd->killed; - if ((null_value= !Geometry::create_from_json(&buffer, &je, options==1, str))) + if (!json_read_value(&je) && + (null_value= !Geometry::create_from_json(&buffer, &je, options==1, str))) { int code= 0; @@ -1530,7 +1531,9 @@ bool Item_func_spatial_precise_rel::val_bool() null_value= true; goto exit; } - /* fall through */ + handle_sp_crosses_func_case(func, trn, g1, g2, + shape_a, shape_b, null_value); + break; case SP_OVERLAPS_FUNC: { // Both geometries must have the same number of dimensions. diff --git a/sql/item_jsonfunc.cc b/sql/item_jsonfunc.cc index 491c6fd3bdcf9..89ab7c468c690 100644 --- a/sql/item_jsonfunc.cc +++ b/sql/item_jsonfunc.cc @@ -572,6 +572,7 @@ void report_path_error_ex(const char *ps, json_path_t *p, Checks if the path has '.*' '[*]' or '**' constructions and sets the NO_WILDCARD_ALLOWED error if the case. */ +__attribute__((nonnull, warn_unused_result)) static int path_setup_nwc(json_path_t *p, CHARSET_INFO *i_cs, const uchar *str, const uchar *end) { @@ -586,6 +587,15 @@ static int path_setup_nwc(json_path_t *p, CHARSET_INFO *i_cs, return 1; } +static inline +CHARSET_INFO *def_path_charset(CHARSET_INFO *cs, CHARSET_INFO *alt) +{ + if (cs) return cs; + if (alt) return alt; + return &my_charset_utf8mb4_bin; +} + + bool Item_func_json_valid::fix_length_and_dec(THD *thd) { mem_root_dynamic_array_init(thd->mem_root, PSI_INSTRUMENT_MEM, @@ -643,14 +653,11 @@ bool Item_func_json_equals::val_bool() String a_tmp, b_tmp; THD *thd; - if ((null_value= args[0]->null_value || args[1]->null_value)) - return 1; - String *a= args[0]->val_json(&a_tmp); - if ((null_value= a == nullptr)) + if ((null_value= a == nullptr || args[0]->null_value)) return 1; String *b= args[1]->val_json(&b_tmp); - if ((null_value= b == nullptr)) + if ((null_value= b == nullptr || args[1]->null_value)) return 1; DYNAMIC_STRING a_res; @@ -849,18 +856,20 @@ bool Json_path_extractor::extract(MEM_ROOT *mem_root, String *str, { String *s_p= item_jp->val_str(&tmp_path); + if (!s_p) + return true; if (allow_wildcard) { - if (s_p && + if (!s_p->charset() || json_path_setup(&p, s_p->charset(), (const uchar *) s_p->ptr(), (const uchar *) s_p->ptr() + s_p->length())) error= true; } else { - if (s_p && - path_setup_nwc(&p, s_p->charset(), (const uchar *) s_p->ptr(), - (const uchar *) s_p->ptr() + s_p->length())) + if (path_setup_nwc(&p, def_path_charset(s_p->charset(), cs), + (const uchar *) s_p->ptr(), + (const uchar *) s_p->ptr() + s_p->length())) error= true; } @@ -1795,8 +1804,11 @@ bool Item_func_json_contains::val_bool() if (!path.parsed) { String *s_p= args[2]->val_str(&tmp_path); - if (s_p && - path_setup_nwc(&path.p,s_p->charset(),(const uchar *) s_p->ptr(), + if (!s_p) + goto return_null; + if (path_setup_nwc(&path.p, + def_path_charset(s_p->charset(), js->charset()), + (const uchar *) s_p->ptr(), (const uchar *) s_p->end())) { report_path_error(s_p, &path.p, 2); @@ -2402,8 +2414,11 @@ String *Item_func_json_array_append::val_str(String *str) if (!c_path->parsed) { String *s_p= args[n_arg]->val_str(tmp_paths+n_path); - if (s_p && - path_setup_nwc(&c_path->p,s_p->charset(),(const uchar *) s_p->ptr(), + if (!s_p) + goto return_null; + if (path_setup_nwc(&c_path->p, + def_path_charset(s_p->charset(), js->charset()), + (const uchar *) s_p->ptr(), (const uchar *) s_p->ptr() + s_p->length())) { report_path_error(s_p, &c_path->p, n_arg); @@ -2538,16 +2553,20 @@ String *Item_func_json_array_insert::val_str(String *str) String *s_p= args[n_arg]->val_str(tmp_paths+n_path); if (!s_p) goto return_null; + if (!s_p->charset()) + goto path_err; - if (path_setup_nwc(&c_path->p,s_p->charset(),(const uchar *) s_p->ptr(), - (const uchar *) s_p->ptr() + s_p->length()) || + if (path_setup_nwc(&c_path->p, + def_path_charset(s_p->charset(), js->charset()), + (const uchar *) s_p->ptr(), + (const uchar *) s_p->ptr() + s_p->length()) || (((json_path_step_t*) (c_path->p.steps.buffer)) + (c_path->p.last_step_idx) -1) < ((json_path_step_t*)(c_path->p.steps.buffer)) || ((json_path_step_t*)(c_path->p.steps.buffer) + c_path->p.last_step_idx)->type != JSON_PATH_ARRAY) { if (c_path->p.s.error == 0) c_path->p.s.error= SHOULD_END_WITH_ARRAY; - +path_err: report_path_error(s_p, &c_path->p, n_arg); goto return_null; @@ -3427,8 +3446,11 @@ longlong Item_func_json_length::val_int() if (!path.parsed) { String *s_p= args[1]->val_str(&tmp_path); - if (s_p && - path_setup_nwc(&path.p, s_p->charset(), (const uchar *) s_p->ptr(), + if (!s_p) + goto null_return; + if (path_setup_nwc(&path.p, + def_path_charset(s_p->charset(), js->charset()), + (const uchar *) s_p->ptr(), (const uchar *) s_p->ptr() + s_p->length())) { report_path_error(s_p, &path.p, 1); @@ -3697,7 +3719,8 @@ String *Item_func_json_insert::val_str(String *str) String *s_p= args[n_arg]->val_str(tmp_paths+n_path); if (s_p) { - if (path_setup_nwc(&c_path->p,s_p->charset(), + if (path_setup_nwc(&c_path->p, + def_path_charset(s_p->charset(), js->charset()), (const uchar *) s_p->ptr(), (const uchar *) s_p->ptr() + s_p->length())) { @@ -3708,6 +3731,8 @@ String *Item_func_json_insert::val_str(String *str) /* We search to the last step. */ c_path->p.last_step_idx--; } + else + goto return_null; c_path->parsed= c_path->constant; } if (args[n_arg]->null_value) @@ -3987,7 +4012,8 @@ String *Item_func_json_remove::val_str(String *str) st_json_path_step_t* last_step= NULL, *initial_step= NULL; if (s_p) { - if (path_setup_nwc(&c_path->p,s_p->charset(), + if (path_setup_nwc(&c_path->p, + def_path_charset(s_p->charset(), js->charset()), (const uchar *) s_p->ptr(), (const uchar *) s_p->ptr() + s_p->length())) { @@ -4015,6 +4041,8 @@ String *Item_func_json_remove::val_str(String *str) goto null_return; } } + else + goto null_return; c_path->parsed= c_path->constant; } if (args[n_arg]->null_value) @@ -4240,13 +4268,16 @@ String *Item_func_json_keys::val_str(String *str) if (!path.parsed) { String *s_p= args[1]->val_str(&tmp_path); - if (s_p && - path_setup_nwc(&path.p, s_p->charset(), (const uchar *) s_p->ptr(), + if (!s_p) + goto null_return; + if (path_setup_nwc(&path.p, + def_path_charset(s_p->charset(), js->charset()), + (const uchar *) s_p->ptr(), (const uchar *) s_p->ptr() + s_p->length())) - { - report_path_error(s_p, &path.p, 1); - goto null_return; - } + { + report_path_error(s_p, &path.p, 1); + goto null_return; + } path.parsed= path.constant; } @@ -5246,20 +5277,18 @@ int compare_nested_object(json_engine_t *js, json_engine_t *value, current_mem_root, temp_je, stack)) { value->s.error= temp_je->s.error; - value->s.c_str= temp_je->s.c_str; goto error; } if (json_normalize(&b_res, b.ptr(), b.length(), value->s.cs, current_mem_root, temp_je, stack)) { js->s.error= temp_je->s.error; - js->s.c_str= temp_je->s.c_str; goto error; } result= strcmp(a_res.str, b_res.str) ? 0 : 1; - error: +error: dynstr_free(&a_res); dynstr_free(&b_res); @@ -5267,7 +5296,8 @@ int compare_nested_object(json_engine_t *js, json_engine_t *value, } -static int json_find_overlap_with_object(json_engine_t *js, json_engine_t *value, +static int json_find_overlap_with_object(json_engine_t *js, + json_engine_t *value, bool compare_whole, MEM_ROOT *current_mem_root, json_engine_t *temp_je, diff --git a/sql/item_strfunc.cc b/sql/item_strfunc.cc index 0ab026df77ebd..553212f81506c 100644 --- a/sql/item_strfunc.cc +++ b/sql/item_strfunc.cc @@ -60,6 +60,38 @@ C_MODE_END #define FMT_HEADER_ONLY 1 #include "fmt/args.h" +/* + an allocator that implements std::allocator interface. + can be used with std or fmt. + but it uses our memory accounting + and allows to limit strings to max_allowed_packet + to be more generic needs to take MY_THREAD_SPECIFIC and PSI key as arguments, + and limit the amount of memory allocated, not just size of one allocation +*/ +template struct my_allocator { + using value_type= T; + std::size_t max_alloc; + + my_allocator(std::size_t max_alloc) : max_alloc(max_alloc) {}; + template my_allocator(const my_allocator&o) + : max_alloc(o.max_alloc) {} + + T* allocate(std::size_t n) + { + void* p= n > max_alloc / sizeof(T) + ? nullptr : my_malloc(PSI_INSTRUMENT_MEM, n * sizeof(T), + MYF(MY_THREAD_SPECIFIC)); + if (!p) + throw std::bad_alloc(); + return static_cast(p); + } + + std::size_t max_size() { return max_alloc / sizeof(T); } + void deallocate(T* p, std::size_t) noexcept { my_free(p); } + bool operator==(const my_allocator&) const noexcept { return true; } + bool operator!=(const my_allocator&) const noexcept { return false; } +}; + size_t username_char_length= USERNAME_CHAR_LENGTH; /* @@ -1514,8 +1546,6 @@ bool Item_func_sformat::fix_length_and_dec(THD *thd) if (!val_arg) return TRUE; - ulonglong char_length= 0; - uint flags= MY_COLL_ALLOW_SUPERSET_CONV | MY_COLL_ALLOW_COERCIBLE_CONV | MY_COLL_ALLOW_NUMERIC_CONV; @@ -1536,8 +1566,8 @@ bool Item_func_sformat::fix_length_and_dec(THD *thd) return TRUE; } - char_length= MAX_BLOB_WIDTH; - fix_char_length_ulonglong(char_length); + max_length= (uint32)MY_MIN((ulonglong)thd->variables.max_allowed_packet, + thd->variables.max_mem_used); return FALSE; } @@ -1569,6 +1599,8 @@ struct fmt_locale_comma : std::numpunct }; static std::locale fmt_locale(std::locale(), new fmt_locale_comma); +using fmt_buffer= fmt::basic_memory_buffer >; /* SFORMAT(format_string, ...) This function receives a formatting specification string and N parameters @@ -1617,25 +1649,19 @@ String *Item_func_sformat::val_str(String *res) } } - null_value= false; /* Create the string output */ try { -#ifdef _MSC_VER -/* - C4834 : "discarding return value of function with [[nodiscard]] attribute" - in fmt 12.1 template code, for isalpha() -*/ -#pragma warning(push) -#pragma warning(disable : 4834) -#endif - auto text = fmt::vformat(fmt_locale, fmt_arg->c_ptr_safe(), arg_store); -#ifdef _MSC_VER -#pragma warning(pop) -#endif - res->length(0); - res->set_charset(collation.collation); - res->append(text.c_str(), text.size(), fmt_arg->charset()); + /* have to use *1.5 to match fmt realloc strategy */ + fmt_buffer text{my_allocator(max_length + max_length/2)}; + fmt::vformat_to(std::back_inserter(text), fmt_locale, + fmt_arg->c_ptr_safe(), arg_store); + if (!((null_value= text.size() > max_length))) + { + res->length(0); + res->set_charset(collation.collation); + res->append(text.data(), text.size(), fmt_arg->charset()); + } } catch (const fmt::format_error &ex) { @@ -1645,6 +1671,10 @@ String *Item_func_sformat::val_str(String *res) ER_THD(thd, WARN_SFORMAT_ERROR), ex.what()); null_value= true; } + catch (const std::bad_alloc&) + { + null_value= true; // OOM + } return null_value ? NULL : res; } diff --git a/sql/item_subselect.cc b/sql/item_subselect.cc index baeb5f204c32a..d026b1f21aae9 100644 --- a/sql/item_subselect.cc +++ b/sql/item_subselect.cc @@ -371,12 +371,41 @@ bool Item_subselect::enumerate_field_refs_processor(void *arg) return FALSE; } + +/** + @brief + Set the "eliminated" flag set by mark_as_eliminated_processor(). + + @details + Table elimination marks every subquery reachable from an eliminated outer + join's ON expression as eliminated. +*/ + bool Item_subselect::mark_as_eliminated_processor(void *arg) { eliminated= TRUE; return FALSE; } +/** + @brief + Clear the "eliminated" flag set by mark_as_eliminated_processor(). + + @details + Equality propagation (build_equal_items()) can inject a reference to a + subquery that lives in another part of the query (e.g. the WHERE clause) + into that ON expression. Such a subquery still has to be executed, + so after elimination we walk the surviving expressions and clear the flag + on any subquery still referenced from them. +*/ + + +bool Item_subselect::unmark_as_eliminated_processor(void *arg) +{ + eliminated= FALSE; + return FALSE; +} + /** Remove a subselect item from its unit so that the unit no longer diff --git a/sql/item_subselect.h b/sql/item_subselect.h index 87a24f55b1f8e..127ed121e5519 100644 --- a/sql/item_subselect.h +++ b/sql/item_subselect.h @@ -239,6 +239,7 @@ class Item_subselect :public Item_result_field, item_walk_flags flags) override; bool unknown_splocal_processor(void *arg) override; bool mark_as_eliminated_processor(void *arg) override; + bool unmark_as_eliminated_processor(void *arg) override; bool eliminate_subselect_processor(void *arg) override; bool enumerate_field_refs_processor(void *arg) override; bool check_vcol_func_processor(void *arg) override diff --git a/sql/lex.h b/sql/lex.h index a8b76dab29506..709d949cd6e8e 100644 --- a/sql/lex.h +++ b/sql/lex.h @@ -281,7 +281,6 @@ SYMBOL symbols[] = { { "HELP", SYM(HELP_SYM)}, { "HIGH_PRIORITY", SYM(HIGH_PRIORITY)}, { "HISTORY", SYM(HISTORY_SYM)}, - { "HOST", SYM(HOST_SYM)}, { "HOSTS", SYM(HOSTS_SYM)}, { "HOUR", SYM(HOUR_SYM)}, { "HOUR_MICROSECOND", SYM(HOUR_MICROSECOND_SYM)}, @@ -468,7 +467,6 @@ SYMBOL symbols[] = { { "OUTFILE", SYM(OUTFILE)}, { "OVER", SYM(OVER_SYM)}, { "OVERLAPS", SYM(OVERLAPS_SYM)}, - { "OWNER", SYM(OWNER_SYM)}, { "PACKAGE", SYM(PACKAGE_MARIADB_SYM)}, { "PACK_KEYS", SYM(PACK_KEYS_SYM)}, { "PAGE", SYM(PAGE_SYM)}, @@ -602,7 +600,6 @@ SYMBOL symbols[] = { { "SLOW", SYM(SLOW)}, { "SNAPSHOT", SYM(SNAPSHOT_SYM)}, { "SMALLINT", SYM(SMALLINT)}, - { "SOCKET", SYM(SOCKET_SYM)}, { "SOFT", SYM(SOFT_SYM)}, { "SOME", SYM(ANY_SYM)}, { "SONAME", SYM(SONAME_SYM)}, diff --git a/sql/log.cc b/sql/log.cc index cb055a5e184f2..5c2e7f3a92c08 100644 --- a/sql/log.cc +++ b/sql/log.cc @@ -1,5 +1,5 @@ /* Copyright (c) 2000, 2018, Oracle and/or its affiliates. - Copyright (c) 2009, 2024, MariaDB Corporation. + Copyright (c) 2009, 2026, MariaDB plc. This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by @@ -68,6 +68,7 @@ #ifdef WITH_WSREP #include "wsrep_trans_observer.h" #include "wsrep_status.h" +#include "wsrep_xid.h" #endif /* WITH_WSREP */ #ifdef HAVE_REPLICATION @@ -84,8 +85,8 @@ LOGGER logger; -const char *log_bin_index= 0; -const char *log_bin_basename= 0; +READ_ONLY_SYSVAR const char *log_bin_index= 0; +READ_ONLY_SYSVAR const char *log_bin_basename= 0; MYSQL_BIN_LOG mysql_bin_log(&sync_binlog_period); @@ -12256,7 +12257,7 @@ ulong tc_log_page_waits= 0; static const uchar tc_log_magic[]={(uchar) 254, 0x23, 0x05, 0x74}; -ulong opt_tc_log_size; +READ_ONLY_SYSVAR ulong opt_tc_log_size; ulong tc_log_max_pages_used=0, tc_log_page_size=0, tc_log_cur_pages_used=0; int TC_LOG_MMAP::open(const char *opt_name) @@ -14618,6 +14619,76 @@ MYSQL_BIN_LOG::recover_gtid_index_abort(Gtid_index_writer *gi) } +#if defined(WITH_WSREP) && defined(HAVE_REPLICATION) +/* + MDEV-38147: A Galera mariabackup SST no longer ships the donor's binary log + (the only thing it carried was a Gtid_list whose position was ahead of the + snapshot, causing error 1950). Instead the joiner starts a fresh binary log, + so its Gtid_list / @@gtid_binlog_pos must be seeded from the recovered wsrep + position - otherwise the joiner would report an empty binlog position until + it re-binlogs new transactions, which breaks its use as an async master. + + The wsrep cluster position lives in the storage-engine checkpoint (restored + by the SST). Async-replica source positions live in mysql.gtid_slave_pos + (also restored from the engine) and are handled separately, so they are not + seeded here. + + This seeding applies only with wsrep_gtid_mode=ON. In that mode cluster + writes are re-tagged to wsrep_gtid_domain_id and binlogged with the cluster + seqno, so @@gtid_binlog_pos in that domain tracks the cluster seqno - which is + exactly the SE checkpoint position, and exactly the position from which IST + resumes re-binlogging. Seeding it keeps the joiner in lockstep and avoids + error 1950 from re-binlogging over an ahead position. + + With wsrep_gtid_mode=OFF cluster writes keep the node's configured + gtid_domain_id and are binlogged with a locally allocated seq_no; the cluster + seqno is not written to the binlog GTID (it lives only in + thd->wsrep_current_gtid_seqno - see the GTID assignment guarded by + wsrep_gtid_mode in wsrep_mysqld.cc). That binlog position is node-local and + unrelated to the checkpoint seqno, so there is nothing cluster-consistent to + seed: a fresh joiner just resumes its own local counter, which cannot produce + an ahead position. We therefore seed nothing in that mode. +*/ +static void wsrep_seed_binlog_gtid_state() +{ + /* + Only wsrep_gtid_mode=ON has a cluster-consistent binlog position that maps + onto the SE checkpoint (see the block comment above). + */ + if (!wsrep_gtid_mode) + return; + + wsrep_server_gtid_t const eng= wsrep_get_SE_checkpoint(); + if (eng.seqno <= 0) + return; /* not a wsrep node / no position */ + + rpl_gtid eng_gtid; + eng_gtid.domain_id= eng.domain_id; /* == wsrep_gtid_domain_id */ + eng_gtid.server_id= eng.server_id; + eng_gtid.seq_no= eng.seqno; + + rpl_gtid *cur= rpl_global_gtid_binlog_state.find_most_recent(eng_gtid.domain_id); + if (cur && cur->seq_no >= eng_gtid.seq_no) + return; /* binlog state already at or ahead of the checkpoint */ + + sql_print_information("WSREP: seeding binlog GTID state to %u-%u-%llu " + "from the storage-engine checkpoint", + eng_gtid.domain_id, eng_gtid.server_id, + (unsigned long long) eng_gtid.seq_no); + /* + Use the locking update() for consistency with the find_most_recent() read + above. With strict=false the only failure is OOM; update() will have called + my_error(ER_OUT_OF_RESOURCES), but there is no current_thd this early in + startup, so report the failure explicitly here as well. + */ + if (rpl_global_gtid_binlog_state.update(&eng_gtid, false)) + sql_print_error("WSREP: failed to seed binlog GTID state to %u-%u-%llu " + "from the storage-engine checkpoint (out of memory)", + eng_gtid.domain_id, eng_gtid.server_id, + (unsigned long long) eng_gtid.seq_no); +} +#endif /* WITH_WSREP && HAVE_REPLICATION */ + int MYSQL_BIN_LOG::do_binlog_recovery(const char *opt_name, bool do_xa_recovery) { @@ -14652,6 +14723,10 @@ MYSQL_BIN_LOG::do_binlog_recovery(const char *opt_name, bool do_xa_recovery) error= 0; } } +#if defined(WITH_WSREP) && defined(HAVE_REPLICATION) + if (!error && WSREP_PROVIDER_EXISTS) + wsrep_seed_binlog_gtid_state(); +#endif return error; } diff --git a/sql/log_event.cc b/sql/log_event.cc index 8e6de277a080e..00134a6fcef2f 100644 --- a/sql/log_event.cc +++ b/sql/log_event.cc @@ -981,7 +981,7 @@ Log_event* Log_event::read_log_event(const uchar *buf, size_t event_len, my_bool crc_check, my_bool print_errors) { - Log_event* ev; + Log_event* ev= nullptr; enum_binlog_checksum_alg alg; DBUG_ENTER("Log_event::read_log_event(char*,...)"); DBUG_ASSERT(fdle != 0); @@ -1000,15 +1000,33 @@ Log_event* Log_event::read_log_event(const uchar *buf, size_t event_len, } uint event_type= buf[EVENT_TYPE_OFFSET]; + switch (event_type) { + case FORMAT_DESCRIPTION_EVENT: + // If event is FD the descriptor is in it. + if (unlikely(get_checksum_alg(buf, event_len, &alg))) + { +#ifdef MYSQL_CLIENT + if (force_opt) + { + event_len-= BINLOG_CHECKSUM_LEN; + ev= new Unknown_log_event(buf, fdle); + goto exit; + } +#endif + *error= "Found invalid event in binary log"; + DBUG_RETURN(nullptr); + } + break; + case START_EVENT_V3: // all following START events in the current file are without checksum - if (event_type == START_EVENT_V3) (const_cast< Format_description_log_event *>(fdle))->used_checksum_alg= BINLOG_CHECKSUM_ALG_OFF; + // fall-through + default: /* CRC verification by SQL and Show-Binlog-Events master side. The caller has to provide @fdle->checksum_alg to be the last seen FD's (A) descriptor. - If event is FD the descriptor is in it. Notice, FD of the binlog can be only in one instance and therefore Show-Binlog-Events executing master side thread needs just to know the only FD's (A) value - whereas RL can contain more. @@ -1023,11 +1041,9 @@ Log_event* Log_event::read_log_event(const uchar *buf, size_t event_len, Notice, a pre-checksum FD version forces alg := BINLOG_CHECKSUM_ALG_UNDEF. */ - alg= (event_type != FORMAT_DESCRIPTION_EVENT) ? - fdle->used_checksum_alg : get_checksum_alg(buf, event_len); + alg= fdle->used_checksum_alg; // Emulate the corruption during reading an event DBUG_EXECUTE_IF("corrupt_read_log_event_char", - if (event_type != FORMAT_DESCRIPTION_EVENT) { uchar *debug_event_buf_c= const_cast(buf); int debug_cor_pos= rand() % (event_len - BINLOG_CHECKSUM_LEN); @@ -1036,23 +1052,23 @@ Log_event* Log_event::read_log_event(const uchar *buf, size_t event_len, DBUG_SET("-d,corrupt_read_log_event_char"); } ); + } if (crc_check && event_checksum_test(const_cast(buf), event_len, alg)) { #ifdef MYSQL_CLIENT - *error= "Event crc check failed! Most likely there is event corruption."; if (force_opt) { + event_len-= BINLOG_CHECKSUM_LEN; ev= new Unknown_log_event(buf, fdle); - DBUG_RETURN(ev); + goto exit; } - else - DBUG_RETURN(NULL); + *error= "Event crc check failed! Most likely there is event corruption."; #else *error= ER_THD_OR_DEFAULT(current_thd, ER_BINLOG_READ_EVENT_CHECKSUM_FAILURE); if (print_errors) sql_print_error("%s", *error); - DBUG_RETURN(NULL); #endif + DBUG_RETURN(NULL); } if (event_type > fdle->number_of_event_types && @@ -1097,14 +1113,15 @@ Log_event* Log_event::read_log_event(const uchar *buf, size_t event_len, ev= Log_event::read_log_event_no_checksum(buf, event_len, error, fdle); } +#ifdef MYSQL_CLIENT +exit: if (ev) { -#ifdef MYSQL_CLIENT ev->read_checksum_alg= alg; if (alg != BINLOG_CHECKSUM_ALG_OFF && alg != BINLOG_CHECKSUM_ALG_UNDEF) ev->read_checksum_value= uint4korr(buf + (event_len)); -#endif } +#endif DBUG_RETURN(ev); } @@ -2325,7 +2342,8 @@ Format_description_log_event(const uchar *buf, uint event_len, { DBUG_ENTER("Format_description_log_event::Format_description_log_event(char*,...)"); used_checksum_alg= BINLOG_CHECKSUM_ALG_UNDEF; - if (event_len < LOG_EVENT_MINIMAL_HEADER_LEN + ST_COMMON_HEADER_LEN_OFFSET) + if (unlikely( + event_len < LOG_EVENT_MINIMAL_HEADER_LEN + ST_POST_HEADER_LEN_OFFSET)) { server_version[0]= 0; DBUG_VOID_RETURN; @@ -2338,32 +2356,36 @@ Format_description_log_event(const uchar *buf, uint event_len, created= uint4korr(buf+ST_CREATED_OFFSET); dont_set_created= 1; - if (server_version[0] == 0) + if (unlikely(server_version[0] == 0)) DBUG_VOID_RETURN; /* sanity check */ - if ((common_header_len=buf[ST_COMMON_HEADER_LEN_OFFSET]) < LOG_EVENT_MINIMAL_HEADER_LEN) + if (unlikely( + (common_header_len=buf[ST_COMMON_HEADER_LEN_OFFSET]) < LOG_EVENT_MINIMAL_HEADER_LEN)) DBUG_VOID_RETURN; /* sanity check */ number_of_event_types= - event_len - (LOG_EVENT_MINIMAL_HEADER_LEN + ST_COMMON_HEADER_LEN_OFFSET + 1); + event_len - (LOG_EVENT_MINIMAL_HEADER_LEN + ST_POST_HEADER_LEN_OFFSET); DBUG_PRINT("info", ("common_header_len=%d number_of_event_types=%d", common_header_len, number_of_event_types)); /* If alloc fails, we'll detect it in is_valid() */ - post_header_len= (uint8*) my_memdup(PSI_INSTRUMENT_ME, - buf+ST_COMMON_HEADER_LEN_OFFSET+1, - number_of_event_types* - sizeof(*post_header_len), - MYF(0)); calc_server_version_split(); + buf+= ST_POST_HEADER_LEN_OFFSET; if (!is_version_before_checksum(&server_version_split)) { - /* the last bytes are the checksum alg desc and value (or value's room) */ + /* the last bytes are the checksum alg desc */ + if (unlikely(number_of_event_types < BINLOG_CHECKSUM_ALG_DESC_LEN)) + DBUG_VOID_RETURN; /* sanity check: But there are no last bytes. */ number_of_event_types -= BINLOG_CHECKSUM_ALG_DESC_LEN; - used_checksum_alg= (enum_binlog_checksum_alg)post_header_len[number_of_event_types]; + used_checksum_alg= (enum_binlog_checksum_alg)buf[number_of_event_types]; } else { used_checksum_alg= BINLOG_CHECKSUM_ALG_OFF; } + post_header_len= (uint8*) my_memdup(PSI_INSTRUMENT_ME, + buf, + number_of_event_types* + sizeof(*post_header_len), + MYF(0)); deduct_options_written_to_bin_log(); reset_crypto(); @@ -2477,35 +2499,50 @@ Format_description_log_event::is_version_before_checksum(const master_version_sp } /** - @param buf buffer holding serialized FD event - @param len netto (possible checksum is stripped off) length of the event buf - - @return the version-safe checksum alg descriptor where zero + @param buf buffer holding serialized FD event including the 4-byte checksum + @param len length of the event buf + @param alg output the version-safe checksum alg descriptor where zero designates no checksum, 255 - the orginator is checksum-unaware (effectively no checksum) and the actual [1-254] range alg descriptor. + @return whether this is an invalid FD event */ -enum_binlog_checksum_alg get_checksum_alg(const uchar *buf, size_t len) +bool get_checksum_alg(const uchar *buf, size_t len, + enum_binlog_checksum_alg *alg) { - enum_binlog_checksum_alg ret; + constexpr ptrdiff_t POST_HEADER_LEN_OFFSET= + LOG_EVENT_MINIMAL_HEADER_LEN + ST_POST_HEADER_LEN_OFFSET; char version[ST_SERVER_VER_LEN]; DBUG_ENTER("get_checksum_alg"); DBUG_ASSERT(buf[EVENT_TYPE_OFFSET] == FORMAT_DESCRIPTION_EVENT); + if (unlikely(len < POST_HEADER_LEN_OFFSET)) + DBUG_RETURN(true); memcpy(version, buf + LOG_EVENT_MINIMAL_HEADER_LEN + ST_SERVER_VER_OFFSET, ST_SERVER_VER_LEN); version[ST_SERVER_VER_LEN - 1]= 0; Format_description_log_event::master_version_split version_split(version); - ret= Format_description_log_event::is_version_before_checksum(&version_split) - ? BINLOG_CHECKSUM_ALG_UNDEF - : (enum_binlog_checksum_alg)buf[len - BINLOG_CHECKSUM_LEN - BINLOG_CHECKSUM_ALG_DESC_LEN]; - DBUG_ASSERT(ret == BINLOG_CHECKSUM_ALG_OFF || - ret == BINLOG_CHECKSUM_ALG_UNDEF || - ret == BINLOG_CHECKSUM_ALG_CRC32); - DBUG_RETURN(ret); + if (Format_description_log_event::is_version_before_checksum(&version_split)) + *alg= BINLOG_CHECKSUM_ALG_UNDEF; + else + { + /* + len >= POST_HEADER_LEN_OFFSET > + BINLOG_CHECKSUM_LEN + BINLOG_CHECKSUM_ALG_DESC_LEN + */ + size_t checksum_alg_offset= + len - BINLOG_CHECKSUM_LEN - BINLOG_CHECKSUM_ALG_DESC_LEN; + if (unlikely(checksum_alg_offset < POST_HEADER_LEN_OFFSET)) + DBUG_RETURN(true); + *alg= static_cast(buf[checksum_alg_offset]); + } + DBUG_ASSERT(*alg == BINLOG_CHECKSUM_ALG_OFF || + *alg == BINLOG_CHECKSUM_ALG_UNDEF || + *alg == BINLOG_CHECKSUM_ALG_CRC32); + DBUG_RETURN(false); } Start_encryption_log_event:: @@ -3296,9 +3333,17 @@ Rows_log_event::Rows_log_event(const uchar *buf, size_t event_len, case RW_V_EXTRAINFO_TAG: { /* Have an 'extra info' section, read it in */ - assert((end - pos) >= EXTRA_ROW_INFO_HDR_BYTES); + if (unlikely((end - pos) <= EXTRA_ROW_INFO_LEN_OFFSET)) + { + m_cols.bitmap= 0; + DBUG_VOID_RETURN; + } uint8 infoLen= pos[EXTRA_ROW_INFO_LEN_OFFSET]; - assert((end - pos) >= infoLen); + if (unlikely(infoLen < EXTRA_ROW_INFO_HDR_BYTES || (end-pos) < infoLen)) + { + m_cols.bitmap= 0; + DBUG_VOID_RETURN; + } /* Just store/use the first tag of this type, skip others */ if (likely(!m_extra_row_data)) { diff --git a/sql/log_event.h b/sql/log_event.h index 43a3b7ff76021..5599dc425915d 100644 --- a/sql/log_event.h +++ b/sql/log_event.h @@ -198,7 +198,7 @@ class String; #define STOP_HEADER_LEN 0 #define LOAD_HEADER_LEN (4 + 4 + 4 + 1 +1 + 4) #define SLAVE_HEADER_LEN 0 -#define START_V3_HEADER_LEN (2 + ST_SERVER_VER_LEN + 4) +#define START_V3_HEADER_LEN ST_COMMON_HEADER_LEN_OFFSET #define ROTATE_HEADER_LEN 8 // this is FROZEN (the Rotate post-header is frozen) #define INTVAR_HEADER_LEN 0 #define CREATE_FILE_HEADER_LEN 4 @@ -280,6 +280,7 @@ class String; #define ST_SERVER_VER_OFFSET 2 #define ST_CREATED_OFFSET (ST_SERVER_VER_OFFSET + ST_SERVER_VER_LEN) #define ST_COMMON_HEADER_LEN_OFFSET (ST_CREATED_OFFSET + 4) +#define ST_POST_HEADER_LEN_OFFSET (ST_COMMON_HEADER_LEN_OFFSET + 1) /* slave event post-header (this event is never written) */ @@ -6204,7 +6205,8 @@ bool slave_execute_deferred_events(THD *thd); bool event_that_should_be_ignored(const uchar *buf); bool event_checksum_test(uchar *buf, size_t event_len, enum_binlog_checksum_alg alg); -enum_binlog_checksum_alg get_checksum_alg(const uchar *buf, size_t len); +bool get_checksum_alg(const uchar *buf, size_t len, + enum_binlog_checksum_alg *alg); extern TYPELIB binlog_checksum_typelib; #ifdef WITH_WSREP enum Log_event_type wsrep_peak_event(rpl_group_info *rgi, ulonglong* event_size); diff --git a/sql/log_event_client.cc b/sql/log_event_client.cc index ab884da52c564..5676a142f1df5 100644 --- a/sql/log_event_client.cc +++ b/sql/log_event_client.cc @@ -2519,13 +2519,18 @@ bool User_var_log_event::print(FILE* file, PRINT_EVENT_INFO* print_event_info) { char str_buf[200]; int str_len= sizeof(str_buf) - 1; - int precision= (int)val[0]; - int scale= (int)val[1]; + decimal_digits_t precision= (uchar)val[0]; + decimal_digits_t scale= (uchar)val[1]; decimal_digit_t dec_buf[10]; decimal_t dec; dec.len= 10; dec.buf= dec_buf; + /* Verify decimal metadata as well as the actual length */ + if (precision == 0 || scale > precision || + val_len < decimal_bin_size(precision, scale) + 2) + goto err; + bin2decimal((uchar*) val+2, &dec, precision, scale); decimal2string(&dec, str_buf, &str_len, 0, 0, 0); str_buf[str_len]= 0; diff --git a/sql/log_event_server.cc b/sql/log_event_server.cc index b6a3b788b4737..ca089705cdc22 100644 --- a/sql/log_event_server.cc +++ b/sql/log_event_server.cc @@ -2459,7 +2459,12 @@ Format_description_log_event::to_packet(String *packet) { uchar *p; uint32 needed_length= - packet->length() + START_V3_HEADER_LEN + 1 + number_of_event_types + 1; + packet->length() + DBUG_IF("truncate_fde_common_header_len") ? + ST_COMMON_HEADER_LEN_OFFSET : ST_POST_HEADER_LEN_OFFSET; + if (!DBUG_IF("truncate_fde_post_header_len")) + needed_length += number_of_event_types; + if (!DBUG_IF("truncate_fde_used_checksum_alg")) + needed_length += BINLOG_CHECKSUM_ALG_DESC_LEN; if (packet->reserve(needed_length)) return true; p= (uchar *)packet->ptr() + packet->length();; @@ -2472,9 +2477,13 @@ Format_description_log_event::to_packet(String *packet) created= get_time(); int4store(p, created); p+= 4; - *p++= common_header_len; - memcpy(p, post_header_len, number_of_event_types); - p+= number_of_event_types; + if (!DBUG_IF("truncate_fde_common_header_len")) + *p++= common_header_len; + if (!DBUG_IF("truncate_fde_post_header_len")) + { + memcpy(p, post_header_len, number_of_event_types); + p+= number_of_event_types; + } /* if checksum is requested @@ -2500,7 +2509,8 @@ Format_description_log_event::to_packet(String *packet) (A), (V) presence in FD of the checksum-aware server makes the event 1 + 4 bytes bigger comparing to the former FD. */ - *p++= checksum_byte; + if (!DBUG_IF("truncate_fde_used_checksum_alg")) + *p++= checksum_byte; return false; } @@ -2519,7 +2529,7 @@ bool Format_description_log_event::write(Log_event_writer *writer) if (to_packet(&packet)) return true; size_t rec_size= packet.length(); - DBUG_ASSERT(needed == rec_size); + DBUG_ASSERT(needed >= rec_size); uint orig_checksum_len= writer->checksum_len; writer->checksum_len= BINLOG_CHECKSUM_LEN; @@ -4098,6 +4108,15 @@ void User_var_log_event::pack_info(Protocol* protocol) char buf2[DECIMAL_MAX_STR_LENGTH+1]; String str(buf2, sizeof(buf2), &my_charset_bin); buf.length(0); + decimal_digits_t precision= (uchar)val[0]; + decimal_digits_t scale= (uchar)val[1]; + /* Values were intentionally corrupted in User_var_log_event::write */ + DBUG_EXECUTE_IF("corrupt_user_var_decimal_precision", + precision = 2; scale= 1;); + + if (precision == 0 || scale > precision || + val_len < decimal_bin_size(precision, scale) + 2) + return; my_decimal((const uchar *) (val + 2), val[0], val[1]).to_string(&str); if (user_var_append_name_part(protocol->thd, &buf, name, name_len, m_data_type_name) || @@ -4193,6 +4212,9 @@ bool User_var_log_event::write(Log_event_writer *writer) buf2[1]= (char)dec->frac; decimal2bin((decimal_t*)val, buf2+2, buf2[0], buf2[1]); val_len= decimal_bin_size(buf2[0], buf2[1]) + 2; + /* Leave val_len honest and lie about the metadata. */ + DBUG_EXECUTE_IF("corrupt_user_var_decimal_precision", + buf2[0]= 65; buf2[1]= 0;); break; } case STRING_RESULT: @@ -4280,7 +4302,7 @@ int User_var_log_event::do_apply_event(rpl_group_info *rgi) rgi->rli->report(ERROR_LEVEL, ER_SLAVE_FATAL_ERROR, ER_THD(thd, ER_SLAVE_FATAL_ERROR), "Invalid variable length at User var event"); - return 1; + DBUG_RETURN(1); } float8get(real_val, val); it= new (thd->mem_root) Item_float(thd, real_val, 0); @@ -4293,7 +4315,7 @@ int User_var_log_event::do_apply_event(rpl_group_info *rgi) rgi->rli->report(ERROR_LEVEL, ER_SLAVE_FATAL_ERROR, ER_THD(thd, ER_SLAVE_FATAL_ERROR), "Invalid variable length at User var event"); - return 1; + DBUG_RETURN(1); } int_val= (longlong) uint8korr(val); it= new (thd->mem_root) Item_int(thd, int_val); @@ -4302,12 +4324,15 @@ int User_var_log_event::do_apply_event(rpl_group_info *rgi) break; case DECIMAL_RESULT: { - if (val_len < 3) + decimal_digits_t precision= (uchar)val[0]; + decimal_digits_t scale= (uchar)val[1]; + if (precision == 0 || scale > precision || + val_len < decimal_bin_size(precision, scale) + 2) { rgi->rli->report(ERROR_LEVEL, ER_SLAVE_FATAL_ERROR, ER_THD(thd, ER_SLAVE_FATAL_ERROR), "Invalid variable length at User var event"); - return 1; + DBUG_RETURN(1); } Item_decimal *dec= new (thd->mem_root) Item_decimal(thd, (uchar*) val+2, val[0], val[1]); it= dec; diff --git a/sql/mdl.cc b/sql/mdl.cc index 944d4cd4d4300..cf48f51ac5bb8 100644 --- a/sql/mdl.cc +++ b/sql/mdl.cc @@ -164,7 +164,7 @@ void MDL_key::init_psi_keys() #endif static bool mdl_initialized= 0; -uint mdl_instances; +READ_ONLY_SYSVAR uint mdl_instances; enum tal_status { TAL_ERROR, TAL_ACQUIRED, TAL_WAIT, TAL_NOWAIT }; diff --git a/sql/mysqld.cc b/sql/mysqld.cc index 2415be4eef5e5..ae5df9761505b 100644 --- a/sql/mysqld.cc +++ b/sql/mysqld.cc @@ -1,5 +1,5 @@ /* Copyright (c) 2000, 2015, Oracle and/or its affiliates. - Copyright (c) 2008, 2023, MariaDB + Copyright (c) 2008, 2026, MariaDB plc. This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by @@ -56,6 +56,7 @@ #include "optimizer_defaults.h" #include +#include #include #include #include "my_cpu.h" @@ -313,7 +314,7 @@ const char *my_localhost= "localhost", *delayed_user= "delayed", *slave_user= "", *wsrep_user= ""; -bool opt_large_files= sizeof(my_off_t) > 4; +READ_ONLY_SYSVAR my_bool opt_large_files; static my_bool opt_autocommit; ///< for --autocommit command-line option /* Used with --help for detailed option @@ -351,7 +352,7 @@ static char *default_character_set_name; static char *character_set_filesystem_name; static char *lc_messages; static char *lc_time_names_name; -char *my_bind_addr_str; +READ_ONLY_SYSVAR char *my_bind_addr_str; static char *default_collation_name; const char *default_storage_engine, *default_tmp_storage_engine; const char *enforced_storage_engine=NULL; @@ -373,11 +374,13 @@ char server_uid[SERVER_UID_SIZE+1]; // server uid will be written here /* Global variables */ -bool opt_bin_log, opt_bin_log_used=0, opt_ignore_builtin_innodb= 0; +READ_ONLY_SYSVAR my_bool opt_bin_log; +bool opt_bin_log_used=0; +READ_ONLY_SYSVAR my_bool opt_ignore_builtin_innodb; static bool opt_bin_log_nonempty, opt_bin_log_path; -char *opt_binlog_storage_engine= const_cast(""); +READ_ONLY_SYSVAR char *opt_binlog_storage_engine= const_cast(""); static plugin_ref opt_binlog_engine_plugin; -const char *opt_binlog_directory; +READ_ONLY_SYSVAR const char *opt_binlog_directory; handlerton *opt_binlog_engine_hton; bool opt_bin_log_compress; uint opt_bin_log_compress_min_len; @@ -389,11 +392,12 @@ static my_bool opt_abort; ulonglong log_output_options; my_bool opt_userstat_running; bool opt_error_log= IF_WIN(1,0); -bool opt_disable_networking=0, opt_skip_show_db=0; -bool opt_skip_name_resolve=0; +READ_ONLY_SYSVAR my_bool opt_disable_networking; +READ_ONLY_SYSVAR my_bool opt_skip_show_db; +READ_ONLY_SYSVAR my_bool opt_skip_name_resolve; my_bool opt_character_set_client_handshake= 1; bool opt_endinfo, using_udf_functions; -my_bool locked_in_memory; +READ_ONLY_SYSVAR my_bool locked_in_memory; bool opt_using_transactions; bool volatile abort_loop; uint volatile global_disable_checkpoint; @@ -403,6 +407,64 @@ ulong slow_start_timeout; static MEM_ROOT startup_root; MEM_ROOT read_only_root; +#if defined(HAVE_RO_AFTER_INIT) && !defined(EMBEDDED_LIBRARY) +// start and end of the ro_after_init section, to test if a variable is in it +extern char ro_after_init_start[] __attribute__((weak)); +extern char ro_after_init_end[] __attribute__((weak)); + +#elif defined(_MSC_VER) && !defined(EMBEDDED_LIBRARY) + +/* + $a/$z bracket the ro_after_init$m section (see READ_ONLY_SYSVAR in + my_global.h): MSVC's linker merges and alphabetically sorts sections by + their full "name$suffix" across every object file being linked +*/ +#pragma section("ro_after_init$a", read, write) +#pragma section("ro_after_init$z", read, write) +__declspec(allocate("ro_after_init$a")) __declspec(align(4096)) +static char ro_after_init_start_marker; +__declspec(allocate("ro_after_init$z")) __declspec(align(4096)) +static char ro_after_init_end_marker; +static char * const ro_after_init_start= &ro_after_init_start_marker; +static char * const ro_after_init_end= &ro_after_init_end_marker; + +#else + +static constexpr char *ro_after_init_start= 0; +static constexpr char *ro_after_init_end= 0; + +#endif + +/* set protection of the __ro_after_init section */ +static void set_ro_after_init_prot(enum my_vmem_prot prot) +{ + if (size_t size= ro_after_init_end - ro_after_init_start) + my_virtual_mem_protect(ro_after_init_start, size, prot); +} + +/* used for asserts, so returns TRUE if mprotect is impossible */ +bool var_is_ro_after_init(const char *addr) +{ + return ! ro_after_init_start || + (addr >= ro_after_init_start && addr < ro_after_init_end); +} + +/* + FLUSH PRIVILEGES leaves --skip-grant-tables mode by clearing opt_noacl. + But opt_noacl is READ_ONLY_SYSVAR, so briefly remove the protection for this + single write. It's done at most once and only when the server was started + with --skip-grant-tables. +*/ +void clear_opt_noacl() +{ + if (opt_noacl) + { + set_ro_after_init_prot(MY_VMEM_READWRITE); + opt_noacl= 0; + set_ro_after_init_prot(MY_VMEM_READONLY); + } +} + /** @brief 'grant_option' is used to indicate if privileges needs to be checked, in which case the lock, LOCK_grant, is used @@ -414,15 +476,15 @@ bool volatile grant_option; my_bool opt_skip_slave_start = 0; ///< If set, slave is not autostarted my_bool opt_reckless_slave = 0; -my_bool opt_enable_named_pipe= 0; +READ_ONLY_SYSVAR my_bool opt_enable_named_pipe= 0; my_bool opt_local_infile, opt_slave_compressed_protocol; my_bool opt_safe_user_create = 0; -my_bool opt_show_slave_auth_info; -my_bool opt_log_slave_updates= 0; -my_bool opt_replicate_annotate_row_events= 0; +READ_ONLY_SYSVAR my_bool opt_show_slave_auth_info; +READ_ONLY_SYSVAR my_bool opt_log_slave_updates; +READ_ONLY_SYSVAR my_bool opt_replicate_annotate_row_events; my_bool opt_mysql56_temporal_format=0, strict_password_validation= 1; -char *opt_slave_skip_errors; -char *opt_slave_transaction_retry_errors; +READ_ONLY_SYSVAR char *opt_slave_skip_errors; +READ_ONLY_SYSVAR char *opt_slave_transaction_retry_errors; /* Legacy global handlerton. These will be removed (please do not add more). @@ -434,12 +496,13 @@ handlerton *partition_hton; ulong read_only= 0, opt_readonly= 0; my_bool use_temp_pool, relay_log_purge; my_bool relay_log_recovery; -my_bool opt_sync_frm, opt_allow_suspicious_udfs; +my_bool opt_sync_frm; +READ_ONLY_SYSVAR my_bool opt_allow_suspicious_udfs; my_bool opt_secure_auth= 0; my_bool opt_require_secure_transport= 0; -char* opt_secure_file_priv; -my_bool lower_case_file_system= 0; -my_bool opt_large_pages= 0; +READ_ONLY_SYSVAR char* opt_secure_file_priv; +READ_ONLY_SYSVAR my_bool lower_case_file_system; +READ_ONLY_SYSVAR my_bool opt_large_pages; #ifdef HAVE_SOLARIS_LARGE_PAGES my_bool opt_super_large_pages= 0; #endif @@ -456,10 +519,10 @@ ulong opt_replicate_events_marked_for_skip; changed). False otherwise. */ volatile bool mqh_used = 0; -my_bool opt_noacl; +READ_ONLY_SYSVAR my_bool opt_noacl; my_bool sp_automatic_privileges= 1; -ulong opt_binlog_rows_event_max_size; +READ_ONLY_SYSVAR ulong opt_binlog_rows_event_max_size; uint opt_binlog_row_event_fragment_threshold; ulong binlog_row_metadata; my_bool opt_binlog_gtid_index= TRUE; @@ -470,12 +533,13 @@ my_bool opt_slave_sql_verify_checksum= 1; const char *binlog_format_names[]= {"MIXED", "STATEMENT", "ROW", NullS}; const char *binlog_formats_create_tmp_names[]= {"MIXED", "STATEMENT", NullS}; volatile sig_atomic_t calling_initgroups= 0; /**< Used in SIGSEGV handler. */ -uint mysqld_port, select_errors, ha_open_options; -uint mysqld_extra_port; +READ_ONLY_SYSVAR uint mysqld_port; +uint select_errors, ha_open_options; +READ_ONLY_SYSVAR uint mysqld_extra_port; uint mysqld_port_timeout; ulong delay_key_write_options; -uint protocol_version; -uint lower_case_table_names; +READ_ONLY_SYSVAR uint protocol_version; +READ_ONLY_SYSVAR uint lower_case_table_names; ulong tc_heuristic_recover= 0; Atomic_counter THD_count::count, CONNECT::count; bool shutdown_wait_for_slaves; @@ -486,10 +550,12 @@ Atomic_counter slave_open_temp_tables; */ Atomic_counter sending_new_binlog_file; ulong thread_created; -ulong back_log, connect_timeout, server_id; +READ_ONLY_SYSVAR ulong back_log; +ulong connect_timeout, server_id; ulong what_to_log; ulong slow_launch_time; -ulong open_files_limit, max_binlog_size; +READ_ONLY_SYSVAR ulong open_files_limit; +ulong max_binlog_size; ulong slave_trans_retries; ulong slave_trans_retry_interval; uint slave_net_timeout; @@ -512,7 +578,7 @@ ulonglong slave_max_statement_time; double slave_abort_blocking_timeout; ulonglong binlog_stmt_cache_size=0; ulonglong max_binlog_stmt_cache_size=0; -ulonglong test_flags; +READ_ONLY_SYSVAR ulonglong test_flags; ulonglong query_cache_size=0; ulong query_cache_limit=0; ulong executed_events=0; @@ -529,7 +595,7 @@ ulong binlog_gtid_index_hit= 0, binlog_gtid_index_miss= 0; ulong max_connections, max_connect_errors; uint max_password_errors; ulong extra_max_connections; -uint max_digest_length= 0; +READ_ONLY_SYSVAR uint max_digest_length; ulong slave_retried_transactions; ulong transactions_multi_engine; ulong rpl_transactions_multi_engine; @@ -538,7 +604,7 @@ ulonglong slave_skipped_errors; ulong feature_files_opened_with_delayed_keys= 0, feature_check_constraint= 0; ulonglong denied_connections; my_decimal decimal_zero; -long opt_secure_timestamp; +READ_ONLY_SYSVAR long opt_secure_timestamp; uint default_password_lifetime; my_bool disconnect_on_expired_password; @@ -620,14 +686,20 @@ const double log_10[] = { time_t server_start_time; -char mysql_home[FN_REFLEN], pidfile_name[FN_REFLEN], system_time_zone[30]; +READ_ONLY_SYSVAR char mysql_home[FN_REFLEN]; +READ_ONLY_SYSVAR char pidfile_name[FN_REFLEN]; +READ_ONLY_SYSVAR char system_time_zone[30]; char *default_tz_name, *opt_path; -char log_error_file[FN_REFLEN], glob_hostname[FN_REFLEN], *opt_log_basename; -char mysql_real_data_home[FN_REFLEN], - lc_messages_dir[FN_REFLEN], reg_ext[FN_EXTLEN], - mysql_charsets_dir[FN_REFLEN], - *opt_init_file, *opt_tc_log_file, *opt_ddl_recovery_file; -char *lc_messages_dir_ptr= lc_messages_dir, *log_error_file_ptr; +READ_ONLY_SYSVAR char log_error_file[FN_REFLEN]; +READ_ONLY_SYSVAR char glob_hostname[FN_REFLEN]; +char *opt_log_basename; +READ_ONLY_SYSVAR char mysql_real_data_home[FN_REFLEN]; +READ_ONLY_SYSVAR char lc_messages_dir[FN_REFLEN]; +char reg_ext[FN_EXTLEN], mysql_charsets_dir[FN_REFLEN]; +char *opt_tc_log_file, *opt_ddl_recovery_file; +READ_ONLY_SYSVAR char *opt_init_file; +READ_ONLY_SYSVAR char *lc_messages_dir_ptr= lc_messages_dir; +READ_ONLY_SYSVAR char *log_error_file_ptr; char mysql_unpacked_real_data_home[FN_REFLEN]; size_t mysql_unpacked_real_data_home_len; uint mysql_real_data_home_len, mysql_data_home_len= 1; @@ -637,16 +709,18 @@ key_map key_map_full(0); // Will be initialized later Time_zone *default_tz; -const char *mysql_real_data_home_ptr= mysql_real_data_home; +READ_ONLY_SYSVAR const char *mysql_real_data_home_ptr= mysql_real_data_home; extern "C" { -char server_version[SERVER_VERSION_LENGTH]; +READ_ONLY_SYSVAR char server_version[SERVER_VERSION_LENGTH]; } -char *server_version_ptr; -char *mysqld_unix_port, *opt_mysql_tmpdir; -ulong thread_handling; +READ_ONLY_SYSVAR char *server_version_ptr; +READ_ONLY_SYSVAR char *mysqld_unix_port; +READ_ONLY_SYSVAR char *opt_mysql_tmpdir; +READ_ONLY_SYSVAR ulong thread_handling; -my_bool encrypt_binlog; -my_bool encrypt_tmp_disk_tables, encrypt_tmp_files; +READ_ONLY_SYSVAR my_bool encrypt_binlog; +my_bool encrypt_tmp_disk_tables; +READ_ONLY_SYSVAR my_bool encrypt_tmp_files; /** name of reference on left expression in rewritten IN subquery */ const Lex_ident_column in_left_expr_name= ""_Lex_ident_column; @@ -711,20 +785,25 @@ uint temp_pool_set_next() return res; } -CHARSET_INFO *system_charset_info, *files_charset_info ; -CHARSET_INFO *system_charset_info_for_i_s; +READ_ONLY_SYSVAR CHARSET_INFO *system_charset_info; +READ_ONLY_SYSVAR CHARSET_INFO *system_charset_info_for_i_s; CHARSET_INFO *national_charset_info, *table_alias_charset; -CHARSET_INFO *character_set_filesystem; +CHARSET_INFO *character_set_filesystem, *files_charset_info; CHARSET_INFO *error_message_charset_info; MY_LOCALE *my_default_lc_messages; MY_LOCALE *my_default_lc_time_names; -SHOW_COMP_OPTION have_ssl, have_symlink, have_dlopen, have_query_cache; -SHOW_COMP_OPTION have_geometry, have_rtree_keys; -SHOW_COMP_OPTION have_crypt, have_compress; -SHOW_COMP_OPTION have_profiling; -SHOW_COMP_OPTION have_openssl; +READ_ONLY_SYSVAR SHOW_COMP_OPTION have_ssl; +READ_ONLY_SYSVAR SHOW_COMP_OPTION have_symlink; +READ_ONLY_SYSVAR SHOW_COMP_OPTION have_dlopen; +READ_ONLY_SYSVAR SHOW_COMP_OPTION have_query_cache; +READ_ONLY_SYSVAR SHOW_COMP_OPTION have_geometry; +READ_ONLY_SYSVAR SHOW_COMP_OPTION have_rtree_keys; +READ_ONLY_SYSVAR SHOW_COMP_OPTION have_crypt; +READ_ONLY_SYSVAR SHOW_COMP_OPTION have_compress; +READ_ONLY_SYSVAR SHOW_COMP_OPTION have_profiling; +READ_ONLY_SYSVAR SHOW_COMP_OPTION have_openssl; #ifndef EMBEDDED_LIBRARY static std::atomic shutdown_user; @@ -796,12 +875,16 @@ int mysqld_server_started=0, mysqld_server_initialized= 0; File_parser_dummy_hook file_parser_dummy_hook; /* replication parameters */ -uint report_port= 0; -char *master_info_file; +READ_ONLY_SYSVAR uint report_port= 0; +READ_ONLY_SYSVAR char *master_info_file; // Options do not reset to default if the default is `nullptr`, so use `auto`. char *master_heartbeat_period_str= autoset_my_option; -char *relay_log_info_file, *report_user, *report_password, *report_host; -char *opt_relay_logname = 0, *opt_relaylog_index_name=0; +READ_ONLY_SYSVAR char *relay_log_info_file; +READ_ONLY_SYSVAR char *report_user; +READ_ONLY_SYSVAR char *report_password; +READ_ONLY_SYSVAR char *report_host; +READ_ONLY_SYSVAR char *opt_relay_logname= 0; +char *opt_relaylog_index_name=0; char *opt_logname, *opt_slow_logname, *opt_bin_logname; char *opt_binlog_index_name=0; my_bool opt_binlog_legacy_event_pos= FALSE; @@ -819,9 +902,12 @@ my_bool opt_expect_abort= 0, opt_bootstrap= 0; static my_bool opt_myisam_log; static int cleanup_done; static ulong opt_specialflag; -char *mysql_home_ptr, *pidfile_name_ptr; +READ_ONLY_SYSVAR char *mysql_home_ptr; +READ_ONLY_SYSVAR char *pidfile_name_ptr; +#ifdef EMBEDDED_LIBRARY /** Initial command line arguments (count), after load_defaults().*/ static int defaults_argc; +#endif /** Initial command line arguments (arguments), after load_defaults(). This memory is allocated by @c load_defaults() and should be freed @@ -1498,7 +1584,6 @@ static void charset_error_reporter(enum loglevel level, C_MODE_END struct passwd *user_info; -static pthread_t select_thread; #endif /* OS specific variables */ @@ -1533,11 +1618,16 @@ int deny_severity = LOG_WARNING; ulong query_cache_min_res_unit= QUERY_CACHE_MIN_RESULT_DATA_SIZE; Query_cache query_cache; -my_bool opt_use_ssl = 1; -char *opt_ssl_ca= NULL, *opt_ssl_capath= NULL, *opt_ssl_cert= NULL, - *opt_ssl_cipher= NULL, *opt_ssl_key= NULL, *opt_ssl_crl= NULL, - *opt_ssl_crlpath= NULL, *opt_tls_version= NULL; -ulonglong tls_version= 0; +my_bool opt_use_ssl= 1; +READ_ONLY_SYSVAR char *opt_ssl_ca= NULL; +READ_ONLY_SYSVAR char *opt_ssl_capath= NULL; +READ_ONLY_SYSVAR char *opt_ssl_cert= NULL; +READ_ONLY_SYSVAR char *opt_ssl_cipher= NULL; +READ_ONLY_SYSVAR char *opt_ssl_key= NULL; +READ_ONLY_SYSVAR char *opt_ssl_crl= NULL; +READ_ONLY_SYSVAR char *opt_ssl_crlpath= NULL; +READ_ONLY_SYSVAR ulonglong tls_version= 0; +char *opt_tls_version= NULL; static scheduler_functions thread_scheduler_struct, extra_thread_scheduler_struct; scheduler_functions *thread_scheduler= &thread_scheduler_struct, @@ -1999,6 +2089,8 @@ static void clean_up(bool print_message, bool use_dummy_thd) if (cleanup_done++) return; /* purecov: inspected */ + set_ro_after_init_prot(MY_VMEM_READWRITE); // to allow cleanup + #ifdef HAVE_REPLICATION // We must call end_slave() as clean_up may have been called during startup end_slave(); @@ -2089,7 +2181,7 @@ static void clean_up(bool print_message, bool use_dummy_thd) mysql_library_end(); finish_client_errs(); free_root(&startup_root, MYF(0)); - protect_root(&read_only_root, PROT_READ | PROT_WRITE); + protect_root(&read_only_root, MY_VMEM_READWRITE); free_root(&read_only_root, MYF(0)); cleanup_errmsgs(); free_error_messages(); @@ -2097,13 +2189,6 @@ static void clean_up(bool print_message, bool use_dummy_thd) logger.cleanup_end(); sys_var_end(); free_charsets(); - - my_free(const_cast(log_bin_basename)); - my_free(const_cast(log_bin_index)); -#ifndef EMBEDDED_LIBRARY - my_free(const_cast(relay_log_basename)); - my_free(const_cast(relay_log_index)); -#endif free_list(opt_plugin_load_list_ptr); destroy_proxy_protocol_networks(); @@ -4005,7 +4090,7 @@ static int init_early_variables() global_status_var.global_memory_used= 0; init_alloc_root(PSI_NOT_INSTRUMENTED, &startup_root, 1024, 0, MYF(0)); init_alloc_root(PSI_NOT_INSTRUMENTED, &read_only_root, 1024, 0, - MYF(MY_ROOT_USE_MPROTECT)); + MYF(MY_ROOT_USE_VMEM)); return 0; } @@ -6001,7 +6086,6 @@ static void test_lc_time_sz() static void run_main_loop() { - select_thread=pthread_self(); mysql_mutex_lock(&LOCK_start_thread); select_thread_in_use=1; mysql_mutex_unlock(&LOCK_start_thread); @@ -6051,7 +6135,9 @@ int mysqld_main(int argc, char **argv) orig_argv= argv; my_defaults_mark_files= TRUE; load_defaults_or_exit(MYSQL_CONFIG_NAME, load_default_groups, &argc, &argv); +#ifdef EMBEDDED_LIBRARY defaults_argc= argc; +#endif defaults_argv= argv; remaining_argc= argc; remaining_argv= argv; @@ -6355,7 +6441,11 @@ int mysqld_main(int argc, char **argv) #endif /* WITH_WSREP */ /* Protect read_only_root against writes */ - protect_root(&read_only_root, PROT_READ); + move_allocated_sysvars_to_root(&read_only_root); + protect_root(&read_only_root, MY_VMEM_READONLY); + + /* Protect read-only sysvars */ + set_ro_after_init_prot(MY_VMEM_READONLY); if (opt_bootstrap) { @@ -8888,7 +8978,6 @@ mysqld_get_one_option(const struct my_option *opt, const char *argument, } break; case OPT_IGNORE_DB_DIRECTORY: - opt_ignore_db_dirs= NULL; // will be set in ignore_db_dirs_process_additions if (*argument == 0) ignore_db_dirs_reset(); else @@ -9489,46 +9578,43 @@ fn_format_relative_to_data_home(char * to, const char *name, bool is_secure_file_path(char *path) { - char buff1[FN_REFLEN], buff2[FN_REFLEN]; - size_t opt_secure_file_priv_len; - /* - All paths are secure if opt_secure_file_path is 0 - */ - if (!opt_secure_file_priv) - return TRUE; + char buf1[FN_REFLEN], buf2[FN_REFLEN]; + const char *cmp; - opt_secure_file_priv_len= strlen(opt_secure_file_priv); + if (opt_secure_file_priv) + cmp= opt_secure_file_priv; + else +#ifdef _WIN32 + return TRUE; // All paths are secure if opt_secure_file_priv is unset +#else + cmp= "/proc/"; // Check that it doesn't start with this prefix +#endif if (strlen(path) >= FN_REFLEN) return FALSE; - if (my_realpath(buff1, path, 0)) + if (my_realpath(buf1, path, 0)) { - /* - The supplied file path might have been a file and not a directory. - */ - size_t length= dirname_length(path); // Guaranteed to be < FN_REFLEN - memcpy(buff2, path, length); - buff2[length]= '\0'; - if (length == 0 || my_realpath(buff1, buff2, 0)) + /* The supplied file path might have been a file and not a directory. */ + size_t length= dirname_length(path); // Guaranteed to be < FN_REFLEN + memcpy(buf2, path, length); + buf2[length]= '\0'; + if (length == 0 || my_realpath(buf1, buf2, 0)) return FALSE; } - convert_dirname(buff2, buff1, NullS); - if (!lower_case_file_system) + convert_dirname(buf2, buf1, NullS); + + size_t cmp_len= strlen(cmp); + bool matched; + if (lower_case_file_system) { - if (strncmp(opt_secure_file_priv, buff2, opt_secure_file_priv_len)) - return FALSE; + my_bool is_prefix; + matched= !files_charset_info->strnncoll(buf2, strlen(buf2), cmp, cmp_len, + &is_prefix); } else - { - my_bool use_prefix; - if (files_charset_info->strnncoll(buff2, strlen(buff2), - opt_secure_file_priv, - opt_secure_file_priv_len, - &use_prefix)) - return FALSE; - } - return TRUE; + matched= !strncmp(cmp, buf2, cmp_len); + return opt_secure_file_priv ? matched : !matched; } diff --git a/sql/mysqld.h b/sql/mysqld.h index 9d8dcb8d76eb2..991893ee8ada9 100644 --- a/sql/mysqld.h +++ b/sql/mysqld.h @@ -1,5 +1,5 @@ /* Copyright (c) 2006, 2016, Oracle and/or its affiliates. - Copyright (c) 2010, 2021, MariaDB Corporation. + Copyright (c) 2010, 2026, MariaDB plc. This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by @@ -83,6 +83,8 @@ void unlink_thd(THD *thd); void refresh_status_legacy(THD *thd); void refresh_session_status(THD *thd); void refresh_global_status(); +void clear_opt_noacl(); +bool var_is_ro_after_init(const char *addr); bool is_secure_file_path(char *path); extern void init_net_server_extension(THD *thd); extern void handle_accepted_socket(MYSQL_SOCKET new_sock, MYSQL_SOCKET sock); @@ -107,8 +109,8 @@ extern CHARSET_INFO *character_set_filesystem; void temp_pool_clear_bit(uint bit); uint temp_pool_set_next(); -extern bool opt_large_files; -extern bool opt_bin_log, opt_error_log, opt_bin_log_compress; +extern my_bool opt_bin_log, opt_large_files; +extern bool opt_error_log, opt_bin_log_compress; extern char *opt_binlog_storage_engine; extern const char *opt_binlog_directory; extern handlerton *opt_binlog_engine_hton; @@ -116,9 +118,9 @@ extern uint opt_bin_log_compress_min_len; extern my_bool opt_log, opt_bootstrap; extern my_bool opt_support_flashback; extern ulonglong log_output_options; -extern bool opt_disable_networking, opt_skip_show_db; -extern bool opt_skip_name_resolve; -extern bool opt_ignore_builtin_innodb; +extern my_bool opt_disable_networking, opt_skip_show_db; +extern my_bool opt_skip_name_resolve; +extern my_bool opt_ignore_builtin_innodb; extern my_bool opt_character_set_client_handshake; extern my_bool debug_assert_on_not_freed_memory; extern MYSQL_PLUGIN_IMPORT bool volatile abort_loop; diff --git a/sql/mysqld_ro.lds b/sql/mysqld_ro.lds new file mode 100644 index 0000000000000..5777feeeabc32 --- /dev/null +++ b/sql/mysqld_ro.lds @@ -0,0 +1,31 @@ +/* + Copyright (c) 2026, MariaDB plc. + + This program is free software; you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation; version 2 of the License. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program; if not, write to the Free Software + Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1335 USA */ + +/* + Give the "ro_after_init" section a page-aligned span of its own, so that the + variables placed there (with READ_ONLY_SYSVAR tag) can be mprotect()-ed + once server initialization is complete, without affecting neighbouring data. +*/ +SECTIONS +{ + ro_after_init ALIGN(CONSTANT(COMMONPAGESIZE)) : + { + ro_after_init_start = .; + *(ro_after_init) + . = ALIGN(CONSTANT(COMMONPAGESIZE)); + ro_after_init_end = .; + } +} INSERT AFTER .data; diff --git a/sql/opt_range.cc b/sql/opt_range.cc index 63fd9333c9803..c8d1d8da84e14 100644 --- a/sql/opt_range.cc +++ b/sql/opt_range.cc @@ -16561,6 +16561,17 @@ int QUICK_GROUP_MIN_MAX_SELECT::get_next() else update_min_result(); } + else + { + /* + If next_min_max returns an unexpected handler error (HA_ERR_...), + then that error will be propagated to the caller of this + method by returning immediately here. + */ + if (first_res != HA_ERR_END_OF_FILE && + first_res != HA_ERR_KEY_NOT_FOUND) + DBUG_RETURN(first_res); // Unrecoverable error. + } } /* If there is no FIRST in the group, there is no LAST either. */ if ((have_last && !have_first) || @@ -16574,7 +16585,26 @@ int QUICK_GROUP_MIN_MAX_SELECT::get_next() else update_max_result(); } - /* If a LAST was found, a FIRST must have been found as well. */ + else + { + /* + If next_min_max returns an unexpected handler error (HA_ERR_...), + then that error will be propagated to the caller of this + method by returning immediately here. + */ + if (last_res != HA_ERR_END_OF_FILE && last_res != HA_ERR_KEY_NOT_FOUND) + DBUG_RETURN(last_res); // Unrecoverable error. + } + + /* + If we're computing both FIRST and LAST (have_first && have_last) then: + - Since execution has reached here, we have a record with a value for + FIRST (first_res==0). + - This means we must also have a record for LAST (last_res==0). It can + be the same record that we've found for FIRST, or a different one. + We could also have encountered an unrecoverable error while reading + the LAST record but that has been handled above. + */ DBUG_ASSERT((have_last && !have_first) || (have_last && have_first && (last_res == 0))); } @@ -16588,6 +16618,15 @@ int QUICK_GROUP_MIN_MAX_SELECT::get_next() make_prev_keypart_map(real_key_parts), HA_READ_KEY_EXACT); + /* + Set the result of checking this GROUP BY group. + 1. If we're computing a FIRST, take first_res. LAST was either not + computed at all or it has been computed successfully. Fatal errors + have already been checked. + 2. Otherwise, if we're computing a LAST only, take last_res. + 3. Otherwise, we're just enumerating GROUP BY groups and the 'result' + variable already has the outcome of the check. + */ result= have_first ? first_res : have_last ? last_res : result; } while (result == HA_ERR_KEY_NOT_FOUND || result == HA_ERR_END_OF_FILE); diff --git a/sql/opt_table_elimination.cc b/sql/opt_table_elimination.cc index 812c206540f17..90c03fba27a69 100644 --- a/sql/opt_table_elimination.cc +++ b/sql/opt_table_elimination.cc @@ -625,6 +625,75 @@ void add_module_expr(Dep_analysis_context *dac, Dep_module_expr **eq_mod, /*****************************************************************************/ +/** + @brief + Clear the "eliminated" flag on every subquery reachable from the given + expression (see Item_subselect::unmark_as_eliminated_processor). +*/ + +static inline void unmark_eliminated_subqueries(Item *expr) +{ + if (expr) + expr->walk(&Item::unmark_as_eliminated_processor, NULL, 0); +} + + +/** + @brief + Clear the "eliminated" flag on subqueries reachable from ON expressions of + outer joins that were NOT eliminated. Such an ON expression is still + evaluated at runtime. +*/ + +static void unmark_live_on_expr_subqueries(JOIN *join, + List *join_list) +{ + TABLE_LIST *tbl; + List_iterator it(*join_list); + while ((tbl= it++)) + { + if (tbl->nested_join) + { + unmark_live_on_expr_subqueries(join, &tbl->nested_join->join_list); + /* Only sweep this nest's ON expr if some of its tables survived. */ + if (tbl->nested_join->used_tables & ~join->eliminated_tables) + unmark_eliminated_subqueries(tbl->on_expr); + } + else if (tbl->table && !(tbl->table->map & join->eliminated_tables)) + unmark_eliminated_subqueries(tbl->on_expr); + } +} + + +/** + @brief + Equality propagation (build_equal_items()) can inject into the ON + expression of an eliminated outer join, a reference to a subquery that + actually lives in another (surviving) part of the query. + mark_as_eliminated() then flags that subquery as eliminated even though + it still has to be executed. + Clear the flag on every subquery that is still reachable. +*/ + +static void unmark_still_reachable(JOIN *join) +{ + unmark_eliminated_subqueries(join->conds); + unmark_eliminated_subqueries(join->having); + + List_iterator live_it(join->fields_list); + Item *item; + while ((item= live_it++)) + unmark_eliminated_subqueries(item); + + for (ORDER *ord= join->order; ord; ord= ord->next) + unmark_eliminated_subqueries(*(ord->item)); + + for (ORDER *ord= join->group_list; ord; ord= ord->next) + unmark_eliminated_subqueries(*(ord->item)); + + unmark_live_on_expr_subqueries(join, join->join_list); +} + /* Perform table elimination @@ -764,6 +833,7 @@ void eliminate_tables(JOIN *join) /* There are some tables that we probably could eliminate. Try it. */ eliminate_tables_for_list(join, join->join_list, all_tables, NULL, used_tables, &trace_eliminated_tables); + unmark_still_reachable(join); } DBUG_VOID_RETURN; } diff --git a/sql/opt_trace.cc b/sql/opt_trace.cc index 4a93ccebf5fd2..12ef16aff6d8e 100644 --- a/sql/opt_trace.cc +++ b/sql/opt_trace.cc @@ -201,10 +201,9 @@ void opt_trace_disable_if_no_security_context_access(THD *thd) existing connection, per the manual. */ if (!(thd->main_security_ctx.check_access(GLOBAL_ACLS & ~GRANT_ACL)) && - (0 != strcmp(thd->main_security_ctx.priv_user, - thd->security_context()->priv_user) || - !Lex_ident_host(Lex_cstring_strlen(thd->main_security_ctx.priv_host)). - streq(Lex_cstring_strlen(thd->security_context()->priv_host)))) + !thd->main_security_ctx.is_priv_user( + Lex_cstring_strlen(thd->security_context()->priv_user), + Lex_cstring_strlen(thd->security_context()->priv_host))) trace->missing_privilege(); } diff --git a/sql/privilege.h b/sql/privilege.h index 4361ee7e31a5e..e16c75f16deb5 100644 --- a/sql/privilege.h +++ b/sql/privilege.h @@ -666,11 +666,10 @@ constexpr privilege_t PRIV_STMT_SHOW_CREATE_SERVER= FEDERATED_ADMIN_ACL; /* Privileges related to processes */ -constexpr privilege_t PRIV_COM_PROCESS_INFO= PROCESS_ACL; -// This privilege applies both for SHOW EXPLAIN and SHOW ANALYZE -constexpr privilege_t PRIV_STMT_SHOW_EXPLAIN= PROCESS_ACL; constexpr privilege_t PRIV_STMT_SHOW_ENGINE_STATUS= PROCESS_ACL; constexpr privilege_t PRIV_STMT_SHOW_ENGINE_MUTEX= PROCESS_ACL; +// This privilege is used in thd_visible_in_processlist() and thus applies +// to SHOW PROCESSLIST, I_S.PROCESSLIST, SHOW EXPLAIN and SHOW ANALYZE constexpr privilege_t PRIV_STMT_SHOW_PROCESSLIST= PROCESS_ACL; diff --git a/sql/set_var.cc b/sql/set_var.cc index fcd3d82150d47..4fe16d54ebf85 100644 --- a/sql/set_var.cc +++ b/sql/set_var.cc @@ -1,5 +1,6 @@ /* Copyright (c) 2002, 2013, Oracle and/or its affiliates. Copyright (c) 2008, 2014, SkySQL Ab. + Copyright (c) 2026, MariaDB plc. This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by @@ -185,6 +186,10 @@ sys_var::sys_var(sys_var_chain *chain, const char *name_arg, option.var_type= flags & AUTO_SET ? GET_AUTO : 0; option.deprecation_substitute= substitute; + /* only check readonly, global, not-plugin sysvars */ + SYSVAR_ASSERT(!is_readonly() || scope() != GLOBAL || offset == 0 || + var_is_ro_after_init((char*)global_var_ptr())); + if (chain->last) chain->last->next= this; else @@ -194,6 +199,35 @@ sys_var::sys_var(sys_var_chain *chain, const char *name_arg, test_load= &static_test_load; } +/* + move a read-only global char* value to a memroot and reset ALLOCATED flag. + used before mprotect()-ing the memroot from changes. +*/ +void sys_var::move_charptr_to_root(MEM_ROOT *root) +{ + char **ptr= (char**)global_var_ptr(); + if (!is_readonly() || scope() != GLOBAL || cast_pluginvar() || is_struct() + || show_type() != SHOW_CHAR_PTR || !*ptr || var_is_ro_after_init(*ptr)) + return; + + char *old_val= *ptr; + *ptr= strdup_root(root, old_val); + if (flags & ALLOCATED) + { + my_free(old_val); + flags&= ~ALLOCATED; + } +} + + +void move_allocated_sysvars_to_root(MEM_ROOT *root) +{ + for (uint i= 0; i < system_variable_hash.records; i++) + ((sys_var*) my_hash_element(&system_variable_hash, i)) + ->move_charptr_to_root(root); +} + + bool sys_var::update(THD *thd, set_var *var) { enum_var_type type= var->type; @@ -757,6 +791,9 @@ int sql_set_variables(THD *thd, List *var_list, bool free) error|= var->update(thd); // Returns 0, -1 or 1 } + DBUG_EXECUTE_IF("set_skip_name_resolve", opt_skip_name_resolve= 0;); + DBUG_EXECUTE_IF("set_version_buf", server_version_ptr[0]= '2';); + err: if (free) free_underlaid_joins(thd, thd->lex->first_select_lex()); diff --git a/sql/set_var.h b/sql/set_var.h index edc1970e1420c..805e389c3b324 100644 --- a/sql/set_var.h +++ b/sql/set_var.h @@ -1,7 +1,7 @@ #ifndef SET_VAR_INCLUDED #define SET_VAR_INCLUDED /* Copyright (c) 2002, 2013, Oracle and/or its affiliates. - Copyright (c) 2009, 2020, MariaDB + Copyright (c) 2009, 2026, MariaDB plc. This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by @@ -115,6 +115,7 @@ class sys_var: protected Value_source // for double_from_string_with_check virtual sys_var_pluginvar *cast_pluginvar() { return 0; } bool check(THD *thd, set_var *var); + void move_charptr_to_root(MEM_ROOT *root); const uchar *value_ptr(THD *thd, enum_var_type type, const LEX_CSTRING *base) const; /** @@ -137,8 +138,6 @@ class sys_var: protected Value_source // for double_from_string_with_check return system_charset_info_for_i_s; } bool is_readonly() const { return flags & READONLY; } - void update_flags(int new_flags) { flags = new_flags; } - int get_flags() const { return flags; } /** the following is only true for keycache variables, that support the syntax @@keycache_name.variable_name @@ -271,6 +270,21 @@ class sys_var: protected Value_source // for double_from_string_with_check friend class Session_tracker; }; +/* + special assert for sysvars. Tells the name of the variable, + and fails even in non-debug builds. + + It is supposed to be used *only* in Sys_var* constructors, + and has name_arg hard-coded to prevent incorrect usage. +*/ +#define SYSVAR_ASSERT(X) \ + while(!(X)) \ + { \ + fprintf(stderr, "Sysvar '%s' failed '%s'\n", name_arg, #X); \ + DBUG_ASSERT(0); \ + exit(255); \ + } + #include "sql_plugin.h" /* SHOW_HA_ROWS, SHOW_MY_BOOL */ @@ -447,6 +461,7 @@ ulong get_system_variable_hash_records(void); ulonglong get_system_variable_hash_version(void); SHOW_VAR* enumerate_sys_vars(THD *thd, bool sorted, enum enum_var_type type); +void move_allocated_sysvars_to_root(MEM_ROOT *root); int fill_sysvars(THD *thd, TABLE_LIST *tables, COND *cond); sys_var *find_sys_var(THD *thd, const char *str, size_t length= 0, diff --git a/sql/signal_handler.cc b/sql/signal_handler.cc index 8f03b7306c613..873675d6ea4e6 100644 --- a/sql/signal_handler.cc +++ b/sql/signal_handler.cc @@ -1,5 +1,5 @@ /* Copyright (c) 2011, 2012, Oracle and/or its affiliates. - Copyright (c) 2011, 2021, MariaDB Corporation. + Copyright (c) 2011, 2026, MariaDB plc. This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by @@ -203,9 +203,11 @@ extern "C" sig_handler handle_fatal_signal(int sig) "Please include the information from the server start above, to the end of the\n" "information below.\n\n"); - set_server_version(server_version, sizeof(server_version)); + /* print the real version on crash */ + char real_version[SERVER_VERSION_LENGTH]; + set_server_version(real_version, sizeof(real_version)); my_safe_printf_stderr("Server version: %s source revision: %s\n\n", - server_version, SOURCE_REVISION); + real_version, SOURCE_REVISION); #ifdef WITH_WSREP Wsrep_server_state::handle_fatal_signal(); diff --git a/sql/slave.cc b/sql/slave.cc index 5521c1b81c168..1a4a302afef1f 100644 --- a/sql/slave.cc +++ b/sql/slave.cc @@ -1,5 +1,5 @@ /* Copyright (c) 2000, 2017, Oracle and/or its affiliates. - Copyright (c) 2009, 2022, MariaDB Corporation + Copyright (c) 2009, 2026, MariaDB plc. This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by @@ -81,14 +81,14 @@ uint *slave_transaction_retry_errors; uint slave_transaction_retry_error_length= 0; char slave_transaction_retry_error_names[SHOW_VAR_FUNC_BUFF_SIZE]; -char* slave_load_tmpdir = 0; +READ_ONLY_SYSVAR char* slave_load_tmpdir; Master_info *active_mi= 0; -my_bool replicate_same_server_id; -ulonglong relay_log_space_limit = 0; +READ_ONLY_SYSVAR my_bool replicate_same_server_id; +READ_ONLY_SYSVAR ulonglong relay_log_space_limit; ulonglong opt_read_binlog_speed_limit = 0; -const char *relay_log_index= 0; -const char *relay_log_basename= 0; +READ_ONLY_SYSVAR const char *relay_log_index; +READ_ONLY_SYSVAR const char *relay_log_basename; LEX_CSTRING default_master_connection_name= { (char*) "", 0 }; @@ -2936,10 +2936,15 @@ void store_master_info(THD *thd, Master_info *mi, TABLE *table, (*field++)->store(mi->connection_name.str, mi->connection_name.length, &my_charset_bin); - mysql_mutex_lock(&mi->run_lock); + /* The SQL thread's THD is protected by rli.run_lock, the IO thread's by + run_lock. Take each separately to avoid reading a freed THD. */ + mysql_mutex_lock(&mi->rli.run_lock); THD *sql_thd= mi->rli.sql_driver_thd; + DEBUG_SYNC(thd, "hold_sss_with_run_lock"); const char *const slave_sql_running_state= sql_thd ? sql_thd->get_proc_info() : ""; + mysql_mutex_unlock(&mi->rli.run_lock); + mysql_mutex_lock(&mi->run_lock); THD *io_thd= mi->io_thd; const char *const slave_io_running_state= io_thd ? io_thd->get_proc_info() : ""; @@ -5885,7 +5890,12 @@ static int queue_event(Master_info* mi, const uchar *buf, ulong event_len) */ if (buf[EVENT_TYPE_OFFSET] == FORMAT_DESCRIPTION_EVENT) { - checksum_alg= get_checksum_alg(buf, event_len); + if (unlikely(get_checksum_alg(buf, event_len, &checksum_alg))) + { + error= ER_SLAVE_RELAY_LOG_WRITE_FAILURE; + unlock_data_lock= FALSE; + goto err; + } } else if (buf[EVENT_TYPE_OFFSET] == START_EVENT_V3) { @@ -5967,6 +5977,17 @@ static int queue_event(Master_info* mi, const uchar *buf, ulong event_len) goto err; case ROTATE_EVENT: { + /* + This is normally done in Log_event::read_log_event(), + but we bypass it here because it's expensive and costs dynamic memory. + */ + if (unlikely(ROTATE_EVENT > + mi->rli.relay_log.description_event_for_queue->number_of_event_types)) + { + // The current FDE does not support `ROTATE_EVENT`. + error= ER_SLAVE_RELAY_LOG_WRITE_FAILURE; + goto err; + } Rotate_log_event rev(buf, checksum_alg != BINLOG_CHECKSUM_ALG_OFF ? event_len - BINLOG_CHECKSUM_LEN : event_len, mi->rli.relay_log.description_event_for_queue); diff --git a/sql/sp_head.cc b/sql/sp_head.cc index c6d6456608aa4..531cb9ab3a028 100644 --- a/sql/sp_head.cc +++ b/sql/sp_head.cc @@ -1,6 +1,6 @@ /* Copyright (c) 2002, 2016, Oracle and/or its affiliates. - Copyright (c) 2011, 2024, MariaDB + Copyright (c) 2011, 2026, MariaDB plc. This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by @@ -3022,10 +3022,8 @@ bool check_show_routine_access(THD *thd, sp_head *sp, bool *full_access) 1, TRUE) && (tables.grant.privilege & SELECT_ACL) != NO_ACL) || /* Check if user owns the routine. */ - (!strcmp(sp->m_definer.user.str, - thd->security_ctx->priv_user) && - !strcmp(sp->m_definer.host.str, - thd->security_ctx->priv_host)) || + thd->security_ctx->is_priv_user(sp->m_definer.user, + sp->m_definer.host) || /* Check if current role or any of the sub-granted roles own the routine. */ (sp->m_definer.host.length == 0 && diff --git a/sql/spatial.cc b/sql/spatial.cc index d30cd58fbf2e6..056dacc2b8f3e 100644 --- a/sql/spatial.cc +++ b/sql/spatial.cc @@ -580,21 +580,17 @@ Geometry *Geometry::create_from_json(Geometry_buffer *buffer, json_engine_t *je, bool er_on_3D, String *res) { Class_info *ci= NULL; - const uchar *coord_start= NULL, *geom_start= NULL, - *features_start= NULL, *geometry_start= NULL; + enum t_enum { T_NONE, T_GEOMETRY, T_GEOMETRIES, T_COORD, T_FEATURE }; + enum t_enum arg= T_NONE; + json_engine_t argje, *je_arg; Geometry *result; uchar key_buf[max_keyname_len]; uint key_len; - int fcoll_type_found= 0, feature_type_found= 0; + bool feature_type_found= false; - if (check_stack_overrun(current_thd, STACK_MIN_SIZE , NULL)) + if (check_stack_overrun(current_thd, STACK_MIN_SIZE, NULL)) return NULL; - const uint32_t *killed_ptr= (uint32_t *) je->killed_ptr; - - if (json_read_value(je)) - goto err_return; - if (je->value_type != JSON_VALUE_OBJECT) { je->s.error= GEOJ_INCORRECT_GEOJSON; @@ -603,7 +599,8 @@ Geometry *Geometry::create_from_json(Geometry_buffer *buffer, while (json_scan_next(je) == 0 && je->state != JST_OBJ_END) { - DBUG_ASSERT(je->state == JST_KEY); + if (je->state != JST_KEY) + break; key_len=0; while (json_read_keyname_chr(je) == 0) @@ -632,31 +629,36 @@ Geometry *Geometry::create_from_json(Geometry_buffer *buffer, if (je->value_type == JSON_VALUE_STRING) { - if ((ci= find_class((const char *) je->value, je->value_len))) + if ((ci= find_class(reinterpret_cast(je->value), je->value_len))) { - if ((coord_start= - (ci == &geometrycollection_class) ? geom_start : coord_start)) + if ((ci == &geometrycollection_class && arg == T_GEOMETRIES) || arg == T_COORD) goto create_geom; + if (arg != T_NONE) + break; /* invalid arg present for current geometry */ } else if (je->value_len == feature_coll_type_len && my_charset_latin1.strnncoll(je->value, je->value_len, - feature_coll_type, feature_coll_type_len) == 0) + feature_coll_type, feature_coll_type_len) == 0) { /* - 'FeatureCollection' type found. Handle the 'Featurecollection'/'features' - GeoJSON construction. + 'FeatureCollection' type found. Handle the 'Featurecollection' + /'features' GeoJSON construction. */ - if (features_start) - goto handle_feature_collection; - fcoll_type_found= 1; + ci= &geometrycollection_class; + if (arg == T_FEATURE) + goto create_geom; + if (arg != T_NONE) + break; /* invalid arg present for current geometry */ } else if (je->value_len == feature_type_len && my_charset_latin1.strnncoll(je->value, je->value_len, - feature_type, feature_type_len) == 0) + feature_type, feature_type_len) == 0) { - if (geometry_start) + if (arg == T_GEOMETRY) goto handle_geometry_key; - feature_type_found= 1; + feature_type_found= true; + if (arg != T_NONE) + break; /* invalid arg present for current geometry */ } else /* can't understand the type. */ break; @@ -667,6 +669,8 @@ Geometry *Geometry::create_from_json(Geometry_buffer *buffer, else if (key_len == coord_keyname_len && memcmp(key_buf, coord_keyname, coord_keyname_len) == 0) { + if (arg != T_NONE) + break; /* previous arg unprocessed */ /* Found the "coordinates" key. Let's check it's an array and remember where it starts. @@ -676,16 +680,25 @@ Geometry *Geometry::create_from_json(Geometry_buffer *buffer, if (je->value_type == JSON_VALUE_ARRAY) { - coord_start= je->value_begin; + arg= T_COORD; if (ci && ci != &geometrycollection_class) - goto create_geom; + { + je_arg= je; + goto create_geom; + } + argje= *je; + je_arg= &argje; if (json_skip_level(je)) goto err_return; } + else + break; /* coordinates needs to be an array */ } else if (key_len == geometries_keyname_len && memcmp(key_buf, geometries_keyname, geometries_keyname_len) == 0) { + if (arg != T_NONE) + break; /* previous arg unprocessed */ /* Found the "geometries" key. Let's check it's an array and remember where it starts. @@ -695,17 +708,28 @@ Geometry *Geometry::create_from_json(Geometry_buffer *buffer, if (je->value_type == JSON_VALUE_ARRAY) { - geom_start= je->value_begin; if (ci == &geometrycollection_class) { - coord_start= geom_start; + je_arg= je; goto create_geom; } + if (ci != nullptr) + break; /* geometries only valid inside geometrycollation */ + arg= T_GEOMETRIES; + argje= *je; + je_arg= &argje; + /* skip geometries for now and search for type */ + if (json_skip_level(je)) + goto err_return; } + else + break; /* geometries needs to be an array */ } else if (key_len == features_keyname_len && memcmp(key_buf, features_keyname, features_keyname_len) == 0) { + if (arg != T_NONE) + break; /* previous arg unprocessed */ /* 'features' key found. Handle the 'Featurecollection'/'features' GeoJSON construction. @@ -714,10 +738,23 @@ Geometry *Geometry::create_from_json(Geometry_buffer *buffer, goto err_return; if (je->value_type == JSON_VALUE_ARRAY) { - features_start= je->value_begin; - if (fcoll_type_found) - goto handle_feature_collection; + if (ci == &geometrycollection_class) + { + je_arg= je; + goto create_geom; + } + if (ci != nullptr) + break; /* features only valid inside featurecollation */ + + arg= T_FEATURE; + argje= *je; + je_arg= &argje; + /* skip features for now and search for type */ + if (json_skip_level(je)) + goto err_return; } + else + break; /* feature collections needs to be an array */ } else if (key_len == geometry_keyname_len && memcmp(key_buf, geometry_keyname, geometry_keyname_len) == 0) @@ -726,12 +763,21 @@ Geometry *Geometry::create_from_json(Geometry_buffer *buffer, goto err_return; if (je->value_type == JSON_VALUE_OBJECT) { - geometry_start= je->value_begin; if (feature_type_found) + { + je_arg= je; goto handle_geometry_key; + } + if (ci != nullptr) + break; /* geometry only valid inside feature */ + arg= T_GEOMETRY; + argje= *je; + je_arg= &argje; + if (json_skip_level(je)) + goto err_return; } else - goto err_return; + break; /* geometry needs to be an object */ } else { @@ -750,31 +796,35 @@ Geometry *Geometry::create_from_json(Geometry_buffer *buffer, } goto err_return; -handle_feature_collection: - ci= &geometrycollection_class; - coord_start= features_start; - create_geom: - json_scan_start(je, je->s.cs, coord_start, je->s.str_end); - je->killed_ptr= killed_ptr; - if (res->reserve(1 + 4, 512)) goto err_return; result= (*ci->m_create_func)(buffer->data); res->q_append((char) wkb_ndr); res->q_append((uint32) result->get_class_info()->m_type_id); - if (result->init_from_json(je, er_on_3D, res)) + if (result->init_from_json(je_arg, er_on_3D, res)) + { + if (je_arg != je) /* copy error out of copied engine */ + *je= *je_arg; + goto err_return; + } + /* finish of the object scan for validation/geomcollection */ + if (json_skip_level(je)) goto err_return; return result; -handle_geometry_key: - json_scan_start(je, je->s.cs, geometry_start, je->s.str_end); - je->killed_ptr= killed_ptr; - return create_from_json(buffer, je, er_on_3D, res); +handle_geometry_key: /* feature */ + + result= create_from_json(buffer, je_arg, er_on_3D, res); + + /* skip rest of feature - can be arbitrary fields */ + if (json_skip_level(je)) + goto err_return; + return result; err_return: return NULL; } @@ -1122,7 +1172,6 @@ static int read_point_from_json(json_engine_t *je, bool er_on_3D, while (json_scan_next(je) == 0 && je->state != JST_ARRAY_END) { - DBUG_ASSERT(je->state == JST_VALUE); if (json_read_value(je)) return 1; @@ -1136,6 +1185,8 @@ static int read_point_from_json(json_engine_t *je, bool er_on_3D, n_coord++; } + if (je->s.error != 0) + return 1; if (n_coord <= 2 || !er_on_3D) return 0; je->s.error= Geometry::GEOJ_DIMENSION_NOT_SUPPORTED; @@ -1149,8 +1200,6 @@ static int read_point_from_json(json_engine_t *je, bool er_on_3D, bool Gis_point::init_from_json(json_engine_t *je, bool er_on_3D, String *wkb) { double x, y; - if (json_read_value(je)) - return TRUE; if (je->value_type != JSON_VALUE_ARRAY) { @@ -1164,6 +1213,7 @@ bool Gis_point::init_from_json(json_engine_t *je, bool er_on_3D, String *wkb) wkb->q_append(x); wkb->q_append(y); + /* Note GEOJSON RFC7946 3.3.1 - 3D possible, but not WKB? */ return FALSE; } @@ -1451,11 +1501,9 @@ bool Gis_line_string::init_from_json(json_engine_t *je, bool er_on_3D, uint32 np_pos= wkb->length(); Gis_point p; - if (json_read_value(je)) - return TRUE; - if (je->value_type != JSON_VALUE_ARRAY) { +err_geoj_incorrect: je->s.error= GEOJ_INCORRECT_GEOJSON; return TRUE; } @@ -1466,7 +1514,13 @@ bool Gis_line_string::init_from_json(json_engine_t *je, bool er_on_3D, while (json_scan_next(je) == 0 && je->state != JST_ARRAY_END) { - DBUG_ASSERT(je->state == JST_VALUE); + if (je->state != JST_VALUE) + goto err_geoj_incorrect; + + if (json_read_value(je)) + return TRUE; + if (je->value_type != JSON_VALUE_ARRAY) + goto err_geoj_incorrect; if (p.init_from_json(je, er_on_3D, wkb)) return TRUE; @@ -1986,11 +2040,9 @@ bool Gis_polygon::init_from_json(json_engine_t *je, bool er_on_3D, String *wkb) uint32 lr_pos= wkb->length(); int closed; - if (json_read_value(je)) - return TRUE; - if (je->value_type != JSON_VALUE_ARRAY) { +err_geoj_incorrect: je->s.error= GEOJ_INCORRECT_GEOJSON; return TRUE; } @@ -2002,11 +2054,18 @@ bool Gis_polygon::init_from_json(json_engine_t *je, bool er_on_3D, String *wkb) while (json_scan_next(je) == 0 && je->state != JST_ARRAY_END) { Gis_line_string ls; - DBUG_ASSERT(je->state == JST_VALUE); + if (je->state != JST_VALUE) + goto err_geoj_incorrect; + + if (json_read_value(je)) + return TRUE; + if (je->value_type != JSON_VALUE_ARRAY) + goto err_geoj_incorrect; uint32 ls_pos=wkb->length(); if (ls.init_from_json(je, er_on_3D, wkb)) return TRUE; + ls.set_data_ptr(wkb->ptr() + ls_pos, wkb->length() - ls_pos); if (ls.is_closed(&closed) || !closed) { @@ -2834,6 +2893,8 @@ uint Gis_multi_point::init_from_wkb(const char *wkb, uint len, wkbByteOrder bo, { res->q_append((char)wkb_ndr); res->q_append((uint32)wkb_point); + if ((uchar) wkb[0] > wkb_ndr) /* invalid */ + return 0; if (!p.init_from_wkb(wkb + WKB_HEADER_SIZE, POINT_DATA_SIZE, (wkbByteOrder) wkb[0], res)) return 0; @@ -2849,11 +2910,9 @@ bool Gis_multi_point::init_from_json(json_engine_t *je, bool er_on_3D, uint32 np_pos= wkb->length(); Gis_point p; - if (json_read_value(je)) - return TRUE; - if (je->value_type != JSON_VALUE_ARRAY) { +err_geoj_incorrect: je->s.error= GEOJ_INCORRECT_GEOJSON; return TRUE; } @@ -2864,7 +2923,12 @@ bool Gis_multi_point::init_from_json(json_engine_t *je, bool er_on_3D, while (json_scan_next(je) == 0 && je->state != JST_ARRAY_END) { - DBUG_ASSERT(je->state == JST_VALUE); + if (je->state != JST_VALUE) + goto err_geoj_incorrect; + if (json_read_value(je)) + return TRUE; + if (je->value_type != JSON_VALUE_ARRAY) + goto err_geoj_incorrect; if (wkb->reserve(1 + 4, 512)) return TRUE; @@ -3230,11 +3294,9 @@ bool Gis_multi_line_string::init_from_json(json_engine_t *je, bool er_on_3D, uint32 n_line_strings= 0; uint32 ls_pos= wkb->length(); - if (json_read_value(je)) - return TRUE; - if (je->value_type != JSON_VALUE_ARRAY) { +err_geoj_incorrect: je->s.error= GEOJ_INCORRECT_GEOJSON; return TRUE; } @@ -3245,9 +3307,14 @@ bool Gis_multi_line_string::init_from_json(json_engine_t *je, bool er_on_3D, while (json_scan_next(je) == 0 && je->state != JST_ARRAY_END) { - Gis_line_string ls; - DBUG_ASSERT(je->state == JST_VALUE); + if (je->state != JST_VALUE) + goto err_geoj_incorrect; + if (json_read_value(je)) + return TRUE; + if (je->value_type != JSON_VALUE_ARRAY) + goto err_geoj_incorrect; + Gis_line_string ls; if (wkb->reserve(1 + 4, 512)) return TRUE; wkb->q_append((char) wkb_ndr); @@ -3702,11 +3769,9 @@ bool Gis_multi_polygon::init_from_json(json_engine_t *je, bool er_on_3D, int np_pos= wkb->length(); Gis_polygon p; - if (json_read_value(je)) - return TRUE; - if (je->value_type != JSON_VALUE_ARRAY) { +err_geoj_incorrect: je->s.error= GEOJ_INCORRECT_GEOJSON; return TRUE; } @@ -3717,7 +3782,12 @@ bool Gis_multi_polygon::init_from_json(json_engine_t *je, bool er_on_3D, while (json_scan_next(je) == 0 && je->state != JST_ARRAY_END) { - DBUG_ASSERT(je->state == JST_VALUE); + if (je->state != JST_VALUE) + goto err_geoj_incorrect; + if (json_read_value(je)) + return TRUE; + if (je->value_type != JSON_VALUE_ARRAY) + goto err_geoj_incorrect; if (wkb->reserve(1 + 4, 512)) return TRUE; @@ -4438,6 +4508,9 @@ uint Gis_geometry_collection::init_from_wkb(const char *wkb, uint len, return 0; n_geom= wkb_get_uint(wkb, bo); + if (check_stack_overrun(current_thd, STACK_MIN_SIZE, (uchar*)&wkb_orig)) + return 1; + if (res->reserve(4, 512)) return 0; res->q_append(n_geom); @@ -4478,11 +4551,9 @@ bool Gis_geometry_collection::init_from_json(json_engine_t *je, bool er_on_3D, Geometry_buffer buffer; Geometry *g; - if (json_read_value(je)) - return TRUE; - if (je->value_type != JSON_VALUE_ARRAY) { +err_geoj_incorrect: je->s.error= GEOJ_INCORRECT_GEOJSON; return TRUE; } @@ -4493,17 +4564,17 @@ bool Gis_geometry_collection::init_from_json(json_engine_t *je, bool er_on_3D, while (json_scan_next(je) == 0 && je->state != JST_ARRAY_END) { - json_engine_t sav_je= *je; - - DBUG_ASSERT(je->state == JST_VALUE); + if (je->state == JST_VALUE) + { + if (json_read_value(je)) + return TRUE; + } + if (je->state != JST_OBJ_START) + goto err_geoj_incorrect; if (!(g= create_from_json(&buffer, je, er_on_3D, wkb))) return TRUE; - *je= sav_je; - if (json_skip_array_item(je)) - return TRUE; - n_objects++; } diff --git a/sql/sql_acl.cc b/sql/sql_acl.cc index d8a22ae6f2ab8..de8a27c99a87e 100644 --- a/sql/sql_acl.cc +++ b/sql/sql_acl.cc @@ -1,5 +1,5 @@ /* Copyright (c) 2000, 2018, Oracle and/or its affiliates. - Copyright (c) 2009, 2023, MariaDB + Copyright (c) 2009, 2026, MariaDB plc. This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by @@ -3884,7 +3884,11 @@ privilege_t acl_get(const char *host, const char *ip, [key_data_size + 1, key_data_size + MY_CS_MBMAXLEN]. */ CharBuffer key; - key.append(Lex_cstring_strlen(safe_str(ip))).append_char('\0') + /* + localhost connections have ip=0, host="localhost", roles have ip="", + host="". Fall back to host, otherwise both get the same key. + */ + key.append(Lex_cstring_strlen(safe_str(ip ? ip : host))).append_char('\0') .append(Lex_cstring_strlen(user)).append_char('\0'); tmp_db= key.end(); key.append_opt_casedn(files_charset_info, Lex_cstring_strlen(db), @@ -9711,8 +9715,7 @@ bool get_show_user(THD *thd, LEX_USER *lex_user, const char **username, { *username= lex_user->user.str; *hostname= lex_user->host.str; - do_check_access= strcmp(*username, sctx->priv_user) || - strcmp(*hostname, sctx->priv_host); + do_check_access= !sctx->is_priv_user(lex_user->user, lex_user->host); } if (do_check_access && check_access(thd, SELECT_ACL, "mysql", 0, 0, 1, 0)) @@ -12118,6 +12121,49 @@ Silence_routine_definer_errors::handle_condition( } +/* + The low level function to revoke routine privileges for the given sp handler + @param thd the thd + @param proc_privs the table mysql.proc_privs + @param sp_db the routine database + @param sp_name the routine name + @param sph the sp handler +*/ +static void sp_revoke_privileges_for_handler(THD *thd, TABLE *proc_privs, + const Lex_ident_db &sp_db, + const Lex_ident_routine &sp_name, + const Sp_handler *sph) +{ + uint counter, revoked; + HASH *hash= sph->get_priv_hash(); + do + { + for (counter= 0, revoked= 0 ; counter < hash->records ; ) + { + GRANT_NAME *grant_proc= (GRANT_NAME*) my_hash_element(hash, counter); + if (sp_db.streq(Lex_cstring_strlen(grant_proc->db)) && + sp_name.streq(Lex_cstring_strlen(grant_proc->tname))) + { + LEX_USER lex_user; + lex_user.user.str= grant_proc->user; + lex_user.user.length= strlen(grant_proc->user); + lex_user.host.str= safe_str(grant_proc->host.hostname); + lex_user.host.length= strlen(lex_user.host.str); + if (replace_routine_table(thd, grant_proc, + proc_privs, lex_user, + grant_proc->db, grant_proc->tname, + sph, ALL_KNOWN_ACL, 1) == 0) + { + revoked= 1; + continue; + } + } + counter++; + } + } while (revoked); +} + + /** Revoke privileges for all users on a stored procedure. Use an error handler that converts errors about missing grants into warnings. @@ -12140,9 +12186,7 @@ bool sp_revoke_privileges(THD *thd, const Lex_ident_routine &sp_name, const Sp_handler *sph) { - uint counter, revoked; int result; - HASH *hash= sph->get_priv_hash(); Silence_routine_definer_errors error_handler; DBUG_ENTER("sp_revoke_privileges"); @@ -12162,31 +12206,12 @@ bool sp_revoke_privileges(THD *thd, mysql_mutex_lock(&acl_cache->lock); /* Remove procedure access */ - do - { - for (counter= 0, revoked= 0 ; counter < hash->records ; ) - { - GRANT_NAME *grant_proc= (GRANT_NAME*) my_hash_element(hash, counter); - if (sp_db.streq(Lex_cstring_strlen(grant_proc->db)) && - sp_name.streq(Lex_cstring_strlen(grant_proc->tname))) - { - LEX_USER lex_user; - lex_user.user.str= grant_proc->user; - lex_user.user.length= strlen(grant_proc->user); - lex_user.host.str= safe_str(grant_proc->host.hostname); - lex_user.host.length= strlen(lex_user.host.str); - if (replace_routine_table(thd, grant_proc, - tables.procs_priv_table().table(), lex_user, - grant_proc->db, grant_proc->tname, - sph, ALL_KNOWN_ACL, 1) == 0) - { - revoked= 1; - continue; - } - } - counter++; - } - } while (revoked); + if (sph == &sp_handler_package_spec) + sp_revoke_privileges_for_handler(thd, tables.procs_priv_table().table(), + sp_db, sp_name, &sp_handler_package_body); + + sp_revoke_privileges_for_handler(thd, tables.procs_priv_table().table(), + sp_db, sp_name, sph); mysql_mutex_unlock(&acl_cache->lock); mysql_rwlock_unlock(&LOCK_grant); diff --git a/sql/sql_base.cc b/sql/sql_base.cc index 88b26535463e0..f189ecef385ab 100644 --- a/sql/sql_base.cc +++ b/sql/sql_base.cc @@ -2278,7 +2278,7 @@ bool open_table(THD *thd, TABLE_LIST *table_list, Open_table_context *ot_ctx) goto retry_share; } - if (thd->open_tables && thd->open_tables->s->tdc->flushed) + if (!table_list->sequence && thd->open_tables && thd->open_tables->s->tdc->flushed) { /* If the version changes while we're opening the tables, @@ -5378,6 +5378,10 @@ bool open_and_lock_internal_tables(TABLE *table, bool lock_table) MYSQL_LOCK_USE_MALLOC)) goto err; + /* no existing lock to merge with */ + if (save_lock == nullptr) + DBUG_RETURN(0); + if (!(new_lock= mysql_lock_merge(save_lock, thd->lock))) { thd->lock= save_lock; @@ -7151,7 +7155,31 @@ find_field_in_tables(THD *thd, Item_ident *item, { if (report_error == REPORT_ALL_ERRORS || report_error == REPORT_EXCEPT_NON_UNIQUE) + { +#ifndef NO_EMBEDDED_ACCESS_CHECKS + /* + If the user has no rights on this column in any candidate table, + check_grant_column() will issue a generic "access denied" error + */ + if (check_privileges) + { + for (TABLE_LIST *tl= first_table; tl != last_table; + tl= tl->next_name_resolution_table) + { + if (db.str && !tl->db.streq(db)) + continue; + if (table_name && + !tl->alias.streq(Lex_cstring_strlen(table_name))) + continue; + if (check_grant_column(thd, &tl->grant, tl->db.str, + tl->table_name.str, name, + thd->security_ctx)) + return found; + } + } +#endif my_error(ER_BAD_FIELD_ERROR, MYF(0), item->full_name(), thd_where(thd)); + } else found= not_found_field; } @@ -9498,18 +9526,9 @@ fill_record_n_invoke_before_triggers(THD *thd, TABLE *table, Re-calculate virtual fields to cater for cases when base columns are updated by the triggers. */ - if (table->vfield && fields.elements && - no_need_to_skip_a_row(skip_row_indicator)) - { - Item *fld= (Item_field*) fields.head(); - Item_field *item_field= fld->field_for_view_update(); - if (item_field) - { - DBUG_ASSERT(table == item_field->field->table); - result|= table->update_virtual_fields(table->file, - VCOL_UPDATE_FOR_WRITE); - } - } + if (table->vfield && no_need_to_skip_a_row(skip_row_indicator)) + result|= table->update_virtual_fields(table->file, + VCOL_UPDATE_FOR_WRITE); } return result; } diff --git a/sql/sql_class.cc b/sql/sql_class.cc index 2d0a41bb7a15c..d15a0f2352ad9 100644 --- a/sql/sql_class.cc +++ b/sql/sql_class.cc @@ -1,6 +1,6 @@ /* Copyright (c) 2000, 2015, Oracle and/or its affiliates. - Copyright (c) 2008, 2024, MariaDB Corporation. + Copyright (c) 2008, 2026, MariaDB plc. This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by @@ -99,7 +99,6 @@ extern "C" void free_user_var(void *entry_) char *pos= (char*) entry+ALIGN_SIZE(sizeof(*entry)); if (entry->value && entry->value != pos) my_free(entry->value); - my_free(entry); } /* Functions for last-value-from-sequence hash */ @@ -840,6 +839,13 @@ THD::THD(my_thread_id id, bool is_wsrep_applier) &main_mem_root, DEFAULT_ROOT_BLOCK_SIZE, 0, MYF(MY_THREAD_SPECIFIC)); + /* + Use MY_ROOT_USE_VMEM to keep user vars away from the heap, so that a heap + buffer overflow couldn't corrupt a user_var_entry and its value pointer + */ + init_sql_alloc(key_memory_user_var_entry, &user_vars_memroot, + 256, 0, MYF(MY_THREAD_SPECIFIC | MY_ROOT_USE_VMEM)); + /* Allocation of user variables for binary logging is always done with main mem root @@ -1750,8 +1756,22 @@ void THD::cleanup(void) } wt_thd_destroy(&transaction->wt); + bool user_vars_used= user_vars.records != 0; my_hash_free(&user_vars); my_hash_free(&sequences); + /* + MY_ROOT_USE_VMEM is expensive. If this connection used user vars, + let's keep one page prealloc so that the next connection on this THD + wouldn't need to allocate. + */ + if (user_vars_used) + { + reset_root_defaults(&user_vars_memroot, user_vars_memroot.block_size, + user_vars_memroot.block_size); + free_root(&user_vars_memroot, MYF(MY_KEEP_PREALLOC)); + } + else + free_root(&user_vars_memroot, MYF(0)); sp_caches_clear(); statement_rcontext_reinit(); auto_inc_intervals_forced.empty(); @@ -1920,6 +1940,7 @@ THD::~THD() #endif main_lex.free_set_stmt_mem_root(); free_root(&main_mem_root, MYF(0)); + free_root(&user_vars_memroot, MYF(0)); my_free(m_token_array); my_free(killed_err); main_da.free_memory(); @@ -5051,9 +5072,8 @@ change_security_context(THD *thd, DBUG_ASSERT(definer_user->str && definer_host->str); *backup= NULL; - needs_change= (strcmp(definer_user->str, thd->security_ctx->priv_user) || - !Lex_ident_host(*definer_host). - streq(Lex_cstring_strlen(thd->security_ctx->priv_host))); + needs_change= !thd->security_ctx->is_priv_user(*definer_user, + *definer_host); if (needs_change) { if (acl_getroot(this, *definer_user, *definer_host, *definer_host, *db)) @@ -5080,14 +5100,21 @@ Security_context::restore_security_context(THD *thd, #endif -bool Security_context::user_matches(Security_context *them) +/** + check that `this` is not system or pre-auth thread and + is owned by the same account as in `them` + + Note that it's not symmetric, it's used in KILL/SHOW commands + to see whether "them" can kill/see "us", never the other way around! +*/ +bool Security_context::priv_user_matches(const Security_context *them) const { - return ((user != NULL) && (them->user != NULL) && - !strcmp(user, them->user)); + return user && is_priv_user(Lex_cstring_strlen(them->priv_user), + Lex_cstring_strlen(them->priv_host)); } bool Security_context::is_priv_user(const LEX_CSTRING &user, - const LEX_CSTRING &host) + const LEX_CSTRING &host) const { return ((user.str != NULL) && (host.str != NULL) && !strcmp(user.str, priv_user) && diff --git a/sql/sql_class.h b/sql/sql_class.h index 077cd0f589a5c..db9b646e3eae6 100644 --- a/sql/sql_class.h +++ b/sql/sql_class.h @@ -1,6 +1,6 @@ /* Copyright (c) 2000, 2016, Oracle and/or its affiliates. - Copyright (c) 2009, 2025, MariaDB Corporation. + Copyright (c) 2009, 2026, MariaDB plc. This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by @@ -2122,7 +2122,7 @@ class Security_context { void restore_security_context(THD *thd, Security_context *backup); #endif - bool user_matches(Security_context *); + bool priv_user_matches(const Security_context *) const; /** Check global access @param want_access The required privileges @@ -2131,7 +2131,7 @@ class Security_context { @return True if the security context fulfills the access requirements. */ bool check_access(const privilege_t want_access, bool match_any = false); - bool is_priv_user(const LEX_CSTRING &user, const LEX_CSTRING &host); + bool is_priv_user(const LEX_CSTRING &user, const LEX_CSTRING &host) const; bool is_user_defined() const { return user && user != delayed_user && user != slave_user && user != wsrep_user; }; }; @@ -5992,6 +5992,7 @@ class THD: public THD_count, /* this must be first */ void mark_transaction_to_rollback(bool all); bool internal_transaction() { return transaction != &default_transaction; } + MEM_ROOT *user_vars_root() { return &user_vars_memroot; } private: /** The current internal error handler for this thread, or NULL. */ @@ -6013,6 +6014,10 @@ class THD: public THD_count, /* this must be first */ tree itself is reused between executions and thus is stored elsewhere. */ MEM_ROOT main_mem_root; + /** + Memory root the user_var_entry objects and their names are allocated on. + */ + MEM_ROOT user_vars_memroot; Diagnostics_area main_da; Diagnostics_area *m_stmt_da; diff --git a/sql/sql_derived.cc b/sql/sql_derived.cc index 4a4346b3f71a5..493cb744de05a 100644 --- a/sql/sql_derived.cc +++ b/sql/sql_derived.cc @@ -904,7 +904,7 @@ bool mysql_derived_prepare(THD *thd, LEX *lex, TABLE_LIST *derived) */ thd->create_tmp_table_for_derived= TRUE; distinct= (unit->first_select()->next_select() ? - unit->union_distinct && !unit->union_distinct->next_select() : + unit->check_distinct_in_union() : unit->distinct); if (!(derived->table) && diff --git a/sql/sql_parse.cc b/sql/sql_parse.cc index 496426cf0447c..4d2c7a28a4f09 100644 --- a/sql/sql_parse.cc +++ b/sql/sql_parse.cc @@ -1,5 +1,5 @@ /* Copyright (c) 2000, 2017, Oracle and/or its affiliates. - Copyright (c) 2008, 2024, MariaDB + Copyright (c) 2008, 2026, MariaDB plc. This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by @@ -2315,13 +2315,8 @@ dispatch_command_return dispatch_command(enum enum_server_command command, break; case COM_PROCESS_INFO: status_var_increment(thd->status_var.com_stat[SQLCOM_SHOW_PROCESSLIST]); - if (!thd->security_ctx->priv_user[0] && - check_global_access(thd, PRIV_COM_PROCESS_INFO)) - break; general_log_print(thd, command, NullS); - mysqld_list_processes(thd, - thd->security_ctx->master_access & PRIV_COM_PROCESS_INFO ? - NullS : thd->security_ctx->priv_user, 0); + mysqld_list_processes(thd, 0); break; case COM_PROCESS_KILL: { @@ -2867,12 +2862,9 @@ bool sp_process_definer(THD *thd) to create a stored routine under another user one must have SUPER privilege). */ - bool curuser= !strcmp(d->user.str, thd->security_ctx->priv_user); + bool curuser= thd->security_ctx->is_priv_user(d->user, d->host); bool currole= !curuser && !strcmp(d->user.str, thd->security_ctx->priv_role); - bool curuserhost= curuser && d->host.str && - Lex_ident_host(d->host). - streq(Lex_cstring_strlen(thd->security_ctx->priv_host)); - if (!curuserhost && !currole && + if (!curuser && !currole && check_global_access(thd, PRIV_DEFINER_CLAUSE, false)) DBUG_RETURN(TRUE); } @@ -3915,7 +3907,7 @@ mysql_execute_command(THD *thd, bool is_called_from_prepared_stmt) case SQLCOM_SHOW_ANALYZE: { if (!thd->security_ctx->priv_user[0] && - check_global_access(thd, PRIV_STMT_SHOW_EXPLAIN)) + check_global_access(thd, PRIV_STMT_SHOW_PROCESSLIST)) break; /* @@ -4847,14 +4839,7 @@ mysql_execute_command(THD *thd, bool is_called_from_prepared_stmt) break; } case SQLCOM_SHOW_PROCESSLIST: - if (!thd->security_ctx->priv_user[0] && - check_global_access(thd, PRIV_STMT_SHOW_PROCESSLIST)) - break; - mysqld_list_processes(thd, - (thd->security_ctx->master_access & PRIV_STMT_SHOW_PROCESSLIST ? - NullS : - thd->security_ctx->priv_user), - lex->verbose); + mysqld_list_processes(thd, lex->verbose); break; case SQLCOM_SHOW_AUTHORS: res= mysqld_show_authors(thd); @@ -9145,31 +9130,10 @@ kill_one_thread(THD *thd, my_thread_id id, killed_state kill_signal, killed_type DEBUG_SYNC(thd, "found_killee"); if (tmp->get_command() != COM_DAEMON) { - /* - If we're SUPER, we can KILL anything, including system-threads. - No further checks. - - KILLer: thd->security_ctx->user could in theory be NULL while - we're still in "unauthenticated" state. This is a theoretical - case (the code suggests this could happen, so we play it safe). - - KILLee: tmp->security_ctx->user will be NULL for system threads. - We need to check so Jane Random User doesn't crash the server - when trying to kill a) system threads or b) unauthenticated users' - threads (Bug#43748). - - If user of both killer and killee are non-NULL, proceed with - slayage if both are string-equal. - - It's ok to also kill DELAYED threads with KILL_CONNECTION instead of - KILL_SYSTEM_THREAD; The difference is that KILL_CONNECTION may be - faster and do a harder kill than KILL_SYSTEM_THREAD; - */ - mysql_mutex_lock(&tmp->LOCK_thd_data); // Lock from concurrent usage if ((thd->security_ctx->master_access & PRIV_KILL_OTHER_USER_PROCESS) || - thd->security_ctx->user_matches(tmp->security_ctx)) + tmp->security_ctx->priv_user_matches(thd->security_ctx)) { #ifdef WITH_WSREP if (wsrep_thd_is_BF(tmp, false) || tmp->wsrep_applier) @@ -9182,21 +9146,14 @@ kill_one_thread(THD *thd, my_thread_id id, killed_state kill_signal, killed_type tmp->thread_id, (tmp->wsrep_applier ? "wsrep applier" : "high priority")); } + else if (WSREP(tmp)) + error = wsrep_kill_thd(thd, tmp, kill_signal); else - { - if (WSREP(tmp)) - { - error = wsrep_kill_thd(thd, tmp, kill_signal); - } - else - { #endif /* WITH_WSREP */ + { tmp->awake_no_mutex(kill_signal); error= 0; -#ifdef WITH_WSREP - } } -#endif /* WITH_WSREP */ } else error= (type == KILL_TYPE_QUERY ? ER_KILL_QUERY_DENIED_ERROR : @@ -9249,7 +9206,7 @@ static my_bool kill_threads_callback(THD *thd, kill_threads_callback_arg *arg) { if (!(arg->thd->security_ctx->master_access & PRIV_KILL_OTHER_USER_PROCESS) && - !arg->thd->security_ctx->user_matches(thd->security_ctx)) + !thd->security_ctx->priv_user_matches(arg->thd->security_ctx)) { return MY_TEST(arg->thd->security_ctx->master_access & PROCESS_ACL); } diff --git a/sql/sql_parse.h b/sql/sql_parse.h index 5f672cbd5ce21..c670a1f0ff6bd 100644 --- a/sql/sql_parse.h +++ b/sql/sql_parse.h @@ -130,8 +130,7 @@ bool check_stack_overrun(THD *thd, long margin, uchar *dummy); /* Variables */ extern const Lex_ident_db_normalized any_db; -extern cf_flags_t sql_command_flags[]; -extern uint server_command_flags[]; +/* sql_command_flags[] is declared in sql_class.h, where cf_flags_t is defined */ extern const LEX_CSTRING command_name[]; extern uint server_command_flags[]; diff --git a/sql/sql_plugin.cc b/sql/sql_plugin.cc index acb4a8c4acfd2..df45a6fbba557 100644 --- a/sql/sql_plugin.cc +++ b/sql/sql_plugin.cc @@ -1,6 +1,6 @@ /* Copyright (c) 2005, 2018, Oracle and/or its affiliates. - Copyright (c) 2010, 2020, MariaDB Corporation. + Copyright (c) 2010, 2026, MariaDB plc. This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by @@ -66,9 +66,9 @@ static TYPELIB global_plugin_typelib= static I_List opt_plugin_load_list; I_List *opt_plugin_load_list_ptr= &opt_plugin_load_list; -char *opt_plugin_dir_ptr; -char opt_plugin_dir[FN_REFLEN]; -ulong plugin_maturity; +READ_ONLY_SYSVAR char *opt_plugin_dir_ptr; +READ_ONLY_SYSVAR char opt_plugin_dir[FN_REFLEN]; +READ_ONLY_SYSVAR ulong plugin_maturity; static LEX_CSTRING MYSQL_PLUGIN_NAME= {STRING_WITH_LEN("plugin") }; diff --git a/sql/sql_reload.cc b/sql/sql_reload.cc index 4572263d8ef79..610a212ca19bb 100644 --- a/sql/sql_reload.cc +++ b/sql/sql_reload.cc @@ -1,5 +1,5 @@ /* Copyright (c) 2010, 2016, Oracle and/or its affiliates. - Copyright (c) 2011, 2016, MariaDB + Copyright (c) 2011, 2026, MariaDB plc. This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by @@ -120,7 +120,7 @@ bool reload_acl_and_cache(THD *thd, unsigned long long options, }); #endif } - opt_noacl= 0; + clear_opt_noacl(); if (unlikely(tmp_thd)) { diff --git a/sql/sql_repl.cc b/sql/sql_repl.cc index b6fc23b51b6b5..8ba89a803911e 100644 --- a/sql/sql_repl.cc +++ b/sql/sql_repl.cc @@ -1956,10 +1956,10 @@ gtid_state_from_pos(const char *name, uint32 offset, goto end; } - current_checksum_alg= get_checksum_alg((uchar*) packet.ptr(), - packet.length()); found_format_description_event= true; - if (unlikely(!(tmp= new Format_description_log_event((uchar*) packet.ptr(), + if (unlikely(get_checksum_alg((uchar*) packet.ptr(), packet.length(), + ¤t_checksum_alg) || + !(tmp= new Format_description_log_event((uchar*)packet.ptr(), packet.length(), fdev)))) { @@ -2773,6 +2773,8 @@ static int init_binlog_sender(binlog_send_info *info, static int send_format_descriptor_event(binlog_send_info *info, IO_CACHE *log, LOG_INFO *linfo, my_off_t start_pos) { + static constexpr const char* CORRUPT_FDE= + "Corrupt Format_description event found or out-of-memory"; int error; ulong ev_offset; THD *thd= info->thd; @@ -2841,9 +2843,14 @@ static int send_format_descriptor_event(binlog_send_info *info, IO_CACHE *log, DBUG_RETURN(1); } - info->current_checksum_alg= get_checksum_alg((uchar*) packet->ptr() + - ev_offset, - packet->length() - ev_offset); + if (unlikely(get_checksum_alg((uchar*) packet->ptr() + ev_offset, + packet->length() - ev_offset, + &(info->current_checksum_alg)))) + { + info->error= ER_MASTER_FATAL_ERROR_READING_BINLOG; + info->errmsg= CORRUPT_FDE; + DBUG_RETURN(1); + } DBUG_ASSERT(info->current_checksum_alg == BINLOG_CHECKSUM_ALG_OFF || info->current_checksum_alg == BINLOG_CHECKSUM_ALG_UNDEF || @@ -2871,8 +2878,7 @@ static int send_format_descriptor_event(binlog_send_info *info, IO_CACHE *log, ev_len, info->fdev))) { info->error= ER_MASTER_FATAL_ERROR_READING_BINLOG; - info->errmsg= "Corrupt Format_description event found " - "or out-of-memory"; + info->errmsg= CORRUPT_FDE; DBUG_RETURN(1); } delete info->fdev; diff --git a/sql/sql_servers.cc b/sql/sql_servers.cc index 7df83eb5ed9fa..62fb12ef00434 100644 --- a/sql/sql_servers.cc +++ b/sql/sql_servers.cc @@ -46,6 +46,7 @@ #include "transaction.h" #include "lock.h" // MYSQL_LOCK_IGNORE_TIMEOUT #include "create_options.h" +#include "strfunc.h" /* We only use 1 mutex to guard the data structures - THR_LOCK_servers. @@ -76,7 +77,7 @@ static int delete_server_record(TABLE *table, LEX_CSTRING *name); static int delete_server_record_in_cache(LEX_SERVER_OPTIONS *server_options); /* update functions */ -static void prepare_server_struct_for_update(LEX_SERVER_OPTIONS *server_options, +static int prepare_server_struct_for_update(LEX_SERVER_OPTIONS *server_options, FOREIGN_SERVER *existing, FOREIGN_SERVER *altered); static int update_server(THD *thd, FOREIGN_SERVER *existing, @@ -586,7 +587,7 @@ store_server_fields(TABLE *table, FOREIGN_SERVER *server) (uint) strlen(server->password), system_charset_info)) goto err; if (server->port > -1 && - table->field[PORT_FIELD]->store(server->port)) + table->field[PORT_FIELD]->store(server->port, true)) goto err; if (server->socket && table->field[SOCKET_FIELD]->store(server->socket, @@ -1104,6 +1105,47 @@ delete_server_record(TABLE *table, LEX_CSTRING *name) DBUG_RETURN(error); } + +static bool set_known_options(LEX_SERVER_OPTIONS *options) +{ + static const char *known_option_names[]= { + "user", "host", "database", "password", "owner", "socket", "port", 0 + }; + static const TYPELIB known_options= CREATE_TYPELIB_FOR(known_option_names); +#define CASE_KNOWN_OPTION(ID, NAME) case ID: options->NAME=opt->value; break; + + for (engine_option_value *opt= options->option_list; opt; opt= opt->next) + { + if (!opt->value.str) + continue; + switch (find_type(&known_options, opt->name.str, opt->name.length, 0)) { + CASE_KNOWN_OPTION(1, username); + CASE_KNOWN_OPTION(2, host); + CASE_KNOWN_OPTION(3, db); + CASE_KNOWN_OPTION(4, password); + CASE_KNOWN_OPTION(5, owner); + CASE_KNOWN_OPTION(6, socket); + case 7: // PORT + { + int error; + char *end= (char *) opt->value.str + opt->value.length; + char *old_end= end; + longlong p= my_strtoll10(opt->value.str, &end, &error); + if (error > 0 || end != old_end || p > INT32_MAX || p < 0) + { + my_error(ER_BAD_OPTION_VALUE, MYF(0), opt->value.str, "PORT"); + return 1; + } + options->port= (int32)p; + break; + } + default: break; + } + } + return 0; +} + + /* SYNOPSIS @@ -1127,6 +1169,9 @@ int create_server(THD *thd, LEX_SERVER_OPTIONS *server_options) DBUG_PRINT("info", ("server_options->server_name %s", server_options->server_name.str)); + if (set_known_options(server_options)) + DBUG_RETURN(ER_BAD_OPTION_VALUE); + mysql_rwlock_wrlock(&THR_LOCK_servers); /* hit the memory first */ @@ -1217,9 +1262,10 @@ int alter_server(THD *thd, LEX_SERVER_OPTIONS *server_options) server_options->server_name.length))) goto end; - prepare_server_struct_for_update(server_options, existing, &altered); + error= prepare_server_struct_for_update(server_options, existing, &altered); - error= update_server(thd, existing, &altered); + if (!error) + error= update_server(thd, existing, &altered); close_mysql_tables(thd); @@ -1272,7 +1318,7 @@ prepare_server_struct_for_insert(LEX_SERVER_OPTIONS *server_options) if (!(server= (FOREIGN_SERVER *)alloc_root(&mem, sizeof(FOREIGN_SERVER)))) DBUG_RETURN(NULL); /* purecov: inspected */ -#define SET_SERVER_OR_RETURN(X, DEFAULT) \ +#define SET_SERVER_OR_RETURN(X) \ do { \ if (!(server->X= server_options->X.str ? \ strmake_root(&mem, server_options->X.str, \ @@ -1281,8 +1327,8 @@ prepare_server_struct_for_insert(LEX_SERVER_OPTIONS *server_options) } while(0) /* name and scheme are always set (the parser guarantees it) */ - SET_SERVER_OR_RETURN(server_name, NULL); - SET_SERVER_OR_RETURN(scheme, NULL); + SET_SERVER_OR_RETURN(server_name); + SET_SERVER_OR_RETURN(scheme); /* scheme-specific checks */ if (!strcasecmp(server->scheme, "mysql")) @@ -1296,12 +1342,12 @@ prepare_server_struct_for_insert(LEX_SERVER_OPTIONS *server_options) } } - SET_SERVER_OR_RETURN(host, ""); - SET_SERVER_OR_RETURN(db, ""); - SET_SERVER_OR_RETURN(username, ""); - SET_SERVER_OR_RETURN(password, ""); - SET_SERVER_OR_RETURN(socket, ""); - SET_SERVER_OR_RETURN(owner, ""); + SET_SERVER_OR_RETURN(host); + SET_SERVER_OR_RETURN(db); + SET_SERVER_OR_RETURN(username); + SET_SERVER_OR_RETURN(password); + SET_SERVER_OR_RETURN(socket); + SET_SERVER_OR_RETURN(owner); copy_option_list(&mem, server, server_options->option_list); server->server_name_length= server_options->server_name.length; @@ -1326,7 +1372,7 @@ prepare_server_struct_for_insert(LEX_SERVER_OPTIONS *server_options) */ -static void +static int prepare_server_struct_for_update(LEX_SERVER_OPTIONS *server_options, FOREIGN_SERVER *existing, FOREIGN_SERVER *altered) @@ -1338,6 +1384,9 @@ prepare_server_struct_for_update(LEX_SERVER_OPTIONS *server_options, DBUG_PRINT("info", ("existing name %s altered name %s", existing->server_name, altered->server_name)); + if (set_known_options(server_options)) + DBUG_RETURN(1); + /* The logic here is this: is this value set AND is it different than the existing value? @@ -1367,7 +1416,7 @@ prepare_server_struct_for_update(LEX_SERVER_OPTIONS *server_options, server_options->port != existing->port) ? server_options->port : -1; - DBUG_VOID_RETURN; + DBUG_RETURN(0); } /* diff --git a/sql/sql_show.cc b/sql/sql_show.cc index dbc8ba539a1d7..b46515f2b7fd5 100644 --- a/sql/sql_show.cc +++ b/sql/sql_show.cc @@ -1,5 +1,5 @@ /* Copyright (c) 2000, 2015, Oracle and/or its affiliates. - Copyright (c) 2009, 2023, MariaDB + Copyright (c) 2009, 2026, MariaDB plc. This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by @@ -638,7 +638,7 @@ static DYNAMIC_ARRAY ignore_db_dirs_array; A value for the read only system variable to show a list of ignored directories. */ -char *opt_ignore_db_dirs= NULL; +READ_ONLY_SYSVAR char *opt_ignore_db_dirs; /** This flag is ON if: @@ -753,11 +753,6 @@ ignore_db_dirs_reset() void ignore_db_dirs_free() { - if (opt_ignore_db_dirs) - { - my_free(opt_ignore_db_dirs); - opt_ignore_db_dirs= NULL; - } ignore_db_dirs_reset(); delete_dynamic(&ignore_db_dirs_array); my_hash_free(&ignore_db_dirs_hash); @@ -868,6 +863,7 @@ ignore_db_dirs_process_additions() len--; /* +1 the terminating zero */ + my_free(opt_ignore_db_dirs); ptr= opt_ignore_db_dirs= (char *) my_malloc(key_memory_ignored_db, len + 1, MYF(0)); if (!ptr) @@ -1576,12 +1572,15 @@ bool mysql_show_create_server(THD *thd, LEX_CSTRING *name) bool first= true; while (option) { - if (!first) - buffer.append(STRING_WITH_LEN(", ")); - buffer.append(option->name); - buffer.append(STRING_WITH_LEN(" ")); - append_unescaped(&buffer, option->value.str, option->value.length); - first= false; + if (option->value.str) + { + if (!first) + buffer.append(STRING_WITH_LEN(", ")); + append_identifier(thd, &buffer, &option->name); + buffer.append(STRING_WITH_LEN(" ")); + append_unescaped(&buffer, option->value.str, option->value.length); + first= false; + } option= option->next; } buffer.append(STRING_WITH_LEN(");")); @@ -2947,37 +2946,31 @@ static const char *thread_state_info(THD *tmp) Privileged users can see all THDs. - @param user - user name or NULL for privileged users. - @param thd - THD + @param caller - security context of the acting thread + @param thd - THD to check if visible to the caller @retval true - THD visible in processlist @retval false - THD not visible in processlist */ -static bool thd_visible_in_processlist(const char *user, THD *thd) +static bool thd_visible_in_processlist(const Security_context *caller, THD *thd) { if (!thd->vio_ok() && !thd->system_thread) return false; // "something bad happened" thread, don't show it - if (!user) + if (caller->master_access & PRIV_STMT_SHOW_PROCESSLIST) return true; // privileged user can see all threads - - const char *thd_user= thd->security_ctx->user; - if (!thd_user) - return false; // dunno if this ever happens, safety first - bool user_or_event_worker_thread= - !thd->system_thread || thd->system_thread & SYSTEM_THREAD_EVENT_WORKER; + !thd->system_thread || thd->system_thread & SYSTEM_THREAD_EVENT_WORKER; - return user_or_event_worker_thread && !strcmp(thd_user, user); + return user_or_event_worker_thread && + thd->security_ctx->priv_user_matches(caller); } struct list_callback_arg { - list_callback_arg(const char *u, THD *t, ulong m): - user(u), thd(t), max_query_length(m) {} + list_callback_arg(THD *t, ulong m): thd(t), max_query_length(m) {} I_List thread_infos; - const char *user; THD *thd; ulong max_query_length; }; @@ -2988,7 +2981,7 @@ static my_bool list_callback(THD *tmp, list_callback_arg *arg) Security_context *tmp_sctx= tmp->security_ctx; bool got_thd_data; - if (thd_visible_in_processlist(arg->user, tmp)) + if (thd_visible_in_processlist(arg->thd->security_ctx, tmp)) { thread_info *thd_info= new (arg->thd->mem_root) thread_info; @@ -3071,17 +3064,21 @@ static my_bool list_callback(THD *tmp, list_callback_arg *arg) } -void mysqld_list_processes(THD *thd,const char *user, bool verbose) +void mysqld_list_processes(THD *thd, bool verbose) { Item *field; List field_list; - list_callback_arg arg(user, thd, - verbose ? thd->variables.max_allowed_packet : - PROCESS_LIST_WIDTH); + list_callback_arg arg(thd, verbose ? thd->variables.max_allowed_packet + : PROCESS_LIST_WIDTH); Protocol *protocol= thd->protocol; MEM_ROOT *mem_root= thd->mem_root; DBUG_ENTER("mysqld_list_processes"); + /* anonymous users cannot see anything */ + if (!thd->security_ctx->priv_user[0] && + check_global_access(thd, PRIV_STMT_SHOW_PROCESSLIST)) + DBUG_VOID_RETURN; + field_list.push_back(new (mem_root) Item_int(thd, "Id", 0, MY_INT32_NUM_DECIMAL_DIGITS), mem_root); @@ -3322,30 +3319,24 @@ void select_result_text_buffer::save_to(String *res) int fill_show_explain_or_analyze(THD *thd, TABLE_LIST *table, COND *cond, bool json_format, bool is_analyze) { - const char *calling_user; THD *tmp; my_thread_id thread_id; DBUG_ENTER("fill_show_explain_or_analyze"); DBUG_ASSERT(cond==NULL); thread_id= thd->lex->value_list.head()->val_int(); - calling_user= (thd->security_ctx->master_access & PRIV_STMT_SHOW_EXPLAIN) ? - NullS : thd->security_ctx->priv_user; if ((tmp= find_thread_by_id(thread_id))) { - Security_context *tmp_sctx= tmp->security_ctx; MEM_ROOT explain_mem_root, *save_mem_root; /* - If calling_user==NULL, calling thread has SUPER or PROCESS - privilege, and so can do SHOW EXPLAIN/SHOW ANALYZE on any user. - - if calling_user!=NULL, he's only allowed to view + Same rule as for SHOW PROCESSLIST: + A thread with PROCESS privilege can do SHOW EXPLAIN/SHOW + ANALYZE on any user, everybody else is only allowed to view SHOW EXPLAIN/SHOW ANALYZE on his own threads. */ - if (calling_user && (!tmp_sctx->user || strcmp(calling_user, - tmp_sctx->user))) + if (!thd_visible_in_processlist(thd->security_ctx, tmp)) { my_error(ER_SPECIFIC_ACCESS_DENIED_ERROR, MYF(0), "PROCESS"); mysql_mutex_unlock(&tmp->LOCK_thd_kill); @@ -3491,11 +3482,8 @@ static my_bool processlist_callback(THD *tmp, processlist_callback_arg *arg) const char *val; ulonglong max_counter; bool got_thd_data; - char *user= - arg->thd->security_ctx->master_access & PRIV_STMT_SHOW_PROCESSLIST ? - NullS : arg->thd->security_ctx->priv_user; - if (!thd_visible_in_processlist(user, tmp)) + if (!thd_visible_in_processlist(arg->thd->security_ctx, tmp)) return 0; restore_record(arg->table, s->default_values); @@ -7551,10 +7539,7 @@ static int get_schema_views_record(THD *thd, TABLE_LIST *tables, Security_context *sctx= thd->security_ctx; if (!tables->allowed_show) { - if (my_charset_bin.streq(tables->definer.user, - Lex_cstring_strlen(sctx->priv_user)) && - Lex_ident_host(tables->definer.host). - streq(Lex_cstring_strlen(sctx->priv_host))) + if (sctx->is_priv_user(tables->definer.user, tables->definer.host)) tables->allowed_show= TRUE; #ifndef NO_EMBEDDED_ACCESS_CHECKS else diff --git a/sql/sql_show.h b/sql/sql_show.h index 892aadfa6202f..0fdfd8248539a 100644 --- a/sql/sql_show.h +++ b/sql/sql_show.h @@ -1,5 +1,5 @@ /* Copyright (c) 2005, 2010, Oracle and/or its affiliates. - Copyright (c) 2012, 2016, MariaDB + Copyright (c) 2012, 2026, MariaDB plc. This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by @@ -113,7 +113,7 @@ bool mysqld_show_create_db(THD *thd, LEX_CSTRING *db_name, const DDL_options_st &options); bool mysql_show_create_server(THD *thd, LEX_CSTRING *name); -void mysqld_list_processes(THD *thd,const char *user,bool verbose); +void mysqld_list_processes(THD *thd, bool verbose); int mysqld_show_status(THD *thd); int mysqld_show_variables(THD *thd,const char *wild); bool mysqld_show_storage_engines(THD *thd); diff --git a/sql/sql_union.cc b/sql/sql_union.cc index 8bf48bc4487ba..4eb704ce43bd3 100644 --- a/sql/sql_union.cc +++ b/sql/sql_union.cc @@ -772,10 +772,22 @@ bool select_unit_ext::send_eof() next_sl && next_sl->get_linkage() == INTERSECT_TYPE && !next_sl->distinct; + /* + True at the last INTERSECT of an inline INTERSECT DISTINCT subsequence: the + records that did not match every operand (their intersect counter differs + from curr_step) must be dropped. Instead of a separate table scan this is + folded into the scan/unfold loops below, so the table is scanned only once + (MDEV-38722). When the subsequence is materialized in a derived table this + filtering is done by select_unit::send_eof() instead. + */ + bool filter_intersect= curr_op_type == INTERSECT_DISTINCT && + !(next_sl && + next_sl->get_linkage() == INTERSECT_TYPE); bool need_unfold= disable_index_if_needed(curr_sl); if (((curr_sl->distinct && !is_next_distinct) || curr_op_type == INTERSECT_ALL || + filter_intersect || is_next_intersect_all) && !need_unfold) { @@ -797,6 +809,13 @@ bool select_unit_ext::send_eof() } break; } + if (filter_intersect && additional_cnt->val_int() != curr_step) + { + /* drop a record that did not match every INTERSECT operand */ + if (unlikely(error= delete_record())) + break; + continue; + } store_record(table, record[1]); if (curr_sl->distinct && !is_next_distinct) @@ -850,6 +869,13 @@ bool select_unit_ext::send_eof() } break; } + if (filter_intersect && additional_cnt->val_int() != curr_step) + { + /* drop a record that did not match every INTERSECT operand */ + if (unlikely(error= delete_record())) + break; + continue; + } dup_cnt= (ha_rows)duplicate_cnt->val_int(); /* delete record if not exist in the second operand */ if (dup_cnt == 0) @@ -2165,6 +2191,15 @@ void st_select_lex_unit::optimize_bag_operation(bool is_outer_distinct) sl->distinct= true; disable_index= sl; } + else if (disable_index) + { + /* + EXCEPT ALL needs the unique index in select_unit_ext::send_data(). + If the index would otherwise be released at an earlier node, + postpone the release until (at least) this operation. + */ + disable_index= sl; + } } } else @@ -3083,7 +3118,8 @@ void st_select_lex_unit::set_unique_exclude() bool st_select_lex_unit::check_distinct_in_union() { - if (union_distinct && !union_distinct->next_select()) + if (union_distinct && !union_distinct->next_select() && + union_distinct->distinct) return true; return false; } diff --git a/sql/sql_yacc.yy b/sql/sql_yacc.yy index 1fdad927ca189..4d87b47a2c128 100644 --- a/sql/sql_yacc.yy +++ b/sql/sql_yacc.yy @@ -929,7 +929,6 @@ bool my_yyoverflow(short **a, YYSTYPE **b, size_t *yystacksize); %token HELP_SYM %token HIGH_PRIORITY %token HISTORY_SYM /* MYSQL */ -%token HOST_SYM %token HOSTS_SYM %token HOUR_SYM /* SQL-2003-R */ %token ID_SYM /* MYSQL */ @@ -1038,7 +1037,6 @@ bool my_yyoverflow(short **a, YYSTYPE **b, size_t *yystacksize); %token OPTIONS_SYM %token OPTION /* SQL-2003-N */ %token OVERLAPS_SYM -%token OWNER_SYM %token PACK_KEYS_SYM %token PAGE_SYM %token PARSER_SYM @@ -1125,7 +1123,6 @@ bool my_yyoverflow(short **a, YYSTYPE **b, size_t *yystacksize); %token SLAVE_POS_SYM %token SLOW %token SNAPSHOT_SYM -%token SOCKET_SYM %token SOFT_SYM %token SONAME_SYM %token SOUNDS_SYM @@ -3044,21 +3041,6 @@ server_options_list: server_option: USER_SYM TEXT_STRING_sys { - MYSQL_YYABORT_UNLESS(Lex->server_options.username.str == 0); - Lex->server_options.username= $2; - engine_option_value *new_option= - new (thd->mem_root) engine_option_value( - engine_option_value::Name( - safe_lexcstrdup_root(thd->mem_root, $1)), - engine_option_value::Value( - safe_lexcstrdup_root(thd->mem_root, $2)), true); - new_option->link(&Lex->server_options.option_list, - &Lex->option_list_last); - } - | HOST_SYM TEXT_STRING_sys - { - MYSQL_YYABORT_UNLESS(Lex->server_options.host.str == 0); - Lex->server_options.host= $2; engine_option_value *new_option= new (thd->mem_root) engine_option_value( engine_option_value::Name( @@ -3070,21 +3052,6 @@ server_option: } | DATABASE TEXT_STRING_sys { - MYSQL_YYABORT_UNLESS(Lex->server_options.db.str == 0); - Lex->server_options.db= $2; - engine_option_value *new_option= - new (thd->mem_root) engine_option_value( - engine_option_value::Name( - safe_lexcstrdup_root(thd->mem_root, $1)), - engine_option_value::Value( - safe_lexcstrdup_root(thd->mem_root, $2)), true); - new_option->link(&Lex->server_options.option_list, - &Lex->option_list_last); - } - | OWNER_SYM TEXT_STRING_sys - { - MYSQL_YYABORT_UNLESS(Lex->server_options.owner.str == 0); - Lex->server_options.owner= $2; engine_option_value *new_option= new (thd->mem_root) engine_option_value( engine_option_value::Name( @@ -3096,21 +3063,6 @@ server_option: } | PASSWORD_SYM TEXT_STRING_sys { - MYSQL_YYABORT_UNLESS(Lex->server_options.password.str == 0); - Lex->server_options.password= $2; - engine_option_value *new_option= - new (thd->mem_root) engine_option_value( - engine_option_value::Name( - safe_lexcstrdup_root(thd->mem_root, $1)), - engine_option_value::Value( - safe_lexcstrdup_root(thd->mem_root, $2)), true); - new_option->link(&Lex->server_options.option_list, - &Lex->option_list_last); - } - | SOCKET_SYM TEXT_STRING_sys - { - MYSQL_YYABORT_UNLESS(Lex->server_options.socket.str == 0); - Lex->server_options.socket= $2; engine_option_value *new_option= new (thd->mem_root) engine_option_value( engine_option_value::Name( @@ -3122,23 +3074,6 @@ server_option: } | PORT_SYM ulong_num { - /* - We especially don't want this to happen: - - The value of $2 is ULONG_MAX, causing - server_options.port to be -1, which means "default - port". - - Because we are doing a check here, we may as well check - against the SQL data type in one go rather than just the - C++ type here and SQL type later in sql_servers.cc. - */ - if ($2 > INT32_MAX) - { - my_error(ER_DATA_OUT_OF_RANGE, myf(0), "port", "INT"); - MYSQL_YYABORT; - } - Lex->server_options.port= $2; engine_option_value *new_option= new (thd->mem_root) engine_option_value( engine_option_value::Name( @@ -3150,16 +3085,6 @@ server_option: /* port can be a quoted number */ | PORT_SYM TEXT_STRING_sys { - int error; - char *end= (char *) $2.str + $2.length; - longlong p= my_strtoll10($2.str, &end, &error); - if (error > 0 || end != (char *) $2.str + $2.length || - p > LONG_MAX || p < LONG_MIN) - { - thd->parse_error(); - MYSQL_YYABORT; - } - Lex->server_options.port= (long) p; engine_option_value *new_option= new (thd->mem_root) engine_option_value( engine_option_value::Name( @@ -16717,12 +16642,10 @@ keyword_sp_var_not_label: | FOLLOWING_SYM | GET_SYM | HELP_SYM - | HOST_SYM | INSTALL_SYM | OPTION | OPTIONS_SYM | OTHERS_MARIADB_SYM - | OWNER_SYM | PARSER_SYM | PERIOD_SYM | PORT_SYM @@ -16732,7 +16655,6 @@ keyword_sp_var_not_label: | RESET_SYM | SECURITY_SYM | SERVER_SYM - | SOCKET_SYM | SLAVE | SLAVES | SONAME_SYM diff --git a/sql/sys_vars.cc b/sql/sys_vars.cc index 99b6af633321c..f73fb93b8fbcf 100644 --- a/sql/sys_vars.cc +++ b/sql/sys_vars.cc @@ -1,5 +1,5 @@ /* Copyright (c) 2002, 2015, Oracle and/or its affiliates. - Copyright (c) 2012, 2022, MariaDB Corporation. + Copyright (c) 2012, 2026, MariaDB plc. This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by @@ -1995,7 +1995,7 @@ static Sys_var_ulonglong Sys_max_heap_table_size( VALID_RANGE(16384, SIZE_T_MAX), DEFAULT(16*1024*1024), BLOCK_SIZE(1024)); -static ulong mdl_locks_cache_size; +READ_ONLY_SYSVAR static ulong mdl_locks_cache_size; static Sys_var_ulong Sys_metadata_locks_cache_size( "metadata_locks_cache_size", UNUSED_HELP, READ_ONLY GLOBAL_VAR(mdl_locks_cache_size), CMD_LINE(REQUIRED_ARG), @@ -2003,7 +2003,7 @@ static Sys_var_ulong Sys_metadata_locks_cache_size( BLOCK_SIZE(1), NO_MUTEX_GUARD, NOT_IN_BINLOG, ON_CHECK(0), ON_UPDATE(0), DEPRECATED(1105, "")); -static ulong mdl_locks_hash_partitions; +READ_ONLY_SYSVAR static ulong mdl_locks_hash_partitions; static Sys_var_ulong Sys_metadata_locks_hash_instances( "metadata_locks_hash_instances", UNUSED_HELP, READ_ONLY GLOBAL_VAR(mdl_locks_hash_partitions), CMD_LINE(REQUIRED_ARG), @@ -2123,7 +2123,7 @@ Sys_gtid_seq_no( #ifdef HAVE_REPLICATION -static unsigned char opt_gtid_binlog_pos_dummy; +READ_ONLY_SYSVAR static unsigned char opt_gtid_binlog_pos_dummy; static Sys_var_gtid_binlog_pos Sys_gtid_binlog_pos( "gtid_binlog_pos", "Last GTID logged to the binary log, per replication " "domain", @@ -2150,7 +2150,7 @@ Sys_var_gtid_binlog_pos::global_value_ptr(THD *thd, } -static unsigned char opt_gtid_current_pos_dummy; +READ_ONLY_SYSVAR static unsigned char opt_gtid_current_pos_dummy; static Sys_var_gtid_current_pos Sys_gtid_current_pos( "gtid_current_pos", "Current GTID position of the server. Per " "replication domain, this is either the last GTID replicated by a " @@ -3649,11 +3649,15 @@ static Sys_var_mybool Sys_require_secure_transport( ON_CHECK(check_require_secure_transport), ON_UPDATE(0)); static Sys_var_charptr_fscs Sys_secure_file_priv( - "secure_file_priv", - "Limit LOAD DATA, SELECT ... OUTFILE, and LOAD_FILE() to files " - "within specified directory", - PREALLOCATED READ_ONLY GLOBAL_VAR(opt_secure_file_priv), - CMD_LINE(REQUIRED_ARG, OPT_SEQURE_FILE_PRIV), DEFAULT(0)); + "secure_file_priv", + "Limit LOAD DATA, SELECT ... OUTFILE, and LOAD_FILE() to files " + "within specified directory." +#ifndef _WIN32 + " Empty value means no limits except /proc" +#endif + , + PREALLOCATED READ_ONLY GLOBAL_VAR(opt_secure_file_priv), + CMD_LINE(REQUIRED_ARG, OPT_SEQURE_FILE_PRIV), DEFAULT(0)); static bool check_server_id(sys_var *self, THD *thd, set_var *var) { @@ -3696,8 +3700,7 @@ Sys_server_id( VALID_RANGE(1, UINT_MAX32), DEFAULT(1), BLOCK_SIZE(1), NO_MUTEX_GUARD, NOT_IN_BINLOG, ON_CHECK(check_server_id), ON_UPDATE(fix_server_id)); -char *server_uid_ptr= &server_uid[0]; - +READ_ONLY_SYSVAR char *server_uid_ptr= &server_uid[0]; static Sys_var_charptr Sys_server_uid( "server_uid", "Automatically calculated server unique id hash", READ_ONLY GLOBAL_VAR(server_uid_ptr), @@ -4324,7 +4327,7 @@ static Sys_var_charptr_fscs Sys_ssl_crlpath( READ_ONLY GLOBAL_VAR(opt_ssl_crlpath), SSL_OPT(OPT_SSL_CRLPATH), DEFAULT(0)); -static char *opt_ssl_passphrase; +READ_ONLY_SYSVAR static char *opt_ssl_passphrase; static Sys_var_charptr Sys_ssl_passphrase( "ssl_passphrase", "SSL certificate key passphrase", @@ -4433,7 +4436,7 @@ static Sys_var_mybool Sys_sync_frm( GLOBAL_VAR(opt_sync_frm), CMD_LINE(OPT_ARG), DEFAULT(TRUE)); -static char *system_time_zone_ptr; +READ_ONLY_SYSVAR static char *system_time_zone_ptr; static Sys_var_charptr Sys_system_time_zone( "system_time_zone", "The server system time zone", READ_ONLY GLOBAL_VAR(system_time_zone_ptr), @@ -4771,7 +4774,7 @@ static Sys_var_charptr Sys_version( CMD_LINE_HELP_ONLY, DEFAULT(server_version)); -static char *server_version_comment_ptr; +READ_ONLY_SYSVAR static char *server_version_comment_ptr; static Sys_var_charptr Sys_version_comment( "version_comment", "Value of the COMPILATION_COMMENT option " "specified by CMake when building MariaDB, for example " @@ -4780,14 +4783,14 @@ static Sys_var_charptr Sys_version_comment( CMD_LINE_HELP_ONLY, DEFAULT(MYSQL_COMPILATION_COMMENT)); -static char *server_version_compile_machine_ptr; +READ_ONLY_SYSVAR static char *server_version_compile_machine_ptr; static Sys_var_charptr Sys_version_compile_machine( "version_compile_machine", "The machine type or architecture " "MariaDB was built on, for example i686", READ_ONLY GLOBAL_VAR(server_version_compile_machine_ptr), CMD_LINE_HELP_ONLY, DEFAULT(DEFAULT_MACHINE)); -static char *server_version_compile_os_ptr; +READ_ONLY_SYSVAR static char *server_version_compile_os_ptr; static Sys_var_charptr Sys_version_compile_os( "version_compile_os", "Operating system that MariaDB was built " "on, for example debian-linux-gnu", @@ -4796,20 +4799,20 @@ static Sys_var_charptr Sys_version_compile_os( DEFAULT(SYSTEM_TYPE)); #include -static char *server_version_source_revision; +READ_ONLY_SYSVAR static char *server_version_source_revision; static Sys_var_charptr Sys_version_source_revision( "version_source_revision", "Source control revision id for MariaDB source code", READ_ONLY GLOBAL_VAR(server_version_source_revision), CMD_LINE_HELP_ONLY, DEFAULT(SOURCE_REVISION)); -static char *malloc_library; +READ_ONLY_SYSVAR static char *malloc_library; static Sys_var_charptr Sys_malloc_library( "version_malloc_library", "Version of the used malloc library", READ_ONLY GLOBAL_VAR(malloc_library), CMD_LINE_HELP_ONLY, DEFAULT(guess_malloc_library())); -static char *ssl_library; +READ_ONLY_SYSVAR static char *ssl_library; static Sys_var_charptr Sys_ssl_library( "version_ssl_library", "Version of the used SSL library", READ_ONLY GLOBAL_VAR(ssl_library), CMD_LINE_HELP_ONLY, @@ -5415,7 +5418,7 @@ static Sys_var_uint Sys_group_concat_max_len( VALID_RANGE(4, MAX_MAX_ALLOWED_PACKET), DEFAULT(1024*1024), BLOCK_SIZE(1)); -static char *glob_hostname_ptr; +READ_ONLY_SYSVAR static char *glob_hostname_ptr; static Sys_var_charptr Sys_hostname( "hostname", "Server host name", READ_ONLY GLOBAL_VAR(glob_hostname_ptr), NO_CMD_LINE, @@ -5472,7 +5475,7 @@ static Sys_var_mybool Sys_keep_files_on_create( NO_MUTEX_GUARD, NOT_IN_BINLOG, ON_CHECK(0), ON_UPDATE(0), DEPRECATED_NO_REPLACEMENT(1008)); -static char *license; +READ_ONLY_SYSVAR static char *license; static Sys_var_charptr Sys_license( "license", "The type of license the server has", READ_ONLY GLOBAL_VAR(license), NO_CMD_LINE, @@ -5708,7 +5711,7 @@ static Sys_var_have Sys_have_symlink( # define SANITIZER_MODE "MSAN" # endif -static char *have_sanitizer; +READ_ONLY_SYSVAR static char *have_sanitizer; static Sys_var_charptr Sys_have_santitizer( "have_sanitizer", "If the server is compiled with sanitize (compiler option), this " @@ -5832,7 +5835,7 @@ static Sys_var_charptr_fscs Sys_relay_log( static Sys_var_charptr_fscs Sys_relay_log_index( "relay_log_index", "The location and name to use for the file " "that keeps a list of the last relay logs", - READ_ONLY GLOBAL_VAR(relay_log_index), NO_CMD_LINE, + PREALLOCATED READ_ONLY GLOBAL_VAR(relay_log_index), NO_CMD_LINE, DEFAULT(0)); /* @@ -5842,19 +5845,19 @@ static Sys_var_charptr_fscs Sys_relay_log_index( */ static Sys_var_charptr_fscs Sys_binlog_index( "log_bin_index", "File that holds the names for last binary log files", - READ_ONLY GLOBAL_VAR(log_bin_index), NO_CMD_LINE, + PREALLOCATED READ_ONLY GLOBAL_VAR(log_bin_index), NO_CMD_LINE, DEFAULT(0)); static Sys_var_charptr_fscs Sys_relay_log_basename( "relay_log_basename", "The full path of the relay log file names, excluding the extension", - READ_ONLY GLOBAL_VAR(relay_log_basename), NO_CMD_LINE, + PREALLOCATED READ_ONLY GLOBAL_VAR(relay_log_basename), NO_CMD_LINE, DEFAULT(0)); static Sys_var_charptr_fscs Sys_log_bin_basename( "log_bin_basename", "The full path of the binary log file names, excluding the extension", - READ_ONLY GLOBAL_VAR(log_bin_basename), NO_CMD_LINE, + PREALLOCATED READ_ONLY GLOBAL_VAR(log_bin_basename), NO_CMD_LINE, DEFAULT(0)); static Sys_var_charptr_fscs Sys_relay_log_info_file( @@ -6822,7 +6825,7 @@ static Sys_var_mybool Sys_wsrep_gtid_mode( "ignored (backward compatibility)", GLOBAL_VAR(wsrep_gtid_mode), CMD_LINE(OPT_ARG), DEFAULT(FALSE)); -static char *wsrep_patch_version_ptr; +READ_ONLY_SYSVAR static char *wsrep_patch_version_ptr; static Sys_var_charptr Sys_wsrep_patch_version( "wsrep_patch_version", "Wsrep patch version, for example wsrep_25.10", READ_ONLY GLOBAL_VAR(wsrep_patch_version_ptr), CMD_LINE_HELP_ONLY, @@ -6911,7 +6914,7 @@ static Sys_var_charptr_fscs Sys_ignore_db_dirs( "Specifies a directory to add to the ignore list when collecting " "database names from the datadir. Put a blank argument to reset " "the list accumulated so far", - READ_ONLY GLOBAL_VAR(opt_ignore_db_dirs), + PREALLOCATED READ_ONLY GLOBAL_VAR(opt_ignore_db_dirs), CMD_LINE(REQUIRED_ARG, OPT_IGNORE_DB_DIRECTORY), DEFAULT(0)); diff --git a/sql/sys_vars.inl b/sql/sys_vars.inl index 1d24cdd823a01..91883853646cd 100644 --- a/sql/sys_vars.inl +++ b/sql/sys_vars.inl @@ -1,5 +1,5 @@ /* Copyright (c) 2002, 2011, Oracle and/or its affiliates. - Copyright (c) 2010, 2020, MariaDB Corporation. + Copyright (c) 2010, 2026, MariaDB plc. This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by @@ -89,22 +89,6 @@ extern const char *UNUSED_HELP; #pragma clang diagnostic ignored "-Winvalid-offsetof" #endif -/* - special assert for sysvars. Tells the name of the variable, - and fails even in non-debug builds. - - It is supposed to be used *only* in Sys_var* constructors, - and has name_arg hard-coded to prevent incorrect usage. -*/ -#define SYSVAR_ASSERT(X) \ - while(!(X)) \ - { \ - fprintf(stderr, "Sysvar '%s' failed '%s'\n", name_arg, #X); \ - DBUG_ASSERT(0); \ - exit(255); \ - } - - static const char *bool_values[3]= {"OFF", "ON", 0}; TYPELIB bool_typelib= CREATE_TYPELIB_FOR(bool_values); diff --git a/sql/table.cc b/sql/table.cc index 1e56e733e8fdf..ac6e04a218919 100644 --- a/sql/table.cc +++ b/sql/table.cc @@ -130,10 +130,8 @@ static std::atomic last_table_id; /* Functions defined in this file */ -static bool fix_type_pointers(const char ***typelib_value_names, - uint **typelib_value_lengths, - TYPELIB *point_to_type, uint types, - char *names, size_t names_length); +static bool fix_type_pointers(const char ***, const char **, uint **, + TYPELIB *, uint, char *, size_t); static field_index_t find_field(Field **fields, uchar *record, uint start, uint length); @@ -798,7 +796,7 @@ static bool create_key_infos(THD *thd, const uchar *strpos, { uint i, j, n_length; uint primary_key_parts= 0; - KEY_PART_INFO *key_part= NULL; + KEY_PART_INFO *key_part= NULL, *key_part_end= NULL; ulong *rec_per_key= NULL; DBUG_ASSERT(keyinfo == first_keyinfo); DBUG_ASSERT(share->keys == 0); @@ -808,7 +806,7 @@ static bool create_key_infos(THD *thd, const uchar *strpos, if (!(keyinfo = (KEY*) alloc_root(&share->mem_root, len))) return 1; bzero((char*) keyinfo, len); - key_part= reinterpret_cast (keyinfo); + key_part= key_part_end= reinterpret_cast (keyinfo); } bzero((char*)first_keyinfo, sizeof(*first_keyinfo)); @@ -878,12 +876,15 @@ static bool create_key_infos(THD *thd, const uchar *strpos, sizeof(ulong) * *ext_key_parts))) return 1; bzero((char*) rec_per_key, sizeof(*rec_per_key) * *ext_key_parts); + key_part_end= key_part + *ext_key_parts; } keyinfo->key_part= key_part; keyinfo->rec_per_key= rec_per_key; for (j=keyinfo->user_defined_key_parts ; j-- ; key_part++) { + if (key_part >= key_part_end) + return 1; if (strpos + (new_frm_ver >= 1 ? 9 : 7) >= frm_image_end) return 1; if (keyinfo->algorithm != HA_KEY_ALG_LONG_HASH && @@ -942,13 +943,17 @@ static bool create_key_infos(THD *thd, const uchar *strpos, share->ext_key_parts+= keyinfo->ext_key_parts; DBUG_ASSERT(share->ext_key_parts <= *ext_key_parts); } + if (key_part > key_part_end) + return 1; + size_t max_keyname_len= MY_MIN(len, (uint)(frm_image_end - strpos)); keynames->str= (char*) key_part; - keynames->length= strnmov(keynames->str, (char *) strpos, - frm_image_end - strpos) - keynames->str; + keynames->length= strnmov(keynames->str, (char *)strpos, + max_keyname_len) - keynames->str; + if (keynames->length >= max_keyname_len) + return 1; // meaning key name was not \0-terminated + keynames->length++; // include '\0', to make fix_type_pointers() happy. strpos+= keynames->length; - if (*strpos++) // key names are \0-terminated - return 1; - keynames->length++; // Include '\0', to make fix_type_pointers() happy. + len-= (uint)keynames->length; //reading index comments for (keyinfo= share->key_info, i=0; i < keys; i++, keyinfo++) @@ -960,11 +965,13 @@ static bool create_key_infos(THD *thd, const uchar *strpos, keyinfo->comment.length= uint2korr(strpos); strpos+= 2; - if (strpos + keyinfo->comment.length >= frm_image_end) + if (strpos + keyinfo->comment.length >= frm_image_end || + keyinfo->comment.length > len) return 1; keyinfo->comment.str= strmake_root(&share->mem_root, (char*) strpos, keyinfo->comment.length); strpos+= keyinfo->comment.length; + len-= (uint)keyinfo->comment.length; } DBUG_ASSERT(MY_TEST(keyinfo->flags & HA_USES_COMMENT) == (keyinfo->comment.length > 0)); @@ -1083,7 +1090,8 @@ static void mysql57_calculate_null_position(TABLE_SHARE *share, uchar **null_pos, uint *null_bit_pos, const uchar *strpos, - const uchar *vcol_screen_pos) + const uchar *vcol_screen_pos, + const uchar *vcol_screen_end) { uint field_pack_length= 17; @@ -1094,6 +1102,8 @@ static void mysql57_calculate_null_position(TABLE_SHARE *share, if ((strpos[10] & MYSQL57_GENERATED_FIELD)) { + if (vcol_screen_pos + MYSQL57_GCOL_HEADER_SIZE >= vcol_screen_end) + return; /* Skip virtual (not stored) generated field */ bool stored_in_db= vcol_screen_pos[3]; vcol_screen_pos+= (uint2korr(vcol_screen_pos + 1) + @@ -1859,6 +1869,8 @@ int TABLE_SHARE::init_from_binary_frm_image(THD *thd, bool write, const uchar *forminfo; const uchar *frm_image_end = frm_image + frm_length; uchar *record, *null_flags, *null_pos, *UNINIT_VAR(mysql57_vcol_null_pos); + uchar *data_start, *data_end; + uchar *first_stored= 0, *first_virtual= 0, *next_stored= 0, *next_virtual= 0; const uchar *disk_buff, *strpos; ulong pos, record_offset; ulong rec_buff_length; @@ -1866,13 +1878,13 @@ int TABLE_SHARE::init_from_binary_frm_image(THD *thd, bool write, KEY *keyinfo; KEY_PART_INFO *key_part= NULL; Field **field_ptr, *reg_field; - const char **interval_array; + const char **interval_array, **interval_array_end; uint *typelib_value_lengths= NULL; enum legacy_db_type legacy_db_type; my_bitmap_map *bitmaps; bool null_bits_are_used; uint vcol_screen_length; - uchar *vcol_screen_pos; + uchar *vcol_screen_pos, *vcol_screen_end; LEX_CUSTRING options; LEX_CSTRING se_name= empty_clex_str; KEY first_keyinfo; @@ -1882,7 +1894,7 @@ int TABLE_SHARE::init_from_binary_frm_image(THD *thd, bool write, bool vers_can_native= false, frm_created= 0; Field_data_type_info_array field_data_type_info_array; MEM_ROOT *old_root= thd->mem_root; - Virtual_column_info **table_check_constraints; + Virtual_column_info **table_check_constr; bool *interval_unescaped= NULL; extra2_fields extra2; bool extra_index_flags_present= FALSE; @@ -1929,6 +1941,14 @@ int TABLE_SHARE::init_from_binary_frm_image(THD *thd, bool write, /* Length of the MariaDB extra2 segment in the form file. */ len = uint2korr(frm_image+4); + if (frm_length < FRM_HEADER_SIZE + len || + !(pos= uint4korr(frm_image + FRM_HEADER_SIZE + len))) + goto err; + + forminfo= frm_image + pos; + if (forminfo + FRM_FORMINFO_SIZE >= frm_image_end) + goto err; + if (read_extra2(frm_image, len, &extra2)) goto err; @@ -1948,17 +1968,7 @@ int TABLE_SHARE::init_from_binary_frm_image(THD *thd, bool write, if (!share->default_part_plugin) goto err; } -#endif - - if (frm_length < FRM_HEADER_SIZE + len || - !(pos= uint4korr(frm_image + FRM_HEADER_SIZE + len))) - goto err; - forminfo= frm_image + pos; - if (forminfo + FRM_FORMINFO_SIZE >= frm_image_end) - goto err; - -#ifdef WITH_PARTITION_STORAGE_ENGINE if (frm_image[61] && !share->default_part_plugin) { enum legacy_db_type db_type= (enum legacy_db_type) (uint) frm_image[61]; @@ -2050,6 +2060,9 @@ int TABLE_SHARE::init_from_binary_frm_image(THD *thd, bool write, share->keys_in_use.init(keys); ext_key_parts= key_parts; + if ((key_parts && !keys) || key_parts < keys) + goto err; + if (extra2.index_flags.str && extra2.index_flags.length != keys) goto err; @@ -2207,12 +2220,10 @@ int TABLE_SHARE::init_from_binary_frm_image(THD *thd, bool write, { //reading long table comment if (next_chunk + 2 > buff_end) - { - DBUG_PRINT("error", - ("long table comment is not defined in .frm")); goto err; - } share->comment.length = uint2korr(next_chunk); + if (next_chunk + 2 + share->comment.length > buff_end) + goto err; if (! (share->comment.str= strmake_root(&share->mem_root, (char*)next_chunk + 2, share->comment.length))) { @@ -2221,17 +2232,18 @@ int TABLE_SHARE::init_from_binary_frm_image(THD *thd, bool write, next_chunk+= 2 + share->comment.length; } - DBUG_ASSERT(next_chunk <= buff_end); - if (share->db_create_options & HA_OPTION_TEXT_CREATE_OPTIONS_legacy) { if (options.str) goto err; + if (next_chunk + 4 > buff_end) + goto err; options.length= uint4korr(next_chunk); options.str= next_chunk + 4; next_chunk+= options.length + 4; + if (next_chunk > buff_end) + goto err; } - DBUG_ASSERT(next_chunk <= buff_end); } else { @@ -2243,7 +2255,6 @@ int TABLE_SHARE::init_from_binary_frm_image(THD *thd, bool write, share->key_block_size= uint2korr(frm_image+62); keyinfo= share->key_info; - if (extra2.index_flags.str) extra_index_flags_present= TRUE; @@ -2288,6 +2299,8 @@ int TABLE_SHARE::init_from_binary_frm_image(THD *thd, bool write, disk_buff= frm_image + pos + FRM_FORMINFO_SIZE; share->fields= uint2korr(forminfo+258); + if (share->fields <= 0 || share->fields > MAX_FIELDS) + goto err; if (extra2.field_flags.str && extra2.field_flags.length != share->fields) goto err; pos= uint2korr(forminfo+260); /* Length of all screens */ @@ -2309,6 +2322,9 @@ int TABLE_SHARE::init_from_binary_frm_image(THD *thd, bool write, share->comment.length); } + if (hash_fields > share->fields || share->null_fields > share->fields) + goto err; + DBUG_PRINT("info",("i_count: %d i_parts: %d index: %d n_length: %d int_length: %d com_length: %d vcol_screen_length: %d", interval_count,interval_parts, keys,n_length,int_length, com_length, vcol_screen_length)); /* @@ -2333,12 +2349,8 @@ int TABLE_SHARE::init_from_binary_frm_image(THD *thd, bool write, &share->field, (uint)(share->fields+1)*sizeof(Field*), &share->intervals, (uint)interval_count*sizeof(TYPELIB), &share->check_constraints, (uint) share->table_check_constraints * sizeof(Virtual_column_info*), - /* - This looks wrong: shouldn't it be (+2+interval_count) - instread of (+3) ? - */ - &interval_array, (uint) (share->fields+interval_parts+ keys+3)*sizeof(char *), - &typelib_value_lengths, total_typelib_value_count * sizeof(uint *), + &interval_array, (uint)total_typelib_value_count * sizeof(char *), + &typelib_value_lengths, (uint)total_typelib_value_count * sizeof(uint), &names, (uint) (n_length+int_length), &comment_pos, (uint) com_length, &vcol_screen_pos, vcol_screen_length, @@ -2354,10 +2366,13 @@ int TABLE_SHARE::init_from_binary_frm_image(THD *thd, bool write, } field_ptr= share->field; - table_check_constraints= share->check_constraints; + interval_array_end= interval_array + total_typelib_value_count; + table_check_constr= share->check_constraints; read_length=(uint) (share->fields * field_pack_length + pos+ (uint) (n_length+int_length+com_length+ vcol_screen_length)); + if (disk_buff + read_length > frm_image_end) + goto err; strpos= disk_buff+pos; if (!interval_count) @@ -2371,21 +2386,24 @@ int TABLE_SHARE::init_from_binary_frm_image(THD *thd, bool write, com_length); memcpy(vcol_screen_pos, disk_buff+read_length-vcol_screen_length, vcol_screen_length); + vcol_screen_end= vcol_screen_pos + vcol_screen_length; - if (fix_type_pointers(&interval_array, &typelib_value_lengths, - &share->fieldnames, 1, names, n_length) || + if (fix_type_pointers(&interval_array, interval_array_end, + &typelib_value_lengths, &share->fieldnames, 1, names, + n_length) || share->fieldnames.count != share->fields) goto err; - if (fix_type_pointers(&interval_array, &typelib_value_lengths, - share->intervals, interval_count, - names + n_length, int_length)) + if (fix_type_pointers(&interval_array, interval_array_end, + &typelib_value_lengths, share->intervals, + interval_count, names + n_length, int_length)) goto err; - if (keynames.length && - (fix_type_pointers(&interval_array, &typelib_value_lengths, - &share->keynames, 1, keynames.str, keynames.length) || - share->keynames.count != keys)) + if ((keynames.length && + fix_type_pointers(&interval_array, interval_array_end, + &typelib_value_lengths, &share->keynames, 1, + keynames.str, keynames.length)) || + share->keynames.count != keys) goto err; /* Allocate handler */ @@ -2409,14 +2427,12 @@ int TABLE_SHARE::init_from_binary_frm_image(THD *thd, bool write, */ share->null_bytes= (share->null_fields + null_bit_pos + 7) / 8; } -#ifndef WE_WANT_TO_SUPPORT_VERY_OLD_FRM_FILES else { share->null_bytes= (share->null_fields+7)/8; null_flags= null_pos= record + 1 + share->reclength - share->null_bytes; null_bit_pos= 0; } -#endif use_hash= share->fields >= MAX_FIELDS_BEFORE_HASH; if (use_hash) @@ -2437,7 +2453,7 @@ int TABLE_SHARE::init_from_binary_frm_image(THD *thd, bool write, mysql57_vcol_null_bit_pos= null_bit_pos; mysql57_calculate_null_position(share, &mysql57_vcol_null_pos, &mysql57_vcol_null_bit_pos, - strpos, vcol_screen_pos); + strpos, vcol_screen_pos, vcol_screen_end); } /* Set system versioning information. */ @@ -2486,20 +2502,22 @@ int TABLE_SHARE::init_from_binary_frm_image(THD *thd, bool write, if (extra2.without_overlaps.str) { - if (extra2.application_period.str == NULL) + if (extra2.application_period.str == NULL || + extra2.without_overlaps.length < frm_keyno_size) goto err; const uchar *key_pos= extra2.without_overlaps.str; period.unique_keys= read_frm_keyno(key_pos); + if (period.unique_keys > keys || + extra2.without_overlaps.length != (period.unique_keys+1)*frm_keyno_size) + goto err; for (uint k= 0; k < period.unique_keys; k++) { key_pos+= frm_keyno_size; uint key_nr= read_frm_keyno(key_pos); + if (key_nr >= keys) + goto err; key_info[key_nr].without_overlaps= true; } - - if ((period.unique_keys + 1) * frm_keyno_size - != extra2.without_overlaps.length) - goto err; } if (extra2.field_data_type_info.length && @@ -2536,6 +2554,9 @@ int TABLE_SHARE::init_from_binary_frm_image(THD *thd, bool write, comment.str= (char*) comment_pos; comment.length= comment_length; comment_pos+= comment_length; + if (com_length < comment_length) + goto err; + com_length-= comment_length; } if (strpos[13] == MYSQL_TYPE_VIRTUAL && @@ -2548,7 +2569,8 @@ int TABLE_SHARE::init_from_binary_frm_image(THD *thd, bool write, */ uint vcol_info_length= (uint) strpos[12]; - if (!vcol_info_length) // Expect non-null expression + if (vcol_info_length < 4 || + vcol_screen_pos + vcol_info_length > vcol_screen_end) goto err; attr.frm_unpack_basic(strpos); @@ -2649,15 +2671,17 @@ int TABLE_SHARE::init_from_binary_frm_image(THD *thd, bool write, byte 4 = stored_in_db byte 5.. = expr */ - if ((uint)(vcol_screen_pos)[0] != 1) + if (vcol_screen_pos + MYSQL57_GCOL_HEADER_SIZE > vcol_screen_end || + vcol_screen_pos[0] != 1) goto err; vcol_info= new (&share->mem_root) Virtual_column_info(); uint vcol_info_length= uint2korr(vcol_screen_pos + 1); - if (!vcol_info_length) // Expect non-empty expression - goto err; vcol_info->set_vcol_type(vcol_screen_pos[3] ? VCOL_GENERATED_STORED : VCOL_GENERATED_VIRTUAL); vcol_info->utf8= 0; vcol_screen_pos+= vcol_info_length + MYSQL57_GCOL_HEADER_SIZE;; + if (!vcol_info_length || // Expect non-empty expression + vcol_screen_pos > vcol_screen_end) + goto err; share->virtual_fields++; } } @@ -2714,6 +2738,9 @@ int TABLE_SHARE::init_from_binary_frm_image(THD *thd, bool write, attr.pack_flag&= ~FIELDFLAG_LONG_DECIMAL; } + if (interval_nr > interval_count) + goto err; + if (interval_nr && attr.charset->mbminlen > 1 && !interval_unescaped[interval_nr - 1]) { @@ -2807,6 +2834,52 @@ int TABLE_SHARE::init_from_binary_frm_image(THD *thd, bool write, if (!reg_field) // Not supported field type goto err; + /* + Verify that fields follow one another in the record without overlaps. + That is the next field starts where the previous field ended. + Virtual fields complicate the check. Stored fields follow each other in + the record order, virtual fields also follow each in the record order + but they're always at the end of the record, and can be interleaved + with normal fields in the table. + + We detect this dynamically - as long as the next field starts where the + previous ended, assume they're normal fields. When the next field starts + with a gap - it's the first virtual (and the future virtual fields must + follow that field). If the next field starts before the previous - it + means the table started from virtual fields and this is the first stored + field. Either ("gap" or "before") can happen only once. + */ + if (!next_stored) // first field, i == 0 + { + first_stored= reg_field->ptr; + next_stored= first_stored + reg_field->pack_length_in_rec(); + } + else if (reg_field->ptr == next_stored) // next stored field + next_stored+= reg_field->pack_length_in_rec(); + else if (reg_field->ptr > next_stored) // virtual + { + if (next_virtual) + { + if (reg_field->ptr != next_virtual) + goto err; + next_virtual+= reg_field->pack_length_in_rec(); + } + else + { + first_virtual= reg_field->ptr; + next_virtual= first_virtual + reg_field->pack_length_in_rec(); + } + } + else // reg_field < next_stored. can happen if the i=0 field was virtual + { + if (next_virtual) + goto err; + first_virtual= first_stored; + next_virtual= next_stored; + first_stored= reg_field->ptr; + next_stored= first_stored + reg_field->pack_length_in_rec(); + } + if (attr.unireg_check == Field::TIMESTAMP_DNUN_FIELD || attr.unireg_check == Field::TIMESTAMP_DN_FIELD) { @@ -2893,7 +2966,8 @@ int TABLE_SHARE::init_from_binary_frm_image(THD *thd, bool write, /* We want to store the value for the last bits */ swap_variables(uchar*, null_pos, mysql57_vcol_null_pos); swap_variables(uint, null_bit_pos, mysql57_vcol_null_bit_pos); - DBUG_ASSERT((null_pos + (null_bit_pos + 7) / 8) <= share->field[0]->ptr); + if ((null_pos + (null_bit_pos + 7) / 8) > share->field[0]->ptr) + goto err; } share->primary_key= MAX_KEY; @@ -2923,7 +2997,9 @@ int TABLE_SHARE::init_from_binary_frm_image(THD *thd, bool write, hash_keypart->fieldnr= hash_field_used_no + 1; hash_field= share->field[hash_field_used_no]; hash_field->flags|= LONG_UNIQUE_HASH_FIELD;//Used in parse_vcol_defs - DBUG_ASSERT(hash_field->invisible == INVISIBLE_FULL); + if (hash_field->invisible != INVISIBLE_FULL || + hash_field->pack_length() != HA_HASH_FIELD_LENGTH) + goto err; keyinfo->flags|= HA_NOSAME; share->virtual_fields++; share->stored_fields--; @@ -2951,7 +3027,8 @@ int TABLE_SHARE::init_from_binary_frm_image(THD *thd, bool write, key_part= keyinfo->key_part; for (i=0 ; i < keyinfo->user_defined_key_parts ;i++) { - DBUG_ASSERT(key_part[i].fieldnr > 0); + if (key_part[i].fieldnr <= 0 || key_part[i].fieldnr > share->fields) + goto err; // Table field corresponding to the i'th key part. Field *table_field= share->field[key_part[i].fieldnr - 1]; @@ -3066,6 +3143,8 @@ int TABLE_SHARE::init_from_binary_frm_image(THD *thd, bool write, for (i= 0; i < first_key_parts; i++) { uint fieldnr= keyinfo[0].key_part[i].fieldnr; + if (fieldnr <= 0 || fieldnr > share->fields) + goto err; if (share->field[fieldnr-1]->key_length() != keyinfo[0].key_part[i].length) { @@ -3118,6 +3197,8 @@ int TABLE_SHARE::init_from_binary_frm_image(THD *thd, bool write, { uint length_bytes= 0; uint fieldnr= keyinfo->key_part[i].fieldnr; + if (fieldnr <= 0 || fieldnr > share->fields) + goto err; field= share->field[fieldnr-1]; if (field->null_ptr) @@ -3198,7 +3279,7 @@ int TABLE_SHARE::init_from_binary_frm_image(THD *thd, bool write, share->default_values, (uint) key_part->offset, (uint) key_part->length); - if (!key_part->fieldnr) + if (key_part->fieldnr <= 0 || key_part->fieldnr > share->fields) goto err; field= key_part->field= share->field[key_part->fieldnr-1]; @@ -3360,7 +3441,7 @@ int TABLE_SHARE::init_from_binary_frm_image(THD *thd, bool write, DBUG_ASSERT(share->primary_key == MAX_KEY); } } - if (new_field_pack_flag <= 1) + if (new_field_pack_flag <= 1 && share->null_fields) { /* Old file format with default as not null */ uint null_length= (share->null_fields+7)/8; @@ -3373,8 +3454,6 @@ int TABLE_SHARE::init_from_binary_frm_image(THD *thd, bool write, /* Handle virtual expressions */ if (vcol_screen_length && share->frm_version >= FRM_VER_EXPRESSSIONS) { - uchar *vcol_screen_end= vcol_screen_pos + vcol_screen_length; - /* Skip header */ vcol_screen_pos+= FRM_VCOL_NEW_BASE_SIZE; share->vcol_defs.str+= FRM_VCOL_NEW_BASE_SIZE; @@ -3387,6 +3466,9 @@ int TABLE_SHARE::init_from_binary_frm_image(THD *thd, bool write, while (vcol_screen_pos < vcol_screen_end) { Virtual_column_info *vcol_info; + if (vcol_screen_end - vcol_screen_pos < FRM_VCOL_NEW_HEADER_SIZE) + goto err; + uint type= (uint) vcol_screen_pos[0]; uint field_nr= uint2korr(vcol_screen_pos+1); uint expr_length= uint2korr(vcol_screen_pos+3); @@ -3399,17 +3481,21 @@ int TABLE_SHARE::init_from_binary_frm_image(THD *thd, bool write, if (field_nr != UINT_MAX16) { - DBUG_ASSERT(field_nr < share->fields); + if (field_nr >= share->fields) + goto err; reg_field= share->field[field_nr]; } else { reg_field= 0; - DBUG_ASSERT(name_length); + if (!name_length || type != VCOL_CHECK_TABLE) + goto err; } vcol_screen_pos+= FRM_VCOL_NEW_HEADER_SIZE; vcol_info->set_vcol_type((enum_vcol_info_type) type); + if ((uint)(vcol_screen_end-vcol_screen_pos) < name_length+expr_length) + goto err; if (name_length) { vcol_info->name.str= strmake_root(&share->mem_root, @@ -3423,7 +3509,8 @@ int TABLE_SHARE::init_from_binary_frm_image(THD *thd, bool write, switch (type) { case VCOL_GENERATED_VIRTUAL: { - uint recpos; + if (reg_field->vcol_info) + goto err; reg_field->vcol_info= vcol_info; share->virtual_fields++; share->stored_fields--; @@ -3432,35 +3519,41 @@ int TABLE_SHARE::init_from_binary_frm_image(THD *thd, bool write, if (reg_field->flags & PART_KEY_FLAG) vcol_info->set_vcol_type(VCOL_GENERATED_VIRTUAL_INDEXED); /* Correct stored_rec_length as non stored fields are last */ - recpos= (uint) (reg_field->ptr - record); + uint recpos= (uint) (reg_field->ptr - record); if (share->stored_rec_length >= recpos) share->stored_rec_length= recpos-1; break; } case VCOL_GENERATED_STORED: - DBUG_ASSERT(!reg_field->vcol_info); + if (reg_field->vcol_info) + goto err; reg_field->vcol_info= vcol_info; share->virtual_fields++; break; case VCOL_DEFAULT: - DBUG_ASSERT(!reg_field->default_value); + if (reg_field->default_value) + goto err; reg_field->default_value= vcol_info; share->default_expressions++; break; case VCOL_CHECK_FIELD: - DBUG_ASSERT(!reg_field->check_constraint); + if (reg_field->check_constraint) + goto err; reg_field->check_constraint= vcol_info; share->field_check_constraints++; break; case VCOL_CHECK_TABLE: - *(table_check_constraints++)= vcol_info; + if ((uint)(table_check_constr - share->check_constraints) >= + share->table_check_constraints) + goto err; + *(table_check_constr++)= vcol_info; break; } } } - DBUG_ASSERT((uint) (table_check_constraints - share->check_constraints) == - (uint) (share->table_check_constraints - - share->field_check_constraints)); + if (table_check_constr - share->check_constraints != + (int)share->table_check_constraints - (int)share->field_check_constraints) + goto err; if (options.str) { @@ -3530,6 +3623,19 @@ int TABLE_SHARE::init_from_binary_frm_image(THD *thd, bool write, share->can_cmp_whole_record= (share->blob_fields == 0 && share->varchar_fields == 0); + data_start= share->default_values; + data_end= data_start + share->reclength; + if (share->null_field_first) + data_start+= share->null_bytes; + else + data_end-= share->null_bytes; + + if (data_end < data_start || + (first_virtual && first_virtual != next_stored) || + first_stored < data_start || + (next_virtual ? next_virtual : next_stored) > data_end) + goto err; + share->column_bitmap_size= bitmap_buffer_size(share->fields); bitmap_count= 1; @@ -5032,19 +5138,20 @@ void open_table_error(TABLE_SHARE *share, enum open_frm_error error, } /* open_table_error */ - /* - ** fix a str_type to a array type - ** typeparts separated with some char. different types are separated - ** with a '\0' - */ +/* + fix a str_type to a array type + typeparts separated with some char. different types are separated + with a '\0' +*/ static bool -fix_type_pointers(const char ***typelib_value_names, +fix_type_pointers(const char ***typelib_value_names, const char **names_end, uint **typelib_value_lengths, TYPELIB *point_to_type, uint types, char *ptr, size_t length) { const char *end= ptr + length; + names_end--; // simplify the check below, reserve place for 0 at the end while (types--) { @@ -5076,7 +5183,7 @@ fix_type_pointers(const char ***typelib_value_names, { // Now scan the next value+sep pair char *vend= (char*) memchr(ptr, sep, end - ptr); - if (!vend) + if (!vend || *typelib_value_names >= names_end) return true; // Bad format *((*typelib_value_names)++)= ptr; *((*typelib_value_lengths)++)= (uint) (vend - ptr); diff --git a/sql/table_cache.cc b/sql/table_cache.cc index 60d26b8c7f3e1..f7f06d95cde2c 100644 --- a/sql/table_cache.cc +++ b/sql/table_cache.cc @@ -1,5 +1,5 @@ /* Copyright (c) 2000, 2012, Oracle and/or its affiliates. - Copyright (c) 2010, 2022, MariaDB Corporation. + Copyright (c) 2010, 2026, MariaDB plc. Copyright (C) 2013 Sergey Vojtovich and MariaDB Foundation This program is free software; you can redistribute it and/or modify @@ -56,7 +56,7 @@ /** Configuration. */ ulong tdc_size; /**< Table definition cache threshold for LRU eviction. */ ulong tc_size; /**< Table cache threshold for LRU eviction. */ -uint32 tc_instances; +READ_ONLY_SYSVAR uint32 tc_instances; static size_t tc_allocated_size; static std::atomic tc_active_instances(1); static std::atomic tc_contention_warning_reported; diff --git a/sql/threadpool_common.cc b/sql/threadpool_common.cc index 04e652ba1f880..93b292176d4d7 100644 --- a/sql/threadpool_common.cc +++ b/sql/threadpool_common.cc @@ -1,4 +1,4 @@ -/* Copyright (C) 2012, 2020, MariaDB +/* Copyright (C) 2012, 2026, MariaDB plc. This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by @@ -43,7 +43,7 @@ uint threadpool_max_size; uint threadpool_stall_limit; uint threadpool_max_threads; uint threadpool_oversubscribe; -uint threadpool_mode; +READ_ONLY_SYSVAR uint threadpool_mode; uint threadpool_prio_kickup_timer; my_bool threadpool_exact_stats; my_bool threadpool_dedicated_listener; diff --git a/sql/vector_mhnsw.cc b/sql/vector_mhnsw.cc index d640363b6e76a..ce0af25f8c04a 100644 --- a/sql/vector_mhnsw.cc +++ b/sql/vector_mhnsw.cc @@ -1012,6 +1012,13 @@ int FVectorNode::load_from_record(TABLE *graph) FVector *vec_ptr= FVector::align_ptr(tref() + tref_len()); memcpy(vec_ptr->data(), v->ptr(), v->length()); vec_ptr->postprocess(ctx->use_subdist, ctx->vec_len); + /* + For COSINE the vector was normalized in FVector::create(), so abs2 is + known to be exactly 0.5. Recomputing it from the quantized int16 + coordinates only adds rounding noise and degrades the recall (MDEV-39858). + */ + if (ctx->metric == COSINE) + vec_ptr->abs2= 0.5f; longlong layer= graph->field[FIELD_LAYER]->val_int(); if (layer > 100) // 10e30 nodes at M=2, more at larger M's diff --git a/sql/wsrep_mysqld.cc b/sql/wsrep_mysqld.cc index 6ea1addf0e6fe..27bba02ee92b8 100644 --- a/sql/wsrep_mysqld.cc +++ b/sql/wsrep_mysqld.cc @@ -73,7 +73,7 @@ const char *wsrep_SR_store_types[]= { "none", "table", NullS }; extern my_bool plugins_are_initialized; /* System variables. */ -const char *wsrep_provider; +READ_ONLY_SYSVAR const char *wsrep_provider; const char *wsrep_provider_options; const char *wsrep_cluster_address; const char *wsrep_cluster_name; @@ -81,11 +81,11 @@ const char *wsrep_node_name; const char *wsrep_node_address; const char *wsrep_node_incoming_address; const char *wsrep_start_position; -const char *wsrep_data_home_dir; +READ_ONLY_SYSVAR const char *wsrep_data_home_dir; const char *wsrep_dbug_option; -const char *wsrep_notify_cmd; -const char *wsrep_status_file; -const char *wsrep_allowlist; +READ_ONLY_SYSVAR const char *wsrep_notify_cmd; +READ_ONLY_SYSVAR const char *wsrep_status_file; +READ_ONLY_SYSVAR const char *wsrep_allowlist; ulong wsrep_debug; // Debug level logging my_bool wsrep_convert_LOCK_to_trx; // Convert locking sessions to trx @@ -93,7 +93,7 @@ my_bool wsrep_auto_increment_control; // Control auto increment variab my_bool wsrep_drupal_282555_workaround; // Retry autoinc insert after dupkey my_bool wsrep_certify_nonPK; // Certify, even when no primary key ulong wsrep_certification_rules = WSREP_CERTIFICATION_RULES_STRICT; -my_bool wsrep_recovery; // Recovery +READ_ONLY_SYSVAR my_bool wsrep_recovery; // Recovery my_bool wsrep_log_conflicts; my_bool wsrep_load_data_splitting= 0; // Commit load data every 10K intervals my_bool wsrep_slave_UK_checks; // Slave thread does UK checks @@ -128,7 +128,7 @@ long wsrep_max_protocol_version= 4; // Maximum protocol version to u long int wsrep_protocol_version= wsrep_max_protocol_version; ulong wsrep_trx_fragment_unit= WSREP_FRAG_BYTES; // unit for fragment size -ulong wsrep_SR_store_type= WSREP_SR_STORE_TABLE; +READ_ONLY_SYSVAR ulong wsrep_SR_store_type= WSREP_SR_STORE_TABLE; uint wsrep_ignore_apply_errors= 0; uint wsrep_applier_retry_count= 0; @@ -3410,10 +3410,11 @@ static void wsrep_mdl_log(wsrep_mdl_log_t level, */ static void wsrep_log_state(const char *msg, const THD *thd, bool granted) { - char buff[2048]; + char buff[2048]={'\0'}; String buffer(buff, sizeof(buff), system_charset_info); + buffer.length(0); wsrep_get_state(thd, granted, &buffer); - WSREP_DEBUG(msg, buffer.c_ptr()); + WSREP_DEBUG("%s %s", msg, buffer.c_ptr()); } /** This function handles MDL-conflict when thread holding MDL-lock @@ -3442,8 +3443,8 @@ static void wsrep_handle_granted_bf( if (wsrep_debug) { - wsrep_log_state("wsrep_handle_granted_bf() : (%s)", request_thd, false); - wsrep_log_state("wsrep_handle_granted_bf() : (%s)", granted_thd, true); + wsrep_log_state("wsrep_handle_granted_bf(): ", request_thd, false); + wsrep_log_state("wsrep_handle_granted_bf(): ", granted_thd, true); } if (wsrep_thd_is_aborting(granted_thd)) @@ -3494,8 +3495,8 @@ static void wsrep_handle_locked(THD* request_thd, if (wsrep_debug) { - wsrep_log_state("wsrep_handle_locked() : (%s)", request_thd, false); - wsrep_log_state("wsrep_handle_locked() : (%s)", granted_thd, true); + wsrep_log_state("wsrep_handle_locked(): ", request_thd, false); + wsrep_log_state("wsrep_handle_locked(): ", granted_thd, true); } if (granted_thd->current_backup_stage != BACKUP_FINISHED && @@ -3534,8 +3535,8 @@ static void wsrep_abort_granted(THD* request_thd, if (wsrep_debug) { - wsrep_log_state("wsrep_abort_granted() : (%s)", request_thd, false); - wsrep_log_state("wsrep_abort_granted() : (%s)", granted_thd, true); + wsrep_log_state("wsrep_abort_granted(): ", request_thd, false); + wsrep_log_state("wsrep_abort_granted(): ", granted_thd, true); } wsrep_mdl_log(WSREP_MDL_DEBUG, "MDL conflict-> BF abort", @@ -3608,8 +3609,8 @@ void wsrep_handle_mdl_conflict(MDL_context *requestor_ctx, if (wsrep_debug) { - wsrep_log_state("wsrep_handle_mdl_conflict() : (%s)", request_thd, false); - wsrep_log_state("wsrep_handle_mdl_conflict() : (%s)", granted_thd, true); + wsrep_log_state("wsrep_handle_mdl_conflict(): ", request_thd, false); + wsrep_log_state("wsrep_handle_mdl_conflict(): ", granted_thd, true); } if (granted_thd->wsrep_aborter != 0) diff --git a/sql/wsrep_plugin.cc b/sql/wsrep_plugin.cc index b6446afe4aa14..1e0e94df8b5fc 100644 --- a/sql/wsrep_plugin.cc +++ b/sql/wsrep_plugin.cc @@ -20,15 +20,11 @@ in favor of single options which are initialized from provider. */ -#include "sql_plugin.h" -#include "sql_priv.h" -#include "sql_class.h" -#include "set_var.h" - #include "my_global.h" #include "mysqld_error.h" #include +#include "wsrep_on.h" // WSREP_ON #include "wsrep_mysqld.h" #include "wsrep/provider_options.hpp" #include "wsrep_server_state.h" @@ -286,21 +282,12 @@ static int wsrep_provider_plugin_init(void *p) } provider_plugin_enabled= true; - - // When plugin-wsrep-provider is enabled we set - // wsrep_provider_options parameter as READ_ONLY - sys_var *my_var= find_sys_var(current_thd, "wsrep_provider_options"); - int flags= my_var->get_flags(); - my_var->update_flags(flags |= (int)sys_var::READONLY); return 0; } static int wsrep_provider_plugin_deinit(void *p) { WSREP_DEBUG("wsrep_provider_plugin_deinit()"); - sys_var *my_var= find_sys_var(current_thd, "wsrep_provider_options"); - int flags= my_var->get_flags(); - my_var->update_flags(flags &= (int)~sys_var::READONLY); return 0; } diff --git a/sql/wsrep_sst.cc b/sql/wsrep_sst.cc index f22a75caa97ef..42a8078d78b78 100644 --- a/sql/wsrep_sst.cc +++ b/sql/wsrep_sst.cc @@ -397,9 +397,6 @@ static bool wsrep_sst_complete (THD* thd, Wsrep_server_state& server_state= Wsrep_server_state::instance(); enum wsrep::server_state::state state= server_state.state(); bool failed= false; - char start_pos_buf[FN_REFLEN]; - ssize_t len= wsrep::print_to_c_str(sst_gtid, start_pos_buf, FN_REFLEN-1); - start_pos_buf[len]='\0'; // Do not call sst_received if we are not in joiner or // initialized state on server. This is because it @@ -414,14 +411,31 @@ static bool wsrep_sst_complete (THD* thd, } else { - WSREP_INFO("SST succeeded for position %s", start_pos_buf); + /* + Note: sst_received() does NOT use sst_gtid (the position reported by + the SST script). It determines the position internally from storage via + Wsrep_server_service::get_position(). + For physical SST methods these two may differ (e.g. the joiner's storage + recovers to an earlier position than the script reported). Log the + position actually adopted, not the script-reported one, to avoid + confusion. + */ + wsrep::gtid const received_gtid= wsrep_get_SE_checkpoint(); + char recv_pos_buf[FN_REFLEN]; + ssize_t const recv_len= + wsrep::print_to_c_str(received_gtid, recv_pos_buf, FN_REFLEN-1); + recv_pos_buf[recv_len > 0 ? recv_len : 0]= '\0'; + WSREP_INFO("SST succeeded for position %s", recv_pos_buf); } } else { + char start_pos_buf[FN_REFLEN]; + ssize_t const len= wsrep::print_to_c_str(sst_gtid, start_pos_buf, FN_REFLEN - 1); + start_pos_buf[len > 0 ? len : 0]= '\0'; + WSREP_ERROR("SST failed for position %s initialized %d server_state %s", - start_pos_buf, - server_state.is_initialized(), + start_pos_buf, server_state.is_initialized(), wsrep::to_c_string(state)); failed= true; } diff --git a/sql/wsrep_var.cc b/sql/wsrep_var.cc index d5eeca3902e6e..d2f52574a609a 100644 --- a/sql/wsrep_var.cc +++ b/sql/wsrep_var.cc @@ -84,7 +84,7 @@ static bool refresh_provider_options() { std::string opts= Wsrep_server_state::instance().provider().options(); wsrep_provider_options_init(opts.c_str()); - get_provider_option_value(wsrep_provider_options, + get_provider_option_value(opts.c_str(), (char*)"repl.max_ws_size", &wsrep_max_ws_size); return false; @@ -534,7 +534,9 @@ bool wsrep_provider_options_check(sys_var *self, THD* thd, set_var* var) } if (wsrep_provider_plugin_enabled()) { - my_error(ER_INCORRECT_GLOBAL_LOCAL_VAR, MYF(0), var->var->name.str, "read only"); + my_message(ER_WRONG_ARGUMENTS, + "wsrep_provider_options cannot be changed while the " + "wsrep-provider plugin is loaded", MYF(0)); return true; } return false; diff --git a/storage/archive/ha_archive.cc b/storage/archive/ha_archive.cc index c4c7046f111fc..1db9f9fedd930 100644 --- a/storage/archive/ha_archive.cc +++ b/storage/archive/ha_archive.cc @@ -945,14 +945,21 @@ uint32 ha_archive::max_row_length(const uchar *record) unsigned int ha_archive::pack_row(const uchar *record, azio_stream *writer) { uchar *ptr; + uint32 max_len; my_ptrdiff_t const rec_offset= record - table->record[0]; DBUG_ENTER("ha_archive::pack_row"); - if (fix_rec_buff(max_row_length(record))) - DBUG_RETURN(HA_ERR_OUT_OF_MEM); /* purecov: inspected */ + max_len= max_row_length(record); if (writer->version == 1) + { + if (fix_rec_buff(max_len)) + DBUG_RETURN(HA_ERR_OUT_OF_MEM); /* purecov: inspected */ DBUG_RETURN(pack_row_v1(record)); + } + + if (fix_rec_buff(max_len + ARCHIVE_ROW_HEADER_SIZE)) + DBUG_RETURN(HA_ERR_OUT_OF_MEM); /* Copy null bits */ memcpy(record_buffer->buffer+ARCHIVE_ROW_HEADER_SIZE, @@ -1616,6 +1623,46 @@ int ha_archive::optimize(THD* thd, HA_CHECK_OPT* check_opt) while (!(rc= get_row(&archive, table->record[0]))) { + /* + If the source is version 3, unpack_row() stores blob data in + record_buffer. Since pack_row() also uses record_buffer for + packing, we must copy blob data out first to avoid corruption. + Version 2 already stores blobs in the 'buffer' member. + */ + if (archive.version >= 3) + { + uint *blob_ptr, *blob_end; + size_t total_blob_length= 0; + for (blob_ptr= table->s->blob_field, + blob_end= blob_ptr + table->s->blob_fields; + blob_ptr != blob_end; blob_ptr++) + { + Field_blob *blob= (Field_blob*) table->field[*blob_ptr]; + if (!blob->is_null()) + total_blob_length+= blob->get_length(); + } + if (buffer.alloc(total_blob_length)) + { + rc= HA_ERR_OUT_OF_MEM; + break; + } + char *pos= (char*) buffer.ptr(); + for (blob_ptr= table->s->blob_field, + blob_end= blob_ptr + table->s->blob_fields; + blob_ptr != blob_end; blob_ptr++) + { + Field_blob *blob= (Field_blob*) table->field[*blob_ptr]; + if (blob->is_null()) + continue; + uint32 length= blob->get_length(); + if (length) + { + memcpy(pos, blob->get_ptr(), length); + blob->set_ptr(length, (uchar*) pos); + pos+= length; + } + } + } real_write_row(table->record[0], &writer); /* Long term it should be possible to optimize this so that @@ -1635,6 +1682,7 @@ int ha_archive::optimize(THD* thd, HA_CHECK_OPT* check_opt) tmp_restore_column_map(&table->read_set, org_bitmap); share->rows_recorded= (ha_rows)writer.rows; + buffer.free(); } DBUG_PRINT("info", ("recovered %llu archive rows", diff --git a/storage/connect/CMakeLists.txt b/storage/connect/CMakeLists.txt index fe4269859fb6a..f11b3dc772ec9 100644 --- a/storage/connect/CMakeLists.txt +++ b/storage/connect/CMakeLists.txt @@ -29,14 +29,14 @@ SET(CONNECT_SOURCES ha_connect.cc connect.cc user_connect.cc mycat.cc fmdlex.c osutil.c rcmsg.c rcmsg.h array.cpp blkfil.cpp colblk.cpp csort.cpp -filamap.cpp filamdbf.cpp filamfix.cpp filamgz.cpp filamtxt.cpp filter.cpp +filamap.cpp filamdbf.cpp filamfix.cpp filamgz.cpp filamtxt.cpp filechk.cpp filter.cpp json.cpp jsonudf.cpp maputil.cpp myconn.cpp myutil.cpp plgdbutl.cpp plugutil.cpp reldef.cpp tabcol.cpp tabdos.cpp tabext.cpp tabfix.cpp tabfmt.cpp tabjson.cpp table.cpp tabmul.cpp tabmysql.cpp taboccur.cpp tabpivot.cpp tabsys.cpp tabtbl.cpp tabutil.cpp tabvir.cpp tabxcl.cpp valblk.cpp value.cpp xindex.cpp xobject.cpp array.h blkfil.h block.h catalog.h checklvl.h colblk.h connect.h csort.h -engmsg.h filamap.h filamdbf.h filamfix.h filamgz.h filamtxt.h +engmsg.h filamap.h filamdbf.h filamfix.h filamgz.h filamtxt.h filechk.h filter.h global.h ha_connect.h inihandl.h json.h jsonudf.h maputil.h msgid.h mycat.h myconn.h myutil.h os.h osutil.h plgcnx.h plgdbsem.h preparse.h reldef.h resource.h tabcol.h tabdos.h tabext.h tabfix.h tabfmt.h tabjson.h tabmul.h diff --git a/storage/connect/bsonudf.cpp b/storage/connect/bsonudf.cpp index c12eefb5b671d..41c721f622782 100644 --- a/storage/connect/bsonudf.cpp +++ b/storage/connect/bsonudf.cpp @@ -16,6 +16,7 @@ #include #include "bsonudf.h" +#include "filechk.h" #if defined(UNIX) || defined(UNIV_LINUX) #define _O_RDONLY O_RDONLY @@ -4478,6 +4479,16 @@ char *bson_file(UDF_INIT *initid, UDF_ARGS *args, char *result, PlugSubSet(g->Sarea, g->Sarea_Size); fn = MakePSZ(g, args, 0); + if (!fn) { + PUSH_WARNING("Missing file name"); + *is_null = 1; + return NULL; + } else if (!connect_can_access_file(fn)) { + PUSH_WARNING("Access denied: FILE privilege or secure_file_priv violation"); + *is_null = 1; + return NULL; + } + if (args->arg_count > 1) { int pretty = 3, pty = 3; size_t len; @@ -4629,7 +4640,10 @@ char *bfile_make(UDF_INIT *initid, UDF_ARGS *args, char *result, } // endswitch arg_type if (fn) { - if (!bnx.Serialize(g, jvp, fn, pretty)) + if (!connect_can_access_file(fn)) { + PUSH_WARNING("Access denied: FILE privilege or secure_file_priv violation"); + fn = NULL; + } else if (!bnx.Serialize(g, jvp, fn, pretty)) PUSH_WARNING(g->Message); } else PUSH_WARNING("Missing file name"); @@ -6065,6 +6079,16 @@ char *bbin_file(UDF_INIT *initid, UDF_ARGS *args, char *result, fn = MakePSZ(g, args, 0); + if (!fn) { + PUSH_WARNING("Missing file name"); + *error = 1; + goto fin; + } else if (!connect_can_access_file(fn)) { + PUSH_WARNING("Access denied: FILE privilege or secure_file_priv violation"); + *error = 1; + goto fin; + } + for (unsigned int i = 1; i < args->arg_count; i++) if (args->arg_type[i] == INT_RESULT && *(longlong*)args->args[i] < 4) { pretty = (int) * (longlong*)args->args[i]; diff --git a/storage/connect/filamdbf.cpp b/storage/connect/filamdbf.cpp index ee65e0cf5dbc0..d0972ac131fa6 100644 --- a/storage/connect/filamdbf.cpp +++ b/storage/connect/filamdbf.cpp @@ -246,7 +246,7 @@ PQRYRES DBFColumns(PGLOBAL g, PCSZ dp, PCSZ fn, PTOS topt, bool info) bool bad, mul; PCSZ target, pwd; DBFHEADER mainhead, *hp = NULL; - DESCRIPTOR thisfield, *tfp; + DESCRIPTOR thisfield, *tfp= &thisfield; FILE *infile = NULL; UNZIPUTL *zutp = NULL; PQRYRES qrp; @@ -308,7 +308,6 @@ PQRYRES DBFColumns(PGLOBAL g, PCSZ dp, PCSZ fn, PTOS topt, bool info) return NULL; } // endif dbfhead - tfp = &thisfield; } // endif zipped /************************************************************************/ diff --git a/storage/connect/filechk.cpp b/storage/connect/filechk.cpp new file mode 100644 index 0000000000000..03e88633e99dd --- /dev/null +++ b/storage/connect/filechk.cpp @@ -0,0 +1,41 @@ +/* Copyright (C) MariaDB Corporation Ab + + This program is free software; you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation; version 2 of the License. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program; if not, write to the Free Software + Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1335 USA +*/ + +#include +#include +#include +#include "filechk.h" + +/* + @return true if the current user may access path. +*/ +bool connect_can_access_file(const char *path) +{ + char real_path[FN_REFLEN]; + + if (!path) + return false; /* callers must check for NULL before calling */ + + if (check_global_access(current_thd, FILE_ACL, true)) + return false; + + /* MY_SAFE_PATH returns NULL on overflow; fail closed rather than validate a truncated path */ + if (!fn_format(real_path, path, mysql_real_data_home, "", + MY_RELATIVE_PATH | MY_UNPACK_FILENAME | MY_RETURN_REAL_PATH | MY_SAFE_PATH)) + return false; + + return is_secure_file_path(real_path); +} diff --git a/storage/connect/filechk.h b/storage/connect/filechk.h new file mode 100644 index 0000000000000..75ab7eda09ec6 --- /dev/null +++ b/storage/connect/filechk.h @@ -0,0 +1,20 @@ +/* Copyright (C) MariaDB Corporation Ab + + This program is free software; you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation; version 2 of the License. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program; if not, write to the Free Software + Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1335 USA +*/ + +/* File access checks for CONNECT file UDFs */ +#pragma once +bool connect_can_access_file(const char *path); + diff --git a/storage/connect/jsonudf.cpp b/storage/connect/jsonudf.cpp index cc916feaeaa7a..e11cb2a466be5 100644 --- a/storage/connect/jsonudf.cpp +++ b/storage/connect/jsonudf.cpp @@ -15,6 +15,7 @@ #include #include "jsonudf.h" +#include "filechk.h" #if defined(UNIX) || defined(UNIV_LINUX) #define _O_RDONLY O_RDONLY @@ -4562,6 +4563,16 @@ char *json_file(UDF_INIT *initid, UDF_ARGS *args, char *result, PlugSubSet(g->Sarea, g->Sarea_Size); fn = MakePSZ(g, args, 0); + if (!fn) { + PUSH_WARNING("Missing file name"); + *is_null = 1; + return NULL; + } else if (!connect_can_access_file(fn)) { + PUSH_WARNING("Access denied: FILE privilege or secure_file_priv violation"); + *is_null = 1; + return NULL; + } + if (args->arg_count > 1) { int pretty = 3, pty = 3; size_t len; @@ -4715,7 +4726,10 @@ char *jfile_make(UDF_INIT *initid, UDF_ARGS *args, char *result, } // endswitch arg_type if (fn) { - if (!Serialize(g, jvp->GetJson(), fn, pretty)) + if (!connect_can_access_file(fn)) { + PUSH_WARNING("Access denied: FILE privilege or secure_file_priv violation"); + fn = NULL; + } else if (!Serialize(g, jvp->GetJson(), fn, pretty)) PUSH_WARNING(g->Message); } else PUSH_WARNING("Missing file name"); @@ -5767,8 +5781,8 @@ my_bool jbin_file_init(UDF_INIT *initid, UDF_ARGS *args, char *message) if (args->arg_count < 1 || args->arg_count > 4) { strcpy(message, "This function only accepts 1 to 4 arguments"); return true; - } else if (args->arg_type[0] != STRING_RESULT || !args->args[0]) { - strcpy(message, "First argument must be a constant string (file name)"); + } else if (args->arg_type[0] != STRING_RESULT) { + strcpy(message, "First argument must be a string (file name)"); return true; } // endifs @@ -5786,7 +5800,10 @@ my_bool jbin_file_init(UDF_INIT *initid, UDF_ARGS *args, char *message) initid->maybe_null = 1; CalcLen(args, false, reslen, memlen); - fl = GetFileLength(args->args[0]); + if (args->args[0]) + fl = GetFileLength(args->args[0]); + else + fl = 100; // What can be done here? reslen += fl; more += fl * M; //memlen += more; @@ -5811,6 +5828,16 @@ char *jbin_file(UDF_INIT *initid, UDF_ARGS *args, char *result, g->Xchk = NULL; fn = MakePSZ(g, args, 0); + if (!fn) { + PUSH_WARNING("Missing file name"); + *error = 1; + goto fin; + } else if (!connect_can_access_file(fn)) { + PUSH_WARNING("Access denied: FILE privilege or secure_file_priv violation"); + *error = 1; + goto fin; + } + for (unsigned int i = 1; i < args->arg_count; i++) if (args->arg_type[i] == INT_RESULT && *(longlong*)args->args[i] < 4) { pretty = (int) * (longlong*)args->args[i]; diff --git a/storage/connect/mysql-test/connect/r/drop-open-error.result b/storage/connect/mysql-test/connect/r/drop-open-error.result index f9b9b7e87d29f..8f5341bbcceb0 100644 --- a/storage/connect/mysql-test/connect/r/drop-open-error.result +++ b/storage/connect/mysql-test/connect/r/drop-open-error.result @@ -5,11 +5,3 @@ drop table mdev9949; Warnings: Warning 1017 Can't find file: 'DATADIR/test/mdev9949.dos' (errno: 2 "No such file or directory") drop table t1; -select @@secure_file_priv 'must be NULL'; -must be NULL -NULL -create table t1 (a char(16)) engine=myisam; -insert into t1 values('Hello World!'); -create table t2 engine=connect file_name='foo/bar.txt' as select * from t1; -ERROR HY000: Got error 174 'Open(a+b) error 2 on foo/bar.txt: No such file or directory' from CONNECT -drop table t1; diff --git a/storage/connect/mysql-test/connect/r/fix.result b/storage/connect/mysql-test/connect/r/fix.result index 87b70ab8b2233..f4cc758abbd39 100644 --- a/storage/connect/mysql-test/connect/r/fix.result +++ b/storage/connect/mysql-test/connect/r/fix.result @@ -129,3 +129,13 @@ Sam Chicago 1979-11-22 2007-10-10 James Dallas 1992-05-13 2009-12-14 Bill Boston 1986-09-11 2008-02-10 DROP TABLE t1; +# +# MDEV-40637 CONNECT crashes on double(255,50) in DOS table +# +create table t (x double(255,30)) engine=connect table_type='dos' file_name='t.dos'; +insert into t values (1e50); +select * from t; +x +1000000000000000000000000000000.000000000000000000000000000000 +drop table t; +# End of 10.6 tests diff --git a/storage/connect/mysql-test/connect/r/json_bson_file_priv.result b/storage/connect/mysql-test/connect/r/json_bson_file_priv.result new file mode 100644 index 0000000000000..34cbf8c972b0e --- /dev/null +++ b/storage/connect/mysql-test/connect/r/json_bson_file_priv.result @@ -0,0 +1,156 @@ +# +# Test FILE privilege and secure_file_priv enforcement for JSON/BSON file UDFs +# +# Privileged user: path inside secure_file_priv MYSQL_TMP_DIR +select json_file('MYSQL_TMP_DIR/bib0.json', 0) IS NOT NULL AS ok; +ok +1 +select bson_file('MYSQL_TMP_DIR/biblio.json', 0) IS NOT NULL AS ok; +ok +1 +Warnings: +Warning 1105 File pretty format doesn't match the specified pretty value +select jfile_make('{"test":1}', 'MYSQL_TMP_DIR/out.json', 0) IS NOT NULL AS ok; +ok +1 +select bfile_make('{"test":1}', 'MYSQL_TMP_DIR/out.json', 0) IS NOT NULL AS ok; +ok +1 +select jbin_file('MYSQL_TMP_DIR/bib0.json') IS NOT NULL AS ok; +ok +1 +select bbin_file('MYSQL_TMP_DIR/biblio.json') IS NOT NULL AS ok; +ok +1 +# Privileged user: path outside secure_file_priv +select json_file('DATADIR/bib0.json') IS NULL AS denied; +denied +1 +Warnings: +Warning 1105 Access denied: FILE privilege or secure_file_priv violation +select bson_file('DATADIR/biblio.json') IS NULL AS denied; +denied +1 +Warnings: +Warning 1105 Access denied: FILE privilege or secure_file_priv violation +select jfile_make('{"test":1}', 'DATADIR/out.json', 0) IS NULL AS denied; +denied +1 +Warnings: +Warning 1105 Access denied: FILE privilege or secure_file_priv violation +select bfile_make('{"test":1}', 'DATADIR/out.json', 0) IS NULL AS denied; +denied +1 +Warnings: +Warning 1105 Access denied: FILE privilege or secure_file_priv violation +select jbin_file('DATADIR/bib0.json') IS NULL AS denied; +denied +1 +Warnings: +Warning 1105 Access denied: FILE privilege or secure_file_priv violation +select bbin_file('DATADIR/biblio.json') IS NULL AS denied; +denied +1 +Warnings: +Warning 1105 Access denied: FILE privilege or secure_file_priv violation +# NULL argument: no "access denied", UDF reports its own error +select json_file(NULL) IS NULL AS ok; +ok +1 +Warnings: +Warning 1105 Missing file name +select bson_file(NULL) IS NULL AS ok; +ok +1 +Warnings: +Warning 1105 Missing file name +select jfile_make('{"test":1}', NULL, 0) IS NULL AS ok; +ok +1 +Warnings: +Warning 1105 Missing file name +select bfile_make('{"test":1}', NULL, 0) IS NULL AS ok; +ok +1 +Warnings: +Warning 1105 Missing file name +select jbin_file(NULL) IS NULL AS ok; +ok +1 +Warnings: +Warning 1105 Missing file name +select bbin_file(NULL) IS NULL AS ok; +ok +1 +Warnings: +Warning 1105 Missing file name +# Privileged user: relative path resolving outside secure_file_priv +select json_file('../../bib0.json') IS NULL AS denied; +denied +1 +Warnings: +Warning 1105 Access denied: FILE privilege or secure_file_priv violation +select bson_file('../../biblio.json') IS NULL AS denied; +denied +1 +Warnings: +Warning 1105 Access denied: FILE privilege or secure_file_priv violation +# Overlong filename: MY_SAFE_PATH rejects it, fail closed +select json_file(concat('MYSQL_TMP_DIR/', repeat('x', 512))) IS NULL AS denied; +denied +1 +Warnings: +Warning 1105 Access denied: FILE privilege or secure_file_priv violation +select bson_file(concat('MYSQL_TMP_DIR/', repeat('x', 512))) IS NULL AS denied; +denied +1 +Warnings: +Warning 1105 Access denied: FILE privilege or secure_file_priv violation +# Non-constant filename expression (args->args[0] is NULL at init time) +set @fname = 'MYSQL_TMP_DIR/bib0.json'; +select jbin_file(@fname) IS NOT NULL AS ok; +ok +1 +set @fname = 'MYSQL_TMP_DIR/biblio.json'; +select bbin_file(@fname) IS NOT NULL AS ok; +ok +1 +# User without FILE privilege +create user unprivileged@localhost; +grant select ON *.* TO unprivileged@localhost; +connect unprivileged,localhost,unprivileged,,; +connection unprivileged; +# inside secure_file_priv: denied without FILE privilege +select json_file('MYSQL_TMP_DIR/bib0.json') IS NULL AS denied;; +denied +1 +Warnings: +Warning 1105 Access denied: FILE privilege or secure_file_priv violation +select bson_file('MYSQL_TMP_DIR/biblio.json') IS NULL AS denied;; +denied +1 +Warnings: +Warning 1105 Access denied: FILE privilege or secure_file_priv violation +select jfile_make('{"test":1}', 'MYSQL_TMP_DIR/out.json', 0) IS NULL AS denied;; +denied +1 +Warnings: +Warning 1105 Access denied: FILE privilege or secure_file_priv violation +select bfile_make('{"test":1}', 'MYSQL_TMP_DIR/out.json', 0) IS NULL AS denied;; +denied +1 +Warnings: +Warning 1105 Access denied: FILE privilege or secure_file_priv violation +select jbin_file('MYSQL_TMP_DIR/bib0.json') IS NULL AS denied;; +denied +1 +Warnings: +Warning 1105 Access denied: FILE privilege or secure_file_priv violation +select bbin_file('MYSQL_TMP_DIR/biblio.json') IS NULL AS denied;; +denied +1 +Warnings: +Warning 1105 Access denied: FILE privilege or secure_file_priv violation +disconnect unprivileged; +connection default; +drop user unprivileged@localhost; diff --git a/storage/connect/mysql-test/connect/t/drop-open-error.opt b/storage/connect/mysql-test/connect/t/drop-open-error.opt deleted file mode 100644 index 22520f0aa9901..0000000000000 --- a/storage/connect/mysql-test/connect/t/drop-open-error.opt +++ /dev/null @@ -1 +0,0 @@ ---secure-file-priv="" diff --git a/storage/connect/mysql-test/connect/t/drop-open-error.test b/storage/connect/mysql-test/connect/t/drop-open-error.test index dd286c9646606..97d938e431087 100644 --- a/storage/connect/mysql-test/connect/t/drop-open-error.test +++ b/storage/connect/mysql-test/connect/t/drop-open-error.test @@ -18,14 +18,3 @@ copy_file $MTR_SUITE_DIR/std_data/mdev9949.frm $datadir/test/mdev9949.frm; --replace_result $MARIADB_DATADIR DATADIR/ './' 'DATADIR/' drop table mdev9949; drop table t1; - -# -# MDEV-7935 CREATE TABLE ... AS SELECT ... can cause a Server crash (Assertion `0' in Protocol::end_statement) -# -select @@secure_file_priv 'must be NULL'; # otherwise foo/bar.txt won't be allowed -create table t1 (a char(16)) engine=myisam; -insert into t1 values('Hello World!'); -replace_regex @on .*/foo/@on foo/@; -error ER_GET_ERRMSG; -create table t2 engine=connect file_name='foo/bar.txt' as select * from t1; -drop table t1; diff --git a/storage/connect/mysql-test/connect/t/fix.test b/storage/connect/mysql-test/connect/t/fix.test index b24e7af086d2d..803d1f7869026 100644 --- a/storage/connect/mysql-test/connect/t/fix.test +++ b/storage/connect/mysql-test/connect/t/fix.test @@ -106,3 +106,14 @@ DROP TABLE t1; --remove_file $MYSQLD_DATADIR/test/dept.dat --remove_file $MYSQLD_DATADIR/test/boys.txt --remove_file $MYSQLD_DATADIR/test/boyswin.txt + +--echo # +--echo # MDEV-40637 CONNECT crashes on double(255,50) in DOS table +--echo # +create table t (x double(255,30)) engine=connect table_type='dos' file_name='t.dos'; +insert into t values (1e50); +select * from t; +drop table t; +--remove_file $MYSQLD_DATADIR/test/t.dos + +--echo # End of 10.6 tests diff --git a/storage/connect/mysql-test/connect/t/json_bson_file_priv.opt b/storage/connect/mysql-test/connect/t/json_bson_file_priv.opt new file mode 100644 index 0000000000000..e9a43a5584d8f --- /dev/null +++ b/storage/connect/mysql-test/connect/t/json_bson_file_priv.opt @@ -0,0 +1 @@ +--secure_file_priv=$MYSQL_TMP_DIR diff --git a/storage/connect/mysql-test/connect/t/json_bson_file_priv.test b/storage/connect/mysql-test/connect/t/json_bson_file_priv.test new file mode 100644 index 0000000000000..04c91184a943b --- /dev/null +++ b/storage/connect/mysql-test/connect/t/json_bson_file_priv.test @@ -0,0 +1,100 @@ +--source include/not_embedded.inc +--source json_udf.inc +--source bson_udf.inc + +let $MYSQLD_DATADIR = `select @@datadir`; + +--copy_file $MTR_SUITE_DIR/std_data/bib0.json $MYSQL_TMP_DIR/bib0.json +--copy_file $MTR_SUITE_DIR/std_data/biblio.json $MYSQL_TMP_DIR/biblio.json + +--echo # +--echo # Test FILE privilege and secure_file_priv enforcement for JSON/BSON file UDFs +--echo # + +--echo # Privileged user: path inside secure_file_priv MYSQL_TMP_DIR +--replace_result $MYSQL_TMP_DIR MYSQL_TMP_DIR +--eval select json_file('$MYSQL_TMP_DIR/bib0.json', 0) IS NOT NULL AS ok +--replace_result $MYSQL_TMP_DIR MYSQL_TMP_DIR +--eval select bson_file('$MYSQL_TMP_DIR/biblio.json', 0) IS NOT NULL AS ok +--replace_result $MYSQL_TMP_DIR MYSQL_TMP_DIR +--eval select jfile_make('{"test":1}', '$MYSQL_TMP_DIR/out.json', 0) IS NOT NULL AS ok +--remove_file $MYSQL_TMP_DIR/out.json +--replace_result $MYSQL_TMP_DIR MYSQL_TMP_DIR +--eval select bfile_make('{"test":1}', '$MYSQL_TMP_DIR/out.json', 0) IS NOT NULL AS ok +--remove_file $MYSQL_TMP_DIR/out.json +--replace_result $MYSQL_TMP_DIR MYSQL_TMP_DIR +--eval select jbin_file('$MYSQL_TMP_DIR/bib0.json') IS NOT NULL AS ok +--replace_result $MYSQL_TMP_DIR MYSQL_TMP_DIR +--eval select bbin_file('$MYSQL_TMP_DIR/biblio.json') IS NOT NULL AS ok + +--echo # Privileged user: path outside secure_file_priv +--replace_result $MYSQLD_DATADIR DATADIR +--eval select json_file('$MYSQLD_DATADIR/bib0.json') IS NULL AS denied +--replace_result $MYSQLD_DATADIR DATADIR +--eval select bson_file('$MYSQLD_DATADIR/biblio.json') IS NULL AS denied +--replace_result $MYSQLD_DATADIR DATADIR +--eval select jfile_make('{"test":1}', '$MYSQLD_DATADIR/out.json', 0) IS NULL AS denied +--replace_result $MYSQLD_DATADIR DATADIR +--eval select bfile_make('{"test":1}', '$MYSQLD_DATADIR/out.json', 0) IS NULL AS denied +--replace_result $MYSQLD_DATADIR DATADIR +--eval select jbin_file('$MYSQLD_DATADIR/bib0.json') IS NULL AS denied +--replace_result $MYSQLD_DATADIR DATADIR +--eval select bbin_file('$MYSQLD_DATADIR/biblio.json') IS NULL AS denied + +--echo # NULL argument: no "access denied", UDF reports its own error +select json_file(NULL) IS NULL AS ok; +select bson_file(NULL) IS NULL AS ok; +select jfile_make('{"test":1}', NULL, 0) IS NULL AS ok; +select bfile_make('{"test":1}', NULL, 0) IS NULL AS ok; +select jbin_file(NULL) IS NULL AS ok; +select bbin_file(NULL) IS NULL AS ok; + +--echo # Privileged user: relative path resolving outside secure_file_priv +select json_file('../../bib0.json') IS NULL AS denied; +select bson_file('../../biblio.json') IS NULL AS denied; + +--echo # Overlong filename: MY_SAFE_PATH rejects it, fail closed +--replace_result $MYSQL_TMP_DIR MYSQL_TMP_DIR +--eval select json_file(concat('$MYSQL_TMP_DIR/', repeat('x', 512))) IS NULL AS denied +--replace_result $MYSQL_TMP_DIR MYSQL_TMP_DIR +--eval select bson_file(concat('$MYSQL_TMP_DIR/', repeat('x', 512))) IS NULL AS denied + +--echo # Non-constant filename expression (args->args[0] is NULL at init time) +--replace_result $MYSQL_TMP_DIR MYSQL_TMP_DIR +--eval set @fname = '$MYSQL_TMP_DIR/bib0.json' +select jbin_file(@fname) IS NOT NULL AS ok; +--replace_result $MYSQL_TMP_DIR MYSQL_TMP_DIR +--eval set @fname = '$MYSQL_TMP_DIR/biblio.json' +select bbin_file(@fname) IS NOT NULL AS ok; + +--echo # User without FILE privilege +create user unprivileged@localhost; +grant select ON *.* TO unprivileged@localhost; + +--connect(unprivileged,localhost,unprivileged,,) +--connection unprivileged + +--echo # inside secure_file_priv: denied without FILE privilege +--replace_result $MYSQL_TMP_DIR MYSQL_TMP_DIR +--eval select json_file('$MYSQL_TMP_DIR/bib0.json') IS NULL AS denied; +--replace_result $MYSQL_TMP_DIR MYSQL_TMP_DIR +--eval select bson_file('$MYSQL_TMP_DIR/biblio.json') IS NULL AS denied; +--replace_result $MYSQL_TMP_DIR MYSQL_TMP_DIR +--eval select jfile_make('{"test":1}', '$MYSQL_TMP_DIR/out.json', 0) IS NULL AS denied; +--replace_result $MYSQL_TMP_DIR MYSQL_TMP_DIR +--eval select bfile_make('{"test":1}', '$MYSQL_TMP_DIR/out.json', 0) IS NULL AS denied; +--replace_result $MYSQL_TMP_DIR MYSQL_TMP_DIR +--eval select jbin_file('$MYSQL_TMP_DIR/bib0.json') IS NULL AS denied; +--replace_result $MYSQL_TMP_DIR MYSQL_TMP_DIR +--eval select bbin_file('$MYSQL_TMP_DIR/biblio.json') IS NULL AS denied; + +--disconnect unprivileged +--connection default + +--remove_file $MYSQL_TMP_DIR/bib0.json +--remove_file $MYSQL_TMP_DIR/biblio.json + +drop user unprivileged@localhost; + +--source json_udf2.inc +--source bson_udf2.inc diff --git a/storage/connect/value.cpp b/storage/connect/value.cpp index b358d00a5d89d..67e978f0a8730 100644 --- a/storage/connect/value.cpp +++ b/storage/connect/value.cpp @@ -748,7 +748,7 @@ bool TYPVAL::SetValue_char(const char *p, int n) for (; n > 0 && *p == ' '; p++) n--; - memcpy(buf, p, MY_MIN(n, 31)); + memcpy(buf, p, n= MY_MIN(n, 31)); buf[n] = '\0'; Tval = atof(buf); diff --git a/storage/csv/ha_tina.cc b/storage/csv/ha_tina.cc index 221fbe3f6fd1f..e489971bf7a06 100644 --- a/storage/csv/ha_tina.cc +++ b/storage/csv/ha_tina.cc @@ -1,4 +1,5 @@ /* Copyright (c) 2004, 2011, Oracle and/or its affiliates. + Copyright (c) 2026, MariaDB plc This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by @@ -646,7 +647,8 @@ int ha_tina::chain_append() if (chain_alloced) { if ((chain= (tina_set *) my_realloc(csv_key_memory_tina_set, - (uchar*)chain, chain_size, + (uchar*)chain, + chain_size * sizeof(tina_set), MYF(MY_WME))) == NULL) return -1; } diff --git a/storage/duckdb/CLAUDE.md b/storage/duckdb/CLAUDE.md index bccdb4bf83402..2f1bdb7de756d 100644 --- a/storage/duckdb/CLAUDE.md +++ b/storage/duckdb/CLAUDE.md @@ -46,6 +46,8 @@ Tests use MariaDB's MTR (MySQL Test Runner) framework. Test files live in `mysql All engine code is in the `myduck` namespace (except `ha_duckdb` which is in global scope per MariaDB handler convention). +> **Doc convention:** when this file (or a code comment) names a specific API to describe control flow (e.g. `SELECT_LEX::print()`), it must also name the file/function where that API is actually called. If no call site can be pointed to, describe the behavior instead of naming the API — do not imply a code path that isn't there. + ### Key components - **`ha_duckdb`** (`ha_duckdb.cc/h`) — MariaDB `handler` subclass. Entry point for all storage engine operations (open, close, read, write, DDL). Implements row-at-a-time interface for MariaDB, translating to DuckDB batch operations. @@ -66,11 +68,20 @@ All engine code is in the `myduck` namespace (except `ha_duckdb` which is in glo ### SQL generation conventions -All generated SQL must use **double quotes** for identifiers (DuckDB follows SQL standard), not backticks. The `SELECT_LEX::print()` output from MariaDB uses backticks and must be post-processed. See `docs/mariadb-duckdb-incompatibilities.md` for known function name rewrites and type mapping issues. +DuckDB follows the SQL standard and delimits identifiers with **double quotes**, not backticks. + +SQL reaches DuckDB by two routes: + +- **Whole queries are forwarded verbatim.** SELECT and INSERT … SELECT are pushed down as the raw `thd->query()` text — see `extract_source_query()` in `ha_duckdb_pushdown.cc`. The engine does **not** re-print the whole query via `SELECT_LEX::print()`. +- **Only fragments are printed.** `Item`/`COND::print()` is used for per-table WHERE conditions in cross-engine scan (`ha_duckdb_pushdown.cc`) and for DDL default / `nextval` expressions (`ddl_convertor.cc`). DDL/DML convertors also build identifier strings directly from `Field`/`TABLE` metadata. + +Both routes then pass through `backticks_to_double_quotes()` (`runtime/duckdb_query.cc`), which is where identifier requoting actually happens: backtick-delimited identifiers are rewritten to double-quoted ones and any embedded double quote is escaped (MDEV-40653). Raw forwarding additionally goes through `mariadb_query_has_lexical_mismatch()`, which refuses to forward SQL whose backslash-escape semantics differ between MariaDB and DuckDB. + +See `docs/mariadb-duckdb-incompatibilities.md` for known function name rewrites and type mapping issues. ### DuckDB source and patches -DuckDB source is at `third_parties/duckdb/` (git submodule). No patches are applied — all compatibility is handled at runtime via `duckdb_mysql_compat.cc`. The build produces a static library; `_GLIBCXX_DEBUG` is explicitly undefined in CMakeLists.txt to avoid ABI mismatch with MariaDB's debug build. +DuckDB source is at `third_parties/duckdb/` (git submodule). One local patch is applied on top of the pinned submodule commit — `patches/duckdb-pr24061-setval.diff` (backport of upstream PR #24061 adding `setval()`, needed for AUTO_INCREMENT counter repositioning), applied via `PATCH_COMMAND` in `cmake/duckdb.cmake` and to be dropped when the submodule is bumped to a release containing the PR. Other compatibility is handled at runtime via `duckdb_mysql_compat.cc`. The build produces a static library; `_GLIBCXX_DEBUG` is explicitly undefined in CMakeLists.txt to avoid ABI mismatch with MariaDB's debug build. ### Configuration diff --git a/storage/duckdb/CMakeLists.txt b/storage/duckdb/CMakeLists.txt index 2105ed5fc71fc..cf4e06a7f8e3a 100644 --- a/storage/duckdb/CMakeLists.txt +++ b/storage/duckdb/CMakeLists.txt @@ -37,7 +37,12 @@ SET(DUCKDB_PLUGIN_SOURCES duckdb_udf.cc ) -MYSQL_ADD_PLUGIN(duckdb ${DUCKDB_PLUGIN_SOURCES} STORAGE_ENGINE MODULE_ONLY) +MYSQL_ADD_PLUGIN(duckdb ${DUCKDB_PLUGIN_SOURCES} STORAGE_ENGINE MODULE_ONLY + COMPONENT duckdb-engine) +SET(CPACK_RPM_duckdb-engine_PACKAGE_SUMMARY "DuckDB storage engine for MariaDB server" PARENT_SCOPE) +SET(CPACK_RPM_duckdb-engine_PACKAGE_DESCRIPTION "The MariaDB DuckDB storage +engine embeds DuckDB as a pluggable storage engine, enabling fast analytical +(OLAP) query processing directly inside MariaDB." PARENT_SCOPE) IF(TARGET duckdb) @@ -58,8 +63,10 @@ IF(TARGET duckdb) ADD_SUBDIRECTORY(convertor) ADD_SUBDIRECTORY(runtime) + # atomic: GCC lowers the 16-byte atomic ops of the AUTO_INCREMENT block + # cache (Duckdb_share::autoinc_range) to libatomic calls. target_link_libraries(duckdb duckdb_runtime duckdb_convertor duckdb_common - ${DUCKDB_LIBRARY} + ${DUCKDB_LIBRARY} atomic ) duckdb_setup_target(duckdb) diff --git a/storage/duckdb/README.md b/storage/duckdb/README.md index cfdf8eb84ee08..f72cd9aa0f64a 100644 --- a/storage/duckdb/README.md +++ b/storage/duckdb/README.md @@ -43,6 +43,13 @@ TPC-H Scale Factor 10 (~11 GB raw data, 86.6M rows total, ~60M in `lineitem`). Each query also pays a small fixed cost (~40 ms) for client connect and pushdown setup — noticeable on the cheapest queries, negligible on the heavy analytical ones. See [`docs/tpch_sf10_query_benchmark.md`](docs/tpch_sf10_query_benchmark.md) and [`docs/tpch_sf10_ingestion_benchmark.md`](docs/tpch_sf10_ingestion_benchmark.md) for full methodology. +## Tutorials + +Step-by-step guides for loading and analyzing open datasets with the DuckDB storage engine: + +- [NYC Taxi Trips](docs/tutorials/nyc-taxi-trips.md) — loading Parquet via `run_in_duckdb()`, then cross-engine joins between `ENGINE=DuckDB` and `ENGINE=InnoDB` tables. Follows the ClickHouse taxi tutorial. +- [OWID CO₂ Emissions](docs/tutorials/owid-co2-emissions.md) — loading a CSV dataset and running aggregations and window functions. + ## How It Works DuckDB is an in-process analytical database. Its performance rests on three pillars: @@ -130,6 +137,7 @@ DuckDB handles the join, aggregation, and sorting; InnoDB rows are produced on d - **Some MariaDB functions are yet not pushdown-compatible** — `GROUP_CONCAT()`, `DATE_FORMAT()`, `JSON_CONTAINS()`, `FOUND_ROWS()`, `LAST_INSERT_ID()`, and a few others have no DuckDB equivalent or differ in syntax. Such queries fall back to MariaDB execution. - **Strict GROUP BY** — DuckDB rejects `SELECT` columns not in `GROUP BY` and not aggregated, even when MariaDB's `sql_mode` allows it. - **XA transactions** — `XA PREPARE` is not supported by the engine. +- **Table partitioning** — `CREATE TABLE ... PARTITION BY` and converting a DuckDB table with `ALTER TABLE ... PARTITION BY` are not supported; the engine declares `HTON_NO_PARTITION`. - **Collations** — MariaDB UCA-based collation rules are approximated via DuckDB's built-in `NOCASE`/`NOACCENT` collations for UTF-8 charsets; non-UTF8 charsets fall back to binary comparison. See [`docs/collation-mapping.md`](docs/collation-mapping.md) for the full mapping and known gaps. - **Cross-engine scan is yet single-threaded** — each external (non-DuckDB) table is produced by a single fiber-driven MariaDB query (`_mdb_scan` reports `MaxThreads() == 1`); only the DuckDB side of the query is parallelized. - **ALTER COLUMN DROP DEFAULT** — not propagated to DuckDB catalog. diff --git a/storage/duckdb/build.sh b/storage/duckdb/build.sh index 18383f0d62275..c9a9956913c5f 100755 --- a/storage/duckdb/build.sh +++ b/storage/duckdb/build.sh @@ -11,7 +11,7 @@ BUILD_PATH=$(realpath "$MDB_SOURCE_PATH"/../DuckdbBuildOf_$(basename "$MDB_SOURC CPUS=$(getconf _NPROCESSORS_ONLN) BUILD_TYPE_OPTIONS=("Debug" "RelWithDebInfo") BUILD_TYPE="${BUILD_TYPE:-}" -DISTRO_OPTIONS=("ubuntu:22.04" "ubuntu:24.04" "debian:12" "rockylinux:8" "rockylinux:9") +DISTRO_OPTIONS=("ubuntu:22.04" "ubuntu:24.04" "debian:12" "debian:13" "rockylinux:8" "rockylinux:9") DEFAULT_MDB_DATADIR="/var/lib/mysql" USER="mysql" GROUP="mysql" @@ -30,6 +30,7 @@ usage() { echo " -D Install build prerequisites (requires root/sudo)" echo " -u Build unit tests" echo " -a Enable ASAN/UBSAN (WITH_ASAN, WITH_ASAN_SCOPE, WITH_UBSAN)" + echo " -m Enable MSAN (WITH_MSAN)" echo " -R Use gcc-toolset-\${GCC_VERSION} on Rocky 8" echo " -h Show this help" exit 0 @@ -43,9 +44,10 @@ INSTALL_DEPS=false GCC_TOOLSET=false UNIT_TESTS=false WITH_ASAN=false +WITH_MSAN=false OS="" -while getopts "t:d:j:cpSnDRuah" opt; do +while getopts "t:d:j:cpSnDRuamh" opt; do case $opt in t) BUILD_TYPE="$OPTARG" ;; d) OS="$OPTARG" ;; @@ -58,6 +60,7 @@ while getopts "t:d:j:cpSnDRuah" opt; do R) GCC_TOOLSET=true ;; u) UNIT_TESTS=true ;; a) WITH_ASAN=true ;; + m) WITH_MSAN=true ;; h) usage ;; *) usage ;; esac @@ -110,6 +113,7 @@ info "Build dir: ${_CLR_YELLOW}$BUILD_PATH" info "Build type: ${_CLR_YELLOW}$BUILD_TYPE" info "Jobs: ${_CLR_YELLOW}$CPUS" info "ASAN/UBSAN: ${_CLR_YELLOW}$WITH_ASAN" +info "MSAN: ${_CLR_YELLOW}$WITH_MSAN" info "Unit tests: ${_CLR_YELLOW}$UNIT_TESTS" if [[ $BUILD_PACKAGES = true ]]; then info "Packages: ${_CLR_YELLOW}$PKG_FORMAT ($OS)" @@ -234,6 +238,13 @@ construct_cmake_flags() { ) fi + if [[ $WITH_MSAN = true ]]; then + MDB_CMAKE_FLAGS+=( + -DWITH_MSAN=ON + -DWITH_UNIT_TESTS=OFF + ) + fi + if [[ $BUILD_PACKAGES = true ]]; then if [[ "$PKG_FORMAT" == "rpm" ]]; then local os_version=${OS//[^0-9]/} @@ -244,6 +255,7 @@ construct_cmake_flags() { local codename="" case "$OS" in debian:12*) codename="bookworm" ;; + debian:13*) codename="trixie" ;; ubuntu:22.04) codename="jammy" ;; ubuntu:24.04) codename="noble" ;; *) fail "Unknown DEB codename for $OS" ;; @@ -274,14 +286,25 @@ install_deps() { ncurses-devel readline-devel openssl-devel zlib-devel bzip2-devel \ libzstd-devel libcurl-devel libaio-devel libxml2-devel pcre2-devel \ libxcrypt-devel xz-devel pam-devel perl-DBI python3 python3-devel \ - ccache rpm-build" + libatomic ccache" + + # rpm-build is only needed to build RPM packages; on some bases (e.g. UBI 9) + # installing it forces an rpm upgrade that conflicts with pinned @System rpm. + if [[ $BUILD_PACKAGES = true ]]; then + RPM_DEPS="$RPM_DEPS rpm-build" + fi local DEB_DEPS="build-essential git cmake ninja-build bison flex \ libncurses-dev libreadline-dev libssl-dev zlib1g-dev libbz2-dev \ libzstd-dev libcurl4-openssl-dev libaio-dev libxml2-dev libpcre2-dev \ - libxcrypt-dev liblzma-dev libpam0g-dev libperl-dev python3 python3-dev \ + liblz4-dev liblzma-dev liblzo2-dev libsnappy-dev libpam0g-dev libperl-dev python3 python3-dev \ ccache devscripts equivs debhelper libdistro-info-perl" + # libxcrypt-dev is not available on Debian 13 (Trixie) + if [[ "$OS" != debian:13* ]]; then + DEB_DEPS="$DEB_DEPS libxcrypt-dev" + fi + local command="" case "$OS" in rockylinux:8|rocky:8) @@ -296,9 +319,16 @@ install_deps() { warn "Rocky 8 default gcc 8 lacks C++20 -- consider re-running with -R" fi ;; - rockylinux:9|rocky:9|rocky:10) + rockylinux:9|rocky:9|rocky:10|rhel:9*|redhat:9*|red:9*) + # Enable EPEL and the CodeReady Builder / PowerTools-equivalent repo. + # Rocky/Alma expose it as the 'crb' alias; genuine RHEL enables it via + # subscription-manager. Enabling is best-effort: if neither mechanism + # is available we warn and continue so the dnf install below still runs. command="$SUDO dnf install -y 'dnf-command(config-manager)' epel-release && \ - $SUDO dnf config-manager --set-enabled crb && \ + { $SUDO dnf config-manager --set-enabled crb 2>/dev/null || \ + $SUDO dnf config-manager --set-enabled ubi-9-codeready-builder-rpms 2>/dev/null || \ + $SUDO subscription-manager repos --enable codeready-builder-for-rhel-9-\$(uname -m)-rpms 2>/dev/null || \ + warn 'Could not enable CRB/CodeReady Builder repo; continuing without it'; } && \ $SUDO dnf install -y gcc gcc-c++ ${RPM_DEPS}" ;; ubuntu:*|debian:*) diff --git a/storage/duckdb/cmake/duckdb.cmake b/storage/duckdb/cmake/duckdb.cmake index 9da9909940bb1..1eb9924924dd7 100644 --- a/storage/duckdb/cmake/duckdb.cmake +++ b/storage/duckdb/cmake/duckdb.cmake @@ -51,6 +51,16 @@ SET(_DUCKDB_STATIC_LIBS MESSAGE(STATUS "=== Building DuckDB from submodule (${DUCKDB_SUBMODULE_DIR}) ===") ADD_SUBMODULE(third_parties/duckdb) +# Local patches applied on top of the pinned submodule commit. +# +# duckdb-pr24061-setval.diff: backport of upstream PR #24061 ("Add SetVal", +# merged 2026-07-23, first release after v1.5.2) rewritten against the v1.5.2 +# API. Provides setval('seq', value[, is_called]) needed for AUTO_INCREMENT +# counter repositioning (TRUNCATE, ALTER ... AUTO_INCREMENT=N, explicit-id +# reconciliation). Drop when the submodule is bumped to a release containing +# the PR: the apply below will fail loudly on an already-present setval. +SET(_DUCKDB_PATCH "${CMAKE_CURRENT_SOURCE_DIR}/patches/duckdb-pr24061-setval.diff") + # Upstream sets DUCKDB_EXTENSION_JEMALLOC_LINKED via add_extension_definitions(), # which runs in extension/ but NOT in src/, so allocator.cpp (in duckdb_static) # compiles the glibc malloc() path even though libjemalloc_extension.a is linked. @@ -79,6 +89,8 @@ ExternalProject_Add(duckdb_build PREFIX "${_DUCKDB_BUILD_DIR}" SOURCE_DIR "${DUCKDB_SUBMODULE_DIR}" BINARY_DIR "${_DUCKDB_BUILD_DIR}" + # Idempotent: skip when the working tree already carries the patch. + PATCH_COMMAND bash -c "git apply --reverse --check '${_DUCKDB_PATCH}' 2>/dev/null || git apply --verbose '${_DUCKDB_PATCH}'" CMAKE_ARGS -DCMAKE_BUILD_TYPE=${_DUCKDB_BUILD_TYPE} -DCMAKE_C_COMPILER=${CMAKE_C_COMPILER} diff --git a/storage/duckdb/common/duckdb_types.cc b/storage/duckdb/common/duckdb_types.cc index 625e1c0b168dc..e95fe198667aa 100644 --- a/storage/duckdb/common/duckdb_types.cc +++ b/storage/duckdb/common/duckdb_types.cc @@ -92,6 +92,21 @@ DatabaseTableNames::DatabaseTableNames(const char *name) db_name= std::string(ori_db_name, db_name_length); } +std::string quote_duckdb_identifier(const char *name, size_t length) +{ + std::string out; + out.reserve(length + 2); + out.push_back('"'); + for (size_t i= 0; i < length; i++) + { + if (name[i] == '"') + out.push_back('"'); + out.push_back(name[i]); + } + out.push_back('"'); + return out; +} + Databasename::Databasename(const char *path_name) { char dbname[FN_REFLEN]; diff --git a/storage/duckdb/common/duckdb_types.h b/storage/duckdb/common/duckdb_types.h index 2e39c1aeb1150..6642467ccdebb 100644 --- a/storage/duckdb/common/duckdb_types.h +++ b/storage/duckdb/common/duckdb_types.h @@ -33,6 +33,22 @@ class DatabaseTableNames std::string table_name; }; +/** + Quote an SQL identifier for DuckDB. + + DuckDB (SQL standard) delimits identifiers with double quotes and escapes + an embedded double quote by doubling it. MariaDB identifiers (column names + in particular) may contain arbitrary characters, so failing to escape lets + a crafted name break out of the quoted identifier and inject DuckDB SQL + (MDEV-40653). Returns the name wrapped in double quotes with any embedded + double quote doubled. +*/ +std::string quote_duckdb_identifier(const char *name, size_t length); +inline std::string quote_duckdb_identifier(const std::string &name) +{ + return quote_duckdb_identifier(name.data(), name.size()); +} + /** Utility class to extract the database name from a path like "./db/". */ diff --git a/storage/duckdb/convertor/ddl_convertor.cc b/storage/duckdb/convertor/ddl_convertor.cc index 9d9433c638d35..2432a95389d5d 100644 --- a/storage/duckdb/convertor/ddl_convertor.cc +++ b/storage/duckdb/convertor/ddl_convertor.cc @@ -152,21 +152,126 @@ static std::string get_default_expr_from_vcol_defs(const TABLE_SHARE *share, return ""; } +/** + Does a default expression reference a MariaDB sequence? + + NEXT VALUE FOR s, PREVIOUS VALUE FOR s and SETVAL() are printed by + Item_func_nextval::print() as nextval(`db`.`seq`), lastval(...) and + setval(...) (sql/item_func.cc). DuckDB binds the quoted argument as a + column reference and rejects the whole DDL with "DEFAULT value cannot + contain column names" (MDEV-40266). + + The sequence lives in MariaDB and has no DuckDB counterpart, so such a + default can never be forwarded. It is omitted instead: MariaDB evaluates + column defaults into record[0] before write_row(), and both write paths + always send the value, so only writes that bypass MariaDB entirely (i.e. + run_in_duckdb) would ever consult the DuckDB-side default. + + This matches on the printed function name because the Item tree in + field->default_value->expr is unusable at ha_duckdb::create() time. +*/ +static bool has_mariadb_sequence_func(const std::string &expr) +{ + static const char *const funcs[]= {"nextval(", "lastval(", "setval("}; + + for (const char *func : funcs) + { + size_t len= strlen(func); + for (size_t pos= 0; pos + len <= expr.length(); pos++) + { + if (strncasecmp(expr.c_str() + pos, func, len) != 0) + continue; + /* Ignore a match that is only the tail of a longer identifier. */ + char prev= pos ? expr[pos - 1] : ' '; + if (!isalnum((uchar) prev) && prev != '_') + return true; + } + } + return false; +} + +/** + Expression default text of a field, or "" if it has none or the text is + not available in this share. +*/ +static std::string get_expr_default_text(const Field *field) +{ + if (!field->default_value) + return ""; + return get_default_expr_from_vcol_defs(field->table->s, field->field_index); +} + +/** Does this field carry a default expression referencing a sequence? */ +static bool is_sequence_default(const Field *field) +{ + return has_mariadb_sequence_func(get_expr_default_text(field)); +} + +static void note_sequence_default_omitted(const char *column) +{ + push_warning_printf(current_thd, Sql_condition::WARN_LEVEL_NOTE, + ER_UNKNOWN_ERROR, + "DuckDB: sequence default of column '%s' is not " + "forwarded to DuckDB; MariaDB supplies the value", + column); +} + +std::string autoinc_sequence_name(const std::string &table_name) +{ + return "mdb_autoinc_" + table_name; +} + +Field *find_autoinc_field(const TABLE *table) +{ + for (Field **ptr= table->field; *ptr; ptr++) + { + if ((*ptr)->flags & AUTO_INCREMENT_FLAG) + return *ptr; + } + return nullptr; +} + +/** Reference to the auto-increment sequence usable as a nextval() argument. */ +static std::string autoinc_nextval_expr(const std::string &schema_name, + const std::string &table_name) +{ + /* + The qualified sequence name is a quoted identifier nested inside a + single-quoted string literal argument to nextval(). Escape the inner + identifiers (doubling ") and then escape the resulting string for the + enclosing literal (doubling '). + */ + std::string qualified= + quote_duckdb_identifier(schema_name) + "." + + quote_duckdb_identifier(autoinc_sequence_name(table_name)); + std::string escaped; + escaped.reserve(qualified.size()); + for (char c : qualified) + { + if (c == '\'') + escaped.push_back('\''); + escaped.push_back(c); + } + return "nextval('" + escaped + "')"; +} + /** Read the literal default value of a field from the default record and return it as a string suitable for DuckDB SQL. - BIT fields are converted to DuckDB blob literal format: '\xHH...'::BLOB. - Other fields use standard quoted literal format: 'value'. + BIT fields are converted to DuckDB blob literal format, optionally with an + explicit BLOB cast. Other fields use standard quoted literal format. @param field Field whose default value to read (must not be at offset) @param offset Offset from record[0] to default_values (s->default_values - record[0]) + @param cast_bit Add an explicit BLOB cast for BIT fields @return Default value string, or "NULL" if field is null at default record */ static std::string get_field_default_for_duckdb(Field *field, - my_ptrdiff_t offset) + my_ptrdiff_t offset, + bool cast_bit= true) { field->move_field_offset(offset); @@ -192,7 +297,9 @@ static std::string get_field_default_for_duckdb(Field *field, ss << hx; } } - ss << "'::BLOB"; + ss << "'"; + if (cast_bit) + ss << "::BLOB"; default_value= ss.str(); } else @@ -200,8 +307,25 @@ static std::string get_field_default_for_duckdb(Field *field, char buf[MAX_FIELD_WIDTH]; String str(buf, sizeof(buf), system_charset_info); String *val= field->val_str(&str); - if (val && val->length() > 0) - default_value= "'" + std::string(val->ptr(), val->length()) + "'"; + if (val) + { + /* + Escape the literal by doubling any embedded single quote so a crafted + DEFAULT string cannot terminate the generated DuckDB literal and inject + further DuckDB statements (MDEV-40610). escape_quotes_for_mysql() is + charset aware and performs exactly this doubling. + */ + std::string escaped(2 * val->length(), '\0'); + if (val->length()) + { + my_bool overflow; + size_t escaped_len= escape_quotes_for_mysql( + val->charset(), &escaped[0], 0, val->ptr(), val->length(), + &overflow); + escaped.resize(escaped_len); + } + default_value= "'" + escaped + "'"; + } else default_value= "NULL"; } @@ -233,8 +357,8 @@ static void append_stmt_alter_table(std::ostringstream &output, const std::string &schema_name, const std::string &table_name) { - output << "USE \"" << schema_name << "\";"; - output << ALTER_TABLE_OP_STR << '"' << table_name << '"'; + output << "USE " << quote_duckdb_identifier(schema_name) << ";"; + output << ALTER_TABLE_OP_STR << quote_duckdb_identifier(table_name); } static void append_stmt_column_add(std::ostringstream &output, @@ -248,7 +372,7 @@ static void append_stmt_column_add(std::ostringstream &output, assert(!schema_name.empty() && !table_name.empty() && !column_name.empty() && !column_type.empty()); append_stmt_alter_table(output, schema_name, table_name); - output << ADD_COLUMN_OP_STR << '"' << column_name << '"' << " " + output << ADD_COLUMN_OP_STR << quote_duckdb_identifier(column_name) << " " << column_type; if (has_default) output << DEFINE_DEFAULT_STR << default_value; @@ -262,7 +386,7 @@ static void append_stmt_column_drop(std::ostringstream &output, { assert(!schema_name.empty() && !table_name.empty() && !column_name.empty()); append_stmt_alter_table(output, schema_name, table_name); - output << DROP_COLUMN_OP_STR << '"' << column_name << '"' << ";"; + output << DROP_COLUMN_OP_STR << quote_duckdb_identifier(column_name) << ";"; } static void append_stmt_column_change_type(std::ostringstream &output, @@ -274,7 +398,7 @@ static void append_stmt_column_change_type(std::ostringstream &output, assert(!schema_name.empty() && !table_name.empty() && !column_name.empty() && !column_type.empty()); append_stmt_alter_table(output, schema_name, table_name); - output << ALTER_COLUMN_OP_STR << '"' << column_name << '"' + output << ALTER_COLUMN_OP_STR << quote_duckdb_identifier(column_name) << SET_DATA_TYPE_STR << column_type << ";"; } @@ -287,8 +411,8 @@ static void append_stmt_column_rename(std::ostringstream &output, assert(!schema_name.empty() && !table_name.empty() && !old_column_name.empty() && !new_column_name.empty()); append_stmt_alter_table(output, schema_name, table_name); - output << RENAME_COLUMN_OP_STR << '"' << old_column_name << '"' << " TO " - << '"' << new_column_name << '"' << ";"; + output << RENAME_COLUMN_OP_STR << quote_duckdb_identifier(old_column_name) + << " TO " << quote_duckdb_identifier(new_column_name) << ";"; } static void append_stmt_column_set_default(std::ostringstream &output, @@ -300,8 +424,8 @@ static void append_stmt_column_set_default(std::ostringstream &output, assert(!schema_name.empty() && !table_name.empty() && !column_name.empty() && !default_value.empty()); append_stmt_alter_table(output, schema_name, table_name); - output << ALTER_COLUMN_OP_STR << '"' << column_name << '"' << SET_DEFAULT_STR - << default_value << ";"; + output << ALTER_COLUMN_OP_STR << quote_duckdb_identifier(column_name) + << SET_DEFAULT_STR << default_value << ";"; } static void append_stmt_column_drop_default(std::ostringstream &output, @@ -311,7 +435,7 @@ static void append_stmt_column_drop_default(std::ostringstream &output, { assert(!schema_name.empty() && !table_name.empty() && !column_name.empty()); append_stmt_alter_table(output, schema_name, table_name); - output << ALTER_COLUMN_OP_STR << '"' << column_name << '"' + output << ALTER_COLUMN_OP_STR << quote_duckdb_identifier(column_name) << DROP_DEFAULT_STR << ";"; } @@ -322,7 +446,7 @@ static void append_stmt_column_set_not_null(std::ostringstream &output, { assert(!schema_name.empty() && !table_name.empty() && !column_name.empty()); append_stmt_alter_table(output, schema_name, table_name); - output << ALTER_COLUMN_OP_STR << '"' << column_name << '"' + output << ALTER_COLUMN_OP_STR << quote_duckdb_identifier(column_name) << SET_NOT_NULL_STR << ";"; } @@ -333,7 +457,7 @@ static void append_stmt_column_drop_not_null(std::ostringstream &output, { assert(!schema_name.empty() && !table_name.empty() && !column_name.empty()); append_stmt_alter_table(output, schema_name, table_name); - output << ALTER_COLUMN_OP_STR << '"' << column_name << '"' + output << ALTER_COLUMN_OP_STR << quote_duckdb_identifier(column_name) << DROP_NOT_NULL_STR << ";"; } @@ -348,18 +472,29 @@ static void append_stmt_table_rename(std::ostringstream &output, !new_schema_name.empty() && !new_table_name.empty()); assert(old_schema_name == new_schema_name); append_stmt_alter_table(output, old_schema_name, old_table_name); - output << RENAME_TABLE_OP_STR << '"' << new_table_name << '"' << ";"; + output << RENAME_TABLE_OP_STR << quote_duckdb_identifier(new_table_name) + << ";"; } /* ----- FieldConvertor ----- */ bool FieldConvertor::check() { - /* not support auto_increment */ - if (m_field->flags & AUTO_INCREMENT_FLAG) - return report_duckdb_table_struct_error( - "AUTO_INCREMENT", "removing AUTO_INCREMENT from column", - m_field->field_name.str, m_ctx); + /* + AUTO_INCREMENT is supported by CREATE TABLE only. Adding it to an + existing table would require seeding the sequence from the current + column maximum, which is not implemented yet. + */ + if ((m_field->flags & AUTO_INCREMENT_FLAG) && + m_ctx != ddl_error_context::CREATE) + { + /* + Same error the server raised while HA_NO_AUTO_INCREMENT was set, so + that the rejection stays indistinguishable from before. + */ + my_error(ER_TABLE_CANT_HANDLE_AUTO_INCREMENT, MYF(0), "DUCKDB"); + return true; + } /* No support for INVISIBLE columns. */ if (m_field->invisible >= INVISIBLE_USER) @@ -397,7 +532,9 @@ std::string FieldConvertor::translate() std::ostringstream result; - result << '"' << field->field_name.str << '"' << " "; + result << quote_duckdb_identifier(field->field_name.str, + field->field_name.length) + << " "; result << convert_type(m_field); if (field->flags & NOT_NULL_FLAG) @@ -419,7 +556,9 @@ std::string FieldConvertor::translate() */ std::string expr_str= get_default_expr_from_vcol_defs(field->table->s, field->field_index); - if (!expr_str.empty()) + if (has_mariadb_sequence_func(expr_str)) + note_sequence_default_omitted(field->field_name.str); + else if (!expr_str.empty()) result << " DEFAULT (" << expr_str << ")"; } else if (field->table->s->default_values && field->table->record[0]) @@ -437,8 +576,6 @@ std::string FieldConvertor::translate() } } - assert(!(field->flags & AUTO_INCREMENT_FLAG)); - return result.str(); } @@ -567,6 +704,20 @@ bool CreateTableConvertor::check() return true; } + /* + An ALTER using ALGORITHM=COPY builds a shadow table through create(), so + it arrives here rather than through the ALTER convertors. Reject + AUTO_INCREMENT on that path: the new sequence would start at 1 while the + copied rows already hold larger ids, silently handing out duplicates. + Seeding from the current column maximum is not implemented. + */ + if (m_thd->lex->sql_command != SQLCOM_CREATE_TABLE && + find_autoinc_field(m_table)) + { + my_error(ER_TABLE_CANT_HANDLE_AUTO_INCREMENT, MYF(0), "DUCKDB"); + return true; + } + /* Check PK. */ TABLE_SHARE *share= m_table->s; @@ -608,17 +759,36 @@ std::string CreateTableConvertor::translate() std::ostringstream result; assert((m_create_info->options & HA_LEX_CREATE_TMP_TABLE) == 0); - result << "CREATE SCHEMA IF NOT EXISTS " << '"' << m_schema_name << '"' - << ";"; + result << "CREATE SCHEMA IF NOT EXISTS " + << quote_duckdb_identifier(m_schema_name) << ";"; - result << "USE " << '"' << m_schema_name << '"' << ";"; + result << "USE " << quote_duckdb_identifier(m_schema_name) << ";"; + + /* + The sequence must exist before the table, because the AUTO_INCREMENT + column default resolves it at CREATE TABLE bind time. + */ + if (find_autoinc_field(m_table)) + { + ulonglong start= m_create_info->auto_increment_value + ? m_create_info->auto_increment_value + : 1; + /* A DuckDB sequence counter is int64; never emit an unrepresentable start. */ + if (start > (ulonglong) INT64_MAX) + start= (ulonglong) INT64_MAX; + + result << "CREATE SEQUENCE IF NOT EXISTS " + << quote_duckdb_identifier(m_schema_name) << "." + << quote_duckdb_identifier(autoinc_sequence_name(m_table_name)) + << " START WITH " << start << ";"; + } result << CREATE_TABLE_STR; /* MariaDB: IF NOT EXISTS is handled at the SQL layer, not in HA_CREATE_INFO. Always use IF NOT EXISTS for safety in DuckDB. */ result << IF_NOT_EXISTS_STR; - result << '"' << m_table_name << '"'; + result << quote_duckdb_identifier(m_table_name); result << " ("; append_column_definition(result); @@ -635,7 +805,16 @@ void CreateTableConvertor::append_column_definition(std::ostringstream &output) { if (ptr != first_field) output << ","; - output << FieldConvertor(field).translate(); + output << FieldConvertor(field, ddl_error_context::CREATE).translate(); + + /* + The AUTO_INCREMENT column draws from a DuckDB sequence. Keeping the + default in DuckDB lets writes that bypass MariaDB still obtain a + non-colliding id from the same counter. + */ + if (field->flags & AUTO_INCREMENT_FLAG) + output << " DEFAULT " + << autoinc_nextval_expr(m_schema_name, m_table_name); } } @@ -676,7 +855,11 @@ void AddColumnConvertor::prepare_columns() m_columns_to_add.emplace_back(new_field, field); if ((new_field->flags & NOT_NULL_FLAG) != 0) + { m_columns_to_set_not_null.emplace_back(new_field, field); + if ((field->flags & NO_DEFAULT_VALUE_FLAG) != 0) + m_columns_to_drop_default.emplace_back(new_field, field); + } } } @@ -711,11 +894,28 @@ std::string AddColumnConvertor::translate() default_value= "CURRENT_TIMESTAMP"; } else if (!(field->flags & NO_DEFAULT_VALUE_FLAG)) + { + /* + A sequence default must not be forwarded. Every other default, + including a constant-foldable expression, keeps using the value + MariaDB already materialised in default_values. + */ + if (is_sequence_default(field)) + note_sequence_default_omitted(field->field_name.str); + else + { + my_ptrdiff_t offset= + field->table->s->default_values - field->table->record[0]; + has_default= true; + default_value= get_field_default_for_duckdb(field, offset, false); + } + } + else if (field->flags & NOT_NULL_FLAG) { my_ptrdiff_t offset= field->table->s->default_values - field->table->record[0]; has_default= true; - default_value= get_field_default_for_duckdb(field, offset); + default_value= get_field_default_for_duckdb(field, offset, false); } append_stmt_column_add(result, m_schema_name, m_table_name, @@ -731,6 +931,13 @@ std::string AddColumnConvertor::translate() new_field->field_name.str); } + for (auto &pair : m_columns_to_drop_default) + { + Create_field *new_field= pair.first; + append_stmt_column_drop_default(result, m_schema_name, m_table_name, + new_field->field_name.str); + } + return result.str(); } @@ -884,10 +1091,39 @@ void ChangeColumnDefaultConvertor::prepare_columns() else if (new_has_default && !old_has_default) { /* Default was added */ - m_columns_to_set_default.emplace_back(nullptr, new_field); + if (is_sequence_default(new_field)) + { + note_sequence_default_omitted(new_field->field_name.str); + m_columns_to_drop_default.emplace_back(nullptr, new_field); + } + else + m_columns_to_set_default.emplace_back(nullptr, new_field); } else if (old_has_default && new_has_default) { + std::string old_expr= get_expr_default_text(old_field); + std::string new_expr= get_expr_default_text(new_field); + + if (!old_expr.empty() || !new_expr.empty()) + { + /* + Expression default. It is not materialised in default_values, so + the byte comparison below would be meaningless — compare the + expression text instead. + */ + if (old_expr != new_expr) + { + if (has_mariadb_sequence_func(new_expr)) + { + note_sequence_default_omitted(new_field->field_name.str); + m_columns_to_drop_default.emplace_back(nullptr, new_field); + } + else + m_columns_to_set_default.emplace_back(nullptr, new_field); + } + continue; + } + /* Both have default — check if value changed */ my_ptrdiff_t old_off= old_field->table->s->default_values - old_field->table->record[0]; @@ -951,9 +1187,24 @@ std::string ChangeColumnDefaultConvertor::translate() { Field *field= pair.second; - my_ptrdiff_t offset= - field->table->s->default_values - field->table->record[0]; - std::string default_value= get_field_default_for_duckdb(field, offset); + std::string default_value; + std::string expr= get_expr_default_text(field); + if (!expr.empty()) + { + /* + Expression default: forward the expression text. It is not + materialised in default_values, so the byte path below would + produce a meaningless literal. Sequence defaults never reach + here — prepare_columns() diverts them to drop-default. + */ + default_value= "(" + expr + ")"; + } + else + { + my_ptrdiff_t offset= + field->table->s->default_values - field->table->record[0]; + default_value= get_field_default_for_duckdb(field, offset); + } append_stmt_column_set_default(result, m_schema_name, m_table_name, field->field_name.str, default_value); @@ -1060,9 +1311,40 @@ std::string ChangeColumnConvertor::translate() if (!field || (field->flags & NO_DEFAULT_VALUE_FLAG)) continue; - my_ptrdiff_t offset= - field->table->s->default_values - field->table->record[0]; - std::string default_value= get_field_default_for_duckdb(field, offset); + std::string new_expr= get_expr_default_text(field); + if (has_mariadb_sequence_func(new_expr)) + { + /* + A sequence default is never forwarded to DuckDB. Act only when the + sequence default is newly introduced by this ALTER: drop whatever + default DuckDB had for the column. An unchanged sequence default + needs no statement (DuckDB never had one) and no repeated note. + */ + if (new_expr != get_expr_default_text(new_field->field)) + { + note_sequence_default_omitted(new_field->field_name.str); + append_stmt_column_drop_default(result, m_schema_name, m_table_name, + new_field->field_name.str); + } + continue; + } + + std::string default_value; + if (!new_expr.empty()) + { + /* + Expression default: forward the expression text. It is not + materialised in default_values, so the byte path below would + produce a meaningless literal. + */ + default_value= "(" + new_expr + ")"; + } + else + { + my_ptrdiff_t offset= + field->table->s->default_values - field->table->record[0]; + default_value= get_field_default_for_duckdb(field, offset); + } append_stmt_column_set_default(result, m_schema_name, m_table_name, new_field->field_name.str, default_value); diff --git a/storage/duckdb/convertor/ddl_convertor.h b/storage/duckdb/convertor/ddl_convertor.h index a3a084ade8851..6cb33633ca173 100644 --- a/storage/duckdb/convertor/ddl_convertor.h +++ b/storage/duckdb/convertor/ddl_convertor.h @@ -49,6 +49,16 @@ static inline bool is_uuid_field(const Field *field) field->type_handler()->type_collection() == uuid_coll; } +/** + Name of the DuckDB sequence backing the AUTO_INCREMENT column of a table. + MariaDB allows at most one AUTO_INCREMENT column per table, so the table + name alone identifies the sequence. +*/ +std::string autoinc_sequence_name(const std::string &table_name); + +/** The AUTO_INCREMENT field of a table, or nullptr if there is none. */ +Field *find_autoinc_field(const TABLE *table); + class BaseConvertor { public: @@ -235,6 +245,9 @@ class AddColumnConvertor : public AlterTableConvertor /** Columns to set not null */ Columns m_columns_to_set_not_null; + /** Columns whose temporary default must be dropped */ + Columns m_columns_to_drop_default; + /** Prepare columns to add and set not null. */ void prepare_columns(); }; diff --git a/storage/duckdb/convertor/dml_convertor.cc b/storage/duckdb/convertor/dml_convertor.cc index 76108e00d53aa..4685ed5f04d46 100644 --- a/storage/duckdb/convertor/dml_convertor.cc +++ b/storage/duckdb/convertor/dml_convertor.cc @@ -32,6 +32,24 @@ namespace myduck { extern my_bool use_double_for_decimal; } static const uint sizeof_trailing_comma= sizeof(", ") - 1; static const uint sizeof_trailing_and= sizeof(" AND ") - 1; +/* + Append an SQL identifier to a String, quoted for DuckDB. DuckDB escapes an + embedded double quote by doubling it; failing to escape lets a crafted + identifier break out of the quoted name and inject SQL (MDEV-40653). +*/ +static void append_quoted_identifier(String &target, const char *name, + size_t length) +{ + target.append(STRING_WITH_LEN("\"")); + for (size_t i= 0; i < length; i++) + { + if (name[i] == '"') + target.append(STRING_WITH_LEN("\"")); + target.append(&name[i], 1); + } + target.append(STRING_WITH_LEN("\"")); +} + void append_field_value_to_sql(String &target_str, Field *field) { if (field->is_null()) @@ -165,13 +183,10 @@ static inline void append_table_name(TABLE *table, String &query) the temp name. */ DatabaseTableNames dt(table->s->normalized_path.str); - query.append(STRING_WITH_LEN("\"")); - query.append(dt.db_name.c_str(), dt.db_name.length()); - query.append(STRING_WITH_LEN("\"")); + append_quoted_identifier(query, dt.db_name.c_str(), dt.db_name.length()); query.append(STRING_WITH_LEN(".")); - query.append(STRING_WITH_LEN("\"")); - query.append(dt.table_name.c_str(), dt.table_name.length()); - query.append(STRING_WITH_LEN("\"")); + append_quoted_identifier(query, dt.table_name.c_str(), + dt.table_name.length()); } static inline void get_write_fields(TABLE *table, std::vector &fields) @@ -232,9 +247,8 @@ void DMLConvertor::generate_where_clause(String &query) for (auto field : fields) { - query.append(STRING_WITH_LEN("\"")); - query.append(field->field_name.str, field->field_name.length); - query.append(STRING_WITH_LEN("\"")); + append_quoted_identifier(query, field->field_name.str, + field->field_name.length); query.append(STRING_WITH_LEN(" = ")); append_where_value(query, field); @@ -260,9 +274,8 @@ void InsertConvertor::generate_fields_and_values(String &query) query.append(STRING_WITH_LEN(" (")); for (auto field : fields) { - query.append(STRING_WITH_LEN("\"")); - query.append(field->field_name.str, field->field_name.length); - query.append(STRING_WITH_LEN("\"")); + append_quoted_identifier(query, field->field_name.str, + field->field_name.length); query.append(STRING_WITH_LEN(", ")); } query.length(query.length() - sizeof_trailing_comma); @@ -293,9 +306,8 @@ void UpdateConvertor::generate_fields_and_values(String &query) for (auto field : fields) { - query.append(STRING_WITH_LEN("\"")); - query.append(field->field_name.str, field->field_name.length); - query.append(STRING_WITH_LEN("\"")); + append_quoted_identifier(query, field->field_name.str, + field->field_name.length); query.append(STRING_WITH_LEN(" = ")); append_field_value_to_sql(query, field); diff --git a/storage/duckdb/debian/mariadb-plugin-duckdb.install b/storage/duckdb/debian/mariadb-plugin-duckdb.install index 248ff92bd02e8..45065aa38542d 100644 --- a/storage/duckdb/debian/mariadb-plugin-duckdb.install +++ b/storage/duckdb/debian/mariadb-plugin-duckdb.install @@ -1 +1,2 @@ +etc/mysql/mariadb.conf.d/duckdb.cnf usr/lib/mysql/plugin/ha_duckdb.so diff --git a/storage/duckdb/docs/architecture.md b/storage/duckdb/docs/architecture.md index 4e773431716f3..b87a027b984fd 100644 --- a/storage/duckdb/docs/architecture.md +++ b/storage/duckdb/docs/architecture.md @@ -190,10 +190,13 @@ check_if_supported_inplace_alter() → HA_ALTER_INPLACE_NO_LOCK commit_inplace_alter_table() → AddColumnConvertor / DropColumnConvertor / ChangeColumnConvertor / ChangeColumnDefaultConvertor / ChangeColumnForPrimaryKeyConvertor - → each operation executes in a separate auto-commit context - (DuckDB v1.5+ disallows compound DDL mixing structural + constraint changes) + → all generated operations execute in one explicit DuckDB transaction ``` +Table partitioning is disabled with `HTON_NO_PARTITION`; both +`CREATE TABLE ... PARTITION BY` and `ALTER TABLE ... PARTITION BY` are rejected +before MariaDB creates a partition handler or starts the table-copy protocol. + DROP DATABASE: `duckdb_drop_database()` → `DROP SCHEMA IF EXISTS "db"`. ### Path 3: Row-by-Row DML diff --git a/storage/duckdb/docs/mariadb-duckdb-incompatibilities.md b/storage/duckdb/docs/mariadb-duckdb-incompatibilities.md index d391478119c5a..512488a5b7de7 100644 --- a/storage/duckdb/docs/mariadb-duckdb-incompatibilities.md +++ b/storage/duckdb/docs/mariadb-duckdb-incompatibilities.md @@ -63,6 +63,12 @@ MariaDB function semantics differ from DuckDB in several areas. These are handle | `POWER()` | `pow()` | Supported natively | | `SUBSTRING()` | `substr()` | Supported natively | +### Not translated by the pushdown layer + +| Function | Issue | Workaround | +|---|---|---| +| `TIMESTAMPDIFF(unit, dt1, dt2)` | The `unit` keyword is not bindable, so the call reaches DuckDB as an unknown `timestampdiff` function and the query fails. DuckDB's own `date_diff()` takes the unit as a string and is not a drop-in match | Use `UNIX_TIMESTAMP()` arithmetic, e.g. `FLOOR((UNIX_TIMESTAMP(dt2) - UNIX_TIMESTAMP(dt1)) / 600) * 10` for minute buckets | + ### Potentially incompatible (not yet triggered) | User writes | MariaDB canonical | DuckDB status | @@ -88,14 +94,15 @@ SELECT pushdown uses the original SQL text from `THD::query()`. MariaDB-specific | `HIGH_PRIORITY`, `SQL_NO_CACHE`, `SQL_CACHE`, `SQL_BUFFER_RESULT`, `SQL_SMALL_RESULT`, `SQL_BIG_RESULT`, `SQL_CALC_FOUND_ROWS` | -- | Stripped | | `FORCE INDEX(...)`, `USE INDEX(...)`, `IGNORE INDEX(...)` | -- | Stripped | -### Known unhandled cases (currently cause query failures) +### Known unhandled cases (currently fail or change semantics) -These MariaDB constructs are **not yet rewritten** and fail when pushed down. Because pushdown forwards the original `THD::query()` text (only backticks are converted to double quotes), MariaDB-specific token semantics survive into DuckDB. Discovered while running an analytical query set (402 queries) against DuckDB-engine tables. +These MariaDB constructs are **not yet rewritten** and either fail or have different semantics when pushed down. Because pushdown forwards the original `THD::query()` text (only backticks are converted to double quotes), MariaDB-specific token semantics survive into DuckDB. Discovered while running an analytical query set (402 queries) against DuckDB-engine tables. | MariaDB construct | Sent to DuckDB as | DuckDB result | Root cause | |---|---|---|---| | Double-quoted **string literal**, e.g. `JSON_OBJECT("month", ...)` | `"month"` (verbatim) | `Binder Error: Referenced column "month" not found` | MariaDB without `ANSI_QUOTES` treats `"x"` as a string literal; DuckDB treats `"x"` as an identifier. The forwarded literal is read as a column reference. | | Unquoted column **alias equal to a DuckDB reserved keyword**, e.g. `SELECT expr name` / `SELECT expr year` | `... name` / `... year` (verbatim) | `Parser Error: syntax error at or near "name"` | DuckDB forbids reserved keywords as unquoted identifiers. `AS name` or `"name"` work; bare `name` / `year` / `month` do not. This is why most implicit aliases pass but keyword aliases fail. | +| MariaDB **executable/versioned comments**, e.g. `/*! + 1 */`, `/*!100000 + 1 */`, or `/*M! + 1 */` | Comment text (verbatim) | Contents are ignored as an ordinary block comment | MariaDB executes eligible `/*! ... */` and `/*M! ... */` contents as SQL, optionally gated by a version number; DuckDB treats the entire region as a comment. Forwarded queries can therefore silently use different predicates or expressions. | Reproductions (against any DuckDB-engine table `t`): @@ -104,9 +111,10 @@ SELECT JSON_OBJECT("k", 1) FROM t; -- Binder Error: column "k" not found SELECT JSON_OBJECT('k', 1) FROM t; -- OK SELECT col name FROM t; -- Parser Error at "name" SELECT col AS name FROM t; -- OK +SELECT 1 /*! + 1 */ FROM t; -- MariaDB: 2; DuckDB pushdown: 1 ``` -**Fix direction**: in `ha_duckdb_pushdown.cc`, convert double-quoted string literals to single-quoted form and quote (or `AS`-prefix) aliases that are DuckDB reserved keywords. Both require lexer-aware handling of the query text, not naive replacement — `backticks_to_double_quotes()` already produces legitimate double-quoted identifiers that must not be altered. +**Fix direction**: in `ha_duckdb_pushdown.cc`, convert double-quoted string literals to single-quoted form and quote (or `AS`-prefix) aliases that are DuckDB reserved keywords. Both require lexer-aware handling of the query text, not naive replacement — `backticks_to_double_quotes()` already produces legitimate double-quoted identifiers that must not be altered. Executable/versioned comments must either be expanded according to MariaDB's version rules or make the query ineligible for raw SQL forwarding. --- diff --git a/storage/duckdb/docs/tutorials/nyc-taxi-trips.md b/storage/duckdb/docs/tutorials/nyc-taxi-trips.md new file mode 100644 index 0000000000000..dc459a2482eca --- /dev/null +++ b/storage/duckdb/docs/tutorials/nyc-taxi-trips.md @@ -0,0 +1,380 @@ +# MariaDB DuckDB Tutorial: NYC Taxi Trips + +## Overview + +This tutorial follows the [ClickHouse taxi tutorial][clickhouse-tutorial] and its +[pg_clickhouse port][pg-clickhouse-tutorial], but runs everything on MariaDB with the DuckDB +storage engine. The trip data goes into an `ENGINE=DuckDB` table, the taxi zone lookup goes into an +`ENGINE=InnoDB` table, and a single MariaDB `SELECT` joins the two. + +The data is published by the [NYC Taxi and Limousine Commission][tlc-data] and is subject to the +[NYC Open Data terms of use][nyc-open-data]. The TLC notes that the trip records come from +authorised technology providers rather than from the TLC itself, and makes no accuracy guarantees. + +## Connect to MariaDB + +The DuckDB plugin must be loaded and `run_in_duckdb()` must be enabled: + +```ini +[mysqld] +plugin-maturity=gamma +plugin-load-add=ha_duckdb.so +duckdb-allow-run-in-duckdb=ON +``` + +See the [DuckDB storage engine README](../../README.md#building) for build and installation +instructions. + +> **Note:** `run_in_duckdb()` is OFF by default and requires the `SUPER` privilege. It executes +> arbitrary DuckDB SQL in-process as the server OS user, and DuckDB applies no access control of its +> own. Enable it only on a host you control, and read +> [`security-model.md`](../security-model.md) before using it in a shared or multi-tenant +> deployment. + +Start the client with UTF-8 and local file loading enabled: + +```sh +mariadb --default-character-set=utf8mb4 --local-infile=1 +``` + +Check that the engine, `run_in_duckdb()`, and local file loading are available: + +```sql +SHOW ENGINES; +SELECT @@duckdb_allow_run_in_duckdb; +SHOW VARIABLES LIKE 'local_infile'; +``` + +`SHOW ENGINES` should list `DUCKDB` with `Support` set to `YES`, +`duckdb_allow_run_in_duckdb` should be `1`, and `local_infile` should be `ON`. + +## Download the Data Set + +Run these commands on the MariaDB server host. DuckDB reads the Parquet file inside the server +process, so the account running `mariadbd` must be able to read it. The lookup CSV is loaded from +the client host via `LOAD DATA LOCAL INFILE`. + +Take one month of Yellow Taxi trips plus the taxi zone lookup: + +```sh +curl --fail --location -o /tmp/yellow_tripdata_2024-01.parquet \ + https://d37ci6vzurychx.cloudfront.net/trip-data/yellow_tripdata_2024-01.parquet + +curl --fail --location -o /tmp/taxi_zone_lookup.csv \ + https://d37ci6vzurychx.cloudfront.net/misc/taxi_zone_lookup.csv +``` + +The Parquet file is about 48 MiB and holds 2,964,624 trips. The lookup CSV holds 265 zones. + +## Create a Table + +```sql +CREATE DATABASE IF NOT EXISTS taxi; +USE taxi; + +CREATE TABLE trips ( + trip_id BIGINT NOT NULL, + vendor_id INT, + pickup_datetime DATETIME, + dropoff_datetime DATETIME, + passenger_count INT, + trip_distance DOUBLE, + pu_location_id INT, + do_location_id INT, + payment_type INT, + fare_amount DECIMAL(10,2), + tip_amount DECIMAL(10,2), + total_amount DECIMAL(10,2), + PRIMARY KEY (trip_id) +) ENGINE=DuckDB DEFAULT CHARSET=utf8mb4; +``` + +With the default `duckdb_require_primary_key=ON`, the table needs a primary key. The TLC files have +no trip identifier, so the load below generates `trip_id`. DuckDB table columns accept `utf8`, +`utf8mb3`, `utf8mb4`, and `ascii`; this example uses `utf8mb4`. + +## Add the Data Set + +`run_in_duckdb()` sends the SQL string to DuckDB's vectorized engine for execution and returns DuckDB's own textual rendering of the result as a single string value. DuckDB reads the Parquet file directly, so the load is a single statement: + +```sql +SELECT run_in_duckdb('INSERT INTO taxi.trips + (trip_id, vendor_id, pickup_datetime, dropoff_datetime, passenger_count, + trip_distance, pu_location_id, do_location_id, payment_type, + fare_amount, tip_amount, total_amount) +SELECT + ROW_NUMBER() OVER () AS trip_id, + "VendorID", tpep_pickup_datetime, tpep_dropoff_datetime, passenger_count, + trip_distance, "PULocationID", "DOLocationID", payment_type, + fare_amount, tip_amount, total_amount +FROM read_parquet(''/tmp/yellow_tripdata_2024-01.parquet'')'); +``` + +The TLC column names are mixed case, so they appear as double-quoted DuckDB identifiers. Only the +twelve selected columns are read from the file. + +`run_in_duckdb()` is the bulk-load path, not the only write path: ordinary `INSERT`, `UPDATE`, and +`DELETE` work against an `ENGINE=DuckDB` table too. With the default `duckdb_dml_in_batch=ON` those +writes are buffered and applied to DuckDB at commit, so a write is not visible to later reads in the +same transaction. This tutorial only reads, so it loads once and queries from there. + +Make sure we can query it: + +```sql +SELECT COUNT(*) AS total_trips FROM trips; +``` + +```text +total_trips +2964624 +``` + +The pickup timestamps are worth a look before trusting any date filter: + +```sql +SELECT MIN(pickup_datetime) AS earliest, MAX(pickup_datetime) AS latest FROM trips; +``` + +```text +earliest latest +2002-12-31 22:59:39 2024-02-01 00:01:15 +``` + +The January 2024 file holds 18 rows with timestamps outside the month, too few to move any of the +averages that follow. The queries that group or rank by time still filter on the month explicitly, +so their buckets cover exactly the period they claim to. + +## Analyze the Data + +Calculate the average tip amount: + +```sql +SELECT ROUND(AVG(tip_amount), 2) AS avg_tip FROM trips; +``` + +```text +avg_tip +3.34 +``` + +Calculate the average cost based on the number of passengers: + +```sql +SELECT passenger_count, + COUNT(*) AS trips, + ROUND(AVG(total_amount), 2) AS avg_total +FROM trips +WHERE passenger_count IS NOT NULL +GROUP BY passenger_count +ORDER BY passenger_count; +``` + +```text +passenger_count trips avg_total +0 31465 25.33 +1 2188739 26.21 +2 405103 29.52 +3 91262 29.14 +4 51974 30.88 +5 33506 26.27 +6 22353 25.80 +7 8 57.74 +8 51 95.67 +9 1 18.45 +``` + +Show the busiest pickup hours: + +```sql +SELECT HOUR(pickup_datetime) AS pickup_hour, + COUNT(*) AS trips, + ROUND(AVG(trip_distance), 2) AS avg_miles +FROM trips +WHERE pickup_datetime >= '2024-01-01' AND pickup_datetime < '2024-02-01' +GROUP BY HOUR(pickup_datetime) +ORDER BY trips DESC +LIMIT 5; +``` + +```text +pickup_hour trips avg_miles +18 212788 2.81 +17 206257 3.01 +16 190201 3.35 +15 189359 3.88 +19 184032 3.11 +``` + +Group trips into ten-minute buckets by duration: + +```sql +SELECT FLOOR((UNIX_TIMESTAMP(dropoff_datetime) - + UNIX_TIMESTAMP(pickup_datetime)) / 600) * 10 AS trip_minutes, + COUNT(*) AS trips, + ROUND(AVG(fare_amount), 2) AS avg_fare, + ROUND(AVG(tip_amount), 2) AS avg_tip +FROM trips +WHERE dropoff_datetime > pickup_datetime +GROUP BY FLOOR((UNIX_TIMESTAMP(dropoff_datetime) - + UNIX_TIMESTAMP(pickup_datetime)) / 600) * 10 +ORDER BY trip_minutes +LIMIT 6; +``` + +```text +trip_minutes trips avg_fare avg_tip +0 1229720 8.77 1.98 +10 1080498 15.77 3.00 +20 382804 29.13 5.06 +30 142500 47.72 7.84 +40 64520 59.82 9.35 +50 33057 65.80 9.64 +``` + +`TIMESTAMPDIFF()` is not pushed down — its `unit` keyword is not bindable, so the call reaches +DuckDB as an unknown function and fails on `ENGINE=DuckDB` tables. This query uses +`UNIX_TIMESTAMP()` arithmetic instead. See +[`mariadb-duckdb-incompatibilities.md`](../mariadb-duckdb-incompatibilities.md) for the full list of +pushdown-incompatible functions and syntax. + +## Add the Zone Lookup Table + +The trips carry zone IDs rather than names. The lookup table maps each ID to a borough and zone +name; `132` is JFK Airport and `138` is LaGuardia Airport. Keep it in InnoDB: + +```sql +CREATE TABLE taxi_zones ( + location_id INT PRIMARY KEY, + borough VARCHAR(30), + zone VARCHAR(60), + service_zone VARCHAR(20) +) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4; + +LOAD DATA LOCAL INFILE '/tmp/taxi_zone_lookup.csv' +INTO TABLE taxi_zones +FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '"' +LINES TERMINATED BY '\r\n' +IGNORE 1 LINES +(location_id, borough, zone, @service_zone) +SET service_zone = NULLIF(@service_zone, ''); +``` + +The lookup CSV uses CRLF line endings, hence `LINES TERMINATED BY '\r\n'`. + +Confirm the row count and the engines: + +```sql +SELECT COUNT(*) AS zones FROM taxi_zones; +SELECT TABLE_NAME, ENGINE +FROM information_schema.TABLES +WHERE TABLE_SCHEMA = 'taxi' +ORDER BY TABLE_NAME; +``` + +```text +zones +265 + +TABLE_NAME ENGINE +taxi_zones InnoDB +trips DUCKDB +``` + +## Perform a Cross-Engine Join + +The zone lookup is small and changes rarely, so it stays in InnoDB. The trip data is large and +analytical, so it lives in DuckDB. A single MariaDB `SELECT` joins them — no `run_in_duckdb()`, no +data copying, no ETL. The engine pushes the entire query down to DuckDB and streams the InnoDB rows +in on demand. + +Count the trips ending at JFK or LaGuardia, grouped by the borough where the passenger got in: + +```sql +SELECT z.borough, + COUNT(*) AS airport_trips, + ROUND(AVG(t.total_amount), 2) AS avg_total +FROM trips t +JOIN taxi_zones z ON t.pu_location_id = z.location_id +WHERE t.do_location_id IN (132, 138) +GROUP BY z.borough +ORDER BY airport_trips DESC; +``` + +```text +borough airport_trips avg_total +Manhattan 48077 78.09 +Queens 13789 43.41 +Brooklyn 738 60.82 +Unknown 109 64.48 +Bronx 27 63.63 +N/A 21 94.95 +EWR 1 135.18 +Staten Island 1 11.50 +``` + +`trips` stays in DuckDB and `taxi_zones` stays in InnoDB. There is no `run_in_duckdb()` call and no +copy of the data: this is a plain MariaDB `SELECT` across two storage engines. + +To see what MariaDB sent to DuckDB, turn on query logging and run the join again: + +```sql +SET GLOBAL duckdb_log_options = 'DUCKDB_QUERY'; +``` + +The server error log then shows the pushed-down statement and the external table: + +```text +[Note] DuckDB: cross-engine pushdown with 1 external table(s) +[Note] DuckDB query: SELECT z.borough, COUNT(*) AS airport_trips, ROUND(AVG(t.total_amount), 2) ... +``` + +DuckDB received the whole query, joins and aggregates included, and MariaDB streamed the InnoDB +rows into it. Turn logging off again: + +```sql +SET GLOBAL duckdb_log_options = ''; +``` + +One more join, this time ranking pickup zones for the month: + +```sql +SELECT z.zone, + COUNT(*) AS pickups, + ROUND(AVG(t.tip_amount), 2) AS avg_tip +FROM trips t +JOIN taxi_zones z ON t.pu_location_id = z.location_id +WHERE t.pickup_datetime >= '2024-01-01' AND t.pickup_datetime < '2024-02-01' +GROUP BY z.zone +ORDER BY pickups DESC +LIMIT 10; +``` + +```text +zone pickups avg_tip +JFK Airport 145240 8.86 +Midtown Center 143469 3.08 +Upper East Side South 142707 2.59 +Upper East Side North 136464 2.64 +Midtown East 106717 3.02 +Times Sq/Theatre District 106324 3.30 +Penn Station/Madison Sq West 104522 3.09 +Lincoln Square East 104080 2.79 +LaGuardia Airport 89530 8.67 +Upper West Side South 88474 2.79 +``` + +Airport pickups tip roughly three times as much as midtown ones. + +## Clean Up + +```sql +DROP DATABASE taxi; +``` + +The [OWID CO₂ tutorial](owid-co2-emissions.md) loads CSV instead of Parquet and computes +year-over-year change with a `LAG()` window function. + +[clickhouse-tutorial]: https://clickhouse.com/docs/tutorial +[nyc-open-data]: https://opendata.cityofnewyork.us/overview/ +[pg-clickhouse-tutorial]: https://github.com/ClickHouse/pg_clickhouse/blob/b2bacc51e1205f840046fe9f48f806baa6ae169e/doc/tutorial.md +[tlc-data]: https://www.nyc.gov/site/tlc/about/tlc-trip-record-data.page diff --git a/storage/duckdb/docs/tutorials/owid-co2-emissions.md b/storage/duckdb/docs/tutorials/owid-co2-emissions.md new file mode 100644 index 0000000000000..5b135cb3d1523 --- /dev/null +++ b/storage/duckdb/docs/tutorials/owid-co2-emissions.md @@ -0,0 +1,219 @@ +# MariaDB DuckDB Tutorial: OWID CO₂ Emissions + +## Overview + +This tutorial loads the [Our World in Data CO₂ and Greenhouse Gas Emissions dataset][owid-data] +into an `ENGINE=DuckDB` table and analyzes it through the regular `mariadb` client. + +OWID publishes its work under [Creative Commons BY 4.0][owid-license] and notes that data from +third-party providers remains subject to the providers' license terms. + +## Connect to MariaDB + +The DuckDB plugin must be loaded and `run_in_duckdb()` must be enabled: + +```ini +[mysqld] +plugin-maturity=gamma +plugin-load-add=ha_duckdb.so +duckdb-allow-run-in-duckdb=ON +``` + +See the [DuckDB storage engine README](../../README.md#building) for build and installation +instructions. + +> **Note:** `run_in_duckdb()` is OFF by default and requires the `SUPER` privilege. It executes +> arbitrary DuckDB SQL in-process as the server OS user, and DuckDB applies no access control of its +> own. Enable it only on a host you control, and read +> [`security-model.md`](../security-model.md) before using it in a shared or multi-tenant +> deployment. + +Start the client with a UTF-8 character set: + +```sh +mariadb --default-character-set=utf8mb4 +``` + +Check that the engine and `run_in_duckdb()` are available: + +```sql +SHOW ENGINES; +SELECT @@duckdb_allow_run_in_duckdb; +``` + +`SHOW ENGINES` should list `DUCKDB` with `Support` set to `YES`. The variable should be `1`. + +## Download the Data Set + +Download a copy pinned to a Git commit so that the results below remain reproducible: + +```sh +curl --fail --location -o /tmp/owid-co2-data.csv \ + https://raw.githubusercontent.com/owid/co2-data/382ee6c662b0ece26e111f263b44c029afad7787/owid-co2-data.csv +``` + +Run this command on the MariaDB server host and make sure the account running `mariadbd` can read +the file. + +## Create a Table + +Create a database and a DuckDB table for a subset of the CSV columns: + +```sql +CREATE DATABASE IF NOT EXISTS owid; +USE owid; + +CREATE TABLE co2_emissions ( + country VARCHAR(60) NOT NULL, + year INT NOT NULL, + iso_code VARCHAR(10), + population BIGINT, + gdp DOUBLE, + co2 DOUBLE, + co2_growth_prct DOUBLE, + co2_per_capita DOUBLE, + coal_co2 DOUBLE, + gas_co2 DOUBLE, + oil_co2 DOUBLE, + cement_co2 DOUBLE, + cumulative_co2 DOUBLE, + share_global_co2 DOUBLE, + share_global_cumulative_co2 DOUBLE, + temperature_change_from_co2 DOUBLE, + PRIMARY KEY (country, year) +) ENGINE=DuckDB DEFAULT CHARSET=utf8mb4; +``` + +With the default `duckdb_require_primary_key=ON`, the table needs a primary key. DuckDB table +columns accept `utf8`, `utf8mb3`, `utf8mb4`, and `ascii`; this example uses `utf8mb4`. +`VARCHAR(60)` keeps the `country` key part below MariaDB's 255-byte key length limit (60 × 4 bytes). + +## Add the Data Set + +`run_in_duckdb()` sends the SQL string to DuckDB's vectorized engine for execution and returns DuckDB's own textual rendering of the result as a single string value. Use DuckDB's CSV reader to load the file inside the server process: + +```sql +SELECT run_in_duckdb('INSERT INTO owid.co2_emissions + (country, year, iso_code, population, gdp, co2, co2_growth_prct, + co2_per_capita, coal_co2, gas_co2, oil_co2, cement_co2, + cumulative_co2, share_global_co2, share_global_cumulative_co2, + temperature_change_from_co2) +SELECT + country, year, iso_code, population, gdp, co2, co2_growth_prct, + co2_per_capita, coal_co2, gas_co2, oil_co2, cement_co2, + cumulative_co2, share_global_co2, share_global_cumulative_co2, + temperature_change_from_co2 +FROM read_csv_auto(''/tmp/owid-co2-data.csv'')'); +``` + +`run_in_duckdb()` is the bulk-load path, not the only write path: ordinary `INSERT`, `UPDATE`, and +`DELETE` work against an `ENGINE=DuckDB` table too. With the default `duckdb_dml_in_batch=ON` those +writes are buffered and applied to DuckDB at commit, so a write is not visible to later reads in the +same transaction. This tutorial only reads, so it loads once and queries from there. + +Make sure the data was imported: + +```sql +SELECT COUNT(*) AS total_rows FROM co2_emissions; +SELECT MIN(year) AS earliest, + MAX(year) AS latest, + COUNT(DISTINCT country) AS locations +FROM co2_emissions; +``` + +```text +total_rows +50411 + +earliest latest locations +1750 2024 254 +``` + +## Analyze the Data + +Start with the ten largest country-level emitters in the latest year: + +```sql +SELECT country, + ROUND(co2, 1) AS co2_mt, + ROUND(co2_per_capita, 2) AS co2_per_capita_t +FROM co2_emissions +WHERE year = (SELECT MAX(year) FROM co2_emissions WHERE co2 IS NOT NULL) + AND co2 IS NOT NULL + AND iso_code IS NOT NULL +ORDER BY co2 DESC +LIMIT 10; +``` + +```text +country co2_mt co2_per_capita_t +China 12289.0 8.66 +United States 4904.1 14.20 +India 3193.5 2.20 +Russia 1780.5 12.30 +Japan 961.9 7.77 +Indonesia 812.2 2.87 +Iran 792.6 8.66 +Saudi Arabia 692.1 20.38 +South Korea 583.7 11.29 +Germany 572.3 6.77 +``` + +`iso_code IS NOT NULL` removes aggregate rows such as `World` and `Europe` from the ranking. + +Next, group country-level emissions by decade: + +```sql +SELECT FLOOR(year / 10) * 10 AS decade, + ROUND(SUM(co2), 1) AS total_co2_mt +FROM co2_emissions +WHERE iso_code IS NOT NULL + AND co2 IS NOT NULL +GROUP BY FLOOR(year / 10) * 10 +ORDER BY decade DESC +LIMIT 5; +``` + +```text +decade total_co2_mt +2020 181243.9 +2010 342507.4 +2000 279781.4 +1990 229151.2 +1980 196853.2 +``` + +Use `LAG()` to calculate China's annual change: + +```sql +SELECT year, + ROUND(co2, 1) AS co2_mt, + ROUND((co2 - LAG(co2) OVER (ORDER BY year)) / + LAG(co2) OVER (ORDER BY year) * 100, 2) AS yoy_growth_pct +FROM co2_emissions +WHERE country = 'China' + AND co2 IS NOT NULL +ORDER BY year DESC +LIMIT 5; +``` + +```text +year co2_mt yoy_growth_pct +2024 12289.0 0.96 +2023 12172.0 3.93 +2022 11711.8 3.79 +2021 11284.4 3.56 +2020 10896.5 1.71 +``` + +## Clean Up + +```sql +DROP DATABASE owid; +``` + +The [NYC taxi tutorial](nyc-taxi-trips.md) loads Parquet instead of CSV and joins DuckDB data with +an InnoDB reference table. + +[owid-data]: https://github.com/owid/co2-data +[owid-license]: https://github.com/owid/co2-data/tree/382ee6c662b0ece26e111f263b44c029afad7787#license diff --git a/storage/duckdb/ha_duckdb.cc b/storage/duckdb/ha_duckdb.cc index 79f5b8c322271..62d08c8a075c2 100644 --- a/storage/duckdb/ha_duckdb.cc +++ b/storage/duckdb/ha_duckdb.cc @@ -37,7 +37,6 @@ #include "duckdb_select.h" #include "ddl_convertor.h" #include "dml_convertor.h" -#include "delta_appender.h" #include "row_helpers.h" #include "ha_duckdb_pushdown.h" #include "duckdb_log.h" @@ -61,6 +60,7 @@ static duckdb_status_t srv_duckdb_status; static my_bool copy_ddl_in_batch= TRUE; static my_bool dml_in_batch= TRUE; static my_bool update_modified_column_only= TRUE; +static ulonglong autoinc_cache_size= 1000; static handler *duckdb_create_handler(handlerton *hton, TABLE_SHARE *table, MEM_ROOT *mem_root); @@ -211,9 +211,7 @@ static void duckdb_drop_database(handlerton *hton, char *path) Databasename db(path); - std::string query= "DROP SCHEMA IF EXISTS \""; - query.append(db.name); - query.append("\""); + std::string query= "DROP SCHEMA IF EXISTS " + quote_duckdb_identifier(db.name); if (duckdb_register_trx(thd)) DBUG_VOID_RETURN; @@ -231,7 +229,7 @@ static int duckdb_init_func(void *p) duckdb_hton= (handlerton *) p; duckdb_hton->db_type= DB_TYPE_AUTOASSIGN; duckdb_hton->create= duckdb_create_handler; - duckdb_hton->flags= HTON_TEMPORARY_NOT_SUPPORTED; + duckdb_hton->flags= HTON_TEMPORARY_NOT_SUPPORTED | HTON_NO_PARTITION; duckdb_hton->prepare= duckdb_prepare; duckdb_hton->commit= duckdb_commit; duckdb_hton->rollback= duckdb_rollback; @@ -262,7 +260,12 @@ static int duckdb_deinit_func(void *p) /* ----- Share management ----- */ -Duckdb_share::Duckdb_share() { thr_lock_init(&lock); } +Duckdb_share::Duckdb_share() +{ + thr_lock_init(&lock); + mysql_mutex_init(PSI_INSTRUMENT_ME, &autoinc_refill_mutex, + MY_MUTEX_INIT_FAST); +} Duckdb_share *ha_duckdb::get_share() { @@ -458,7 +461,7 @@ static void build_duckdb_blob_map(Field **field_list, MY_BITMAP *map) /* ----- DML operations ----- */ -int ha_duckdb::write_row(const uchar *) +int ha_duckdb::write_row(const uchar *buf) { DBUG_ENTER("ha_duckdb::write_row"); int ret= 0; @@ -474,6 +477,19 @@ int ha_duckdb::write_row(const uchar *) DBUG_RETURN(ret); } + /* + Fill the AUTO_INCREMENT field. Every row reaches DuckDB with all columns + materialized (both the Appender and the SQL insert paths), so the value + must be assigned here; the sequence-backed column default in DuckDB is + never consulted on this path. + */ + if (table->next_number_field && buf == table->record[0] && + (ret= update_auto_increment())) + { + dbug_tmp_restore_column_map(&table->read_set, org_bitmap); + DBUG_RETURN(ret); + } + myduck::BatchState batch_state= get_batch_state(thd); if (batch_state == myduck::BatchState::NOT_IN_BATCH) @@ -510,6 +526,136 @@ int ha_duckdb::write_row(const uchar *) DBUG_RETURN(ret); } +/* + Consume `want` ids from the table's AUTO_INCREMENT sequence in one query + and return the first id of the block, or 0 on failure (a sequence value is + never 0: CREATE SEQUENCE is emitted with START WITH >= 1). + + nextval() calls of a concurrent DuckDB-side writer (batch ingestion through + run_in_duckdb relying on the column default) may interleave with ours + inside this query. The ids missing from our span belong to that writer, so + such a block must not be handed out; detect it by comparing the span width + with the block size and retry. MariaDB-side callers cannot cause this: they + are serialized on the share's autoinc_refill_mutex. +*/ +static ulonglong reserve_autoinc_block(THD *thd, const char *db_name, + const char *table_name, ulonglong want) +{ + /* + The qualified sequence name is a quoted identifier nested inside a + single-quoted string literal argument to nextval(). Escape the inner + identifiers (doubling ") and then escape the result for the enclosing + literal (doubling ') so a crafted schema/table name cannot inject SQL + (MDEV-40653). + */ + std::string qualified= + quote_duckdb_identifier(std::string(db_name)) + "." + + quote_duckdb_identifier(autoinc_sequence_name(table_name)); + std::string escaped; + escaped.reserve(qualified.size()); + for (char c : qualified) + { + if (c == '\'') + escaped.push_back('\''); + escaped.push_back(c); + } + std::string query= + "SELECT min(x), max(x) FROM (SELECT nextval('" + escaped + + "') AS x FROM range(" + std::to_string(want) + ")) t"; + + auto *ctx= get_duckdb_context(thd); + for (int attempt= 0; attempt < 3; attempt++) + { + auto result= myduck::duckdb_query(ctx->get_connection(), query); + if (result == nullptr || result->HasError()) + return 0; + auto chunk= result->Fetch(); + if (chunk == nullptr || chunk->size() == 0) + return 0; + int64_t lo= chunk->GetValue(0, 0).GetValue(); + int64_t hi= chunk->GetValue(1, 0).GetValue(); + if ((ulonglong) (hi - lo) + 1 == want) + return (ulonglong) lo; + } + return 0; +} + +void ha_duckdb::get_auto_increment(ulonglong offset, ulonglong increment, + ulonglong nb_desired_values, + ulonglong *first_value, + ulonglong *nb_reserved_values) +{ + DBUG_ENTER("ha_duckdb::get_auto_increment"); + + /* + The server rounds the first value up to the offset/increment grid + (handler.cc:4389: at most first + increment - 1) and then steps + `increment` at a time, so it uses ids <= first + nb_desired * increment + - 1. Consume exactly that many from the cache to keep the server inside + our block while leaving no unused ids behind. + */ + ulonglong need= nb_desired_values * increment; + if (need / increment != nb_desired_values) /* multiplication overflow */ + { + *first_value= ULONGLONG_MAX; + *nb_reserved_values= 0; + DBUG_VOID_RETURN; + } + + Duckdb_share::Autoinc_range cur= share->autoinc_range.load(); + for (;;) + { + /* Fast path: claim [cur.next, cur.next + need) with one CAS. */ + while (cur.end - cur.next >= need) + { + Duckdb_share::Autoinc_range claimed= {cur.next + need, cur.end}; + if (share->autoinc_range.compare_exchange_weak(cur, claimed)) + { + *first_value= cur.next; + *nb_reserved_values= nb_desired_values; + DBUG_VOID_RETURN; + } + /* CAS failure reloaded cur. */ + } + + mysql_mutex_lock(&share->autoinc_refill_mutex); + cur= share->autoinc_range.load(); + if (cur.end - cur.next >= need) + { + /* Someone refilled while we waited; back to the fast path. */ + mysql_mutex_unlock(&share->autoinc_refill_mutex); + continue; + } + + ulonglong want= std::max(need, autoinc_cache_size); + DatabaseTableNames dt(table->s->normalized_path.str); + ulonglong first= reserve_autoinc_block(ha_thd(), dt.db_name.c_str(), + dt.table_name.c_str(), want); + if (first == 0) + { + mysql_mutex_unlock(&share->autoinc_refill_mutex); + /* The only failure signal the server understands. */ + *first_value= ULONGLONG_MAX; + *nb_reserved_values= 0; + DBUG_VOID_RETURN; + } + /* + Publish the new block minus our own share. A plain store cannot clash + with concurrent fast-path claims: those still come from the old range, + which lies entirely below `first` because the sequence is monotonic. + The remainder of the old block is dropped, not merged: ids between the + two blocks may already belong to a DuckDB-side writer. The unused ids + become gaps, which AUTO_INCREMENT permits. + */ + share->autoinc_range.store({first + need, first + want}); + mysql_mutex_unlock(&share->autoinc_refill_mutex); + + *first_value= first; + *nb_reserved_values= nb_desired_values; + DBUG_VOID_RETURN; + } +} + int ha_duckdb::update_row(const uchar *old_row, const uchar *new_row) { DBUG_ENTER("ha_duckdb::update_row"); @@ -643,8 +789,8 @@ int ha_duckdb::rnd_init(bool) else DBUG_RETURN(HA_ERR_INTERNAL_ERROR); - std::string query= - "SELECT * FROM \"" + schema_name + "\".\"" + table_name + "\""; + std::string query= "SELECT * FROM " + quote_duckdb_identifier(schema_name) + + "." + quote_duckdb_identifier(table_name); auto *ctx= get_duckdb_context(thd); query_result= myduck::duckdb_query(ctx->get_connection(), query); @@ -806,8 +952,8 @@ int ha_duckdb::delete_all_rows() ctx->delete_appender(dt.db_name, dt.table_name); /* Execute DELETE FROM "schema"."table" */ - std::string query= - "DELETE FROM \"" + dt.db_name + "\".\"" + dt.table_name + "\""; + std::string query= "DELETE FROM " + quote_duckdb_identifier(dt.db_name) + + "." + quote_duckdb_identifier(dt.table_name); auto query_result= myduck::duckdb_query(ctx->get_connection(), query); if (query_result->HasError()) @@ -819,13 +965,9 @@ int ha_duckdb::delete_all_rows() DBUG_RETURN(0); } -const COND *ha_duckdb::cond_push(const COND *cond) +const COND *ha_duckdb::cond_push(const COND *) { DBUG_ENTER("ha_duckdb::cond_push"); - /* - Accept all conditions — DuckDB will evaluate the WHERE clause - from the original SQL query in direct_delete_rows(). - */ DBUG_RETURN(NULL); } @@ -840,6 +982,15 @@ int ha_duckdb::direct_delete_rows(ha_rows *delete_rows) DBUG_ENTER("ha_duckdb::direct_delete_rows"); int ret= 0; THD *thd= ha_thd(); + LEX_STRING *source_query= thd_query_string(thd); + if (myduck::mariadb_query_has_unsafe_quote_escape( + thd, source_query->str, source_query->length)) + { + my_error(ER_GET_ERRMSG, MYF(0), HA_DUCKDB_DML_ERROR, + "Unsafe MariaDB backslash quote escape in forwarded SQL", + "DuckDB"); + DBUG_RETURN(HA_DUCKDB_DML_ERROR); + } ret= duckdb_register_trx(thd); if (ret) @@ -889,6 +1040,15 @@ int ha_duckdb::direct_update_rows(ha_rows *update_rows, ha_rows *found_rows) DBUG_ENTER("ha_duckdb::direct_update_rows"); int ret= 0; THD *thd= ha_thd(); + LEX_STRING *source_query= thd_query_string(thd); + if (myduck::mariadb_query_has_unsafe_quote_escape( + thd, source_query->str, source_query->length)) + { + my_error(ER_GET_ERRMSG, MYF(0), HA_DUCKDB_DML_ERROR, + "Unsafe MariaDB backslash quote escape in forwarded SQL", + "DuckDB"); + DBUG_RETURN(HA_DUCKDB_DML_ERROR); + } ret= duckdb_register_trx(thd); if (ret) @@ -1003,8 +1163,9 @@ int ha_duckdb::delete_table(const char *name) DatabaseTableNames dt(name); - std::string query= - "DROP TABLE IF EXISTS \"" + dt.db_name + "\".\"" + dt.table_name + "\""; + std::string query= "DROP TABLE IF EXISTS " + + quote_duckdb_identifier(dt.db_name) + "." + + quote_duckdb_identifier(dt.table_name); auto *ctx= get_duckdb_context(thd); auto query_result= myduck::duckdb_query(ctx->get_connection(), query); @@ -1012,6 +1173,21 @@ int ha_duckdb::delete_table(const char *name) if (query_result == nullptr || query_result->HasError()) DBUG_RETURN(HA_DUCKDB_DROP_TABLE_ERROR); + /* + Drop the AUTO_INCREMENT sequence, if any. It must go after the table: + while the column default exists the catalog dependency blocks the drop. + The name is derived from the table alone, so no lookup is needed and + the statement is harmless for tables without AUTO_INCREMENT. + */ + std::string seq_query= "DROP SEQUENCE IF EXISTS " + + quote_duckdb_identifier(dt.db_name) + "." + + quote_duckdb_identifier( + autoinc_sequence_name(dt.table_name)); + auto seq_result= myduck::duckdb_query(ctx->get_connection(), seq_query); + + if (seq_result == nullptr || seq_result->HasError()) + DBUG_RETURN(HA_DUCKDB_DROP_TABLE_ERROR); + DBUG_RETURN(0); } @@ -1063,8 +1239,8 @@ int ha_duckdb::truncate() table->s->table_name.length); std::ostringstream query; - query << "USE \"" << schema_name << "\";"; - query << "TRUNCATE TABLE \"" << table_name << "\";"; + query << "USE " << quote_duckdb_identifier(schema_name) << ";"; + query << "TRUNCATE TABLE " << quote_duckdb_identifier(table_name) << ";"; auto *ctx= get_duckdb_context(thd); auto query_result= myduck::duckdb_query(ctx->get_connection(), query.str()); @@ -1098,6 +1274,9 @@ ha_duckdb::check_if_supported_inplace_alter(TABLE *altered_table, if (ha_alter_info->alter_info->flags & ALTER_COLUMN_ORDER) DBUG_RETURN(HA_ALTER_INPLACE_NOT_SUPPORTED); + if (ha_alter_info->error_if_not_empty) + DBUG_RETURN(HA_ALTER_INPLACE_NOT_SUPPORTED); + /* Reject ALTER on tables without PK when require_primary_key is ON */ if (myduck::require_primary_key && table->s->primary_key == MAX_KEY) { @@ -1177,29 +1356,53 @@ bool ha_duckdb::commit_inplace_alter_table(TABLE *altered_table, if (convertors.empty()) DBUG_RETURN(false); - /* Execute each ALTER operation in its own auto-commit context. - DuckDB v1.5+ does not allow compound DDL that mixes structural - changes (ADD COLUMN) with constraint updates (SET DEFAULT) - within the same transaction. */ - auto con= myduck::DuckdbManager::CreateConnection(); - + std::vector statements; for (auto &conv : convertors) { if (!conv || conv->check()) DBUG_RETURN(true); std::string sql= conv->translate(); - if (sql.empty()) - continue; + if (!sql.empty()) + statements.push_back(std::move(sql)); + } + + if (statements.empty()) + DBUG_RETURN(false); + + /* A single MariaDB ALTER TABLE can produce multiple DuckDB statements. + Execute the generated operations atomically on a dedicated connection. */ + auto con= myduck::DuckdbManager::CreateConnection(); + auto query_result= myduck::duckdb_query(*con, "BEGIN"); + if (query_result->HasError()) + { + my_error(ER_GET_ERRMSG, MYF(0), HA_ERR_GENERIC, + query_result->GetError().c_str(), "DuckDB"); + DBUG_RETURN(true); + } - auto query_result= myduck::duckdb_query(*con, sql); + for (const auto &sql : statements) + { + query_result= myduck::duckdb_query(*con, sql); if (query_result->HasError()) { - my_error(ER_GET_ERRMSG, MYF(0), HA_ERR_GENERIC, query_result->GetError().c_str(), "DuckDB"); + std::string error= query_result->GetError(); + myduck::duckdb_query(*con, "ROLLBACK"); + my_error(ER_GET_ERRMSG, MYF(0), HA_ERR_GENERIC, error.c_str(), "DuckDB"); DBUG_RETURN(true); } } + query_result= myduck::duckdb_query(*con, "COMMIT"); + if (query_result->HasError()) + { + std::string error= query_result->GetError(); + if (con->HasActiveTransaction()) + myduck::duckdb_query(*con, "ROLLBACK"); + my_error(ER_GET_ERRMSG, MYF(0), HA_ERR_GENERIC, error.c_str(), "DuckDB"); + DBUG_RETURN(true); + } + DBUG_RETURN(false); } @@ -1229,6 +1432,12 @@ static MYSQL_SYSVAR_BOOL(update_modified_column_only, "Whether to only update modified columns", NULL, NULL, TRUE); +static MYSQL_SYSVAR_ULONGLONG(autoinc_cache_size, autoinc_cache_size, + PLUGIN_VAR_RQCMDARG, + "Number of AUTO_INCREMENT ids reserved from the " + "backing DuckDB sequence per query", + NULL, NULL, 1000, 1, 1048576, 0); + /* ---- Global proxy variables (pushed into DuckDB) ---- */ static MYSQL_SYSVAR_ULONGLONG(memory_limit, myduck::global_memory_limit, @@ -1355,6 +1564,7 @@ static struct st_mysql_sys_var *duckdb_system_variables[]= { MYSQL_SYSVAR(allow_run_in_duckdb), MYSQL_SYSVAR(copy_ddl_in_batch), MYSQL_SYSVAR(dml_in_batch), MYSQL_SYSVAR(update_modified_column_only), + MYSQL_SYSVAR(autoinc_cache_size), /* Global proxy */ MYSQL_SYSVAR(memory_limit), MYSQL_SYSVAR(temp_directory), MYSQL_SYSVAR(max_temp_directory_size), MYSQL_SYSVAR(max_threads), diff --git a/storage/duckdb/ha_duckdb.h b/storage/duckdb/ha_duckdb.h index b0ff93d247185..4f32bbebe1064 100644 --- a/storage/duckdb/ha_duckdb.h +++ b/storage/duckdb/ha_duckdb.h @@ -21,6 +21,7 @@ #define HA_DUCKDB_H #include +#include #include #include "my_global.h" @@ -47,8 +48,30 @@ class Duckdb_share : public Handler_share { public: THR_LOCK lock; + /* + AUTO_INCREMENT block cache. Blocks are drawn from the table's DuckDB + sequence in get_auto_increment(); the cache spreads the cost of that + query over duckdb_autoinc_cache_size ids. + + The fast path claims ids with one CAS on the {next, end} pair. Both + fields must move together: with independent atomics a reader could pair + a stale bound with a fresh one and claim ids of a dropped block gap, + which may belong to a DuckDB-side writer. The mutex only serializes + block refills. + */ + struct Autoinc_range + { + ulonglong next; ///< next unused id of the cached block + ulonglong end; ///< one past the last id of the cached block + }; + alignas(16) std::atomic autoinc_range{{0, 0}}; + mysql_mutex_t autoinc_refill_mutex; Duckdb_share(); - ~Duckdb_share() { thr_lock_delete(&lock); } + ~Duckdb_share() + { + thr_lock_delete(&lock); + mysql_mutex_destroy(&autoinc_refill_mutex); + } }; /** @brief @@ -69,7 +92,7 @@ class ha_duckdb : public handler ulonglong table_flags() const override { return (HA_BINLOG_STMT_CAPABLE | HA_BINLOG_ROW_CAPABLE | - HA_NO_AUTO_INCREMENT | HA_NULL_IN_KEY | HA_CAN_INDEX_BLOBS | + HA_NULL_IN_KEY | HA_CAN_INDEX_BLOBS | HA_CAN_DIRECT_UPDATE_AND_DELETE); } @@ -111,6 +134,9 @@ class ha_duckdb : public handler int close(void) override; int write_row(const uchar *buf) override; + void get_auto_increment(ulonglong offset, ulonglong increment, + ulonglong nb_desired_values, ulonglong *first_value, + ulonglong *nb_reserved_values) override; int update_row(const uchar *old_data, const uchar *new_data) override; int delete_row(const uchar *buf) override; diff --git a/storage/duckdb/ha_duckdb_pushdown.cc b/storage/duckdb/ha_duckdb_pushdown.cc index c865162243ca3..74e00f823ad1b 100644 --- a/storage/duckdb/ha_duckdb_pushdown.cc +++ b/storage/duckdb/ha_duckdb_pushdown.cc @@ -106,6 +106,8 @@ static bool is_query_token(const std::string &sql, size_t start, size_t length) static bool extract_source_query(THD *thd, std::string &source) { const std::string sql(thd->query(), thd->query_length()); + if (myduck::mariadb_query_has_unsafe_quote_escape(thd, sql.data(), sql.size())) + return false; const bool strip_prefix= thd->lex->sql_command == SQLCOM_INSERT_SELECT; const bool backslash_escapes= thd->backslash_escapes(); size_t i= 0; @@ -119,48 +121,21 @@ static bool extract_source_query(THD *thd, std::string &source) i++; continue; } - if (c == '/' && i + 1 < sql.size() && sql[i + 1] == '*') - { - size_t end= sql.find("*/", i + 2); - if (end == std::string::npos) - return false; - i= end + 2; - continue; - } - if (c == '#' || - (c == '-' && i + 1 < sql.size() && sql[i + 1] == '-' && - (i + 2 == sql.size() || isspace((unsigned char) sql[i + 2])))) + size_t end; + myduck::SqlRegionType region= + myduck::scan_sql_region(sql, i, backslash_escapes, end); + if (region == myduck::SqlRegionType::UNTERMINATED) + return false; + if (region == myduck::SqlRegionType::COMMENT) { - size_t end= sql.find('\n', i + (c == '#' ? 1 : 2)); - i= end == std::string::npos ? sql.size() : end + 1; + i= end; continue; } - if (c == '\'' || c == '"' || c == '`') + if (region == myduck::SqlRegionType::QUOTED) { if (!strip_prefix && depth == 0) return false; - const char quote= (char) c; - bool closed= false; - for (i++; i < sql.size(); i++) - { - if (sql[i] == '\\' && backslash_escapes && i + 1 < sql.size()) - { - i++; - continue; - } - if (sql[i] != quote) - continue; - if (i + 1 < sql.size() && sql[i + 1] == quote) - { - i++; - continue; - } - i++; - closed= true; - break; - } - if (!closed) - return false; + i= end; continue; } if (c == '(') diff --git a/storage/duckdb/mysql-test/duckdb/include/alter_duckdb_column.inc b/storage/duckdb/mysql-test/duckdb/include/alter_duckdb_column.inc index 0883ef3745453..4e28aa30087cf 100644 --- a/storage/duckdb/mysql-test/duckdb/include/alter_duckdb_column.inc +++ b/storage/duckdb/mysql-test/duckdb/include/alter_duckdb_column.inc @@ -292,11 +292,14 @@ eval ALTER TABLE t ADD COLUMN d INT INVISIBLE, ALGORITHM = $algorithm; --error ER_PARSE_ERROR eval ALTER TABLE t ALTER COLUMN b SET INVISIBLE, ALGORITHM = $algorithm; -# AUTO_INCREMENT +# AUTO_INCREMENT is supported by CREATE TABLE only. +# 'c INT AUTO_INCREMENT KEY' would be rejected by the parser instead (bare KEY +# means PRIMARY KEY and t already has one), and AUTO_INCREMENT without any key +# is rejected by the server, so neither would reach the engine. --error ER_TABLE_CANT_HANDLE_AUTO_INCREMENT -eval ALTER TABLE t MODIFY COLUMN c INT AUTO_INCREMENT KEY, ALGORITHM = $algorithm; +eval ALTER TABLE t MODIFY COLUMN id INT AUTO_INCREMENT, ALGORITHM = $algorithm; --error ER_TABLE_CANT_HANDLE_AUTO_INCREMENT -eval ALTER TABLE t ADD COLUMN d INT AUTO_INCREMENT, ALGORITHM = $algorithm; +eval ALTER TABLE t ADD COLUMN d INT AUTO_INCREMENT UNIQUE, ALGORITHM = $algorithm; # ENGINE_ATTRIBUTE --error ER_UNKNOWN_OPTION diff --git a/storage/duckdb/mysql-test/duckdb/r/alter_duckdb_column.result b/storage/duckdb/mysql-test/duckdb/r/alter_duckdb_column.result index 2c8cf374fd821..7be07dd029436 100644 --- a/storage/duckdb/mysql-test/duckdb/r/alter_duckdb_column.result +++ b/storage/duckdb/mysql-test/duckdb/r/alter_duckdb_column.result @@ -568,12 +568,12 @@ table_schema table_name column_name column_default is_nullable data_type COLUMN_ VARCHAR VARCHAR VARCHAR VARCHAR VARCHAR VARCHAR VARCHAR [ Rows: 7] db_alter_col t id NULL NO INTEGER NULL -db_alter_col t a '6' YES INTEGER NULL +db_alter_col t a (3 + 3) YES INTEGER NULL db_alter_col t b '3+3' YES VARCHAR NULL -db_alter_col t c '6' YES VARCHAR NULL -db_alter_col t d '4' YES INTEGER NULL +db_alter_col t c (3 + 3) YES VARCHAR NULL +db_alter_col t d (2 + 2) YES INTEGER NULL db_alter_col t e '2+2' YES VARCHAR NULL -db_alter_col t f '4' YES VARCHAR NULL +db_alter_col t f (2 + 2) YES VARCHAR NULL SELECT TABLE_SCHEMA, TABLE_NAME, COLUMN_NAME, COLUMN_DEFAULT, IS_NULLABLE, DATA_TYPE, COLUMN_COMMENT FROM information_schema.columns WHERE TABLE_NAME = 't'; @@ -625,7 +625,7 @@ VARCHAR VARCHAR VARCHAR VARCHAR VARCHAR VARCHAR VARCHAR db_alter_col t id NULL NO INTEGER NULL db_alter_col t B0 CAST('\x00' AS "BLOB") NO BLOB NULL db_alter_col t B1 CAST('\x00\x00\x0D\x05' AS "BLOB") NO BLOB NULL -db_alter_col t B2 CAST('\x00\x00\x00\x00\x00\x00\x00\x1F' AS "BLOB") NO BLOB NULL +db_alter_col t B2 '\x00\x00\x00\x00\x00\x00\x00\x1F' NO BLOB NULL SELECT TABLE_SCHEMA, TABLE_NAME, COLUMN_NAME, COLUMN_DEFAULT, IS_NULLABLE, DATA_TYPE, COLUMN_COMMENT FROM information_schema.columns WHERE TABLE_NAME = 't'; @@ -718,9 +718,9 @@ ALTER TABLE t ADD COLUMN d INT INVISIBLE, ALGORITHM = INSTANT; ERROR HY000: Got error 168 'INVISIBLE column is not supported. Try removing INVISIBLE from column 'd'' from DuckDB ALTER TABLE t ALTER COLUMN b SET INVISIBLE, ALGORITHM = INSTANT; ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'INVISIBLE, ALGORITHM = INSTANT' at line 1 -ALTER TABLE t MODIFY COLUMN c INT AUTO_INCREMENT KEY, ALGORITHM = INSTANT; +ALTER TABLE t MODIFY COLUMN id INT AUTO_INCREMENT, ALGORITHM = INSTANT; ERROR 42000: Storage engine DUCKDB doesn't support AUTO_INCREMENT columns -ALTER TABLE t ADD COLUMN d INT AUTO_INCREMENT, ALGORITHM = INSTANT; +ALTER TABLE t ADD COLUMN d INT AUTO_INCREMENT UNIQUE, ALGORITHM = INSTANT; ERROR 42000: Storage engine DUCKDB doesn't support AUTO_INCREMENT columns CREATE TABLE t1(id INT PRIMARY KEY, a INT) ENGINE = DuckDB ENGINE_ATTRIBUTE='{"KEY":"VALUE"}'; ERROR HY000: Unknown option 'ENGINE_ATTRIBUTE' @@ -1446,12 +1446,12 @@ table_schema table_name column_name column_default is_nullable data_type COLUMN_ VARCHAR VARCHAR VARCHAR VARCHAR VARCHAR VARCHAR VARCHAR [ Rows: 7] db_alter_col t id NULL NO INTEGER NULL -db_alter_col t a '6' YES INTEGER NULL +db_alter_col t a (3 + 3) YES INTEGER NULL db_alter_col t b '3+3' YES VARCHAR NULL -db_alter_col t c '6' YES VARCHAR NULL -db_alter_col t d '4' YES INTEGER NULL +db_alter_col t c (3 + 3) YES VARCHAR NULL +db_alter_col t d (2 + 2) YES INTEGER NULL db_alter_col t e '2+2' YES VARCHAR NULL -db_alter_col t f '4' YES VARCHAR NULL +db_alter_col t f (2 + 2) YES VARCHAR NULL SELECT TABLE_SCHEMA, TABLE_NAME, COLUMN_NAME, COLUMN_DEFAULT, IS_NULLABLE, DATA_TYPE, COLUMN_COMMENT FROM information_schema.columns WHERE TABLE_NAME = 't'; @@ -1503,7 +1503,7 @@ VARCHAR VARCHAR VARCHAR VARCHAR VARCHAR VARCHAR VARCHAR db_alter_col t id NULL NO INTEGER NULL db_alter_col t B0 CAST('\x00' AS "BLOB") NO BLOB NULL db_alter_col t B1 CAST('\x00\x00\x0D\x05' AS "BLOB") NO BLOB NULL -db_alter_col t B2 CAST('\x00\x00\x00\x00\x00\x00\x00\x1F' AS "BLOB") NO BLOB NULL +db_alter_col t B2 '\x00\x00\x00\x00\x00\x00\x00\x1F' NO BLOB NULL SELECT TABLE_SCHEMA, TABLE_NAME, COLUMN_NAME, COLUMN_DEFAULT, IS_NULLABLE, DATA_TYPE, COLUMN_COMMENT FROM information_schema.columns WHERE TABLE_NAME = 't'; @@ -1596,9 +1596,9 @@ ALTER TABLE t ADD COLUMN d INT INVISIBLE, ALGORITHM = INPLACE; ERROR HY000: Got error 168 'INVISIBLE column is not supported. Try removing INVISIBLE from column 'd'' from DuckDB ALTER TABLE t ALTER COLUMN b SET INVISIBLE, ALGORITHM = INPLACE; ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'INVISIBLE, ALGORITHM = INPLACE' at line 1 -ALTER TABLE t MODIFY COLUMN c INT AUTO_INCREMENT KEY, ALGORITHM = INPLACE; +ALTER TABLE t MODIFY COLUMN id INT AUTO_INCREMENT, ALGORITHM = INPLACE; ERROR 42000: Storage engine DUCKDB doesn't support AUTO_INCREMENT columns -ALTER TABLE t ADD COLUMN d INT AUTO_INCREMENT, ALGORITHM = INPLACE; +ALTER TABLE t ADD COLUMN d INT AUTO_INCREMENT UNIQUE, ALGORITHM = INPLACE; ERROR 42000: Storage engine DUCKDB doesn't support AUTO_INCREMENT columns CREATE TABLE t1(id INT PRIMARY KEY, a INT) ENGINE = DuckDB ENGINE_ATTRIBUTE='{"KEY":"VALUE"}'; ERROR HY000: Unknown option 'ENGINE_ATTRIBUTE' diff --git a/storage/duckdb/mysql-test/duckdb/r/alter_duckdb_column_copy.result b/storage/duckdb/mysql-test/duckdb/r/alter_duckdb_column_copy.result index a8158a2afb010..3dc59f19a2d02 100644 --- a/storage/duckdb/mysql-test/duckdb/r/alter_duckdb_column_copy.result +++ b/storage/duckdb/mysql-test/duckdb/r/alter_duckdb_column_copy.result @@ -718,9 +718,9 @@ ALTER TABLE t ADD COLUMN d INT INVISIBLE, ALGORITHM = COPY; ERROR HY000: Table storage engine 'DuckDB' does not support the create option 'INVISIBLE column' ALTER TABLE t ALTER COLUMN b SET INVISIBLE, ALGORITHM = COPY; ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'INVISIBLE, ALGORITHM = COPY' at line 1 -ALTER TABLE t MODIFY COLUMN c INT AUTO_INCREMENT KEY, ALGORITHM = COPY; +ALTER TABLE t MODIFY COLUMN id INT AUTO_INCREMENT, ALGORITHM = COPY; ERROR 42000: Storage engine DUCKDB doesn't support AUTO_INCREMENT columns -ALTER TABLE t ADD COLUMN d INT AUTO_INCREMENT, ALGORITHM = COPY; +ALTER TABLE t ADD COLUMN d INT AUTO_INCREMENT UNIQUE, ALGORITHM = COPY; ERROR 42000: Storage engine DUCKDB doesn't support AUTO_INCREMENT columns CREATE TABLE t1(id INT PRIMARY KEY, a INT) ENGINE = DuckDB ENGINE_ATTRIBUTE='{"KEY":"VALUE"}'; ERROR HY000: Unknown option 'ENGINE_ATTRIBUTE' diff --git a/storage/duckdb/mysql-test/duckdb/r/cross_engine_ryow_dup.result b/storage/duckdb/mysql-test/duckdb/r/cross_engine_ryow_dup.result new file mode 100644 index 0000000000000..6a8e42c778ae2 --- /dev/null +++ b/storage/duckdb/mysql-test/duckdb/r/cross_engine_ryow_dup.result @@ -0,0 +1,83 @@ +# +# Cross-engine RYOW: the same external table referenced more than once. +# +# The external table registry is keyed by table name, so two references +# to one table would collapse onto a single TABLE/handler and produce two +# concurrent scans over one cursor. Such queries must not be pushed +# down; the server executes them itself and the results must still show +# the uncommitted writes of the current transaction. +# +DROP DATABASE IF EXISTS cross_engine_ryow_dup; +CREATE DATABASE cross_engine_ryow_dup CHARACTER SET utf8mb4; +USE cross_engine_ryow_dup; +SET @old_max_threads= @@global.duckdb_max_threads; +SET GLOBAL duckdb_max_threads= 4; +CREATE TABLE t_duck (id INT PRIMARY KEY, val VARCHAR(50)) ENGINE=DuckDB; +CREATE TABLE t_inno (id INT PRIMARY KEY, score INT) ENGINE=InnoDB; +INSERT INTO t_duck VALUES (1,'alpha'),(2,'beta'),(3,'gamma'); +INSERT INTO t_inno VALUES (1,10),(2,20),(3,30); +SET SESSION duckdb_cross_engine_ryow=1; + +# (1) Self-join of the external table under two aliases + +BEGIN; +UPDATE t_inno SET score=999 WHERE id=2; +SELECT d.id, a.score AS a_score, b.score AS b_score +FROM t_duck d +JOIN t_inno a ON d.id = a.id +JOIN t_inno b ON a.id = b.id; +id a_score b_score +1 10 10 +2 999 999 +3 30 30 +ROLLBACK; + +# (2) The external table used both in the join and in a subquery + +BEGIN; +UPDATE t_inno SET score=999 WHERE id=3; +SELECT d.id, i.score +FROM t_duck d JOIN t_inno i ON d.id = i.id +WHERE i.score = (SELECT MAX(score) FROM t_inno); +id score +3 999 +ROLLBACK; + +# (3) The external table twice in a UNION + +BEGIN; +UPDATE t_inno SET score=999 WHERE id=1; +SELECT d.id, i.score FROM t_duck d JOIN t_inno i ON d.id = i.id WHERE i.id = 1 +UNION ALL +SELECT d.id, i.score FROM t_duck d JOIN t_inno i ON d.id = i.id WHERE i.id = 2; +id score +1 999 +2 20 +ROLLBACK; + +# (4) A single reference is still pushed down and still sees own writes + +BEGIN; +UPDATE t_inno SET score=555 WHERE id=2; +SELECT d.id, i.score FROM t_duck d JOIN t_inno i ON d.id = i.id; +id score +1 10 +2 555 +3 30 +ROLLBACK; + +# (5) Committed state is intact + +SELECT d.id, i.score FROM t_duck d JOIN t_inno i ON d.id = i.id; +id score +1 10 +2 20 +3 30 + +# Cleanup + +SET SESSION duckdb_cross_engine_ryow=DEFAULT; +SET GLOBAL duckdb_max_threads= @old_max_threads; +DROP TABLE t_duck; +DROP TABLE t_inno; +DROP DATABASE cross_engine_ryow_dup; diff --git a/storage/duckdb/mysql-test/duckdb/r/cross_engine_ryow_limit.result b/storage/duckdb/mysql-test/duckdb/r/cross_engine_ryow_limit.result new file mode 100644 index 0000000000000..a869558da8aa3 --- /dev/null +++ b/storage/duckdb/mysql-test/duckdb/r/cross_engine_ryow_limit.result @@ -0,0 +1,74 @@ +# +# Cross-engine RYOW: early query termination. +# +# With LIMIT (or any other early stop) DuckDB stops pulling chunks from +# the external table scan, so the scan is never driven to end-of-file. +# The handler scan started behind the server's back must still be +# finished, otherwise the table cannot be closed cleanly. +# +# FLUSH TABLES below forces the table to be closed inside the test +# instead of at shutdown, so that a leaked scan is detected here. +# +# NOTE: the leak itself is caught by DBUG_ASSERT(inited == NONE) in +# handler::~handler(), which is compiled out when DBUG_OFF is set. This +# test therefore only proves the absence of a leak in a debug build; in a +# release build it merely checks the query results. +# +DROP DATABASE IF EXISTS cross_engine_ryow_limit; +CREATE DATABASE cross_engine_ryow_limit CHARACTER SET utf8mb4; +USE cross_engine_ryow_limit; +SET @old_max_threads= @@global.duckdb_max_threads; +SET GLOBAL duckdb_max_threads= 4; +CREATE TABLE t_duck (id INT PRIMARY KEY, val VARCHAR(50)) ENGINE=DuckDB; +CREATE TABLE t_inno (id INT PRIMARY KEY, score INT) ENGINE=InnoDB; +INSERT INTO t_duck SELECT seq, CONCAT('v', seq) FROM seq_1_to_3000; +INSERT INTO t_inno SELECT seq, seq FROM seq_1_to_3000; +SET SESSION duckdb_cross_engine_ryow=1; + +# (1) LIMIT stops the scan long before end-of-file + +BEGIN; +UPDATE t_inno SET score=999 WHERE id=1; +SELECT d.id, i.score +FROM t_duck d JOIN t_inno i ON d.id = i.id +ORDER BY d.id LIMIT 1; +id score +1 999 + +# (2) The same external table is usable again in the same transaction + +SELECT d.id, i.score +FROM t_duck d JOIN t_inno i ON d.id = i.id +ORDER BY d.id LIMIT 2; +id score +1 999 +2 2 +SELECT score FROM t_inno WHERE id = 1; +score +999 +ROLLBACK; + +# (3) Closing the table must not hit a leaked scan + +FLUSH TABLES; +SELECT COUNT(*) FROM t_inno; +COUNT(*) +3000 + +# (4) Early termination combined with an aggregate that stops early + +BEGIN; +UPDATE t_inno SET score=999 WHERE id=2; +SELECT EXISTS (SELECT 1 FROM t_duck d JOIN t_inno i ON d.id = i.id WHERE i.score = 999) AS found; +found +1 +ROLLBACK; +FLUSH TABLES; + +# Cleanup + +SET SESSION duckdb_cross_engine_ryow=DEFAULT; +SET GLOBAL duckdb_max_threads= @old_max_threads; +DROP TABLE t_duck; +DROP TABLE t_inno; +DROP DATABASE cross_engine_ryow_limit; diff --git a/storage/duckdb/mysql-test/duckdb/r/duckdb_auto_increment.result b/storage/duckdb/mysql-test/duckdb/r/duckdb_auto_increment.result new file mode 100644 index 0000000000000..20faf8a00c119 --- /dev/null +++ b/storage/duckdb/mysql-test/duckdb/r/duckdb_auto_increment.result @@ -0,0 +1,204 @@ +SET NAMES utf8mb4; +# +# MDEV-40266: AUTO_INCREMENT backed by a DuckDB sequence +# +# --- CREATE TABLE creates the sequence ------------------------------- +CREATE TABLE t (id BIGINT NOT NULL AUTO_INCREMENT PRIMARY KEY, a INT) ENGINE=DuckDB; +SHOW CREATE TABLE t; +Table Create Table +t CREATE TABLE `t` ( + `id` bigint(20) NOT NULL AUTO_INCREMENT, + `a` int(11) DEFAULT NULL, + PRIMARY KEY (`id`) +) ENGINE=DUCKDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_uca1400_ai_ci +# The sequence must exist in DuckDB, named after the table. +SELECT run_in_duckdb("SELECT schema_name, sequence_name, start_value FROM duckdb_sequences() WHERE schema_name = 'db_autoinc' ORDER BY sequence_name"); +run_in_duckdb("SELECT schema_name, sequence_name, start_value FROM duckdb_sequences() WHERE schema_name = 'db_autoinc' ORDER BY sequence_name") +schema_name sequence_name start_value +VARCHAR VARCHAR BIGINT +[ Rows: 1] +db_autoinc mdb_autoinc_t 1 + + +# The column default must reference that sequence, so that writes which +# bypass MariaDB still draw a non-colliding id from the same counter. +SELECT run_in_duckdb("SELECT column_name, column_default FROM information_schema.columns WHERE table_schema = 'db_autoinc' AND table_name = 't' ORDER BY column_name"); +run_in_duckdb("SELECT column_name, column_default FROM information_schema.columns WHERE table_schema = 'db_autoinc' AND table_name = 't' ORDER BY column_name") +column_name column_default +VARCHAR VARCHAR +[ Rows: 2] +a NULL +id nextval('"db_autoinc"."mdb_autoinc_t"') + + +# --- an insert bypassing MariaDB uses the sequence -------------------- +SELECT run_in_duckdb("INSERT INTO db_autoinc.t(a) VALUES(10)"); +run_in_duckdb("INSERT INTO db_autoinc.t(a) VALUES(10)") +Count +BIGINT +[ Rows: 1] +1 + + +SELECT run_in_duckdb("INSERT INTO db_autoinc.t(a) VALUES(20)"); +run_in_duckdb("INSERT INTO db_autoinc.t(a) VALUES(20)") +Count +BIGINT +[ Rows: 1] +1 + + +SELECT id, a FROM t ORDER BY id; +id a +1 10 +2 20 +# --- MariaDB-side inserts draw from the same sequence ------------------ +INSERT INTO t(a) VALUES(30); +SELECT LAST_INSERT_ID(); +LAST_INSERT_ID() +3 +INSERT INTO t(a) VALUES(40); +SELECT LAST_INSERT_ID(); +LAST_INSERT_ID() +4 +# A multi-row insert reserves its ids in one go. +INSERT INTO t(a) VALUES(50),(60),(70); +SELECT LAST_INSERT_ID(); +LAST_INSERT_ID() +5 +SELECT id, a FROM t ORDER BY id; +id a +1 10 +2 20 +3 30 +4 40 +5 50 +6 60 +7 70 +# A DuckDB-side insert after MariaDB ones must not collide: the block +# reservation advanced the shared counter past the cached ids. +SELECT run_in_duckdb("INSERT INTO db_autoinc.t(a) VALUES(80)"); +run_in_duckdb("INSERT INTO db_autoinc.t(a) VALUES(80)") +Count +BIGINT +[ Rows: 1] +1 + + +SELECT count(*), count(DISTINCT id) FROM t; +count(*) count(DISTINCT id) +8 8 +# --- START WITH honours CREATE TABLE ... AUTO_INCREMENT=N ------------- +CREATE TABLE t2 (id BIGINT NOT NULL AUTO_INCREMENT PRIMARY KEY, a INT) ENGINE=DuckDB AUTO_INCREMENT=100; +SELECT run_in_duckdb("SELECT sequence_name, start_value FROM duckdb_sequences() WHERE schema_name = 'db_autoinc' AND sequence_name = 'mdb_autoinc_t2'"); +run_in_duckdb("SELECT sequence_name, start_value FROM duckdb_sequences() WHERE schema_name = 'db_autoinc' AND sequence_name = 'mdb_autoinc_t2'") +sequence_name start_value +VARCHAR BIGINT +[ Rows: 1] +mdb_autoinc_t2 100 + + +SELECT run_in_duckdb("INSERT INTO db_autoinc.t2(a) VALUES(1)"); +run_in_duckdb("INSERT INTO db_autoinc.t2(a) VALUES(1)") +Count +BIGINT +[ Rows: 1] +1 + + +INSERT INTO t2(a) VALUES(2); +SELECT id, a FROM t2 ORDER BY id; +id a +100 1 +101 2 +# --- a table without AUTO_INCREMENT gets no sequence ------------------ +CREATE TABLE t3 (id INT PRIMARY KEY, a INT) ENGINE=DuckDB; +SELECT run_in_duckdb("SELECT count(*) FROM duckdb_sequences() WHERE schema_name = 'db_autoinc' AND sequence_name = 'mdb_autoinc_t3'"); +run_in_duckdb("SELECT count(*) FROM duckdb_sequences() WHERE schema_name = 'db_autoinc' AND sequence_name = 'mdb_autoinc_t3'") +count_star() +BIGINT +[ Rows: 1] +0 + + +# AUTO_INCREMENT is accepted by CREATE TABLE only; seeding an existing +# column from its current maximum is not implemented. +ALTER TABLE t3 MODIFY COLUMN id INT AUTO_INCREMENT; +ERROR 42000: Storage engine DUCKDB doesn't support AUTO_INCREMENT columns +ALTER TABLE t3 ADD COLUMN b INT AUTO_INCREMENT UNIQUE; +ERROR 42000: Storage engine DUCKDB doesn't support AUTO_INCREMENT columns +# The rejected ALTER must leave the table untouched. +SHOW CREATE TABLE t3; +Table Create Table +t3 CREATE TABLE `t3` ( + `id` int(11) NOT NULL, + `a` int(11) DEFAULT NULL, + PRIMARY KEY (`id`) +) ENGINE=DUCKDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_uca1400_ai_ci +# --- setval() is available (local backport of DuckDB PR #24061) -------- +SELECT run_in_duckdb("CREATE SEQUENCE db_autoinc.check_setval"); +run_in_duckdb("CREATE SEQUENCE db_autoinc.check_setval") +Count +BIGINT +[ Rows: 0] + + +SELECT run_in_duckdb("SELECT setval('db_autoinc.check_setval', 42)"); +run_in_duckdb("SELECT setval('db_autoinc.check_setval', 42)") +setval('db_autoinc.check_setval', 42) +BIGINT +[ Rows: 1] +42 + + +SELECT run_in_duckdb("SELECT nextval('db_autoinc.check_setval')"); +run_in_duckdb("SELECT nextval('db_autoinc.check_setval')") +nextval('db_autoinc.check_setval') +BIGINT +[ Rows: 1] +43 + + +SELECT run_in_duckdb("SELECT setval('db_autoinc.check_setval', 100, false)"); +run_in_duckdb("SELECT setval('db_autoinc.check_setval', 100, false)") +setval('db_autoinc.check_setval', 100, CAST('f' AS BOOLEAN)) +BIGINT +[ Rows: 1] +100 + + +SELECT run_in_duckdb("SELECT nextval('db_autoinc.check_setval')"); +run_in_duckdb("SELECT nextval('db_autoinc.check_setval')") +nextval('db_autoinc.check_setval') +BIGINT +[ Rows: 1] +100 + + +SELECT run_in_duckdb("DROP SEQUENCE db_autoinc.check_setval"); +run_in_duckdb("DROP SEQUENCE db_autoinc.check_setval") +Success +BOOLEAN +[ Rows: 0] + + +# --- DROP TABLE removes the sequence ---------------------------------- +DROP TABLE t; +SELECT run_in_duckdb("SELECT count(*) FROM duckdb_sequences() WHERE schema_name = 'db_autoinc' AND sequence_name = 'mdb_autoinc_t'"); +run_in_duckdb("SELECT count(*) FROM duckdb_sequences() WHERE schema_name = 'db_autoinc' AND sequence_name = 'mdb_autoinc_t'") +count_star() +BIGINT +[ Rows: 1] +0 + + +DROP TABLE t2; +DROP TABLE t3; +SELECT run_in_duckdb("SELECT count(*) FROM duckdb_sequences() WHERE schema_name = 'db_autoinc'"); +run_in_duckdb("SELECT count(*) FROM duckdb_sequences() WHERE schema_name = 'db_autoinc'") +count_star() +BIGINT +[ Rows: 1] +0 + + diff --git a/storage/duckdb/mysql-test/duckdb/r/duckdb_default_expr.result b/storage/duckdb/mysql-test/duckdb/r/duckdb_default_expr.result new file mode 100644 index 0000000000000..694f69aa03166 --- /dev/null +++ b/storage/duckdb/mysql-test/duckdb/r/duckdb_default_expr.result @@ -0,0 +1,153 @@ +SET NAMES utf8mb4; +# +# MDEV-40266: DEFAULT(NEXT VALUE FOR seq) must not break the DuckDB DDL +# +CREATE SEQUENCE s START WITH 1 INCREMENT BY 1; +# CREATE TABLE +CREATE TABLE t (id BIGINT NOT NULL DEFAULT(NEXT VALUE FOR s) PRIMARY KEY, a INT) ENGINE=DuckDB; +Warnings: +Note 1105 DuckDB: sequence default of column 'id' is not forwarded to DuckDB; MariaDB supplies the value +SHOW WARNINGS; +Level Code Message +Note 1105 DuckDB: sequence default of column 'id' is not forwarded to DuckDB; MariaDB supplies the value +SHOW CREATE TABLE t; +Table Create Table +t CREATE TABLE `t` ( + `id` bigint(20) NOT NULL DEFAULT nextval(`db_default_expr`.`s`), + `a` int(11) DEFAULT NULL, + PRIMARY KEY (`id`) +) ENGINE=DUCKDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_uca1400_ai_ci +# The DuckDB column must carry no default: MariaDB always supplies the value. +SELECT run_in_duckdb("SELECT column_name, column_default FROM information_schema.columns WHERE table_schema = 'db_default_expr' AND table_name = 't' ORDER BY column_name"); +run_in_duckdb("SELECT column_name, column_default FROM information_schema.columns WHERE table_schema = 'db_default_expr' AND table_name = 't' ORDER BY column_name") +column_name column_default +VARCHAR VARCHAR +[ Rows: 2] +a NULL +id NULL + + +INSERT INTO t (a) VALUES (10); +INSERT INTO t (a) VALUES (20); +SELECT id, a FROM t ORDER BY id; +id a +1 10 +2 20 +# ADD COLUMN +CREATE SEQUENCE s2 START WITH 100 INCREMENT BY 1; +ALTER TABLE t ADD COLUMN b BIGINT DEFAULT(NEXT VALUE FOR s2); +Warnings: +Note 1105 DuckDB: sequence default of column 'b' is not forwarded to DuckDB; MariaDB supplies the value +SHOW WARNINGS; +Level Code Message +Note 1105 DuckDB: sequence default of column 'b' is not forwarded to DuckDB; MariaDB supplies the value +SELECT run_in_duckdb("SELECT column_name, column_default FROM information_schema.columns WHERE table_schema = 'db_default_expr' AND table_name = 't' ORDER BY column_name"); +run_in_duckdb("SELECT column_name, column_default FROM information_schema.columns WHERE table_schema = 'db_default_expr' AND table_name = 't' ORDER BY column_name") +column_name column_default +VARCHAR VARCHAR +[ Rows: 3] +a NULL +b NULL +id NULL + + +INSERT INTO t (a) VALUES (30); +SELECT id, a, b FROM t ORDER BY id; +id a b +1 10 NULL +2 20 NULL +3 30 100 +# MODIFY COLUMN, adding a sequence default to a plain column +ALTER TABLE t MODIFY COLUMN a INT DEFAULT(NEXT VALUE FOR s2); +Warnings: +Note 1105 DuckDB: sequence default of column 'a' is not forwarded to DuckDB; MariaDB supplies the value +SHOW WARNINGS; +Level Code Message +Note 1105 DuckDB: sequence default of column 'a' is not forwarded to DuckDB; MariaDB supplies the value +SELECT run_in_duckdb("SELECT column_name, column_default FROM information_schema.columns WHERE table_schema = 'db_default_expr' AND table_name = 't' ORDER BY column_name"); +run_in_duckdb("SELECT column_name, column_default FROM information_schema.columns WHERE table_schema = 'db_default_expr' AND table_name = 't' ORDER BY column_name") +column_name column_default +VARCHAR VARCHAR +[ Rows: 3] +a NULL +b NULL +id NULL + + +INSERT INTO t () VALUES (); +SELECT id, a, b FROM t ORDER BY id; +id a b +1 10 NULL +2 20 NULL +3 30 100 +4 101 102 +# MODIFY COLUMN, replacing a sequence default with a literal +ALTER TABLE t MODIFY COLUMN a INT DEFAULT 7; +SHOW WARNINGS; +Level Code Message +SELECT run_in_duckdb("SELECT column_name, column_default FROM information_schema.columns WHERE table_schema = 'db_default_expr' AND table_name = 't' ORDER BY column_name"); +run_in_duckdb("SELECT column_name, column_default FROM information_schema.columns WHERE table_schema = 'db_default_expr' AND table_name = 't' ORDER BY column_name") +column_name column_default +VARCHAR VARCHAR +[ Rows: 3] +a '7' +b NULL +id NULL + + +# MODIFY COLUMN, replacing a literal default with an expression +ALTER TABLE t MODIFY COLUMN a INT DEFAULT (1+1); +SHOW WARNINGS; +Level Code Message +SELECT run_in_duckdb("SELECT column_name, column_default FROM information_schema.columns WHERE table_schema = 'db_default_expr' AND table_name = 't' AND column_name = 'a'"); +run_in_duckdb("SELECT column_name, column_default FROM information_schema.columns WHERE table_schema = 'db_default_expr' AND table_name = 't' AND column_name = 'a'") +column_name column_default +VARCHAR VARCHAR +[ Rows: 1] +a (1 + 1) + + +# Insert bypassing MariaDB: relies on the DuckDB-side expression default. +SELECT run_in_duckdb("INSERT INTO db_default_expr.t(id, b) VALUES(1000, 1000)"); +run_in_duckdb("INSERT INTO db_default_expr.t(id, b) VALUES(1000, 1000)") +Count +BIGINT +[ Rows: 1] +1 + + +SELECT a FROM t WHERE id = 1000; +a +2 +DROP TABLE t; +DROP SEQUENCE s; +DROP SEQUENCE s2; +# +# Non-sequence expression defaults keep reaching DuckDB +# +CREATE TABLE t2 (id INT PRIMARY KEY, a INT DEFAULT (1+1), b VARCHAR(10) DEFAULT ('1+1')) ENGINE=DuckDB; +SELECT run_in_duckdb("SELECT column_name, column_default FROM information_schema.columns WHERE table_schema = 'db_default_expr' AND table_name = 't2' ORDER BY column_name"); +run_in_duckdb("SELECT column_name, column_default FROM information_schema.columns WHERE table_schema = 'db_default_expr' AND table_name = 't2' ORDER BY column_name") +column_name column_default +VARCHAR VARCHAR +[ Rows: 3] +a (1 + 1) +b '1+1' +id NULL + + +INSERT INTO t2 (id) VALUES (1); +# Insert bypassing MariaDB: relies on the DuckDB-side default. +SELECT run_in_duckdb("INSERT INTO db_default_expr.t2(id) VALUES(2)"); +run_in_duckdb("INSERT INTO db_default_expr.t2(id) VALUES(2)") +Count +BIGINT +[ Rows: 1] +1 + + +SELECT * FROM t2 ORDER BY id; +id a b +1 2 1+1 +2 2 1+1 +DROP TABLE t2; diff --git a/storage/duckdb/mysql-test/duckdb/r/duckdb_default_value_injection.result b/storage/duckdb/mysql-test/duckdb/r/duckdb_default_value_injection.result new file mode 100644 index 0000000000000..cb96fbbf5ba76 --- /dev/null +++ b/storage/duckdb/mysql-test/duckdb/r/duckdb_default_value_injection.result @@ -0,0 +1,25 @@ +# +# MDEV-40610: an unescaped column DEFAULT string must not inject DuckDB SQL +# +# get_field_default_for_duckdb() wrapped a string DEFAULT in single quotes +# without doubling an embedded quote, so a crafted default closed the +# generated DuckDB literal and the rest was parsed as DuckDB SQL. The gate +# below fails while the injection works and passes once literals are escaped. +SET NAMES utf8mb4; +# The crafted DEFAULT closes the generated literal and appends a DuckDB COPY +# that would write a file in the data directory. With the literal escaped the +# payload is stored as data and no statement is injected. +CREATE TABLE t (id INT PRIMARY KEY, c VARCHAR(200) DEFAULT 'a'');COPY(SELECT 2)TO ''zz_def.csv'';--') ENGINE=DuckDB CHARSET=utf8mb4| +# No injected COPY ran: zz_def.csv must not exist anywhere under the vardir. +# The default round-trips through MariaDB as plain data. +INSERT INTO t (id) VALUES (1); +SELECT c FROM t WHERE id = 1; +c +a');COPY(SELECT 2)TO 'zz_def.csv';-- +# A legitimate quote in a DEFAULT is stored and returned verbatim. +ALTER TABLE t ADD COLUMN d VARCHAR(20) DEFAULT 'a''b'; +INSERT INTO t (id) VALUES (2); +SELECT d FROM t WHERE id = 2; +d +a'b +DROP TABLE t; diff --git a/storage/duckdb/mysql-test/duckdb/r/duckdb_identifier_escaping.result b/storage/duckdb/mysql-test/duckdb/r/duckdb_identifier_escaping.result new file mode 100644 index 0000000000000..7e7a441c402ca --- /dev/null +++ b/storage/duckdb/mysql-test/duckdb/r/duckdb_identifier_escaping.result @@ -0,0 +1,102 @@ +CREATE DATABASE db_ident_esc; +USE db_ident_esc; +# +# 1) Column names containing a double quote (CREATE/INSERT/SELECT) +# +CREATE TABLE t1 (`a"b` INT PRIMARY KEY, `c"d` INT) ENGINE=DuckDB; +INSERT INTO t1 VALUES (1, 10), (2, 20); +SELECT * FROM t1 ORDER BY `a"b`; +a"b c"d +1 10 +2 20 +SELECT /* user's ' column */ * FROM t1 ORDER BY `a"b`; +a"b c"d +1 10 +2 20 +SELECT * -- user's ' columns +FROM t1 ORDER BY `a"b`; +a"b c"d +1 10 +2 20 +# +# 2) UPDATE/DELETE route the quoted names through the WHERE/SET builders +# +UPDATE t1 SET `c"d` = 99 WHERE `a"b` = 1; +SELECT * FROM t1 ORDER BY `a"b`; +a"b c"d +1 99 +2 20 +DELETE FROM t1 WHERE `a"b` = 2; +SELECT * FROM t1 ORDER BY `a"b`; +a"b c"d +1 99 +# +# 3) ALTER RENAME/ADD/DROP COLUMN with quoted names +# +ALTER TABLE t1 RENAME COLUMN `c"d` TO `e""f`; +SHOW CREATE TABLE t1; +Table Create Table +t1 CREATE TABLE `t1` ( + `a"b` int(11) NOT NULL, + `e""f` int(11) DEFAULT NULL, + PRIMARY KEY (`a"b`) +) ENGINE=DUCKDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_uca1400_ai_ci +SELECT `e""f` FROM t1; +e""f +99 +ALTER TABLE t1 ADD COLUMN `g"h` INT; +SELECT * FROM t1; +a"b e""f g"h +1 99 NULL +ALTER TABLE t1 DROP COLUMN `g"h`; +# +# 4) Table name containing a double quote +# +CREATE TABLE `x"y` (`id` INT PRIMARY KEY) ENGINE=DuckDB; +INSERT INTO `x"y` VALUES (1), (2); +SELECT * FROM `x"y` ORDER BY `id`; +id +1 +2 +TRUNCATE TABLE `x"y`; +SELECT COUNT(*) FROM `x"y`; +COUNT(*) +0 +DROP TABLE `x"y`; +# +# 5) Backslash quote-boundary mismatches disable raw SQL forwarding +# +CREATE TABLE lexical_guard (id INT PRIMARY KEY, v VARCHAR(100)) ENGINE=DuckDB; +INSERT INTO lexical_guard VALUES (1, 'safe'); +SELECT COUNT(*) FROM lexical_guard +WHERE v = 'missing\' OR true -- '; +COUNT(*) +0 +UPDATE lexical_guard SET v = 'changed\' WHERE true -- ' WHERE id = 999; +ERROR HY000: Got error 168 'Unsafe MariaDB backslash quote escape in forwarded SQL' from DuckDB +SELECT * FROM lexical_guard; +id v +1 safe +DELETE FROM lexical_guard WHERE v = 'missing\' OR true -- '; +ERROR HY000: Got error 168 'Unsafe MariaDB backslash quote escape in forwarded SQL' from DuckDB +SELECT COUNT(*) FROM lexical_guard; +COUNT(*) +1 +UPDATE lexical_guard SET v = 'safe\''; +ERROR HY000: Got error 168 'Unsafe MariaDB backslash quote escape in forwarded SQL' from DuckDB +SELECT * FROM lexical_guard; +id v +1 safe +UPDATE /* user's \' comment */ lexical_guard SET v = 'unchanged' WHERE id = 999; +UPDATE lexical_guard SET v = 'unchanged\n' WHERE id = 999; +UPDATE lexical_guard SET v = "changed\" WHERE true -- " WHERE id = 999; +ERROR HY000: Got error 168 'Unsafe MariaDB backslash quote escape in forwarded SQL' from DuckDB +SELECT * FROM lexical_guard; +id v +1 safe +DROP TABLE lexical_guard; +# +# 6) Cleanup +# +DROP TABLE t1; +DROP DATABASE db_ident_esc; diff --git a/storage/duckdb/mysql-test/duckdb/r/mdev_40651.result b/storage/duckdb/mysql-test/duckdb/r/mdev_40651.result new file mode 100644 index 0000000000000..53ec412f791be --- /dev/null +++ b/storage/duckdb/mysql-test/duckdb/r/mdev_40651.result @@ -0,0 +1,25 @@ +CREATE TABLE t (c1 INT KEY) ENGINE=DuckDB; +INSERT INTO t VALUES (1),(1); +ALTER TABLE t ADD c2 INT NOT NULL; +SELECT * FROM t ORDER BY c1, c2; +c1 c2 +1 0 +1 0 +INSERT INTO t (c1) VALUES (2); +ERROR HY000: Field 'c2' doesn't have a default value +ALTER TABLE t PARTITION BY HASH (c1) (PARTITION p1, PARTITION p2); +ERROR HY000: Engine cannot be used in partitioned tables +SELECT * FROM t ORDER BY c1, c2; +c1 c2 +1 0 +1 0 +DROP TABLE t; +CREATE TABLE t (id INT PRIMARY KEY, c1 INT) ENGINE=DuckDB; +INSERT INTO t VALUES (1, NULL); +ALTER TABLE t ADD c2 INT, MODIFY c1 INT NOT NULL; +ERROR HY000: Got error 168 'Constraint Error: NOT NULL constraint failed: t.c1' from DuckDB +ALTER TABLE t ADD c2 INT; +SELECT * FROM t ORDER BY id; +id c1 c2 +1 NULL NULL +DROP TABLE t; diff --git a/storage/duckdb/mysql-test/duckdb/t/alter_default_debug.test b/storage/duckdb/mysql-test/duckdb/t/alter_default_debug.test index a1a5cec984c74..7ebcc4d25edfd 100644 --- a/storage/duckdb/mysql-test/duckdb/t/alter_default_debug.test +++ b/storage/duckdb/mysql-test/duckdb/t/alter_default_debug.test @@ -1,4 +1,5 @@ --source ../include/have_duckdb.inc +--source include/not_msan.inc CREATE TABLE t(id INT PRIMARY KEY, a INT, b INT, c INT) ENGINE = DuckDB; INSERT INTO t VALUES(1, 1, 1, 1); diff --git a/storage/duckdb/mysql-test/duckdb/t/alter_duckdb_column.test b/storage/duckdb/mysql-test/duckdb/t/alter_duckdb_column.test index 0ae239d698c48..2e5076e8b1912 100644 --- a/storage/duckdb/mysql-test/duckdb/t/alter_duckdb_column.test +++ b/storage/duckdb/mysql-test/duckdb/t/alter_duckdb_column.test @@ -1,3 +1,4 @@ +--source include/not_msan.inc --echo #################### --echo # TEST FOR INSTANT # --echo #################### diff --git a/storage/duckdb/mysql-test/duckdb/t/alter_duckdb_column_copy.test b/storage/duckdb/mysql-test/duckdb/t/alter_duckdb_column_copy.test index 2a88f36f9c890..dd862e488dcf2 100644 --- a/storage/duckdb/mysql-test/duckdb/t/alter_duckdb_column_copy.test +++ b/storage/duckdb/mysql-test/duckdb/t/alter_duckdb_column_copy.test @@ -1,3 +1,4 @@ +--source include/not_msan.inc --echo ################# --echo # TEST FOR COPY # --echo ################# diff --git a/storage/duckdb/mysql-test/duckdb/t/alter_duckdb_index.test b/storage/duckdb/mysql-test/duckdb/t/alter_duckdb_index.test index 8df5bdd951bc5..5d08f86baa4cd 100644 --- a/storage/duckdb/mysql-test/duckdb/t/alter_duckdb_index.test +++ b/storage/duckdb/mysql-test/duckdb/t/alter_duckdb_index.test @@ -1,3 +1,4 @@ +--source include/not_msan.inc --echo #################### --echo # TEST FOR INSTANT # --echo #################### diff --git a/storage/duckdb/mysql-test/duckdb/t/alter_engine_duckdb.test b/storage/duckdb/mysql-test/duckdb/t/alter_engine_duckdb.test index 7d047686055cd..6240c455bf79a 100644 --- a/storage/duckdb/mysql-test/duckdb/t/alter_engine_duckdb.test +++ b/storage/duckdb/mysql-test/duckdb/t/alter_engine_duckdb.test @@ -1,3 +1,4 @@ +--source include/not_msan.inc SET GLOBAL duckdb_copy_ddl_in_batch=ON; --source ../include/alter_engine_duckdb.inc diff --git a/storage/duckdb/mysql-test/duckdb/t/batch_disconnect.test b/storage/duckdb/mysql-test/duckdb/t/batch_disconnect.test index 6a4de6d2282a0..0ef4f782ea75e 100644 --- a/storage/duckdb/mysql-test/duckdb/t/batch_disconnect.test +++ b/storage/duckdb/mysql-test/duckdb/t/batch_disconnect.test @@ -1,4 +1,5 @@ --source ../include/have_duckdb.inc +--source include/not_msan.inc --disable_query_log SET @saved_duckdb_dml_in_batch = @@GLOBAL.duckdb_dml_in_batch; diff --git a/storage/duckdb/mysql-test/duckdb/t/batch_rollback.test b/storage/duckdb/mysql-test/duckdb/t/batch_rollback.test index 60bc3e591d6b4..94bf421487c91 100644 --- a/storage/duckdb/mysql-test/duckdb/t/batch_rollback.test +++ b/storage/duckdb/mysql-test/duckdb/t/batch_rollback.test @@ -1,4 +1,5 @@ --source ../include/have_duckdb.inc +--source include/not_msan.inc --disable_query_log SET @saved_duckdb_dml_in_batch = @@GLOBAL.duckdb_dml_in_batch; diff --git a/storage/duckdb/mysql-test/duckdb/t/bugfix_crash_after_commit_error.test b/storage/duckdb/mysql-test/duckdb/t/bugfix_crash_after_commit_error.test index bc6757d240197..da058f620269f 100644 --- a/storage/duckdb/mysql-test/duckdb/t/bugfix_crash_after_commit_error.test +++ b/storage/duckdb/mysql-test/duckdb/t/bugfix_crash_after_commit_error.test @@ -1,4 +1,5 @@ --source ../include/have_duckdb.inc +--source include/not_msan.inc --skip TODO create table t1(id int primary key) engine = duckdb; diff --git a/storage/duckdb/mysql-test/duckdb/t/bugfix_temp_and_system_database.test b/storage/duckdb/mysql-test/duckdb/t/bugfix_temp_and_system_database.test index e802f5c4c354f..420fd8c26aae8 100644 --- a/storage/duckdb/mysql-test/duckdb/t/bugfix_temp_and_system_database.test +++ b/storage/duckdb/mysql-test/duckdb/t/bugfix_temp_and_system_database.test @@ -1,4 +1,5 @@ --source ../include/have_duckdb.inc +--source include/not_msan.inc # DuckDB has default databases named 'temp' and 'system', if we try to create # schema named 'temp' or 'system', we get an error. diff --git a/storage/duckdb/mysql-test/duckdb/t/charset_and_collation.test b/storage/duckdb/mysql-test/duckdb/t/charset_and_collation.test index 38ee163154c6f..54ea1077c1d98 100644 --- a/storage/duckdb/mysql-test/duckdb/t/charset_and_collation.test +++ b/storage/duckdb/mysql-test/duckdb/t/charset_and_collation.test @@ -1,4 +1,5 @@ --source ../include/have_duckdb.inc +--source include/not_msan.inc --echo # --echo # check collation config when execute --echo # diff --git a/storage/duckdb/mysql-test/duckdb/t/create_table_column.test b/storage/duckdb/mysql-test/duckdb/t/create_table_column.test index 4d56714c4845d..fadf29f6c0d6c 100644 --- a/storage/duckdb/mysql-test/duckdb/t/create_table_column.test +++ b/storage/duckdb/mysql-test/duckdb/t/create_table_column.test @@ -1,3 +1,4 @@ +--source include/not_msan.inc # file: duckdb_create_table.test # # Test case for creating tables with various MySQL field types in DuckDB. diff --git a/storage/duckdb/mysql-test/duckdb/t/create_table_column_timestamp.test b/storage/duckdb/mysql-test/duckdb/t/create_table_column_timestamp.test index 1b9ecfc4bcde3..ffdc20b657d17 100644 --- a/storage/duckdb/mysql-test/duckdb/t/create_table_column_timestamp.test +++ b/storage/duckdb/mysql-test/duckdb/t/create_table_column_timestamp.test @@ -1,3 +1,4 @@ +--source include/not_msan.inc --source include/have_debug.inc --disable_query_log diff --git a/storage/duckdb/mysql-test/duckdb/t/create_table_constraint.test b/storage/duckdb/mysql-test/duckdb/t/create_table_constraint.test index cccf1e06d35fb..3875082690b01 100644 --- a/storage/duckdb/mysql-test/duckdb/t/create_table_constraint.test +++ b/storage/duckdb/mysql-test/duckdb/t/create_table_constraint.test @@ -1,4 +1,5 @@ --source ../include/have_duckdb.inc +--source include/not_msan.inc --source include/have_debug.inc diff --git a/storage/duckdb/mysql-test/duckdb/t/cross_engine_join.test b/storage/duckdb/mysql-test/duckdb/t/cross_engine_join.test index 8fb4ea50aac30..43d1e2fa2db8b 100644 --- a/storage/duckdb/mysql-test/duckdb/t/cross_engine_join.test +++ b/storage/duckdb/mysql-test/duckdb/t/cross_engine_join.test @@ -1,3 +1,4 @@ +--source include/not_msan.inc --echo # --echo # Cross-engine join: DuckDB + InnoDB via _mdb_scan replacement scan --echo # diff --git a/storage/duckdb/mysql-test/duckdb/t/cross_engine_ryow.test b/storage/duckdb/mysql-test/duckdb/t/cross_engine_ryow.test index 2936b1dd9d2dc..61dbe0076f2ad 100644 --- a/storage/duckdb/mysql-test/duckdb/t/cross_engine_ryow.test +++ b/storage/duckdb/mysql-test/duckdb/t/cross_engine_ryow.test @@ -1,3 +1,4 @@ +--source include/not_msan.inc --source include/have_sequence.inc --echo # diff --git a/storage/duckdb/mysql-test/duckdb/t/cross_engine_ryow_dup.test b/storage/duckdb/mysql-test/duckdb/t/cross_engine_ryow_dup.test new file mode 100644 index 0000000000000..d572787dfa04f --- /dev/null +++ b/storage/duckdb/mysql-test/duckdb/t/cross_engine_ryow_dup.test @@ -0,0 +1,89 @@ +--source ../include/have_duckdb.inc +--source include/not_msan.inc + +--echo # +--echo # Cross-engine RYOW: the same external table referenced more than once. +--echo # +--echo # The external table registry is keyed by table name, so two references +--echo # to one table would collapse onto a single TABLE/handler and produce two +--echo # concurrent scans over one cursor. Such queries must not be pushed +--echo # down; the server executes them itself and the results must still show +--echo # the uncommitted writes of the current transaction. +--echo # + +--disable_warnings +DROP DATABASE IF EXISTS cross_engine_ryow_dup; +--enable_warnings +CREATE DATABASE cross_engine_ryow_dup CHARACTER SET utf8mb4; +USE cross_engine_ryow_dup; + +SET @old_max_threads= @@global.duckdb_max_threads; +SET GLOBAL duckdb_max_threads= 4; + +CREATE TABLE t_duck (id INT PRIMARY KEY, val VARCHAR(50)) ENGINE=DuckDB; +CREATE TABLE t_inno (id INT PRIMARY KEY, score INT) ENGINE=InnoDB; + +INSERT INTO t_duck VALUES (1,'alpha'),(2,'beta'),(3,'gamma'); +INSERT INTO t_inno VALUES (1,10),(2,20),(3,30); + +SET SESSION duckdb_cross_engine_ryow=1; + +--echo +--echo # (1) Self-join of the external table under two aliases +--echo +BEGIN; +UPDATE t_inno SET score=999 WHERE id=2; +--sorted_result +SELECT d.id, a.score AS a_score, b.score AS b_score + FROM t_duck d + JOIN t_inno a ON d.id = a.id + JOIN t_inno b ON a.id = b.id; +ROLLBACK; + +--echo +--echo # (2) The external table used both in the join and in a subquery +--echo +BEGIN; +UPDATE t_inno SET score=999 WHERE id=3; +--sorted_result +SELECT d.id, i.score + FROM t_duck d JOIN t_inno i ON d.id = i.id + WHERE i.score = (SELECT MAX(score) FROM t_inno); +ROLLBACK; + +--echo +--echo # (3) The external table twice in a UNION +--echo +BEGIN; +UPDATE t_inno SET score=999 WHERE id=1; +--sorted_result +SELECT d.id, i.score FROM t_duck d JOIN t_inno i ON d.id = i.id WHERE i.id = 1 +UNION ALL +SELECT d.id, i.score FROM t_duck d JOIN t_inno i ON d.id = i.id WHERE i.id = 2; +ROLLBACK; + +--echo +--echo # (4) A single reference is still pushed down and still sees own writes +--echo +BEGIN; +UPDATE t_inno SET score=555 WHERE id=2; +--sorted_result +SELECT d.id, i.score FROM t_duck d JOIN t_inno i ON d.id = i.id; +ROLLBACK; + +--echo +--echo # (5) Committed state is intact +--echo +--sorted_result +SELECT d.id, i.score FROM t_duck d JOIN t_inno i ON d.id = i.id; + +--echo +--echo # Cleanup +--echo +SET SESSION duckdb_cross_engine_ryow=DEFAULT; +SET GLOBAL duckdb_max_threads= @old_max_threads; +DROP TABLE t_duck; +DROP TABLE t_inno; +DROP DATABASE cross_engine_ryow_dup; + +--source ../include/cleanup_duckdb.inc diff --git a/storage/duckdb/mysql-test/duckdb/t/cross_engine_ryow_limit.test b/storage/duckdb/mysql-test/duckdb/t/cross_engine_ryow_limit.test new file mode 100644 index 0000000000000..84cc677bd9cfe --- /dev/null +++ b/storage/duckdb/mysql-test/duckdb/t/cross_engine_ryow_limit.test @@ -0,0 +1,81 @@ +--source ../include/have_duckdb.inc +--source include/not_msan.inc +--source include/have_sequence.inc + +--echo # +--echo # Cross-engine RYOW: early query termination. +--echo # +--echo # With LIMIT (or any other early stop) DuckDB stops pulling chunks from +--echo # the external table scan, so the scan is never driven to end-of-file. +--echo # The handler scan started behind the server's back must still be +--echo # finished, otherwise the table cannot be closed cleanly. +--echo # +--echo # FLUSH TABLES below forces the table to be closed inside the test +--echo # instead of at shutdown, so that a leaked scan is detected here. +--echo # +--echo # NOTE: the leak itself is caught by DBUG_ASSERT(inited == NONE) in +--echo # handler::~handler(), which is compiled out when DBUG_OFF is set. This +--echo # test therefore only proves the absence of a leak in a debug build; in a +--echo # release build it merely checks the query results. +--echo # + +--disable_warnings +DROP DATABASE IF EXISTS cross_engine_ryow_limit; +--enable_warnings +CREATE DATABASE cross_engine_ryow_limit CHARACTER SET utf8mb4; +USE cross_engine_ryow_limit; + +SET @old_max_threads= @@global.duckdb_max_threads; +SET GLOBAL duckdb_max_threads= 4; + +CREATE TABLE t_duck (id INT PRIMARY KEY, val VARCHAR(50)) ENGINE=DuckDB; +CREATE TABLE t_inno (id INT PRIMARY KEY, score INT) ENGINE=InnoDB; + +INSERT INTO t_duck SELECT seq, CONCAT('v', seq) FROM seq_1_to_3000; +INSERT INTO t_inno SELECT seq, seq FROM seq_1_to_3000; + +SET SESSION duckdb_cross_engine_ryow=1; + +--echo +--echo # (1) LIMIT stops the scan long before end-of-file +--echo +BEGIN; +UPDATE t_inno SET score=999 WHERE id=1; +SELECT d.id, i.score + FROM t_duck d JOIN t_inno i ON d.id = i.id + ORDER BY d.id LIMIT 1; + +--echo +--echo # (2) The same external table is usable again in the same transaction +--echo +SELECT d.id, i.score + FROM t_duck d JOIN t_inno i ON d.id = i.id + ORDER BY d.id LIMIT 2; +SELECT score FROM t_inno WHERE id = 1; +ROLLBACK; + +--echo +--echo # (3) Closing the table must not hit a leaked scan +--echo +FLUSH TABLES; +SELECT COUNT(*) FROM t_inno; + +--echo +--echo # (4) Early termination combined with an aggregate that stops early +--echo +BEGIN; +UPDATE t_inno SET score=999 WHERE id=2; +SELECT EXISTS (SELECT 1 FROM t_duck d JOIN t_inno i ON d.id = i.id WHERE i.score = 999) AS found; +ROLLBACK; +FLUSH TABLES; + +--echo +--echo # Cleanup +--echo +SET SESSION duckdb_cross_engine_ryow=DEFAULT; +SET GLOBAL duckdb_max_threads= @old_max_threads; +DROP TABLE t_duck; +DROP TABLE t_inno; +DROP DATABASE cross_engine_ryow_limit; + +--source ../include/cleanup_duckdb.inc diff --git a/storage/duckdb/mysql-test/duckdb/t/cross_engine_union.test b/storage/duckdb/mysql-test/duckdb/t/cross_engine_union.test index d303b974cb0f9..0ae97958ccd06 100644 --- a/storage/duckdb/mysql-test/duckdb/t/cross_engine_union.test +++ b/storage/duckdb/mysql-test/duckdb/t/cross_engine_union.test @@ -1,3 +1,4 @@ +--source include/not_msan.inc --echo # --echo # Cross-engine UNION: DuckDB + InnoDB via select_handler unit pushdown --echo # diff --git a/storage/duckdb/mysql-test/duckdb/t/cross_engine_where.test b/storage/duckdb/mysql-test/duckdb/t/cross_engine_where.test index 4736fb64a97d3..51102d6b29f27 100644 --- a/storage/duckdb/mysql-test/duckdb/t/cross_engine_where.test +++ b/storage/duckdb/mysql-test/duckdb/t/cross_engine_where.test @@ -1,3 +1,4 @@ +--source include/not_msan.inc --source include/have_sequence.inc --echo # diff --git a/storage/duckdb/mysql-test/duckdb/t/decimal_high_precision.test b/storage/duckdb/mysql-test/duckdb/t/decimal_high_precision.test index ec7bb24c7bd26..3570bf4a2dbd0 100644 --- a/storage/duckdb/mysql-test/duckdb/t/decimal_high_precision.test +++ b/storage/duckdb/mysql-test/duckdb/t/decimal_high_precision.test @@ -1,4 +1,5 @@ --source ../include/have_duckdb.inc +--source include/not_msan.inc --echo # diff --git a/storage/duckdb/mysql-test/duckdb/t/decimal_precision_all_possibilities.test b/storage/duckdb/mysql-test/duckdb/t/decimal_precision_all_possibilities.test index 2ea4459947590..1ec9348e5d059 100644 --- a/storage/duckdb/mysql-test/duckdb/t/decimal_precision_all_possibilities.test +++ b/storage/duckdb/mysql-test/duckdb/t/decimal_precision_all_possibilities.test @@ -1,3 +1,4 @@ +--source include/not_msan.inc set global duckdb_dml_in_batch = OFF; --source ../include/decimal_precision_all_possibilities.inc diff --git a/storage/duckdb/mysql-test/duckdb/t/dml_delete.test b/storage/duckdb/mysql-test/duckdb/t/dml_delete.test index 9d8e4322cd7a1..8fedb60aaed32 100644 --- a/storage/duckdb/mysql-test/duckdb/t/dml_delete.test +++ b/storage/duckdb/mysql-test/duckdb/t/dml_delete.test @@ -1,3 +1,4 @@ +--source include/not_msan.inc --echo # --echo # DuckDB DELETE operations test --echo # diff --git a/storage/duckdb/mysql-test/duckdb/t/dml_update.test b/storage/duckdb/mysql-test/duckdb/t/dml_update.test index 360dcfc14361f..681c86315a9f6 100644 --- a/storage/duckdb/mysql-test/duckdb/t/dml_update.test +++ b/storage/duckdb/mysql-test/duckdb/t/dml_update.test @@ -1,3 +1,4 @@ +--source include/not_msan.inc --echo # --echo # DuckDB UPDATE operations test --echo # diff --git a/storage/duckdb/mysql-test/duckdb/t/drop_database.test b/storage/duckdb/mysql-test/duckdb/t/drop_database.test index 75877300a67ec..9f3ed4955b092 100644 --- a/storage/duckdb/mysql-test/duckdb/t/drop_database.test +++ b/storage/duckdb/mysql-test/duckdb/t/drop_database.test @@ -1,3 +1,4 @@ +--source include/not_msan.inc --echo # --echo # DuckDB DROP DATABASE operations test --echo # diff --git a/storage/duckdb/mysql-test/duckdb/t/duckdb_add_backticks.test b/storage/duckdb/mysql-test/duckdb/t/duckdb_add_backticks.test index e354231ed18e5..a38ef13a7a93c 100644 --- a/storage/duckdb/mysql-test/duckdb/t/duckdb_add_backticks.test +++ b/storage/duckdb/mysql-test/duckdb/t/duckdb_add_backticks.test @@ -1,3 +1,4 @@ +--source include/not_msan.inc SET GLOBAL duckdb_require_primary_key=OFF; --echo # diff --git a/storage/duckdb/mysql-test/duckdb/t/duckdb_agg_func.test b/storage/duckdb/mysql-test/duckdb/t/duckdb_agg_func.test index bc50c1bf8cd11..8a97c53915632 100644 --- a/storage/duckdb/mysql-test/duckdb/t/duckdb_agg_func.test +++ b/storage/duckdb/mysql-test/duckdb/t/duckdb_agg_func.test @@ -1,4 +1,5 @@ --source ../include/have_duckdb.inc +--source include/not_msan.inc CREATE TABLE t_innodb ( id INT PRIMARY KEY, diff --git a/storage/duckdb/mysql-test/duckdb/t/duckdb_allow_encryption.test b/storage/duckdb/mysql-test/duckdb/t/duckdb_allow_encryption.test index 8f7ba7574ed86..f0144c977ee4b 100644 --- a/storage/duckdb/mysql-test/duckdb/t/duckdb_allow_encryption.test +++ b/storage/duckdb/mysql-test/duckdb/t/duckdb_allow_encryption.test @@ -1,3 +1,4 @@ +--source include/not_msan.inc --disable_warnings DROP DATABASE IF EXISTS encryption_test; diff --git a/storage/duckdb/mysql-test/duckdb/t/duckdb_alter_table_engine.test b/storage/duckdb/mysql-test/duckdb/t/duckdb_alter_table_engine.test index 80ef329792733..52a5cdb78d6fd 100644 --- a/storage/duckdb/mysql-test/duckdb/t/duckdb_alter_table_engine.test +++ b/storage/duckdb/mysql-test/duckdb/t/duckdb_alter_table_engine.test @@ -1,4 +1,5 @@ --source ../include/have_duckdb.inc +--source include/not_msan.inc --echo # --echo # Test DDL: alter table engine = innodb from duckdb tables diff --git a/storage/duckdb/mysql-test/duckdb/t/duckdb_appender_allocator_flush_threshold.test b/storage/duckdb/mysql-test/duckdb/t/duckdb_appender_allocator_flush_threshold.test index 3a154d82679a6..d3d8a8a2b7782 100644 --- a/storage/duckdb/mysql-test/duckdb/t/duckdb_appender_allocator_flush_threshold.test +++ b/storage/duckdb/mysql-test/duckdb/t/duckdb_appender_allocator_flush_threshold.test @@ -1,4 +1,5 @@ --source ../include/have_duckdb.inc +--source include/not_msan.inc SHOW GLOBAL VARIABLES LIKE "duckdb_appender_allocator_flush_threshold"; SELECT run_in_duckdb("FROM duckdb_settings() WHERE name = 'allocator_flush_threshold'"); diff --git a/storage/duckdb/mysql-test/duckdb/t/duckdb_auto_increment.test b/storage/duckdb/mysql-test/duckdb/t/duckdb_auto_increment.test new file mode 100644 index 0000000000000..4648aa05efa41 --- /dev/null +++ b/storage/duckdb/mysql-test/duckdb/t/duckdb_auto_increment.test @@ -0,0 +1,85 @@ +--source ../include/have_duckdb.inc +--source include/not_msan.inc + +--disable_query_log +CREATE DATABASE IF NOT EXISTS db_autoinc CHARACTER SET utf8mb4; +USE db_autoinc; +--enable_query_log +SET NAMES utf8mb4; + +--echo # +--echo # MDEV-40266: AUTO_INCREMENT backed by a DuckDB sequence +--echo # + +--echo # --- CREATE TABLE creates the sequence ------------------------------- +CREATE TABLE t (id BIGINT NOT NULL AUTO_INCREMENT PRIMARY KEY, a INT) ENGINE=DuckDB; + +SHOW CREATE TABLE t; + +--echo # The sequence must exist in DuckDB, named after the table. +SELECT run_in_duckdb("SELECT schema_name, sequence_name, start_value FROM duckdb_sequences() WHERE schema_name = 'db_autoinc' ORDER BY sequence_name"); + +--echo # The column default must reference that sequence, so that writes which +--echo # bypass MariaDB still draw a non-colliding id from the same counter. +SELECT run_in_duckdb("SELECT column_name, column_default FROM information_schema.columns WHERE table_schema = 'db_autoinc' AND table_name = 't' ORDER BY column_name"); + +--echo # --- an insert bypassing MariaDB uses the sequence -------------------- +SELECT run_in_duckdb("INSERT INTO db_autoinc.t(a) VALUES(10)"); +SELECT run_in_duckdb("INSERT INTO db_autoinc.t(a) VALUES(20)"); +SELECT id, a FROM t ORDER BY id; + +--echo # --- MariaDB-side inserts draw from the same sequence ------------------ +INSERT INTO t(a) VALUES(30); +SELECT LAST_INSERT_ID(); +INSERT INTO t(a) VALUES(40); +SELECT LAST_INSERT_ID(); + +--echo # A multi-row insert reserves its ids in one go. +INSERT INTO t(a) VALUES(50),(60),(70); +SELECT LAST_INSERT_ID(); +SELECT id, a FROM t ORDER BY id; + +--echo # A DuckDB-side insert after MariaDB ones must not collide: the block +--echo # reservation advanced the shared counter past the cached ids. +SELECT run_in_duckdb("INSERT INTO db_autoinc.t(a) VALUES(80)"); +SELECT count(*), count(DISTINCT id) FROM t; + +--echo # --- START WITH honours CREATE TABLE ... AUTO_INCREMENT=N ------------- +CREATE TABLE t2 (id BIGINT NOT NULL AUTO_INCREMENT PRIMARY KEY, a INT) ENGINE=DuckDB AUTO_INCREMENT=100; +SELECT run_in_duckdb("SELECT sequence_name, start_value FROM duckdb_sequences() WHERE schema_name = 'db_autoinc' AND sequence_name = 'mdb_autoinc_t2'"); +SELECT run_in_duckdb("INSERT INTO db_autoinc.t2(a) VALUES(1)"); +INSERT INTO t2(a) VALUES(2); +SELECT id, a FROM t2 ORDER BY id; + +--echo # --- a table without AUTO_INCREMENT gets no sequence ------------------ +CREATE TABLE t3 (id INT PRIMARY KEY, a INT) ENGINE=DuckDB; +SELECT run_in_duckdb("SELECT count(*) FROM duckdb_sequences() WHERE schema_name = 'db_autoinc' AND sequence_name = 'mdb_autoinc_t3'"); + +--echo # AUTO_INCREMENT is accepted by CREATE TABLE only; seeding an existing +--echo # column from its current maximum is not implemented. +--error ER_TABLE_CANT_HANDLE_AUTO_INCREMENT +ALTER TABLE t3 MODIFY COLUMN id INT AUTO_INCREMENT; +--error ER_TABLE_CANT_HANDLE_AUTO_INCREMENT +ALTER TABLE t3 ADD COLUMN b INT AUTO_INCREMENT UNIQUE; +--echo # The rejected ALTER must leave the table untouched. +SHOW CREATE TABLE t3; + +--echo # --- setval() is available (local backport of DuckDB PR #24061) -------- +SELECT run_in_duckdb("CREATE SEQUENCE db_autoinc.check_setval"); +SELECT run_in_duckdb("SELECT setval('db_autoinc.check_setval', 42)"); +SELECT run_in_duckdb("SELECT nextval('db_autoinc.check_setval')"); +SELECT run_in_duckdb("SELECT setval('db_autoinc.check_setval', 100, false)"); +SELECT run_in_duckdb("SELECT nextval('db_autoinc.check_setval')"); +SELECT run_in_duckdb("DROP SEQUENCE db_autoinc.check_setval"); + +--echo # --- DROP TABLE removes the sequence ---------------------------------- +DROP TABLE t; +SELECT run_in_duckdb("SELECT count(*) FROM duckdb_sequences() WHERE schema_name = 'db_autoinc' AND sequence_name = 'mdb_autoinc_t'"); + +DROP TABLE t2; +DROP TABLE t3; +SELECT run_in_duckdb("SELECT count(*) FROM duckdb_sequences() WHERE schema_name = 'db_autoinc'"); + +--disable_query_log +DROP DATABASE db_autoinc; +--enable_query_log diff --git a/storage/duckdb/mysql-test/duckdb/t/duckdb_bit_string.test b/storage/duckdb/mysql-test/duckdb/t/duckdb_bit_string.test index 22e82e4c3e5e6..f7c8543a54d2e 100644 --- a/storage/duckdb/mysql-test/duckdb/t/duckdb_bit_string.test +++ b/storage/duckdb/mysql-test/duckdb/t/duckdb_bit_string.test @@ -1,4 +1,5 @@ --source ../include/have_duckdb.inc +--source include/not_msan.inc CREATE TABLE t1 (id INT PRIMARY KEY, col1 VARCHAR(100)) ENGINE=DuckDB; CREATE TABLE t2 (id INT PRIMARY KEY, col1 BLOB) ENGINE=DuckDB; diff --git a/storage/duckdb/mysql-test/duckdb/t/duckdb_collate.test b/storage/duckdb/mysql-test/duckdb/t/duckdb_collate.test index c575cbc92015d..1d29484ce322f 100644 --- a/storage/duckdb/mysql-test/duckdb/t/duckdb_collate.test +++ b/storage/duckdb/mysql-test/duckdb/t/duckdb_collate.test @@ -1,3 +1,4 @@ +--source include/not_msan.inc --disable_query_log --disable_warnings DROP TABLE IF EXISTS t1; diff --git a/storage/duckdb/mysql-test/duckdb/t/duckdb_cte.test b/storage/duckdb/mysql-test/duckdb/t/duckdb_cte.test index b0210d0b55808..c714205c845a8 100644 --- a/storage/duckdb/mysql-test/duckdb/t/duckdb_cte.test +++ b/storage/duckdb/mysql-test/duckdb/t/duckdb_cte.test @@ -1,3 +1,4 @@ +--source include/not_msan.inc CREATE DATABASE IF NOT EXISTS duckdb_cte; USE duckdb_cte; diff --git a/storage/duckdb/mysql-test/duckdb/t/duckdb_db_table_strconvert.test b/storage/duckdb/mysql-test/duckdb/t/duckdb_db_table_strconvert.test index 0d50b5a1c9bbe..884e10f721f63 100644 --- a/storage/duckdb/mysql-test/duckdb/t/duckdb_db_table_strconvert.test +++ b/storage/duckdb/mysql-test/duckdb/t/duckdb_db_table_strconvert.test @@ -1,3 +1,4 @@ +--source include/not_msan.inc # Test for escape characters in the table name or database name. --echo # diff --git a/storage/duckdb/mysql-test/duckdb/t/duckdb_ddl_during_transaction.test b/storage/duckdb/mysql-test/duckdb/t/duckdb_ddl_during_transaction.test index b0146bf54cb86..1d192f2e2145a 100644 --- a/storage/duckdb/mysql-test/duckdb/t/duckdb_ddl_during_transaction.test +++ b/storage/duckdb/mysql-test/duckdb/t/duckdb_ddl_during_transaction.test @@ -1,4 +1,5 @@ --source ../include/have_duckdb.inc +--source include/not_msan.inc --echo ########################################################################### --echo # Prepare diff --git a/storage/duckdb/mysql-test/duckdb/t/duckdb_default_expr.test b/storage/duckdb/mysql-test/duckdb/t/duckdb_default_expr.test new file mode 100644 index 0000000000000..c5a8ae25ec3df --- /dev/null +++ b/storage/duckdb/mysql-test/duckdb/t/duckdb_default_expr.test @@ -0,0 +1,79 @@ +--source ../include/have_duckdb.inc +--source include/not_msan.inc + +--disable_query_log +CREATE DATABASE IF NOT EXISTS db_default_expr CHARACTER SET utf8mb4; +USE db_default_expr; +--enable_query_log +SET NAMES utf8mb4; + +--echo # +--echo # MDEV-40266: DEFAULT(NEXT VALUE FOR seq) must not break the DuckDB DDL +--echo # + +CREATE SEQUENCE s START WITH 1 INCREMENT BY 1; + +--echo # CREATE TABLE +CREATE TABLE t (id BIGINT NOT NULL DEFAULT(NEXT VALUE FOR s) PRIMARY KEY, a INT) ENGINE=DuckDB; +SHOW WARNINGS; +SHOW CREATE TABLE t; + +--echo # The DuckDB column must carry no default: MariaDB always supplies the value. +SELECT run_in_duckdb("SELECT column_name, column_default FROM information_schema.columns WHERE table_schema = 'db_default_expr' AND table_name = 't' ORDER BY column_name"); + +INSERT INTO t (a) VALUES (10); +INSERT INTO t (a) VALUES (20); +SELECT id, a FROM t ORDER BY id; + +--echo # ADD COLUMN +CREATE SEQUENCE s2 START WITH 100 INCREMENT BY 1; +ALTER TABLE t ADD COLUMN b BIGINT DEFAULT(NEXT VALUE FOR s2); +SHOW WARNINGS; +SELECT run_in_duckdb("SELECT column_name, column_default FROM information_schema.columns WHERE table_schema = 'db_default_expr' AND table_name = 't' ORDER BY column_name"); + +INSERT INTO t (a) VALUES (30); +SELECT id, a, b FROM t ORDER BY id; + +--echo # MODIFY COLUMN, adding a sequence default to a plain column +ALTER TABLE t MODIFY COLUMN a INT DEFAULT(NEXT VALUE FOR s2); +SHOW WARNINGS; +SELECT run_in_duckdb("SELECT column_name, column_default FROM information_schema.columns WHERE table_schema = 'db_default_expr' AND table_name = 't' ORDER BY column_name"); + +INSERT INTO t () VALUES (); +SELECT id, a, b FROM t ORDER BY id; + +--echo # MODIFY COLUMN, replacing a sequence default with a literal +ALTER TABLE t MODIFY COLUMN a INT DEFAULT 7; +SHOW WARNINGS; +SELECT run_in_duckdb("SELECT column_name, column_default FROM information_schema.columns WHERE table_schema = 'db_default_expr' AND table_name = 't' ORDER BY column_name"); + +--echo # MODIFY COLUMN, replacing a literal default with an expression +ALTER TABLE t MODIFY COLUMN a INT DEFAULT (1+1); +SHOW WARNINGS; +SELECT run_in_duckdb("SELECT column_name, column_default FROM information_schema.columns WHERE table_schema = 'db_default_expr' AND table_name = 't' AND column_name = 'a'"); + +--echo # Insert bypassing MariaDB: relies on the DuckDB-side expression default. +SELECT run_in_duckdb("INSERT INTO db_default_expr.t(id, b) VALUES(1000, 1000)"); +SELECT a FROM t WHERE id = 1000; + +DROP TABLE t; +DROP SEQUENCE s; +DROP SEQUENCE s2; + +--echo # +--echo # Non-sequence expression defaults keep reaching DuckDB +--echo # + +CREATE TABLE t2 (id INT PRIMARY KEY, a INT DEFAULT (1+1), b VARCHAR(10) DEFAULT ('1+1')) ENGINE=DuckDB; +SELECT run_in_duckdb("SELECT column_name, column_default FROM information_schema.columns WHERE table_schema = 'db_default_expr' AND table_name = 't2' ORDER BY column_name"); + +INSERT INTO t2 (id) VALUES (1); +--echo # Insert bypassing MariaDB: relies on the DuckDB-side default. +SELECT run_in_duckdb("INSERT INTO db_default_expr.t2(id) VALUES(2)"); +SELECT * FROM t2 ORDER BY id; + +DROP TABLE t2; + +--disable_query_log +DROP DATABASE db_default_expr; +--enable_query_log diff --git a/storage/duckdb/mysql-test/duckdb/t/duckdb_default_value_injection.test b/storage/duckdb/mysql-test/duckdb/t/duckdb_default_value_injection.test new file mode 100644 index 0000000000000..170e8b809c6d4 --- /dev/null +++ b/storage/duckdb/mysql-test/duckdb/t/duckdb_default_value_injection.test @@ -0,0 +1,46 @@ +--source ../include/have_duckdb.inc +--source include/not_msan.inc + +--echo # +--echo # MDEV-40610: an unescaped column DEFAULT string must not inject DuckDB SQL +--echo # +--echo # get_field_default_for_duckdb() wrapped a string DEFAULT in single quotes +--echo # without doubling an embedded quote, so a crafted default closed the +--echo # generated DuckDB literal and the rest was parsed as DuckDB SQL. The gate +--echo # below fails while the injection works and passes once literals are escaped. + +--disable_query_log +CREATE DATABASE IF NOT EXISTS injdb CHARACTER SET utf8mb4; +USE injdb; +--enable_query_log +SET NAMES utf8mb4; + +--echo # The crafted DEFAULT closes the generated literal and appends a DuckDB COPY +--echo # that would write a file in the data directory. With the literal escaped the +--echo # payload is stored as data and no statement is injected. +--delimiter | +CREATE TABLE t (id INT PRIMARY KEY, c VARCHAR(200) DEFAULT 'a'');COPY(SELECT 2)TO ''zz_def.csv'';--') ENGINE=DuckDB CHARSET=utf8mb4| +--delimiter ; + +--echo # No injected COPY ran: zz_def.csv must not exist anywhere under the vardir. +perl; + use File::Find; + my $found= 0; + find(sub { $found= 1 if $_ eq 'zz_def.csv' }, $ENV{MYSQLTEST_VARDIR}); + die "MDEV-40610: crafted DEFAULT injected a COPY (zz_def.csv written)\n" if $found; +EOF + +--echo # The default round-trips through MariaDB as plain data. +INSERT INTO t (id) VALUES (1); +SELECT c FROM t WHERE id = 1; + +--echo # A legitimate quote in a DEFAULT is stored and returned verbatim. +ALTER TABLE t ADD COLUMN d VARCHAR(20) DEFAULT 'a''b'; +INSERT INTO t (id) VALUES (2); +SELECT d FROM t WHERE id = 2; + +DROP TABLE t; + +--disable_query_log +DROP DATABASE injdb; +--enable_query_log diff --git a/storage/duckdb/mysql-test/duckdb/t/duckdb_fix_sql.test b/storage/duckdb/mysql-test/duckdb/t/duckdb_fix_sql.test index a005b8027034d..38acb14279748 100644 --- a/storage/duckdb/mysql-test/duckdb/t/duckdb_fix_sql.test +++ b/storage/duckdb/mysql-test/duckdb/t/duckdb_fix_sql.test @@ -1,4 +1,5 @@ --source ../include/have_duckdb.inc +--source include/not_msan.inc CREATE TABLE fake_innodb_table (id INT PRIMARY KEY) ENGINE=InnoDB; INSERT INTO fake_innodb_table VALUES (0); diff --git a/storage/duckdb/mysql-test/duckdb/t/duckdb_identifier_escaping.test b/storage/duckdb/mysql-test/duckdb/t/duckdb_identifier_escaping.test new file mode 100644 index 0000000000000..3ffeb7adef4d8 --- /dev/null +++ b/storage/duckdb/mysql-test/duckdb/t/duckdb_identifier_escaping.test @@ -0,0 +1,88 @@ +# MDEV-40653: identifiers (schema/table/column names) must be escaped when +# building DuckDB SQL. DuckDB delimits identifiers with double quotes and +# escapes an embedded double quote by doubling it. A MariaDB identifier may +# contain a literal double quote, so without escaping a crafted name breaks +# out of the quoted identifier (parser error / SQL injection). + +--disable_query_log +SET @saved_duckdb_require_primary_key = @@GLOBAL.duckdb_require_primary_key; +SET @saved_sql_mode = @@SESSION.sql_mode; +SET GLOBAL duckdb_require_primary_key = OFF; +SET SESSION sql_mode = ''; +--enable_query_log + +CREATE DATABASE db_ident_esc; +USE db_ident_esc; + +--echo # +--echo # 1) Column names containing a double quote (CREATE/INSERT/SELECT) +--echo # +CREATE TABLE t1 (`a"b` INT PRIMARY KEY, `c"d` INT) ENGINE=DuckDB; +INSERT INTO t1 VALUES (1, 10), (2, 20); +SELECT * FROM t1 ORDER BY `a"b`; +SELECT /* user's ' column */ * FROM t1 ORDER BY `a"b`; +SELECT * -- user's ' columns +FROM t1 ORDER BY `a"b`; + +--echo # +--echo # 2) UPDATE/DELETE route the quoted names through the WHERE/SET builders +--echo # +UPDATE t1 SET `c"d` = 99 WHERE `a"b` = 1; +SELECT * FROM t1 ORDER BY `a"b`; +DELETE FROM t1 WHERE `a"b` = 2; +SELECT * FROM t1 ORDER BY `a"b`; + +--echo # +--echo # 3) ALTER RENAME/ADD/DROP COLUMN with quoted names +--echo # +ALTER TABLE t1 RENAME COLUMN `c"d` TO `e""f`; +SHOW CREATE TABLE t1; +SELECT `e""f` FROM t1; +ALTER TABLE t1 ADD COLUMN `g"h` INT; +SELECT * FROM t1; +ALTER TABLE t1 DROP COLUMN `g"h`; + +--echo # +--echo # 4) Table name containing a double quote +--echo # +CREATE TABLE `x"y` (`id` INT PRIMARY KEY) ENGINE=DuckDB; +INSERT INTO `x"y` VALUES (1), (2); +SELECT * FROM `x"y` ORDER BY `id`; +TRUNCATE TABLE `x"y`; +SELECT COUNT(*) FROM `x"y`; +DROP TABLE `x"y`; + +--echo # +--echo # 5) Backslash quote-boundary mismatches disable raw SQL forwarding +--echo # +CREATE TABLE lexical_guard (id INT PRIMARY KEY, v VARCHAR(100)) ENGINE=DuckDB; +INSERT INTO lexical_guard VALUES (1, 'safe'); +SELECT COUNT(*) FROM lexical_guard +WHERE v = 'missing\' OR true -- '; +--error ER_GET_ERRMSG +UPDATE lexical_guard SET v = 'changed\' WHERE true -- ' WHERE id = 999; +SELECT * FROM lexical_guard; +--error ER_GET_ERRMSG +DELETE FROM lexical_guard WHERE v = 'missing\' OR true -- '; +SELECT COUNT(*) FROM lexical_guard; +--error ER_GET_ERRMSG +UPDATE lexical_guard SET v = 'safe\''; +SELECT * FROM lexical_guard; +UPDATE /* user's \' comment */ lexical_guard SET v = 'unchanged' WHERE id = 999; +UPDATE lexical_guard SET v = 'unchanged\n' WHERE id = 999; +--error ER_GET_ERRMSG +UPDATE lexical_guard SET v = "changed\" WHERE true -- " WHERE id = 999; +SELECT * FROM lexical_guard; +DROP TABLE lexical_guard; + +--echo # +--echo # 6) Cleanup +--echo # +DROP TABLE t1; +DROP DATABASE db_ident_esc; + +--disable_query_log +SET GLOBAL duckdb_require_primary_key = @saved_duckdb_require_primary_key; +SET SESSION sql_mode = @saved_sql_mode; +--enable_query_log +--source ../include/cleanup_duckdb.inc diff --git a/storage/duckdb/mysql-test/duckdb/t/duckdb_json.test b/storage/duckdb/mysql-test/duckdb/t/duckdb_json.test index f607d971e4fa1..7bd522a04f7ce 100644 --- a/storage/duckdb/mysql-test/duckdb/t/duckdb_json.test +++ b/storage/duckdb/mysql-test/duckdb/t/duckdb_json.test @@ -1,4 +1,5 @@ --source ../include/have_duckdb.inc +--source include/not_msan.inc CREATE TABLE t1_innodb (id INT PRIMARY KEY, col1 JSON) ENGINE=InnoDB; CREATE TABLE t1_duckdb (id INT PRIMARY KEY, col1 JSON) ENGINE=DuckDB; diff --git a/storage/duckdb/mysql-test/duckdb/t/duckdb_kill.test b/storage/duckdb/mysql-test/duckdb/t/duckdb_kill.test index 2915977740bb0..a9344b6be5b2b 100644 --- a/storage/duckdb/mysql-test/duckdb/t/duckdb_kill.test +++ b/storage/duckdb/mysql-test/duckdb/t/duckdb_kill.test @@ -1,4 +1,5 @@ --source ../include/have_duckdb.inc +--source include/not_msan.inc --source include/have_debug.inc --source include/have_debug_sync.inc diff --git a/storage/duckdb/mysql-test/duckdb/t/duckdb_monitor.test b/storage/duckdb/mysql-test/duckdb/t/duckdb_monitor.test index 00293b87cc20f..9480f9a588d15 100644 --- a/storage/duckdb/mysql-test/duckdb/t/duckdb_monitor.test +++ b/storage/duckdb/mysql-test/duckdb/t/duckdb_monitor.test @@ -1,4 +1,7 @@ --source ../include/have_duckdb.inc +# MDEV-40449: MSan reports use-of-uninitialized-value in +# duckdb::JemallocExtension::ThreadFlush (false positive from uninstrumented jemalloc). +--source include/not_msan.inc # # DuckDB monitoring status variables test. diff --git a/storage/duckdb/mysql-test/duckdb/t/duckdb_numeric_func.test b/storage/duckdb/mysql-test/duckdb/t/duckdb_numeric_func.test index 5f992cb23b505..3d0fdaf1ac856 100644 --- a/storage/duckdb/mysql-test/duckdb/t/duckdb_numeric_func.test +++ b/storage/duckdb/mysql-test/duckdb/t/duckdb_numeric_func.test @@ -1,4 +1,5 @@ --source ../include/have_duckdb.inc +--source include/not_msan.inc # 1. CONV(), CRC32(), MOD, DIV, TRUNCATE() is not support yet. # 2. The RAND() function in duckdb and mysql uses different random number generation algorithms, so it cannot support the seed argument. diff --git a/storage/duckdb/mysql-test/duckdb/t/duckdb_refuse_xa.test b/storage/duckdb/mysql-test/duckdb/t/duckdb_refuse_xa.test index 0e34bd12c6125..4489a54fd3317 100644 --- a/storage/duckdb/mysql-test/duckdb/t/duckdb_refuse_xa.test +++ b/storage/duckdb/mysql-test/duckdb/t/duckdb_refuse_xa.test @@ -1,4 +1,5 @@ --source ../include/have_duckdb.inc +--source include/not_msan.inc CREATE TABLE t1 (c1 INT PRIMARY KEY, c2 VARCHAR(5)) ENGINE=duckdb; diff --git a/storage/duckdb/mysql-test/duckdb/t/duckdb_require_primary_key.test b/storage/duckdb/mysql-test/duckdb/t/duckdb_require_primary_key.test index cc8ffb15acb31..f972509960a99 100644 --- a/storage/duckdb/mysql-test/duckdb/t/duckdb_require_primary_key.test +++ b/storage/duckdb/mysql-test/duckdb/t/duckdb_require_primary_key.test @@ -1,4 +1,5 @@ --source ../include/have_duckdb.inc +--source include/not_msan.inc diff --git a/storage/duckdb/mysql-test/duckdb/t/duckdb_set_operation.test b/storage/duckdb/mysql-test/duckdb/t/duckdb_set_operation.test index e245e7c9575fc..04db6e8cc5398 100644 --- a/storage/duckdb/mysql-test/duckdb/t/duckdb_set_operation.test +++ b/storage/duckdb/mysql-test/duckdb/t/duckdb_set_operation.test @@ -1,3 +1,4 @@ +--source include/not_msan.inc CREATE TABLE t1 (id INT PRIMARY KEY, col1 INT) ENGINE=DuckDB; INSERT INTO t1 VALUES (1, 1), (2, 1), (3, 2), (4, 2); CREATE TABLE t2 (id INT PRIMARY KEY, col1 INT) ENGINE=DuckDB; diff --git a/storage/duckdb/mysql-test/duckdb/t/duckdb_sql_mode.test b/storage/duckdb/mysql-test/duckdb/t/duckdb_sql_mode.test index baeca66b9dbc9..772254a650e59 100644 --- a/storage/duckdb/mysql-test/duckdb/t/duckdb_sql_mode.test +++ b/storage/duckdb/mysql-test/duckdb/t/duckdb_sql_mode.test @@ -1,4 +1,5 @@ --source ../include/have_duckdb.inc +--source include/not_msan.inc --echo --echo 1. ONLY_FULL_GROUP_BY diff --git a/storage/duckdb/mysql-test/duckdb/t/duckdb_sql_syntax.test b/storage/duckdb/mysql-test/duckdb/t/duckdb_sql_syntax.test index c9db95ddd25ec..aee922d26f15b 100644 --- a/storage/duckdb/mysql-test/duckdb/t/duckdb_sql_syntax.test +++ b/storage/duckdb/mysql-test/duckdb/t/duckdb_sql_syntax.test @@ -1,3 +1,4 @@ +--source include/not_msan.inc # WITH ROLLUP CREATE TABLE t1 (id INT PRIMARY KEY, col1 INT, col2 INT); SELECT id, col1, col2 FROM t1 GROUP BY id, col1, col2 WITH ROLLUP; diff --git a/storage/duckdb/mysql-test/duckdb/t/duckdb_string_func.test b/storage/duckdb/mysql-test/duckdb/t/duckdb_string_func.test index a14478edc6578..59489ef568b28 100644 --- a/storage/duckdb/mysql-test/duckdb/t/duckdb_string_func.test +++ b/storage/duckdb/mysql-test/duckdb/t/duckdb_string_func.test @@ -1,4 +1,5 @@ --source ../include/have_duckdb.inc +--source include/not_msan.inc # 1. In Duckdb, some string function does not support blob fields as input # 2. Regex-related functions are basically incompatible, and regex functions will be processed uniformly in the future. diff --git a/storage/duckdb/mysql-test/duckdb/t/duckdb_time_func.test b/storage/duckdb/mysql-test/duckdb/t/duckdb_time_func.test index bc437121db825..8a57fd81aa55e 100644 --- a/storage/duckdb/mysql-test/duckdb/t/duckdb_time_func.test +++ b/storage/duckdb/mysql-test/duckdb/t/duckdb_time_func.test @@ -1,4 +1,5 @@ --source ../include/have_duckdb.inc +--source include/not_msan.inc # Compared with MySQL, duckdb's date functions generally have the following incompatibilities: # 1. Conversion between integer type and date/timestamp/time type diff --git a/storage/duckdb/mysql-test/duckdb/t/duckdb_time_func_unsupported.test b/storage/duckdb/mysql-test/duckdb/t/duckdb_time_func_unsupported.test index 0c21437ed4d8e..61b390edfd7fc 100644 --- a/storage/duckdb/mysql-test/duckdb/t/duckdb_time_func_unsupported.test +++ b/storage/duckdb/mysql-test/duckdb/t/duckdb_time_func_unsupported.test @@ -1,4 +1,5 @@ --source ../include/have_duckdb.inc +--source include/not_msan.inc # Unsupported / incompatible date-time functions for DuckDB pushdown. # This is the companion of duckdb_time_func.test which keeps only the diff --git a/storage/duckdb/mysql-test/duckdb/t/duckdb_uuid.test b/storage/duckdb/mysql-test/duckdb/t/duckdb_uuid.test index 161d588487f88..c65fca8b8a5c1 100644 --- a/storage/duckdb/mysql-test/duckdb/t/duckdb_uuid.test +++ b/storage/duckdb/mysql-test/duckdb/t/duckdb_uuid.test @@ -1,4 +1,5 @@ --source ../include/have_duckdb.inc +--source include/not_msan.inc --source include/have_sequence.inc --echo # diff --git a/storage/duckdb/mysql-test/duckdb/t/feature_duckdb_data_type.test b/storage/duckdb/mysql-test/duckdb/t/feature_duckdb_data_type.test index e21b2cb311e89..06f0163991b37 100644 --- a/storage/duckdb/mysql-test/duckdb/t/feature_duckdb_data_type.test +++ b/storage/duckdb/mysql-test/duckdb/t/feature_duckdb_data_type.test @@ -1,4 +1,5 @@ --source ../include/have_duckdb.inc +--source include/not_msan.inc # Feature ORC data type test # Prepare diff --git a/storage/duckdb/mysql-test/duckdb/t/ha_duckdb.test b/storage/duckdb/mysql-test/duckdb/t/ha_duckdb.test index ffdfff0e07e92..14c5ce546ec78 100644 --- a/storage/duckdb/mysql-test/duckdb/t/ha_duckdb.test +++ b/storage/duckdb/mysql-test/duckdb/t/ha_duckdb.test @@ -1,3 +1,4 @@ +--source include/not_msan.inc --echo # --echo # Basic DuckDB handler test: DDL and DML operations --echo # diff --git a/storage/duckdb/mysql-test/duckdb/t/mdev_40379.test b/storage/duckdb/mysql-test/duckdb/t/mdev_40379.test index 0c21a7d0c60fb..82a48ceac59d4 100644 --- a/storage/duckdb/mysql-test/duckdb/t/mdev_40379.test +++ b/storage/duckdb/mysql-test/duckdb/t/mdev_40379.test @@ -1,3 +1,4 @@ +--source include/not_msan.inc # # MDEV-40379: Assertion failure in CreateTableConvertor::translate # upon CREATE TEMPORARY TABLE with ENGINE=DuckDB diff --git a/storage/duckdb/mysql-test/duckdb/t/mdev_40651.test b/storage/duckdb/mysql-test/duckdb/t/mdev_40651.test new file mode 100644 index 0000000000000..086242246619a --- /dev/null +++ b/storage/duckdb/mysql-test/duckdb/t/mdev_40651.test @@ -0,0 +1,24 @@ +--source ../include/have_duckdb.inc +--source include/have_partition.inc +--source include/not_msan.inc + +# MDEV-40651: keep DuckDB DDL atomic and preserve implicit defaults. +CREATE TABLE t (c1 INT KEY) ENGINE=DuckDB; +INSERT INTO t VALUES (1),(1); +ALTER TABLE t ADD c2 INT NOT NULL; +SELECT * FROM t ORDER BY c1, c2; +--error ER_NO_DEFAULT_FOR_FIELD +INSERT INTO t (c1) VALUES (2); +--error ER_PARTITION_MERGE_ERROR +ALTER TABLE t PARTITION BY HASH (c1) (PARTITION p1, PARTITION p2); +SELECT * FROM t ORDER BY c1, c2; +DROP TABLE t; + +# A later DuckDB DDL error rolls back preceding generated statements. +CREATE TABLE t (id INT PRIMARY KEY, c1 INT) ENGINE=DuckDB; +INSERT INTO t VALUES (1, NULL); +--error ER_GET_ERRMSG +ALTER TABLE t ADD c2 INT, MODIFY c1 INT NOT NULL; +ALTER TABLE t ADD c2 INT; +SELECT * FROM t ORDER BY id; +DROP TABLE t; diff --git a/storage/duckdb/mysql-test/duckdb/t/pushdown_rewrite.test b/storage/duckdb/mysql-test/duckdb/t/pushdown_rewrite.test index bfc8cbf47506e..d01a7e29a9e5a 100644 --- a/storage/duckdb/mysql-test/duckdb/t/pushdown_rewrite.test +++ b/storage/duckdb/mysql-test/duckdb/t/pushdown_rewrite.test @@ -1,4 +1,5 @@ --source ../include/have_duckdb.inc +--source include/not_msan.inc --disable_warnings DROP DATABASE IF EXISTS pushdown_rewrite; diff --git a/storage/duckdb/mysql-test/duckdb/t/rename_duckdb_table.test b/storage/duckdb/mysql-test/duckdb/t/rename_duckdb_table.test index 6ea744b0fd0fb..2511b3b41c1ef 100644 --- a/storage/duckdb/mysql-test/duckdb/t/rename_duckdb_table.test +++ b/storage/duckdb/mysql-test/duckdb/t/rename_duckdb_table.test @@ -1,4 +1,5 @@ --source ../include/have_duckdb.inc +--source include/not_msan.inc --echo # --echo # 1) Prepare diff --git a/storage/duckdb/mysql-test/duckdb/t/run_in_duckdb_access.test b/storage/duckdb/mysql-test/duckdb/t/run_in_duckdb_access.test index 268b9858205c5..1e7afdabdfa8e 100644 --- a/storage/duckdb/mysql-test/duckdb/t/run_in_duckdb_access.test +++ b/storage/duckdb/mysql-test/duckdb/t/run_in_duckdb_access.test @@ -1,4 +1,5 @@ --source ../include/have_duckdb.inc +--source include/not_msan.inc SET @saved_allow_run_in_duckdb = @@GLOBAL.duckdb_allow_run_in_duckdb; # The suite enables duckdb_allow_run_in_duckdb globally (suite.opt), and MTR may diff --git a/storage/duckdb/mysql-test/duckdb/t/space_reclaim.test b/storage/duckdb/mysql-test/duckdb/t/space_reclaim.test index f23b82603fd38..2748253d487c8 100644 --- a/storage/duckdb/mysql-test/duckdb/t/space_reclaim.test +++ b/storage/duckdb/mysql-test/duckdb/t/space_reclaim.test @@ -1,4 +1,5 @@ --source ../include/have_duckdb.inc +--source include/not_msan.inc --source include/have_sequence.inc --echo # diff --git a/storage/duckdb/mysql-test/duckdb/t/streaming_select.test b/storage/duckdb/mysql-test/duckdb/t/streaming_select.test index bf4e0d8fffc36..4999fa997d58a 100644 --- a/storage/duckdb/mysql-test/duckdb/t/streaming_select.test +++ b/storage/duckdb/mysql-test/duckdb/t/streaming_select.test @@ -1,3 +1,4 @@ +--source include/not_msan.inc --source include/have_sequence.inc --disable_warnings diff --git a/storage/duckdb/mysql-test/duckdb/t/supported_copy_ddl.test b/storage/duckdb/mysql-test/duckdb/t/supported_copy_ddl.test index 97069c1b09f4a..ad37d55159bb5 100644 --- a/storage/duckdb/mysql-test/duckdb/t/supported_copy_ddl.test +++ b/storage/duckdb/mysql-test/duckdb/t/supported_copy_ddl.test @@ -1,4 +1,5 @@ --source ../include/have_duckdb.inc +--source include/not_msan.inc # Test for DDL which are supported by DuckDB using COPY algorithm --echo # --echo # RENAME TABLE WITH DIFFERENT DATABASES diff --git a/storage/duckdb/mysql-test/duckdb/t/system_timezone.test b/storage/duckdb/mysql-test/duckdb/t/system_timezone.test index 59aa9d106a51c..1d034ce0d6e6d 100644 --- a/storage/duckdb/mysql-test/duckdb/t/system_timezone.test +++ b/storage/duckdb/mysql-test/duckdb/t/system_timezone.test @@ -1,3 +1,4 @@ +--source include/not_msan.inc --source ../include/have_mysqld_safe.inc # 1) Set valiables to be used in parameters of mysqld_safe. diff --git a/storage/duckdb/mysql-test/duckdb/t/transaction.test b/storage/duckdb/mysql-test/duckdb/t/transaction.test index 7f080ce7456ca..84688d7a37981 100644 --- a/storage/duckdb/mysql-test/duckdb/t/transaction.test +++ b/storage/duckdb/mysql-test/duckdb/t/transaction.test @@ -1,3 +1,4 @@ +--source include/not_msan.inc --echo # --echo # DuckDB transaction tests --echo # diff --git a/storage/duckdb/mysql-test/duckdb/t/truncate_and_maintenance_duckdb_table.test b/storage/duckdb/mysql-test/duckdb/t/truncate_and_maintenance_duckdb_table.test index 3c2108ff356ca..d3f9ee37e4d43 100644 --- a/storage/duckdb/mysql-test/duckdb/t/truncate_and_maintenance_duckdb_table.test +++ b/storage/duckdb/mysql-test/duckdb/t/truncate_and_maintenance_duckdb_table.test @@ -1,3 +1,4 @@ +--source include/not_msan.inc --echo # --echo # 1) PREPARE --echo # diff --git a/storage/duckdb/patches/duckdb-pr24061-setval.diff b/storage/duckdb/patches/duckdb-pr24061-setval.diff new file mode 100644 index 0000000000000..68965fa35041e --- /dev/null +++ b/storage/duckdb/patches/duckdb-pr24061-setval.diff @@ -0,0 +1,179 @@ +diff --git a/src/catalog/catalog_entry/sequence_catalog_entry.cpp b/src/catalog/catalog_entry/sequence_catalog_entry.cpp +index d6a548a267..dbec7a91ac 100644 +--- a/src/catalog/catalog_entry/sequence_catalog_entry.cpp ++++ b/src/catalog/catalog_entry/sequence_catalog_entry.cpp +@@ -81,6 +81,28 @@ int64_t SequenceCatalogEntry::NextValue(DuckTransaction &transaction) { + return result; + } + ++int64_t SequenceCatalogEntry::SetValue(DuckTransaction &transaction, int64_t value, bool is_called) { ++ { ++ lock_guard seqlock(lock); ++ if (value < data.min_value || value > data.max_value) { ++ throw SequenceException("setval: value %lld is out of bounds for sequence \"%s\" (%lld..%lld)", value, name, ++ data.min_value, data.max_value); ++ } ++ ++ data.counter = value; ++ if (!is_called) { ++ data.usage_count++; ++ if (!temporary) { ++ transaction.PushSequenceUsage(*this, data); ++ } ++ return value; ++ } ++ } ++ ++ // is_called: behave as if nextval() was just invoked and returned `value`. ++ return NextValue(transaction); ++} ++ + void SequenceCatalogEntry::ReplayValue(uint64_t v_usage_count, int64_t v_counter) { + if (v_usage_count > data.usage_count) { + data.usage_count = v_usage_count; +diff --git a/src/function/function_list.cpp b/src/function/function_list.cpp +index 4e269f0604..58bf369e01 100644 +--- a/src/function/function_list.cpp ++++ b/src/function/function_list.cpp +@@ -153,6 +153,7 @@ static const StaticFunctionDefinition function[] = { + DUCKDB_SCALAR_FUNCTION_SET_ALIAS(RegexpSplitToArrayFun), + DUCKDB_SCALAR_FUNCTION(RemapStructFun), + DUCKDB_SCALAR_FUNCTION(RowFun), ++ DUCKDB_SCALAR_FUNCTION_SET(SetvalFun), + DUCKDB_SCALAR_FUNCTION_SET(SHA1Fun), + DUCKDB_SCALAR_FUNCTION_SET(SHA256Fun), + DUCKDB_SCALAR_FUNCTION_ALIAS(SplitFun), +diff --git a/src/function/scalar/sequence/nextval.cpp b/src/function/scalar/sequence/nextval.cpp +index 8a53af7bbb..366afcbc3d 100644 +--- a/src/function/scalar/sequence/nextval.cpp ++++ b/src/function/scalar/sequence/nextval.cpp +@@ -20,17 +20,24 @@ namespace duckdb { + namespace { + + struct CurrentSequenceValueOperator { +- static int64_t Operation(DuckTransaction &, SequenceCatalogEntry &seq) { ++ static int64_t Operation(DuckTransaction &, SequenceCatalogEntry &seq, const int64_t, const bool) { + return seq.CurrentValue(); + } + }; + + struct NextSequenceValueOperator { +- static int64_t Operation(DuckTransaction &transaction, SequenceCatalogEntry &seq) { ++ static int64_t Operation(DuckTransaction &transaction, SequenceCatalogEntry &seq, const int64_t, const bool) { + return seq.NextValue(transaction); + } + }; + ++struct SetValValueOperator { ++ static int64_t Operation(DuckTransaction &transaction, SequenceCatalogEntry &seq, const int64_t value, ++ const bool is_called) { ++ return seq.SetValue(transaction, value, is_called); ++ } ++}; ++ + SequenceCatalogEntry &BindSequence(Binder &binder, string &catalog, string &schema, const string &name) { + // fetch the sequence from the catalog + Binder::BindSchemaOrCatalog(binder.context, catalog, schema); +@@ -83,10 +90,44 @@ void NextValFunction(DataChunk &args, ExpressionState &state, Vector &result) { + // sequence to use is hard coded + // increment the sequence + result.SetVectorType(VectorType::FLAT_VECTOR); ++ ++ // setval takes the new value (and optionally is_called) as extra arguments ++ UnifiedVectorFormat new_val_data; ++ UnifiedVectorFormat is_called_data; ++ bool has_new_val = false; ++ bool has_is_called = false; ++ if (std::is_same::value) { ++ args.data[1].ToUnifiedFormat(args.size(), new_val_data); ++ has_new_val = true; ++ if (args.ColumnCount() == 3) { ++ args.data[2].ToUnifiedFormat(args.size(), is_called_data); ++ has_is_called = true; ++ } ++ } ++ + auto result_data = FlatVector::GetData(result); ++ auto &result_validity = FlatVector::Validity(result); + for (idx_t i = 0; i < args.size(); i++) { ++ int64_t value = 0; ++ bool is_called = true; ++ if (has_new_val) { ++ auto idx = new_val_data.sel->get_index(i); ++ if (!new_val_data.validity.RowIsValid(idx)) { ++ result_validity.SetInvalid(i); ++ continue; ++ } ++ value = UnifiedVectorFormat::GetData(new_val_data)[idx]; ++ } ++ if (has_is_called) { ++ auto idx = is_called_data.sel->get_index(i); ++ if (!is_called_data.validity.RowIsValid(idx)) { ++ result_validity.SetInvalid(i); ++ continue; ++ } ++ is_called = UnifiedVectorFormat::GetData(is_called_data)[idx]; ++ } + // get the next value from the sequence +- result_data[i] = OP::Operation(lstate.transaction, lstate.sequence); ++ result_data[i] = OP::Operation(lstate.transaction, lstate.sequence, value, is_called); + } + } + +@@ -162,4 +203,25 @@ ScalarFunction CurrvalFun::GetFunction() { + return curr_val; + } + ++ScalarFunctionSet SetvalFun::GetFunctions() { ++ ScalarFunction set_val("setval", {LogicalType::VARCHAR, LogicalType::BIGINT}, LogicalType::BIGINT, ++ NextValFunction, nullptr, nullptr); ++ set_val.SetBindExtendedCallback(NextValBind); ++ set_val.SetSerializeCallback(Serialize); ++ set_val.SetDeserializeCallback(Deserialize); ++ set_val.SetModifiedDatabasesCallback(NextValModifiedDatabases); ++ set_val.SetInitStateCallback(NextValLocalFunction); ++ set_val.SetVolatile(); ++ set_val.SetFallible(); ++ ++ ScalarFunctionSet set_val_set; ++ set_val_set.AddFunction(set_val); ++ ++ // Add an overload that takes an additional boolean parameter ++ set_val.arguments.push_back(LogicalType::BOOLEAN); ++ set_val_set.AddFunction(set_val); ++ ++ return set_val_set; ++} ++ + } // namespace duckdb +diff --git a/src/include/duckdb/catalog/catalog_entry/sequence_catalog_entry.hpp b/src/include/duckdb/catalog/catalog_entry/sequence_catalog_entry.hpp +index ed12b77a1c..19d3737fb8 100644 +--- a/src/include/duckdb/catalog/catalog_entry/sequence_catalog_entry.hpp ++++ b/src/include/duckdb/catalog/catalog_entry/sequence_catalog_entry.hpp +@@ -61,6 +61,7 @@ public: + SequenceData GetData() const; + int64_t CurrentValue(); + int64_t NextValue(DuckTransaction &transaction); ++ int64_t SetValue(DuckTransaction &transaction, int64_t value, bool is_called); + void ReplayValue(uint64_t usage_count, int64_t counter); + + string ToSQL() const override; +diff --git a/src/include/duckdb/function/scalar/sequence_functions.hpp b/src/include/duckdb/function/scalar/sequence_functions.hpp +index 99766a3326..44b8224006 100644 +--- a/src/include/duckdb/function/scalar/sequence_functions.hpp ++++ b/src/include/duckdb/function/scalar/sequence_functions.hpp +@@ -35,4 +35,14 @@ struct NextvalFun { + static ScalarFunction GetFunction(); + }; + ++struct SetvalFun { ++ static constexpr const char *Name = "setval"; ++ static constexpr const char *Parameters = "sequence_name,value[,is_called]"; ++ static constexpr const char *Description = "Set the value of the sequence to the specified value. If is_called is true, the sequence is called. If is_called is false, the sequence is not called."; ++ static constexpr const char *Example = "setval('my_sequence_name', 100, true)"; ++ static constexpr const char *Categories = ""; ++ ++ static ScalarFunctionSet GetFunctions(); ++}; ++ + } // namespace duckdb diff --git a/storage/duckdb/runtime/delta_appender.cc b/storage/duckdb/runtime/delta_appender.cc index 3d6b91623b0b3..46c4259b64915 100644 --- a/storage/duckdb/runtime/delta_appender.cc +++ b/storage/duckdb/runtime/delta_appender.cc @@ -17,8 +17,9 @@ Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1335 USA */ +#define MYSQL_SERVER 1 + #include -#include "sql_class.h" #include "log.h" #undef UNKNOWN @@ -29,7 +30,6 @@ #include "ddl_convertor.h" #include "duckdb_timezone.h" #include "duckdb_handler_errors.h" -#include "tztime.h" #include "my_decimal.h" #include "duckdb/common/hugeint.hpp" @@ -214,15 +214,16 @@ bool DeltaAppender::Initialize(TABLE *table) m_tmp_table_name= buf_table_name(m_schema_name, m_table_name); std::stringstream ss; - ss << "CREATE TEMPORARY TABLE IF NOT EXISTS main.\"" << m_tmp_table_name - << "\" AS FROM \"" << m_schema_name << "\".\"" << m_table_name - << "\" LIMIT 0;"; - ss << "ALTER TABLE main.\"" << m_tmp_table_name - << "\" ADD COLUMN \"#mdb_delete_flag\" BOOL;"; - ss << "ALTER TABLE main.\"" << m_tmp_table_name - << "\" ADD COLUMN \"#mdb_row_no\" INT;"; - ss << "ALTER TABLE main.\"" << m_tmp_table_name - << "\" ADD COLUMN \"#mdb_trx_no\" INT;"; + ss << "CREATE TEMPORARY TABLE IF NOT EXISTS main." + << quote_duckdb_identifier(m_tmp_table_name) << " AS FROM " + << quote_duckdb_identifier(m_schema_name) << "." + << quote_duckdb_identifier(m_table_name) << " LIMIT 0;"; + ss << "ALTER TABLE main." << quote_duckdb_identifier(m_tmp_table_name) + << " ADD COLUMN \"#mdb_delete_flag\" BOOL;"; + ss << "ALTER TABLE main." << quote_duckdb_identifier(m_tmp_table_name) + << " ADD COLUMN \"#mdb_row_no\" INT;"; + ss << "ALTER TABLE main." << quote_duckdb_identifier(m_tmp_table_name) + << " ADD COLUMN \"#mdb_trx_no\" INT;"; auto ret= myduck::duckdb_query(*m_con, ss.str()); if (ret->HasError()) @@ -249,9 +250,8 @@ bool DeltaAppender::Initialize(TABLE *table) { if (i) m_pk_list+= ", "; - m_pk_list+= "\""; - m_pk_list+= key_part->field->field_name.str; - m_pk_list+= "\""; + m_pk_list+= quote_duckdb_identifier(key_part->field->field_name.str, + key_part->field->field_name.length); bitmap_set_bit(&m_pk_bitmap, key_part->field->field_index); } @@ -259,9 +259,8 @@ bool DeltaAppender::Initialize(TABLE *table) { if (i) m_col_list+= ", "; - m_col_list+= "\""; - m_col_list+= table->field[i]->field_name.str; - m_col_list+= "\""; + m_col_list+= quote_duckdb_identifier(table->field[i]->field_name.str, + table->field[i]->field_name.length); } } else @@ -479,8 +478,8 @@ static void appendSelectQuery(std::stringstream &ss, ss << "SELECT UNNEST(r) FROM (SELECT LAST(ROW(" << select_list << ") ORDER BY \"#mdb_row_no\") AS r, " "LAST(\"#mdb_delete_flag\" ORDER BY \"#mdb_row_no\") AS " - "\"#mdb_delete_flag\" FROM main.\"" - << table_name << "\" GROUP BY " << pk_list << ")"; + "\"#mdb_delete_flag\" FROM main." + << quote_duckdb_identifier(table_name) << " GROUP BY " << pk_list << ")"; if (!delete_flag) ss << " WHERE \"#mdb_delete_flag\" = " << delete_flag; } @@ -488,20 +487,21 @@ static void appendSelectQuery(std::stringstream &ss, void DeltaAppender::generateQuery(std::stringstream &ss, bool delete_flag) { ss.str(""); - ss << "USE \"" << m_schema_name << "\"; "; + ss << "USE " << quote_duckdb_identifier(m_schema_name) << "; "; if (!delete_flag) { - ss << "INSERT INTO \"" << m_schema_name << "\".\"" << m_table_name - << "\" "; + ss << "INSERT INTO " << quote_duckdb_identifier(m_schema_name) << "." + << quote_duckdb_identifier(m_table_name) << " "; appendSelectQuery(ss, m_col_list, m_pk_list, m_tmp_table_name, delete_flag); ss << ";"; } else { - ss << "DELETE FROM \"" << m_schema_name << "\".\"" << m_table_name - << "\" WHERE (" << m_pk_list << ") IN ("; + ss << "DELETE FROM " << quote_duckdb_identifier(m_schema_name) << "." + << quote_duckdb_identifier(m_table_name) << " WHERE (" << m_pk_list + << ") IN ("; appendSelectQuery(ss, m_pk_list, m_pk_list, m_tmp_table_name, delete_flag); ss << ");"; } @@ -532,7 +532,7 @@ bool DeltaAppender::flush(bool idempotent_flag) } ss.str(""); - ss << "DROP TABLE main.\"" << m_tmp_table_name << "\""; + ss << "DROP TABLE main." << quote_duckdb_identifier(m_tmp_table_name); auto ret= myduck::duckdb_query(*m_con, ss.str()); if (ret->HasError()) return true; @@ -553,7 +553,8 @@ void DeltaAppender::cleanup() { my_bitmap_free(&m_pk_bitmap); std::stringstream ss; - ss << "DROP TABLE IF EXISTS main.\"" << m_tmp_table_name << "\";"; + ss << "DROP TABLE IF EXISTS main." << quote_duckdb_identifier(m_tmp_table_name) + << ";"; myduck::duckdb_query(*m_con, ss.str()); } } diff --git a/storage/duckdb/runtime/duckdb_context.cc b/storage/duckdb/runtime/duckdb_context.cc index 9d1f05557c216..03d8da01ddc49 100644 --- a/storage/duckdb/runtime/duckdb_context.cc +++ b/storage/duckdb/runtime/duckdb_context.cc @@ -65,8 +65,9 @@ void DuckdbThdContext::config_duckdb_env(const std::string &schema) if (schema.empty() || schema == m_current_schema) return; - std::string sql1= "CREATE SCHEMA IF NOT EXISTS \"" + schema + "\""; - std::string sql2= "USE \"" + schema + "\""; + std::string sql1= "CREATE SCHEMA IF NOT EXISTS " + + quote_duckdb_identifier(schema); + std::string sql2= "USE " + quote_duckdb_identifier(schema); m_current_schema= schema; for (auto &sql : {sql1, sql2}) diff --git a/storage/duckdb/runtime/duckdb_query.cc b/storage/duckdb/runtime/duckdb_query.cc index 9c177679a01f9..fc0c39a171597 100644 --- a/storage/duckdb/runtime/duckdb_query.cc +++ b/storage/duckdb/runtime/duckdb_query.cc @@ -26,21 +26,190 @@ #include "duckdb_query.h" #include "duckdb/common/exception.hpp" +#include "duckdb/main/pending_query_result.hpp" #include "duckdb_context.h" #include "duckdb_manager.h" #include "duckdb_log.h" +#include + extern handlerton *duckdb_hton; namespace myduck { +SqlRegionType scan_sql_region(const std::string &sql, size_t start, + bool backslash_escapes, size_t &end) +{ + end= start; + if (start >= sql.size()) + return SqlRegionType::NONE; + + char c= sql[start]; + if (c == '/' && start + 1 < sql.size() && sql[start + 1] == '*') + { + size_t close= sql.find("*/", start + 2); + if (close == std::string::npos) + { + end= sql.size(); + return SqlRegionType::UNTERMINATED; + } + end= close + 2; + return SqlRegionType::COMMENT; + } + + if (c == '#' || + (c == '-' && start + 1 < sql.size() && sql[start + 1] == '-' && + (start + 2 == sql.size() || + isspace(static_cast(sql[start + 2]))))) + { + size_t newline= sql.find('\n', start + (c == '#' ? 1 : 2)); + end= newline == std::string::npos ? sql.size() : newline + 1; + return SqlRegionType::COMMENT; + } + + if (c != '\'' && c != '"' && c != '`') + return SqlRegionType::NONE; + + for (size_t i= start + 1; i < sql.size(); i++) + { + if (sql[i] == '\\' && backslash_escapes && i + 1 < sql.size()) + { + i++; + continue; + } + if (sql[i] != c) + continue; + if (i + 1 < sql.size() && sql[i + 1] == c) + { + i++; + continue; + } + end= i + 1; + return SqlRegionType::QUOTED; + } + + end= sql.size(); + return SqlRegionType::UNTERMINATED; +} + +bool mariadb_query_has_unsafe_quote_escape(THD *thd, const char *query, + size_t length) +{ + if (!thd->backslash_escapes() || length == 0) + return false; + + const std::string sql(query, length); + const bool ansi_quotes= thd->variables.sql_mode & MODE_ANSI_QUOTES; + for (size_t i= 0; i < sql.size();) + { + size_t duckdb_end; + SqlRegionType duckdb_region= scan_sql_region(sql, i, false, duckdb_end); + if (duckdb_region == SqlRegionType::COMMENT) + { + i= duckdb_end; + continue; + } + + const bool string_literal= + sql[i] == '\'' || (sql[i] == '"' && !ansi_quotes); + if (string_literal) + { + size_t mariadb_end; + SqlRegionType mariadb_region= + scan_sql_region(sql, i, true, mariadb_end); + if (mariadb_region != duckdb_region || mariadb_end != duckdb_end) + return true; + i= mariadb_end; + continue; + } + + if (duckdb_region == SqlRegionType::QUOTED) + i= duckdb_end; + else if (duckdb_region == SqlRegionType::UNTERMINATED) + return true; + else + i++; + } + return false; +} + +/* + Convert forwarded MariaDB SQL (the raw thd->query() text, plus any + Item::print() fragments) from backtick-quoted identifiers into DuckDB SQL + (double-quoted identifiers). + + MariaDB delimits identifiers with backticks and doubles an embedded backtick; + DuckDB delimits with double quotes and doubles an embedded double quote. A + naive character-by-character swap breaks identifiers that contain a double + quote (MDEV-40653) and also corrupts backticks that appear inside string + literals. Walk the string instead: copy string literals and already + double-quoted identifiers verbatim, and rewrite only backtick-delimited + identifiers, escaping any embedded double quote. +*/ static std::string backticks_to_double_quotes(const std::string &sql) { - std::string out(sql); - for (auto &ch : out) - if (ch == '`') - ch= '"'; + std::string out; + out.reserve(sql.size()); + const size_t n= sql.size(); + size_t i= 0; + + while (i < n) + { + char c= sql[i]; + size_t end; + SqlRegionType region= scan_sql_region(sql, i, false, end); + + if (region == SqlRegionType::COMMENT || + region == SqlRegionType::UNTERMINATED) + { + out.append(sql, i, end - i); + i= end; + continue; + } + + /* Single-quoted string literal: copy verbatim ('' and \' escapes). */ + if (region == SqlRegionType::QUOTED && c == '\'') + { + out.append(sql, i, end - i); + i= end; + continue; + } + + /* Already double-quoted identifier: copy verbatim ("" escape). */ + if (region == SqlRegionType::QUOTED && c == '"') + { + out.append(sql, i, end - i); + i= end; + continue; + } + + /* Backtick identifier: rewrite as a double-quoted identifier. */ + if (region == SqlRegionType::QUOTED && c == '`') + { + out.push_back('"'); + for (i++; i + 1 < end; i++) + { + char d= sql[i]; + if (d == '`' && i + 2 < end && sql[i + 1] == '`') + { + out.push_back('`'); + i++; + continue; + } + if (d == '"') + out.push_back('"'); /* escape " inside a DuckDB identifier */ + out.push_back(d); + } + out.push_back('"'); + i= end; + continue; + } + + out.push_back(c); + i++; + } + return out; } @@ -77,8 +246,9 @@ duckdb_query(duckdb::Connection &connection, const std::string &query) } } -duckdb::unique_ptr -duckdb_stream_query(duckdb::Connection &connection, const std::string &query) +static duckdb::unique_ptr +duckdb_pending_query(duckdb::Connection &connection, const std::string &query, + duckdb::QueryResultOutputType output_type) { const std::string q= backticks_to_double_quotes(query); @@ -87,7 +257,13 @@ duckdb_stream_query(duckdb::Connection &connection, const std::string &query) try { - auto res= connection.SendQuery(q, duckdb::QueryResultOutputType::ALLOW_STREAMING); + auto pending= connection.PendingQuery(q, output_type); + duckdb::unique_ptr res; + if (pending->HasError()) + res= duckdb::make_uniq( + pending->GetErrorObject()); + else + res= pending->Execute(); if ((myduck::duckdb_log_options & LOG_DUCKDB_QUERY_RESULT) && res->HasError()) @@ -106,6 +282,24 @@ duckdb_stream_query(duckdb::Connection &connection, const std::string &query) } } +static duckdb::unique_ptr +duckdb_query_single(duckdb::Connection &connection, const std::string &query) +{ + auto res= duckdb_pending_query( + connection, query, duckdb::QueryResultOutputType::FORCE_MATERIALIZED); + DBUG_ASSERT(res->type == duckdb::QueryResultType::MATERIALIZED_RESULT); + return duckdb::unique_ptr_cast( + std::move(res)); +} + +duckdb::unique_ptr +duckdb_stream_query(duckdb::Connection &connection, const std::string &query) +{ + return duckdb_pending_query( + connection, query, duckdb::QueryResultOutputType::ALLOW_STREAMING); +} + static std::string get_thd_schema(THD *thd) { if (thd->db.str && thd->db.length > 0) @@ -116,6 +310,10 @@ static std::string get_thd_schema(THD *thd) duckdb::unique_ptr duckdb_query(THD *thd, const std::string &query, bool need_config) { + if (mariadb_query_has_unsafe_quote_escape(thd, query.data(), query.size())) + return duckdb::make_uniq( + duckdb::ErrorData("Unsafe MariaDB backslash quote escape in forwarded SQL")); + auto *ctx= static_cast(thd_get_ha_data(thd, duckdb_hton)); if (!ctx) @@ -130,12 +328,16 @@ duckdb_query(THD *thd, const std::string &query, bool need_config) ctx->config_duckdb_session(thd); } - return duckdb_query(ctx->get_connection(), query); + return duckdb_query_single(ctx->get_connection(), query); } duckdb::unique_ptr duckdb_stream_query(THD *thd, const std::string &query, bool need_config) { + if (mariadb_query_has_unsafe_quote_escape(thd, query.data(), query.size())) + return duckdb::make_uniq( + duckdb::ErrorData("Unsafe MariaDB backslash quote escape in forwarded SQL")); + auto *ctx= static_cast(thd_get_ha_data(thd, duckdb_hton)); if (!ctx) diff --git a/storage/duckdb/runtime/duckdb_query.h b/storage/duckdb/runtime/duckdb_query.h index 21f8cd12f9491..6ec9db6da7897 100644 --- a/storage/duckdb/runtime/duckdb_query.h +++ b/storage/duckdb/runtime/duckdb_query.h @@ -19,6 +19,7 @@ #pragma once +#include #include #include #include @@ -31,6 +32,20 @@ class THD; namespace myduck { +enum class SqlRegionType +{ + NONE, + COMMENT, + QUOTED, + UNTERMINATED +}; + +SqlRegionType scan_sql_region(const std::string &sql, size_t start, + bool backslash_escapes, size_t &end); + +bool mariadb_query_has_unsafe_quote_escape(THD *thd, const char *query, + size_t length); + duckdb::unique_ptr duckdb_query(duckdb::Connection &connection, const std::string &query); diff --git a/storage/duckdb/runtime/fiber_scan.cc b/storage/duckdb/runtime/fiber_scan.cc index 6ed90b22f29ce..6cac271daec8c 100644 --- a/storage/duckdb/runtime/fiber_scan.cc +++ b/storage/duckdb/runtime/fiber_scan.cc @@ -163,7 +163,13 @@ int FiberScanState::init(TABLE *tbl, buffer.Initialize(duckdb::Allocator::DefaultAllocator(), types); - if (fiber_context_init(&ctx, FIBER_STACK_SIZE)) + size_t fiber_stack_size= FIBER_STACK_SIZE; +#if defined(FIBER_ASAN_ENABLED) || defined(WITH_UBSAN) + size_t configured_stack_size= static_cast(my_thread_stack_size); + if (configured_stack_size > fiber_stack_size) + fiber_stack_size= configured_stack_size; +#endif + if (fiber_context_init(&ctx, fiber_stack_size)) return 1; fiber_thd= create_background_thd(); diff --git a/storage/heap/_check.c b/storage/heap/_check.c index 1a640fa13da86..3663eada0b268 100644 --- a/storage/heap/_check.c +++ b/storage/heap/_check.c @@ -17,6 +17,7 @@ /* Check that heap-structure is ok */ #include "heapdef.h" +#include "my_compare.h" static int check_one_key(HP_KEYDEF *, uint, ulong, ulong, my_bool); static int check_one_rb_key(const HP_INFO *, uint, ulong, my_bool); diff --git a/storage/heap/heapdef.h b/storage/heap/heapdef.h index e51fe88d8e2b7..1880a9839cf41 100644 --- a/storage/heap/heapdef.h +++ b/storage/heap/heapdef.h @@ -53,7 +53,7 @@ typedef struct st_hp_hash_info } HASH_INFO; typedef struct { - HA_KEYSEG *keyseg; + struct st_HA_KEYSEG *keyseg; uint key_length; uint search_flag; } heap_rb_param; diff --git a/storage/heap/hp_create.c b/storage/heap/hp_create.c index 20a099d415951..079838b618489 100644 --- a/storage/heap/hp_create.c +++ b/storage/heap/hp_create.c @@ -15,6 +15,7 @@ Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1335 USA */ #include "heapdef.h" +#include "my_compare.h" #include static int keys_compare(void *heap_rb, const void *key1, const void *key2); diff --git a/storage/heap/hp_hash.c b/storage/heap/hp_hash.c index a221a2098fb67..9e33e69fa01e7 100644 --- a/storage/heap/hp_hash.c +++ b/storage/heap/hp_hash.c @@ -18,6 +18,7 @@ /* The hash functions used for saveing keys */ #include "heapdef.h" +#include "my_compare.h" #include diff --git a/storage/heap/hp_rfirst.c b/storage/heap/hp_rfirst.c index 60596a2c650fd..1b1610ac52d6e 100644 --- a/storage/heap/hp_rfirst.c +++ b/storage/heap/hp_rfirst.c @@ -34,8 +34,7 @@ int heap_rfirst(HP_INFO *info, uchar *record, int inx) if ((pos = tree_search_edge(&keyinfo->rb_tree, info->parents, &info->last_pos, offsetof(TREE_ELEMENT, left)))) { - memcpy(&pos, pos + (*keyinfo->get_key_length)(keyinfo, pos), - sizeof(uchar*)); + memcpy(&pos, pos + keyinfo->get_key_length(keyinfo, pos), sizeof(uchar*)); info->current_ptr = pos; memcpy(record, pos, (size_t)share->reclength); /* diff --git a/storage/heap/hp_rkey.c b/storage/heap/hp_rkey.c index 2d9fae4c52097..3e4b4dee6ed85 100644 --- a/storage/heap/hp_rkey.c +++ b/storage/heap/hp_rkey.c @@ -55,7 +55,10 @@ int heap_rkey(HP_INFO *info, uchar *record, int inx, const uchar *key, info->update= HA_STATE_NO_KEY; DBUG_RETURN(my_errno= HA_ERR_KEY_NOT_FOUND); } - memcpy(&pos, pos + (*keyinfo->get_key_length)(keyinfo, pos), sizeof(uchar*)); + info->lastkey_len= keyinfo->get_key_length(keyinfo, pos); + if ((keyinfo->flag & (HA_NOSAME | HA_NULL_PART_KEY)) != HA_NOSAME) + memcpy(info->lastkey, pos, info->lastkey_len + sizeof(uchar*)); + memcpy(&pos, pos + info->lastkey_len, sizeof(uchar*)); info->current_ptr= pos; } else diff --git a/storage/heap/hp_rlast.c b/storage/heap/hp_rlast.c index ed9c3499d5e84..c95d8af5d4786 100644 --- a/storage/heap/hp_rlast.c +++ b/storage/heap/hp_rlast.c @@ -34,8 +34,7 @@ int heap_rlast(HP_INFO *info, uchar *record, int inx) if ((pos = tree_search_edge(&keyinfo->rb_tree, info->parents, &info->last_pos, offsetof(TREE_ELEMENT, right)))) { - memcpy(&pos, pos + (*keyinfo->get_key_length)(keyinfo, pos), - sizeof(uchar*)); + memcpy(&pos, pos + keyinfo->get_key_length(keyinfo, pos), sizeof(uchar*)); info->current_ptr = pos; memcpy(record, pos, (size_t)share->reclength); info->update = HA_STATE_AKTIV; diff --git a/storage/heap/hp_rnext.c b/storage/heap/hp_rnext.c index ac21ed83da271..1f04f94d478ec 100644 --- a/storage/heap/hp_rnext.c +++ b/storage/heap/hp_rnext.c @@ -53,9 +53,9 @@ int heap_rnext(HP_INFO *info, uchar *record) or heap_rfirst(). As last key position (info->last_pos) is available, we only need to climb the tree using tree_search_next(). */ - pos = tree_search_next(&keyinfo->rb_tree, &info->last_pos, - offsetof(TREE_ELEMENT, left), - offsetof(TREE_ELEMENT, right)); + pos= tree_search_next(&keyinfo->rb_tree, &info->last_pos, + offsetof(TREE_ELEMENT, left), + offsetof(TREE_ELEMENT, right)); } else if (!info->lastkey_len) { @@ -84,17 +84,19 @@ int heap_rnext(HP_INFO *info, uchar *record) */ custom_arg.keyseg = keyinfo->seg; custom_arg.key_length = info->lastkey_len; - custom_arg.search_flag = SEARCH_SAME | SEARCH_FIND; + custom_arg.search_flag = SEARCH_BIGGER; info->last_find_flag= HA_READ_KEY_OR_NEXT; - pos = tree_search_key(&keyinfo->rb_tree, info->lastkey, info->parents, + pos= tree_search_key(&keyinfo->rb_tree, info->lastkey, info->parents, &info->last_pos, info->last_find_flag, &custom_arg); info->key_version= info->s->key_version; } if (pos) { - memcpy(&pos, pos + (*keyinfo->get_key_length)(keyinfo, pos), - sizeof(uchar*)); - info->current_ptr = pos; + info->lastkey_len= keyinfo->get_key_length(keyinfo, pos); + if ((keyinfo->flag & (HA_NOSAME | HA_NULL_PART_KEY)) != HA_NOSAME) + memcpy(info->lastkey, pos, info->lastkey_len + sizeof(uchar*)); + memcpy(&pos, pos + info->lastkey_len, sizeof(uchar*)); + info->current_ptr= pos; } else { diff --git a/storage/heap/hp_rprev.c b/storage/heap/hp_rprev.c index cc81d179570aa..eb816ed676a42 100644 --- a/storage/heap/hp_rprev.c +++ b/storage/heap/hp_rprev.c @@ -54,7 +54,7 @@ int heap_rprev(HP_INFO *info, uchar *record) { custom_arg.keyseg = keyinfo->seg; custom_arg.key_length = keyinfo->length; - custom_arg.search_flag = SEARCH_SAME; + custom_arg.search_flag = SEARCH_SMALLER; info->last_find_flag= HA_READ_KEY_OR_PREV; pos = tree_search_key(&keyinfo->rb_tree, info->lastkey, info->parents, &info->last_pos, info->last_find_flag, &custom_arg); @@ -62,8 +62,10 @@ int heap_rprev(HP_INFO *info, uchar *record) } if (pos) { - memcpy(&pos, pos + (*keyinfo->get_key_length)(keyinfo, pos), - sizeof(uchar*)); + info->lastkey_len= keyinfo->get_key_length(keyinfo, pos); + if ((keyinfo->flag & (HA_NOSAME | HA_NULL_PART_KEY)) != HA_NOSAME) + memcpy(info->lastkey, pos, info->lastkey_len + sizeof(uchar*)); + memcpy(&pos, pos + info->lastkey_len, sizeof(uchar*)); info->current_ptr = pos; } else diff --git a/storage/heap/hp_test1.c b/storage/heap/hp_test1.c index 9f87ea6b33a4f..8cc62362b626f 100644 --- a/storage/heap/hp_test1.c +++ b/storage/heap/hp_test1.c @@ -23,6 +23,7 @@ #include #include #include "heap.h" +#include "my_compare.h" static int get_options(int argc, char *argv[]); diff --git a/storage/heap/hp_test2.c b/storage/heap/hp_test2.c index 2ca1dd77fbb7e..a5b03354b9fb8 100644 --- a/storage/heap/hp_test2.c +++ b/storage/heap/hp_test2.c @@ -17,6 +17,7 @@ /* Test av isam-databas: stor test */ #include "heapdef.h" /* Because of hp_find_block */ +#include "my_compare.h" #include #define MAX_RECORDS 100000 diff --git a/storage/heap/hp_update.c b/storage/heap/hp_update.c index ad56ca979deb6..c99ac370ac8b8 100644 --- a/storage/heap/hp_update.c +++ b/storage/heap/hp_update.c @@ -47,6 +47,7 @@ int heap_update(HP_INFO *info, const uchar *old, const uchar *heap_new) if ((*keydef->delete_key)(info, keydef, old, pos, keydef == p_lastinx) || (*keydef->write_key)(info, keydef, heap_new, pos)) goto err; + key_changed= 1; if (share->auto_key == (uint) (keydef - share->keydef + 1)) auto_key_changed= 1; } diff --git a/storage/innobase/buf/buf0buf.cc b/storage/innobase/buf/buf0buf.cc index 5d9429bd0cc4c..45b40a7e68d3a 100644 --- a/storage/innobase/buf/buf0buf.cc +++ b/storage/innobase/buf/buf0buf.cc @@ -1,7 +1,7 @@ /***************************************************************************** Copyright (c) 1995, 2018, Oracle and/or its affiliates. All Rights Reserved. -Copyright (c) 2013, 2023, MariaDB Corporation. +Copyright (c) 2013, 2026, MariaDB plc. This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software @@ -1371,14 +1371,9 @@ bool buf_pool_t::create() noexcept retry: { NUMA_MEMPOLICY_INTERLEAVE_IN_SCOPE; -#ifdef _WIN32 memory_unaligned= my_virtual_mem_reserve(&size); - if (!memory_unaligned) - goto oom; -#else - memory_unaligned= my_large_virtual_alloc(&size); if (memory_unaligned); -# if defined __aarch64__ || defined __riscv || defined __mips__ || defined __loongarch64 +#if defined __aarch64__ || defined __riscv || defined __mips__ || defined __loongarch64 else if (size_in_bytes_max_default != 0 && size_in_bytes_max == size_in_bytes_max_default) { @@ -1399,10 +1394,9 @@ bool buf_pool_t::create() noexcept size_in_bytes_max= std::max(size_t(1ULL << 37), size_in_bytes_requested); goto init; } -# endif +#endif else goto oom; -#endif } const size_t alignment_waste= diff --git a/storage/innobase/dict/dict0stats.cc b/storage/innobase/dict/dict0stats.cc index a03dac7f10af5..ac6b31eb2b030 100644 --- a/storage/innobase/dict/dict0stats.cc +++ b/storage/innobase/dict/dict0stats.cc @@ -335,7 +335,6 @@ static int dtype_sql_name(unsigned mtype, unsigned prtype, unsigned len, return snprintf(name, name_sz, "%s%s%s", Main, Unsigned, Not_null); } -static bool innodb_table_stats_not_found; static bool innodb_index_stats_not_found; static bool innodb_table_stats_not_found_reported; static bool innodb_index_stats_not_found_reported; @@ -366,7 +365,6 @@ dict_table_schema_check( if (innodb_table_stats_not_found_reported) { return DB_STATS_DO_NOT_EXIST; } - innodb_table_stats_not_found = true; innodb_table_stats_not_found_reported = true; } else { ut_ad(req_schema == &index_stats_schema); diff --git a/storage/maria/ma_checkpoint.c b/storage/maria/ma_checkpoint.c index f8db94d8950f8..22a1f05d83019 100644 --- a/storage/maria/ma_checkpoint.c +++ b/storage/maria/ma_checkpoint.c @@ -53,9 +53,6 @@ static PAGECACHE_FILE *dfiles, /**< data files to flush in background */ *dfiles_end; /**< list of data files ends here */ static PAGECACHE_FILE *kfiles, /**< index files to flush in background */ *kfiles_end; /**< list of index files ends here */ -/* those two statistics below could serve in SHOW GLOBAL STATUS */ -static uint checkpoints_total= 0, /**< all checkpoint requests made */ - checkpoints_ok_total= 0; /**< all checkpoints which succeeded */ struct st_filter_param { @@ -354,8 +351,6 @@ static int really_execute_checkpoint(void) my_afree(record_pieces); mysql_mutex_lock(&LOCK_checkpoint); checkpoint_in_progress= CHECKPOINT_NONE; - checkpoints_total++; - checkpoints_ok_total+= !error; mysql_mutex_unlock(&LOCK_checkpoint); DBUG_RETURN(error); } diff --git a/storage/maria/ma_delete.c b/storage/maria/ma_delete.c index 9f313397b35ae..7c688e0bf72ed 100644 --- a/storage/maria/ma_delete.c +++ b/storage/maria/ma_delete.c @@ -1333,6 +1333,9 @@ static int underflow(MARIA_HA *info, MARIA_KEYDEF *keyinfo, @retval # How many chars was removed */ +#if defined(_MSC_VER) && defined(_M_X64) && _MSC_VER >= 1930 && _MSC_VER < 1951 +#pragma optimize("g", off) +#endif static uint remove_key(MARIA_KEYDEF *keyinfo, uint page_flag, uint nod_flag, uchar *keypos, uchar *lastkey, uchar *page_end, my_off_t *next_block, @@ -1473,6 +1476,9 @@ static uint remove_key(MARIA_KEYDEF *keyinfo, uint page_flag, uint nod_flag, DBUG_RETURN((uint) s_length); } /* remove_key */ +#if defined(_MSC_VER) && defined(_M_X64) && _MSC_VER >= 1930 && _MSC_VER < 1951 +#pragma optimize("", on) +#endif /**************************************************************************** Logging of redos diff --git a/storage/mroonga/ha_mroonga.cpp b/storage/mroonga/ha_mroonga.cpp index d71d01c605327..3bb8197e5a105 100644 --- a/storage/mroonga/ha_mroonga.cpp +++ b/storage/mroonga/ha_mroonga.cpp @@ -4288,18 +4288,15 @@ int ha_mroonga::wrapper_open(const char *name, int mode, uint open_options) if (error) DBUG_RETURN(error); - if (!(open_options & HA_OPEN_FOR_REPAIR)) { - error = open_table(name); - if (error) - DBUG_RETURN(error); + error = open_table(name); + if (error && !(open_options & HA_OPEN_FOR_REPAIR)) + goto error_exit; - error = wrapper_open_indexes(name); - if (error) { - grn_obj_unlink(ctx, grn_table); - grn_table = NULL; - DBUG_RETURN(error); - } - } + error = wrapper_open_indexes(name); + if (error && !(open_options & HA_OPEN_FOR_REPAIR)) + goto error_exit; + + error = 0; mrn_init_alloc_root(&mem_root, 1024, 0, MYF(0)); wrap_key_info = mrn_create_key_info_for_table(share, table, &error); @@ -4388,6 +4385,7 @@ int ha_mroonga::wrapper_open(const char *name, int mode, uint open_options) } } +error_exit: if (error) { grn_obj_unlink(ctx, grn_table); @@ -4660,37 +4658,35 @@ int ha_mroonga::storage_open(const char *name, int mode, uint open_options) DBUG_RETURN(error); } - if (!(open_options & HA_OPEN_FOR_REPAIR)) { - error = storage_open_indexes(name); - if (error) { - storage_close_columns(); - grn_obj_unlink(ctx, grn_table); - grn_table = NULL; - DBUG_RETURN(error); - } + error = storage_open_indexes(name); + if (error && !(open_options & HA_OPEN_FOR_REPAIR)) { + storage_close_columns(); + grn_obj_unlink(ctx, grn_table); + grn_table = NULL; + DBUG_RETURN(error); + } - storage_set_keys_in_use(); + storage_set_keys_in_use(); - { - mrn::Lock lock(&mrn_operations_mutex); - mrn::PathMapper mapper(name); - const char *table_name = mapper.table_name(); - size_t table_name_size = strlen(table_name); - if (db->is_broken_table(table_name, table_name_size)) { - GRN_LOG(ctx, GRN_LOG_NOTICE, - "Auto repair is started: <%s>", - name); - error = operations_->repair(table_name, table_name_size); + if (!(open_options & HA_OPEN_FOR_REPAIR)) { + mrn::Lock lock(&mrn_operations_mutex); + mrn::PathMapper mapper(name); + const char *table_name = mapper.table_name(); + size_t table_name_size = strlen(table_name); + if (db->is_broken_table(table_name, table_name_size)) { + GRN_LOG(ctx, GRN_LOG_NOTICE, + "Auto repair is started: <%s>", + name); + error = operations_->repair(table_name, table_name_size); + if (!error) + db->mark_table_repaired(table_name, table_name_size); + if (!share->disable_keys) { if (!error) - db->mark_table_repaired(table_name, table_name_size); - if (!share->disable_keys) { - if (!error) - error = storage_reindex(); - } - GRN_LOG(ctx, GRN_LOG_NOTICE, - "Auto repair is done: <%s>: %s", - name, error == 0 ? "success" : "failure"); + error = storage_reindex(); } + GRN_LOG(ctx, GRN_LOG_NOTICE, + "Auto repair is done: <%s>: %s", + name, error == 0 ? "success" : "failure"); } } @@ -5234,7 +5230,7 @@ void ha_mroonga::storage_set_keys_in_use() if (i == table_share->primary_key) { continue; } - if (!grn_index_tables[i]) { + if (grn_index_tables && !grn_index_tables[i]) { /* disabled */ table_share->keys_in_use.clear_bit(i); DBUG_PRINT("info", ("mroonga: key %u disabled", i)); diff --git a/storage/mroonga/mysql-test/mroonga/storage/r/check_table_broken.result b/storage/mroonga/mysql-test/mroonga/storage/r/check_table_broken.result index 4926a72a77af4..af85e783cd502 100644 --- a/storage/mroonga/mysql-test/mroonga/storage/r/check_table_broken.result +++ b/storage/mroonga/mysql-test/mroonga/storage/r/check_table_broken.result @@ -9,10 +9,17 @@ INSERT INTO diaries VALUES ('Hello'); FLUSH TABLES; CHECK TABLE diaries; Table Op Msg_type Msg_text +check_test.diaries check Error system call error: No such file or directory: failed to open path: check_test.diaries check error Corrupt REPAIR TABLE diaries; Table Op Msg_type Msg_text +check_test.diaries repair Error system call error: No such file or directory: failed to open path: check_test.diaries repair status OK +SELECT * +FROM diaries +WHERE MATCH(title) AGAINST('+Hello' IN BOOLEAN MODE); +title +Hello DROP TABLE diaries; DROP DATABASE check_test; USE test; diff --git a/storage/mroonga/mysql-test/mroonga/storage/r/check_table_innodb.result b/storage/mroonga/mysql-test/mroonga/storage/r/check_table_innodb.result new file mode 100644 index 0000000000000..628a33af7f965 --- /dev/null +++ b/storage/mroonga/mysql-test/mroonga/storage/r/check_table_innodb.result @@ -0,0 +1,13 @@ +# +# MDEV-39556 SIGSEGV in ha_mroonga::storage_set_keys_in_use on SELECT +# +CREATE TABLE t (c INT KEY,c2 GEOMETRY NOT NULL,SPATIAL INDEX idx_sp (c2)) ENGINE=InnoDB; +ALTER TABLE t ENGINE=Mroonga; +CHECK TABLE t; +Table Op Msg_type Msg_text +test.t check status OK +SELECT COUNT(*) > 0 AS r FROM information_schema.STATISTICS; +r +1 +DROP TABLE t; +# End of 10.6 tests diff --git a/storage/mroonga/mysql-test/mroonga/storage/r/repair_table_no_index_file.result b/storage/mroonga/mysql-test/mroonga/storage/r/repair_table_no_index_file.result index 24d427ed2ab72..f58b3916eaf79 100644 --- a/storage/mroonga/mysql-test/mroonga/storage/r/repair_table_no_index_file.result +++ b/storage/mroonga/mysql-test/mroonga/storage/r/repair_table_no_index_file.result @@ -17,6 +17,7 @@ SELECT * FROM diaries WHERE MATCH(body) AGAINST("+starting" IN BOOLEAN MODE); ERROR HY000: system call error: No such file or directory: failed to open path: REPAIR TABLE diaries; Table Op Msg_type Msg_text +repair_test.diaries repair Error system call error: No such file or directory: failed to open path: repair_test.diaries repair status OK SELECT * FROM diaries; id title body diff --git a/storage/mroonga/mysql-test/mroonga/storage/t/check_table_broken.test b/storage/mroonga/mysql-test/mroonga/storage/t/check_table_broken.test index d438a912ae28a..d4666e600559e 100644 --- a/storage/mroonga/mysql-test/mroonga/storage/t/check_table_broken.test +++ b/storage/mroonga/mysql-test/mroonga/storage/t/check_table_broken.test @@ -31,12 +31,16 @@ CREATE TABLE diaries ( INSERT INTO diaries VALUES ('Hello'); --remove_file $MYSQLD_DATADIR/check_test.mrn.000010C - FLUSH TABLES; CHECK TABLE diaries; REPAIR TABLE diaries; + +SELECT * + FROM diaries + WHERE MATCH(title) AGAINST('+Hello' IN BOOLEAN MODE); + DROP TABLE diaries; DROP DATABASE check_test; diff --git a/storage/mroonga/mysql-test/mroonga/storage/t/check_table_innodb.test b/storage/mroonga/mysql-test/mroonga/storage/t/check_table_innodb.test new file mode 100644 index 0000000000000..e34d30ec29d2b --- /dev/null +++ b/storage/mroonga/mysql-test/mroonga/storage/t/check_table_innodb.test @@ -0,0 +1,16 @@ +--source include/have_innodb.inc +--source ../../include/mroonga/have_mroonga.inc + +--echo # +--echo # MDEV-39556 SIGSEGV in ha_mroonga::storage_set_keys_in_use on SELECT +--echo # + +CREATE TABLE t (c INT KEY,c2 GEOMETRY NOT NULL,SPATIAL INDEX idx_sp (c2)) ENGINE=InnoDB; + +ALTER TABLE t ENGINE=Mroonga; +CHECK TABLE t; +SELECT COUNT(*) > 0 AS r FROM information_schema.STATISTICS; + +DROP TABLE t; + +--echo # End of 10.6 tests diff --git a/storage/mroonga/mysql-test/mroonga/storage/t/repair_table_no_index_file.test b/storage/mroonga/mysql-test/mroonga/storage/t/repair_table_no_index_file.test index 0f04bd3e4232e..63624a9d56e4b 100644 --- a/storage/mroonga/mysql-test/mroonga/storage/t/repair_table_no_index_file.test +++ b/storage/mroonga/mysql-test/mroonga/storage/t/repair_table_no_index_file.test @@ -43,6 +43,7 @@ FLUSH TABLES; --error ER_CANT_OPEN_FILE SELECT * FROM diaries WHERE MATCH(body) AGAINST("+starting" IN BOOLEAN MODE); +--replace_result "[io][open] " "" REPAIR TABLE diaries; SELECT * FROM diaries; diff --git a/storage/mroonga/mysql-test/mroonga/wrapper/r/repair_table_no_files.result b/storage/mroonga/mysql-test/mroonga/wrapper/r/repair_table_no_files.result index 8258a03bf42c1..98de2e895188a 100644 --- a/storage/mroonga/mysql-test/mroonga/wrapper/r/repair_table_no_files.result +++ b/storage/mroonga/mysql-test/mroonga/wrapper/r/repair_table_no_files.result @@ -17,6 +17,7 @@ SELECT * FROM diaries WHERE MATCH(body) AGAINST("starting"); ERROR HY000: mroonga: failed to open table: REPAIR TABLE diaries; Table Op Msg_type Msg_text +repair_test.diaries repair Error mroonga: failed to open table: repair_test.diaries repair status OK SELECT * FROM diaries; id title body diff --git a/storage/mroonga/mysql-test/mroonga/wrapper/r/repair_table_no_index_file.result b/storage/mroonga/mysql-test/mroonga/wrapper/r/repair_table_no_index_file.result index cca7aee95de8a..271c086c4b3d6 100644 --- a/storage/mroonga/mysql-test/mroonga/wrapper/r/repair_table_no_index_file.result +++ b/storage/mroonga/mysql-test/mroonga/wrapper/r/repair_table_no_index_file.result @@ -17,6 +17,7 @@ SELECT * FROM diaries WHERE MATCH(body) AGAINST("starting"); ERROR HY000: system call error: No such file or directory: failed to open path: REPAIR TABLE diaries; Table Op Msg_type Msg_text +repair_test.diaries repair Error system call error: No such file or directory: failed to open path: repair_test.diaries repair status OK SELECT * FROM diaries; id title body diff --git a/storage/mroonga/mysql-test/mroonga/wrapper/t/repair_table_no_index_file.test b/storage/mroonga/mysql-test/mroonga/wrapper/t/repair_table_no_index_file.test index 2fefadc254d0e..5061a268d8abf 100644 --- a/storage/mroonga/mysql-test/mroonga/wrapper/t/repair_table_no_index_file.test +++ b/storage/mroonga/mysql-test/mroonga/wrapper/t/repair_table_no_index_file.test @@ -44,6 +44,7 @@ FLUSH TABLES; --error ER_CANT_OPEN_FILE SELECT * FROM diaries WHERE MATCH(body) AGAINST("starting"); +--replace_result "[io][open] " "" REPAIR TABLE diaries; SELECT * FROM diaries; diff --git a/storage/myisam/ft_static.c b/storage/myisam/ft_static.c index 78123cdf261cf..2b84efb767327 100644 --- a/storage/myisam/ft_static.c +++ b/storage/myisam/ft_static.c @@ -1,4 +1,5 @@ /* Copyright (c) 2000-2008 MySQL AB, 2009 Sun Microsystems, Inc. + Copyright (c) 2026, MariaDB plc. Use is subject to license terms. This program is free software; you can redistribute it and/or modify @@ -18,9 +19,9 @@ #include "ftdefs.h" -ulong ft_min_word_len= 4; -ulong ft_max_word_len= HA_FT_MAXCHARLEN; -ulong ft_query_expansion_limit= 5; +READ_ONLY_SYSVAR ulong ft_min_word_len= 4; +READ_ONLY_SYSVAR ulong ft_max_word_len= HA_FT_MAXCHARLEN; +READ_ONLY_SYSVAR ulong ft_query_expansion_limit= 5; const char *ft_boolean_syntax= DEFAULT_FTB_SYNTAX; const HA_KEYSEG ft_keysegs[FT_SEGS]= { @@ -55,7 +56,7 @@ const struct _ft_vft _ft_vft_boolean= { ft_boolean_get_relevance, ft_boolean_reinit_search }; -const char *ft_stopword_file= 0; +READ_ONLY_SYSVAR const char *ft_stopword_file= 0; const char *ft_precompiled_stopwords[]= { #ifdef COMPILE_STOPWORDS_IN diff --git a/storage/myisam/mi_delete.c b/storage/myisam/mi_delete.c index 456b3e488c98e..b7db752be2778 100644 --- a/storage/myisam/mi_delete.c +++ b/storage/myisam/mi_delete.c @@ -766,7 +766,7 @@ static int underflow(register MI_INFO *info, register MI_KEYDEF *keyinfo, returns how many chars was removed or 0 on error */ -#if defined(_MSC_VER) && defined(_M_X64) && _MSC_VER >= 1930 +#if defined(_MSC_VER) && defined(_M_X64) && _MSC_VER >= 1930 && _MSC_VER < 1951 #pragma optimize("g", off) #endif @@ -895,6 +895,6 @@ static uint remove_key(MI_KEYDEF *keyinfo, uint nod_flag, DBUG_RETURN((uint) s_length); } /* remove_key */ -#if defined(_MSC_VER) && defined(_M_X64) && _MSC_VER >= 1930 +#if defined(_MSC_VER) && defined(_M_X64) && _MSC_VER >= 1930 && _MSC_VER < 1951 #pragma optimize("",on) #endif diff --git a/storage/perfschema/pfs.cc b/storage/perfschema/pfs.cc index 0a6b0ecb907e8..31765a56bb279 100644 --- a/storage/perfschema/pfs.cc +++ b/storage/perfschema/pfs.cc @@ -3303,7 +3303,7 @@ pfs_get_thread_file_name_locker_v1(PSI_file_locker_state *state, wait->m_class= klass; wait->m_timer_start= 0; wait->m_timer_end= 0; - wait->m_object_instance_addr= NULL; + wait->m_object_instance_addr= 0; wait->m_weak_file= NULL; wait->m_weak_version= 0; wait->m_event_id= pfs_thread->m_event_id++; @@ -3383,7 +3383,7 @@ pfs_get_thread_file_stream_locker_v1(PSI_file_locker_state *state, wait->m_class= klass; wait->m_timer_start= 0; wait->m_timer_end= 0; - wait->m_object_instance_addr= pfs_file; + wait->m_object_instance_addr= pfs_file->m_identity; wait->m_weak_file= pfs_file; wait->m_weak_version= pfs_file->get_version(); wait->m_event_id= pfs_thread->m_event_id++; @@ -3492,7 +3492,7 @@ pfs_get_thread_file_descriptor_locker_v1(PSI_file_locker_state *state, wait->m_class= klass; wait->m_timer_start= 0; wait->m_timer_end= 0; - wait->m_object_instance_addr= pfs_file; + wait->m_object_instance_addr= pfs_file->m_identity; wait->m_weak_file= pfs_file; wait->m_weak_version= pfs_file->get_version(); wait->m_event_id= pfs_thread->m_event_id++; @@ -4672,7 +4672,7 @@ void pfs_end_file_wait_v1(PSI_file_locker *locker, wait->m_timer_end= timer_end; wait->m_number_of_bytes= bytes; wait->m_end_event_id= thread->m_event_id; - wait->m_object_instance_addr= file; + wait->m_object_instance_addr= (file ? file->m_identity : 0); wait->m_weak_file= file; wait->m_weak_version= (file ? file->get_version() : 0); diff --git a/storage/perfschema/pfs.h b/storage/perfschema/pfs.h index f41299fc4e866..a83ffac9d4251 100644 --- a/storage/perfschema/pfs.h +++ b/storage/perfschema/pfs.h @@ -23,11 +23,15 @@ #ifndef PFS_H #define PFS_H +#include + /** @file storage/perfschema/pfs.h Performance schema instrumentation (declarations). */ +typedef std::uint64_t pfs_identity; + #define HAVE_PSI_1 #include @@ -59,4 +63,3 @@ extern MY_THREAD_LOCAL void* THR_PFS_SBH; // status_by_account #define PSI_COUNT_VOLATILITY 2 #endif - diff --git a/storage/perfschema/pfs_account.cc b/storage/perfschema/pfs_account.cc index 2a87783664908..88038e947e126 100644 --- a/storage/perfschema/pfs_account.cc +++ b/storage/perfschema/pfs_account.cc @@ -180,7 +180,7 @@ find_or_create_account(PFS_thread *thread, lf_hash_search_unpin(pins); - pfs= global_account_container.allocate(& dirty_state); + pfs= global_account_container.allocate(& dirty_state, NULL); if (pfs != NULL) { pfs->m_key= key; diff --git a/storage/perfschema/pfs_buffer_container.cc b/storage/perfschema/pfs_buffer_container.cc index 04f6f5219a336..4b96366c4a948 100644 --- a/storage/perfschema/pfs_buffer_container.cc +++ b/storage/perfschema/pfs_buffer_container.cc @@ -29,6 +29,26 @@ #include "pfs_buffer_container.h" #include "pfs_builtin_memory.h" +pfs_identity make_identity(pfs_container_id container_id, pfs_page_id page_id, + pfs_object_id object_id, + pfs_dirty_state *dirty_state) { + /* See pfs_lock, the version number is stored in the high 30 bits. */ + uint32_t version_32 = (dirty_state->m_version_state & VERSION_MASK) >> 2; + /* Isolate 16 least significant bits from the version number. */ + uint16_t version_16 = version_32 & 0xFFFF; + pfs_identity id; + id = container_id; + id <<= 16; + id += page_id; + id <<= 16; + id += object_id; + id <<= 16; + id += version_16; + return id; +} + +std::atomic global_container_id{1}; + PFS_buffer_default_allocator default_mutex_allocator(& builtin_memory_mutex); PFS_mutex_container global_mutex_container(& default_mutex_allocator); @@ -880,4 +900,3 @@ void PFS_user_allocator::free_array(PFS_user_array *array) PFS_user_allocator user_allocator; PFS_user_container global_user_container(& user_allocator); - diff --git a/storage/perfschema/pfs_buffer_container.h b/storage/perfschema/pfs_buffer_container.h index c1cd9e644efe4..c4305795892c0 100644 --- a/storage/perfschema/pfs_buffer_container.h +++ b/storage/perfschema/pfs_buffer_container.h @@ -33,7 +33,26 @@ #include "pfs_prepared_stmt.h" #include "pfs_builtin_memory.h" -#define USE_SCALABLE +typedef std::uint16_t pfs_container_id; +typedef std::uint16_t pfs_page_id; +typedef std::uint16_t pfs_object_id; + +/** + * Build an artificial object identity, for OBJECT_INSTANCE_BEGIN columns. + * To be globally unique, identity consists of: + * - the container id, so objects A and B from different containers + * (instrument classes) will not collide. Note that partitioned containers + * get a container id per partition. + * - the page id within the container + * - the object index within the page + * - the version number from the pfs_lock dirty_state, to resolve ABA + * problems + */ +extern pfs_identity make_identity(pfs_container_id container_id, + pfs_page_id page_id, pfs_object_id object_id, + pfs_dirty_state *dirty_state); + +extern std::atomic global_container_id; class PFS_opaque_container_page; class PFS_opaque_container; @@ -77,7 +96,9 @@ class PFS_buffer_default_array public: typedef T value_type; - value_type *allocate(pfs_dirty_state *dirty_state) + value_type *allocate(pfs_dirty_state *dirty_state, + pfs_container_id container_id, pfs_page_id page_id, + pfs_identity *id) { uint index; uint monotonic; @@ -97,10 +118,11 @@ class PFS_buffer_default_array if (pfs->m_lock.free_to_dirty(dirty_state)) { + if (id != nullptr) + *id = make_identity(container_id, page_id, index, dirty_state); return pfs; } monotonic= m_monotonic.m_u32.fetch_add(1); - } m_full= true; @@ -400,6 +422,11 @@ template ::max(), + "pfs_object_id field is only 16 bits"); + static_assert(PFS_PAGE_COUNT <= std::numeric_limits::max(), + "pfs_page_id field is only 16 bits"); + friend class PFS_buffer_scalable_iterator; /** @@ -427,6 +454,7 @@ class PFS_buffer_scalable_container PFS_buffer_scalable_container(allocator_type *allocator) { + m_container_id = global_container_id++; m_allocator= allocator; m_initialized= false; m_lost= 0; @@ -533,7 +561,7 @@ class PFS_buffer_scalable_container return get_row_count() * get_row_size(); } - value_type *allocate(pfs_dirty_state *dirty_state) + value_type *allocate(pfs_dirty_state *dirty_state, pfs_identity *id) { if (m_full) { @@ -578,7 +606,7 @@ class PFS_buffer_scalable_container if (array != NULL) { - pfs= array->allocate(dirty_state); + pfs = array->allocate(dirty_state, m_container_id, index, id); if (pfs != NULL) { /* Keep a pointer to the parent page, for deallocate(). */ @@ -695,7 +723,7 @@ class PFS_buffer_scalable_container } assert(array != NULL); - pfs= array->allocate(dirty_state); + pfs= array->allocate(dirty_state, m_container_id, current_page_count, id); if (pfs != NULL) { /* Keep a pointer to the parent page, for deallocate(). */ @@ -1019,6 +1047,7 @@ class PFS_buffer_scalable_container array_type * m_pages[PFS_PAGE_COUNT]; allocator_type *m_allocator; pthread_mutex_t m_critical_section; + pfs_container_id m_container_id; }; template @@ -1178,11 +1207,12 @@ class PFS_partitioned_buffer_scalable_container return sum; } - value_type *allocate(pfs_dirty_state *dirty_state, uint partition) + value_type *allocate(pfs_dirty_state *dirty_state, uint partition, + pfs_identity *id) { assert(partition < PFS_PARTITION_COUNT); - return m_partitions[partition]->allocate(dirty_state); + return m_partitions[partition]->allocate(dirty_state, id); } void deallocate(value_type *safe_pfs) @@ -1373,116 +1403,60 @@ class PFS_partitioned_buffer_scalable_iterator uint m_sub_index; }; -#ifdef USE_SCALABLE typedef PFS_buffer_scalable_container PFS_mutex_basic_container; typedef PFS_partitioned_buffer_scalable_container PFS_mutex_container; -#else -typedef PFS_buffer_container PFS_mutex_container; -#endif typedef PFS_mutex_container::iterator_type PFS_mutex_iterator; extern PFS_mutex_container global_mutex_container; -#ifdef USE_SCALABLE typedef PFS_buffer_scalable_container PFS_rwlock_container; -#else -typedef PFS_buffer_container PFS_rwlock_container; -#endif typedef PFS_rwlock_container::iterator_type PFS_rwlock_iterator; extern PFS_rwlock_container global_rwlock_container; -#ifdef USE_SCALABLE typedef PFS_buffer_scalable_container PFS_cond_container; -#else -typedef PFS_buffer_container PFS_cond_container; -#endif typedef PFS_cond_container::iterator_type PFS_cond_iterator; extern PFS_cond_container global_cond_container; -#ifdef USE_SCALABLE typedef PFS_buffer_scalable_container PFS_file_container; -#else -typedef PFS_buffer_container PFS_file_container; -#endif typedef PFS_file_container::iterator_type PFS_file_iterator; extern PFS_file_container global_file_container; -#ifdef USE_SCALABLE typedef PFS_buffer_scalable_container PFS_socket_container; -#else -typedef PFS_buffer_container PFS_socket_container; -#endif typedef PFS_socket_container::iterator_type PFS_socket_iterator; extern PFS_socket_container global_socket_container; -#ifdef USE_SCALABLE typedef PFS_buffer_scalable_container PFS_mdl_container; -#else -typedef PFS_buffer_container PFS_mdl_container; -#endif typedef PFS_mdl_container::iterator_type PFS_mdl_iterator; extern PFS_mdl_container global_mdl_container; -#ifdef USE_SCALABLE typedef PFS_buffer_scalable_container PFS_setup_actor_container; -#else -typedef PFS_buffer_container PFS_setup_actor_container; -#endif typedef PFS_setup_actor_container::iterator_type PFS_setup_actor_iterator; extern PFS_setup_actor_container global_setup_actor_container; -#ifdef USE_SCALABLE typedef PFS_buffer_scalable_container PFS_setup_object_container; -#else -typedef PFS_buffer_container PFS_setup_object_container; -#endif typedef PFS_setup_object_container::iterator_type PFS_setup_object_iterator; extern PFS_setup_object_container global_setup_object_container; -#ifdef USE_SCALABLE typedef PFS_buffer_scalable_container PFS_table_container; -#else -typedef PFS_buffer_container PFS_table_container; -#endif typedef PFS_table_container::iterator_type PFS_table_iterator; extern PFS_table_container global_table_container; -#ifdef USE_SCALABLE typedef PFS_buffer_scalable_container PFS_table_share_container; -#else -typedef PFS_buffer_container PFS_table_share_container; -#endif typedef PFS_table_share_container::iterator_type PFS_table_share_iterator; extern PFS_table_share_container global_table_share_container; -#ifdef USE_SCALABLE typedef PFS_buffer_scalable_container PFS_table_share_index_container; -#else -typedef PFS_buffer_container PFS_table_share_index_container; -#endif typedef PFS_table_share_index_container::iterator_type PFS_table_share_index_iterator; extern PFS_table_share_index_container global_table_share_index_container; -#ifdef USE_SCALABLE typedef PFS_buffer_scalable_container PFS_table_share_lock_container; -#else -typedef PFS_buffer_container PFS_table_share_lock_container; -#endif typedef PFS_table_share_lock_container::iterator_type PFS_table_share_lock_iterator; extern PFS_table_share_lock_container global_table_share_lock_container; -#ifdef USE_SCALABLE typedef PFS_buffer_scalable_container PFS_program_container; -#else -typedef PFS_buffer_container PFS_program_container; -#endif typedef PFS_program_container::iterator_type PFS_program_iterator; extern PFS_program_container global_program_container; -#ifdef USE_SCALABLE typedef PFS_buffer_scalable_container PFS_prepared_stmt_container; -#else -typedef PFS_buffer_container PFS_prepared_stmt_container; -#endif typedef PFS_prepared_stmt_container::iterator_type PFS_prepared_stmt_iterator; extern PFS_prepared_stmt_container global_prepared_stmt_container; @@ -1503,17 +1477,9 @@ class PFS_account_allocator void free_array(PFS_account_array *array); }; -#ifdef USE_SCALABLE -typedef PFS_buffer_scalable_container PFS_account_container; -#else -typedef PFS_buffer_container PFS_account_container; -#endif typedef PFS_account_container::iterator_type PFS_account_iterator; extern PFS_account_container global_account_container; @@ -1534,17 +1500,11 @@ class PFS_host_allocator void free_array(PFS_host_array *array); }; -#ifdef USE_SCALABLE typedef PFS_buffer_scalable_container PFS_host_container; -#else -typedef PFS_buffer_container PFS_host_container; -#endif typedef PFS_host_container::iterator_type PFS_host_iterator; extern PFS_host_container global_host_container; @@ -1577,17 +1537,8 @@ class PFS_thread_allocator void free_array(PFS_thread_array *array); }; -#ifdef USE_SCALABLE -typedef PFS_buffer_scalable_container PFS_thread_container; -#else -typedef PFS_buffer_container PFS_thread_container; -#endif typedef PFS_thread_container::iterator_type PFS_thread_iterator; extern PFS_thread_container global_thread_container; @@ -1608,17 +1559,8 @@ class PFS_user_allocator void free_array(PFS_user_array *array); }; -#ifdef USE_SCALABLE -typedef PFS_buffer_scalable_container PFS_user_container; -#else -typedef PFS_buffer_container PFS_user_container; -#endif typedef PFS_user_container::iterator_type PFS_user_iterator; extern PFS_user_container global_user_container; diff --git a/storage/perfschema/pfs_events_waits.h b/storage/perfschema/pfs_events_waits.h index 93dcb136834de..0c950c84a338d 100644 --- a/storage/perfschema/pfs_events_waits.h +++ b/storage/perfschema/pfs_events_waits.h @@ -29,6 +29,7 @@ Events waits data structures (declarations). */ +#include "pfs.h" #include "pfs_column_types.h" #include "pfs_lock.h" #include "pfs_events.h" @@ -88,8 +89,8 @@ struct PFS_events_waits : public PFS_events PFS_metadata_lock *m_weak_metadata_lock; /** For weak pointers, target object version. */ uint32 m_weak_version; - /** Address in memory of the object instance waited on. */ - const void *m_object_instance_addr; + /** Identity of the object instance waited on. */ + pfs_identity m_object_instance_addr; /** Operation performed. */ enum_operation_type m_operation; /** @@ -154,4 +155,3 @@ void reset_table_io_waits_by_table_handle(); void reset_table_lock_waits_by_table_handle(); #endif - diff --git a/storage/perfschema/pfs_host.cc b/storage/perfschema/pfs_host.cc index bf8421909bab9..408a45c4d849a 100644 --- a/storage/perfschema/pfs_host.cc +++ b/storage/perfschema/pfs_host.cc @@ -169,7 +169,7 @@ PFS_host *find_or_create_host(PFS_thread *thread, lf_hash_search_unpin(pins); - pfs= global_host_container.allocate(& dirty_state); + pfs= global_host_container.allocate(& dirty_state, NULL); if (pfs != NULL) { pfs->m_key= key; diff --git a/storage/perfschema/pfs_instr.cc b/storage/perfschema/pfs_instr.cc index c9fb7ca7a3aa6..fdcd6443b708a 100644 --- a/storage/perfschema/pfs_instr.cc +++ b/storage/perfschema/pfs_instr.cc @@ -297,18 +297,18 @@ void cleanup_file_hash(void) /** Create instrumentation for a mutex instance. @param klass the mutex class - @param identity the mutex address @return a mutex instance, or NULL */ -PFS_mutex* create_mutex(PFS_mutex_class *klass, const void *identity) +PFS_mutex* create_mutex(PFS_mutex_class *klass, const void * /* identity */) { PFS_mutex *pfs; pfs_dirty_state dirty_state; + pfs_identity id; - pfs= global_mutex_container.allocate(& dirty_state, klass->m_volatility); + pfs= global_mutex_container.allocate(& dirty_state, klass->m_volatility, &id); if (pfs != NULL) { - pfs->m_identity= identity; + pfs->m_identity= id; pfs->m_class= klass; pfs->m_enabled= klass->m_enabled && flag_global_instrumentation; pfs->m_timed= klass->m_timed; @@ -343,18 +343,18 @@ void destroy_mutex(PFS_mutex *pfs) /** Create instrumentation for a rwlock instance. @param klass the rwlock class - @param identity the rwlock address @return a rwlock instance, or NULL */ -PFS_rwlock* create_rwlock(PFS_rwlock_class *klass, const void *identity) +PFS_rwlock* create_rwlock(PFS_rwlock_class *klass, const void * /* identity */) { PFS_rwlock *pfs; pfs_dirty_state dirty_state; + pfs_identity id; - pfs= global_rwlock_container.allocate(& dirty_state); + pfs= global_rwlock_container.allocate(& dirty_state, &id); if (pfs != NULL) { - pfs->m_identity= identity; + pfs->m_identity= id; pfs->m_class= klass; pfs->m_enabled= klass->m_enabled && flag_global_instrumentation; pfs->m_timed= klass->m_timed; @@ -391,18 +391,18 @@ void destroy_rwlock(PFS_rwlock *pfs) /** Create instrumentation for a condition instance. @param klass the condition class - @param identity the condition address @return a condition instance, or NULL */ -PFS_cond* create_cond(PFS_cond_class *klass, const void *identity) +PFS_cond* create_cond(PFS_cond_class *klass, const void * /* identity */) { PFS_cond *pfs; pfs_dirty_state dirty_state; + pfs_identity id; - pfs= global_cond_container.allocate(& dirty_state); + pfs= global_cond_container.allocate(& dirty_state, &id); if (pfs != NULL) { - pfs->m_identity= identity; + pfs->m_identity= id; pfs->m_class= klass; pfs->m_enabled= klass->m_enabled && flag_global_instrumentation; pfs->m_timed= klass->m_timed; @@ -523,7 +523,7 @@ PFS_thread* create_thread(PFS_thread_class *klass, const void *identity, PFS_thread *pfs; pfs_dirty_state dirty_state; - pfs= global_thread_container.allocate(& dirty_state); + pfs= global_thread_container.allocate(& dirty_state, NULL); if (pfs != NULL) { pfs->m_thread_internal_id= @@ -848,6 +848,7 @@ find_or_create_file(PFS_thread *thread, PFS_file_class *klass, uint retry_count= 0; const uint retry_max= 3; pfs_dirty_state dirty_state; + pfs_identity id; search: @@ -870,9 +871,10 @@ find_or_create_file(PFS_thread *thread, PFS_file_class *klass, return NULL; } - pfs= global_file_container.allocate(& dirty_state); + pfs= global_file_container.allocate(& dirty_state, &id); if (pfs != NULL) { + pfs->m_identity = id; pfs->m_class= klass; pfs->m_enabled= klass->m_enabled && flag_global_instrumentation; pfs->m_timed= klass->m_timed; @@ -881,7 +883,6 @@ find_or_create_file(PFS_thread *thread, PFS_file_class *klass, pfs->m_filename_length= normalized_length; pfs->m_file_stat.m_open_count= 1; pfs->m_file_stat.m_io_stat.reset(); - pfs->m_identity= (const void *)pfs; pfs->m_temporary= false; int res; @@ -1110,19 +1111,19 @@ void destroy_file(PFS_thread *thread, PFS_file *pfs) Create instrumentation for a table instance. @param share the table share @param opening_thread the opening thread - @param identity the table address @return a table instance, or NULL */ PFS_table* create_table(PFS_table_share *share, PFS_thread *opening_thread, - const void *identity) + const void * /* identity */) { PFS_table *pfs; pfs_dirty_state dirty_state; + pfs_identity id; - pfs= global_table_container.allocate(& dirty_state); + pfs= global_table_container.allocate(& dirty_state, &id); if (pfs != NULL) { - pfs->m_identity= identity; + pfs->m_identity= id; pfs->m_share= share; pfs->m_io_enabled= share->m_enabled && flag_global_instrumentation && global_table_io_class.m_enabled; @@ -1294,6 +1295,7 @@ PFS_socket* create_socket(PFS_socket_class *klass, const my_socket *fd, { PFS_socket *pfs; pfs_dirty_state dirty_state; + pfs_identity id; uint fd_used= 0; uint addr_len_used= addr_len; @@ -1304,13 +1306,13 @@ PFS_socket* create_socket(PFS_socket_class *klass, const my_socket *fd, if (addr_len_used > sizeof(sockaddr_storage)) addr_len_used= sizeof(sockaddr_storage); - pfs= global_socket_container.allocate(& dirty_state); + pfs= global_socket_container.allocate(& dirty_state, &id); if (pfs != NULL) { + pfs->m_identity= id; pfs->m_fd= fd_used; /* There is no socket object, so we use the instrumentation. */ - pfs->m_identity= pfs; pfs->m_class= klass; pfs->m_enabled= klass->m_enabled && flag_global_instrumentation; pfs->m_timed= klass->m_timed; @@ -1378,7 +1380,7 @@ void destroy_socket(PFS_socket *pfs) global_socket_container.deallocate(pfs); } -PFS_metadata_lock* create_metadata_lock(void *identity, +PFS_metadata_lock* create_metadata_lock(void * /* identity */, const MDL_key *mdl_key, opaque_mdl_type mdl_type, opaque_mdl_duration mdl_duration, @@ -1388,11 +1390,12 @@ PFS_metadata_lock* create_metadata_lock(void *identity, { PFS_metadata_lock *pfs; pfs_dirty_state dirty_state; + pfs_identity id; - pfs= global_mdl_container.allocate(& dirty_state); + pfs= global_mdl_container.allocate(& dirty_state, &id); if (pfs != NULL) { - pfs->m_identity= identity; + pfs->m_identity= id; pfs->m_enabled= global_metadata_class.m_enabled && flag_global_instrumentation; pfs->m_timed= global_metadata_class.m_timed; pfs->m_mdl_key.mdl_key_init(mdl_key); diff --git a/storage/perfschema/pfs_instr.h b/storage/perfschema/pfs_instr.h index 868f772267c80..2fe5b05f7b85c 100644 --- a/storage/perfschema/pfs_instr.h +++ b/storage/perfschema/pfs_instr.h @@ -91,8 +91,8 @@ struct PFS_instr /** Instrumented mutex implementation. @see PSI_mutex. */ struct PFS_ALIGNED PFS_mutex : public PFS_instr { - /** Mutex identity, typically a pthread_mutex_t. */ - const void *m_identity; + /** Mutex identity. */ + pfs_identity m_identity; /** Mutex class. */ PFS_mutex_class *m_class; /** Instrument statistics. */ @@ -109,8 +109,8 @@ struct PFS_ALIGNED PFS_mutex : public PFS_instr /** Instrumented rwlock implementation. @see PSI_rwlock. */ struct PFS_ALIGNED PFS_rwlock : public PFS_instr { - /** RWLock identity, typically a pthread_rwlock_t. */ - const void *m_identity; + /** RWLock identity. */ + pfs_identity m_identity; /** RWLock class. */ PFS_rwlock_class *m_class; /** Instrument statistics. */ @@ -134,8 +134,8 @@ struct PFS_ALIGNED PFS_rwlock : public PFS_instr /** Instrumented cond implementation. @see PSI_cond. */ struct PFS_ALIGNED PFS_cond : public PFS_instr { - /** Condition identity, typically a pthread_cond_t. */ - const void *m_identity; + /** Condition identity. */ + pfs_identity m_identity; /** Condition class. */ PFS_cond_class *m_class; /** Condition instance usage statistics. */ @@ -148,8 +148,8 @@ struct PFS_ALIGNED PFS_file : public PFS_instr uint32 get_version() { return m_lock.get_version(); } - /** File identity */ - const void *m_identity; + /** File identity. */ + pfs_identity m_identity; /** File name. */ char m_filename[FN_REFLEN]; /** File name length in bytes. */ @@ -241,8 +241,8 @@ struct PFS_ALIGNED PFS_table ulonglong m_owner_event_id; /** Table share. */ PFS_table_share *m_share; - /** Table identity, typically a handler. */ - const void *m_identity; + /** Table identity. */ + pfs_identity m_identity; /** Table statistics. */ PFS_table_stat m_table_stat; /** Current internal lock. */ @@ -266,8 +266,8 @@ struct PFS_ALIGNED PFS_socket : public PFS_instr uint32 get_version() { return m_lock.get_version(); } - /** Socket identity, typically int */ - const void *m_identity; + /** Socket identity. */ + pfs_identity m_identity; /** Owning thread, if applicable */ PFS_thread *m_thread_owner; /** Socket file descriptor */ @@ -291,7 +291,7 @@ struct PFS_ALIGNED PFS_metadata_lock : public PFS_instr { return m_lock.get_version(); } /** Lock identity. */ - const void *m_identity; + pfs_identity m_identity; MDL_key m_mdl_key; opaque_mdl_type m_mdl_type; opaque_mdl_duration m_mdl_duration; @@ -814,4 +814,3 @@ extern LF_HASH pfs_filename_hash; /** @} */ #endif - diff --git a/storage/perfschema/pfs_instr_class.cc b/storage/perfschema/pfs_instr_class.cc index 84cc452c0225d..6ff893d4d0ee1 100644 --- a/storage/perfschema/pfs_instr_class.cc +++ b/storage/perfschema/pfs_instr_class.cc @@ -683,7 +683,7 @@ create_table_share_lock_stat() pfs_dirty_state dirty_state; /* Create a new record in table stat array. */ - pfs= global_table_share_lock_container.allocate(& dirty_state); + pfs= global_table_share_lock_container.allocate(& dirty_state, NULL); if (pfs != NULL) { /* Reset the stats. */ @@ -736,13 +736,11 @@ create_table_share_index_stat(const TABLE_SHARE *server_share, uint server_index pfs_dirty_state dirty_state; /* Create a new record in index stat array. */ - pfs= global_table_share_index_container.allocate(& dirty_state); + pfs= global_table_share_index_container.allocate(& dirty_state, NULL); if (pfs != NULL) { if (server_index == MAX_INDEXES) - { pfs->m_key.m_name_length= 0; - } else { KEY *key_info= server_share->key_info + server_index; @@ -1762,7 +1760,7 @@ PFS_table_share* find_or_create_table_share(PFS_thread *thread, */ } - pfs= global_table_share_container.allocate(& dirty_state); + pfs= global_table_share_container.allocate(& dirty_state, NULL); if (pfs != NULL) { pfs->m_key= key; @@ -2026,4 +2024,3 @@ void update_program_share_derived_flags(PFS_thread *thread) } /** @} */ - diff --git a/storage/perfschema/pfs_prepared_stmt.cc b/storage/perfschema/pfs_prepared_stmt.cc index 50e4e27bb1ed0..6985720eef112 100644 --- a/storage/perfschema/pfs_prepared_stmt.cc +++ b/storage/perfschema/pfs_prepared_stmt.cc @@ -75,23 +75,23 @@ void reset_prepared_stmt_instances() } PFS_prepared_stmt* -create_prepared_stmt(void *identity, +create_prepared_stmt(void * /* identity */, PFS_thread *thread, PFS_program *pfs_program, PFS_events_statements *pfs_stmt, uint stmt_id, const char* stmt_name, uint stmt_name_length) { PFS_prepared_stmt *pfs= NULL; pfs_dirty_state dirty_state; + pfs_identity id; /* Create a new record in prepared stmt stat array. */ - pfs= global_prepared_stmt_container.allocate(& dirty_state); + pfs= global_prepared_stmt_container.allocate(& dirty_state, &id); if (pfs != NULL) { + pfs->m_identity= id; /* Reset the stats. */ pfs->reset_data(); /* Do the assignments. */ - pfs->m_identity= identity; - pfs->m_sqltext_length= 0; if (stmt_name != NULL) diff --git a/storage/perfschema/pfs_prepared_stmt.h b/storage/perfschema/pfs_prepared_stmt.h index 1a06122382c29..395b6e2c0153b 100644 --- a/storage/perfschema/pfs_prepared_stmt.h +++ b/storage/perfschema/pfs_prepared_stmt.h @@ -38,7 +38,7 @@ struct PFS_ALIGNED PFS_prepared_stmt : public PFS_instr { /** Column OBJECT_INSTANCE_BEGIN */ - const void *m_identity; + pfs_identity m_identity; /** STATEMENT_ID */ ulonglong m_stmt_id; diff --git a/storage/perfschema/pfs_program.cc b/storage/perfschema/pfs_program.cc index ee4e23a59b0ec..b0eee7694bae6 100644 --- a/storage/perfschema/pfs_program.cc +++ b/storage/perfschema/pfs_program.cc @@ -222,7 +222,7 @@ find_or_create_program(PFS_thread *thread, &is_enabled, &is_timed); /* Else create a new record in program stat array. */ - pfs= global_program_container.allocate(& dirty_state); + pfs= global_program_container.allocate(& dirty_state, NULL); if (pfs != NULL) { /* Do the assignments. */ diff --git a/storage/perfschema/pfs_server.cc b/storage/perfschema/pfs_server.cc index 2a691a8f3bb14..c1b7ccd9df0e7 100644 --- a/storage/perfschema/pfs_server.cc +++ b/storage/perfschema/pfs_server.cc @@ -1,4 +1,5 @@ /* Copyright (c) 2008, 2023, Oracle and/or its affiliates. + Copyright (c) 2026, MariaDB plc. This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License, version 2.0, @@ -50,7 +51,7 @@ //#include "template_utils.h" #include "pfs_prepared_stmt.h" -PFS_global_param pfs_param; +READ_ONLY_SYSVAR PFS_global_param pfs_param; PFS_table_stat PFS_table_stat::g_reset_template; diff --git a/storage/perfschema/pfs_setup_actor.cc b/storage/perfschema/pfs_setup_actor.cc index 638b2e5648d92..07b2dbd21cd60 100644 --- a/storage/perfschema/pfs_setup_actor.cc +++ b/storage/perfschema/pfs_setup_actor.cc @@ -154,7 +154,7 @@ int insert_setup_actor(const String *user, const String *host, const String *rol PFS_setup_actor *pfs; pfs_dirty_state dirty_state; - pfs= global_setup_actor_container.allocate(& dirty_state); + pfs= global_setup_actor_container.allocate(& dirty_state, NULL); if (pfs != NULL) { set_setup_actor_key(&pfs->m_key, diff --git a/storage/perfschema/pfs_setup_object.cc b/storage/perfschema/pfs_setup_object.cc index a961c51d0e9df..989c49979e836 100644 --- a/storage/perfschema/pfs_setup_object.cc +++ b/storage/perfschema/pfs_setup_object.cc @@ -157,7 +157,7 @@ int insert_setup_object(enum_object_type object_type, const String *schema, PFS_setup_object *pfs; pfs_dirty_state dirty_state; - pfs= global_setup_object_container.allocate(& dirty_state); + pfs= global_setup_object_container.allocate(& dirty_state, NULL); if (pfs != NULL) { set_setup_object_key(&pfs->m_key, object_type, diff --git a/storage/perfschema/pfs_user.cc b/storage/perfschema/pfs_user.cc index d83ee52a77dec..e3ec2992fa15d 100644 --- a/storage/perfschema/pfs_user.cc +++ b/storage/perfschema/pfs_user.cc @@ -166,7 +166,7 @@ find_or_create_user(PFS_thread *thread, lf_hash_search_unpin(pins); - pfs= global_user_container.allocate(& dirty_state); + pfs= global_user_container.allocate(& dirty_state, NULL); if (pfs != NULL) { pfs->m_key= key; diff --git a/storage/perfschema/table_events_waits.cc b/storage/perfschema/table_events_waits.cc index 2f6049f0ebb00..3b3ae6311c127 100644 --- a/storage/perfschema/table_events_waits.cc +++ b/storage/perfschema/table_events_waits.cc @@ -39,8 +39,6 @@ THR_LOCK table_events_waits_current::m_table_lock; -#define OBJECT_INSTANCE_BEGIN(X) (((intptr)X) - ((intptr) &pfs_truncatable_acl)) - PFS_engine_table_share_state table_events_waits_current::m_share_state = { false /* m_checked */ @@ -255,7 +253,7 @@ int table_events_waits_common::make_table_object_columns(PFS_events_waits *wait) m_row.m_index_name_length= 0; } - m_row.m_object_instance_addr= OBJECT_INSTANCE_BEGIN(wait->m_object_instance_addr); + m_row.m_object_instance_addr= wait->m_object_instance_addr; return 0; } @@ -270,7 +268,7 @@ int table_events_waits_common::make_file_object_columns(PFS_events_waits *wait) m_row.m_object_type= "FILE"; m_row.m_object_type_length= 4; m_row.m_object_schema_length= 0; - m_row.m_object_instance_addr= OBJECT_INSTANCE_BEGIN(wait->m_object_instance_addr); + m_row.m_object_instance_addr= wait->m_object_instance_addr; if (safe_file->get_version() == wait->m_weak_version) { @@ -302,7 +300,7 @@ int table_events_waits_common::make_socket_object_columns(PFS_events_waits *wait m_row.m_object_type= "SOCKET"; m_row.m_object_type_length= 6; m_row.m_object_schema_length= 0; - m_row.m_object_instance_addr= OBJECT_INSTANCE_BEGIN(wait->m_object_instance_addr); + m_row.m_object_instance_addr= wait->m_object_instance_addr; if (safe_socket->get_version() == wait->m_weak_version) { @@ -436,7 +434,7 @@ int table_events_waits_common::make_metadata_lock_object_columns(PFS_events_wait if (m_row.m_object_name_length > 0) memcpy(m_row.m_object_name, mdl->name(), m_row.m_object_name_length); - m_row.m_object_instance_addr= OBJECT_INSTANCE_BEGIN(wait->m_object_instance_addr); + m_row.m_object_instance_addr= wait->m_object_instance_addr; } else { @@ -503,17 +501,17 @@ void table_events_waits_common::make_row(PFS_events_waits *wait) break; case WAIT_CLASS_MUTEX: clear_object_columns(); - m_row.m_object_instance_addr= OBJECT_INSTANCE_BEGIN(wait->m_object_instance_addr); + m_row.m_object_instance_addr= wait->m_object_instance_addr; safe_class= sanitize_mutex_class((PFS_mutex_class*) wait->m_class); break; case WAIT_CLASS_RWLOCK: clear_object_columns(); - m_row.m_object_instance_addr= OBJECT_INSTANCE_BEGIN(wait->m_object_instance_addr); + m_row.m_object_instance_addr= wait->m_object_instance_addr; safe_class= sanitize_rwlock_class((PFS_rwlock_class*) wait->m_class); break; case WAIT_CLASS_COND: clear_object_columns(); - m_row.m_object_instance_addr= OBJECT_INSTANCE_BEGIN(wait->m_object_instance_addr); + m_row.m_object_instance_addr= wait->m_object_instance_addr; safe_class= sanitize_cond_class((PFS_cond_class*) wait->m_class); break; case WAIT_CLASS_TABLE: @@ -1174,4 +1172,3 @@ table_events_waits_history_long::get_row_count(void) { return events_waits_history_long_size; } - diff --git a/storage/perfschema/table_events_waits.h b/storage/perfschema/table_events_waits.h index 85a9b1467454c..5ddf57582d153 100644 --- a/storage/perfschema/table_events_waits.h +++ b/storage/perfschema/table_events_waits.h @@ -79,7 +79,7 @@ struct row_events_waits /** Length in bytes of @c m_index_name. */ uint m_index_name_length; /** Column OBJECT_INSTANCE_BEGIN. */ - intptr m_object_instance_addr; + pfs_identity m_object_instance_addr; /** Column SOURCE. */ char m_source[COL_SOURCE_SIZE]; /** Length in bytes of @c m_source. */ diff --git a/storage/perfschema/table_events_waits_summary.cc b/storage/perfschema/table_events_waits_summary.cc index c153057d50f87..465dad6a7cbee 100644 --- a/storage/perfschema/table_events_waits_summary.cc +++ b/storage/perfschema/table_events_waits_summary.cc @@ -85,7 +85,7 @@ ::table_events_waits_summary_by_instance() void table_events_waits_summary_by_instance ::make_instr_row(PFS_instr *pfs, PFS_instr_class *klass, - const void *object_instance_begin, + pfs_identity object_instance_begin, PFS_single_stat *pfs_stat) { pfs_optimistic_state lock; @@ -99,7 +99,7 @@ ::make_instr_row(PFS_instr *pfs, PFS_instr_class *klass, m_row.m_name= klass->m_name; m_row.m_name_length= klass->m_name_length; - m_row.m_object_instance_addr= (intptr) object_instance_begin; + m_row.m_object_instance_addr= object_instance_begin; m_row.m_stat.set(m_normalizer, pfs_stat); @@ -162,11 +162,7 @@ void table_events_waits_summary_by_instance::make_file_row(PFS_file *pfs) PFS_single_stat sum; pfs->m_file_stat.m_io_stat.sum_waits(& sum); - /* - Files don't have a in memory structure associated to it, - so we use the address of the PFS_file buffer as object_instance_begin - */ - make_instr_row(pfs, safe_class, pfs, & sum); + make_instr_row(pfs, safe_class, pfs->m_identity, & sum); } /** @@ -187,11 +183,7 @@ void table_events_waits_summary_by_instance::make_socket_row(PFS_socket *pfs) PFS_byte_stat pfs_stat; pfs->m_socket_stat.m_io_stat.sum(&pfs_stat); - /* - Sockets don't have an associated in-memory structure, so use the address of - the PFS_socket buffer as object_instance_begin. - */ - make_instr_row(pfs, safe_class, pfs, &pfs_stat); + make_instr_row(pfs, safe_class, pfs->m_identity, &pfs_stat); } int table_events_waits_summary_by_instance @@ -241,4 +233,3 @@ ::read_row_values(TABLE *table, unsigned char *, Field **fields, return 0; } - diff --git a/storage/perfschema/table_events_waits_summary.h b/storage/perfschema/table_events_waits_summary.h index 74f3a9ac959a0..0135b5e9aa987 100644 --- a/storage/perfschema/table_events_waits_summary.h +++ b/storage/perfschema/table_events_waits_summary.h @@ -48,7 +48,7 @@ struct row_events_waits_summary_by_instance /** Length in bytes of @c m_name. */ uint m_name_length; /** Column OBJECT_INSTANCE_BEGIN. */ - intptr m_object_instance_addr; + pfs_identity m_object_instance_addr; /** Columns COUNT_STAR, SUM/MIN/AVG/MAX TIMER_WAIT. */ PFS_stat_row m_stat; }; @@ -65,7 +65,7 @@ class table_events_waits_summary_by_instance : public table_all_instr protected: void make_instr_row(PFS_instr *pfs, PFS_instr_class *klass, - const void *object_instance_begin, + pfs_identity object_instance_begin, PFS_single_stat *pfs_stat); void make_mutex_row(PFS_mutex *pfs) override; void make_rwlock_row(PFS_rwlock *pfs) override; diff --git a/storage/perfschema/table_file_summary_by_instance.cc b/storage/perfschema/table_file_summary_by_instance.cc index ae32893926046..bd9ee3281cfeb 100644 --- a/storage/perfschema/table_file_summary_by_instance.cc +++ b/storage/perfschema/table_file_summary_by_instance.cc @@ -203,7 +203,7 @@ int table_file_summary_by_instance::read_row_values(TABLE *table, m_row.m_event_name.set_field(f); break; case 2: /* OBJECT_INSTANCE */ - set_field_ulonglong(f, (ulonglong)m_row.m_identity); + set_field_ulonglong(f, m_row.m_identity); break; case 3:/* COUNT_STAR */ diff --git a/storage/perfschema/table_file_summary_by_instance.h b/storage/perfschema/table_file_summary_by_instance.h index 09f4383394d24..34a91663b8e05 100644 --- a/storage/perfschema/table_file_summary_by_instance.h +++ b/storage/perfschema/table_file_summary_by_instance.h @@ -50,8 +50,9 @@ struct row_file_summary_by_instance /** Column EVENT_NAME. */ PFS_event_name_row m_event_name; - /** Column OBJECT_INSTANCE_BEGIN */ - const void *m_identity; + /** Column OBJECT_INSTANCE_BEGIN. */ + pfs_identity m_identity; + /** Columns COUNT_STAR, SUM/MIN/AVG/MAX TIMER and NUMBER_OF_BYTES for READ, WRITE and MISC operation types. diff --git a/storage/perfschema/table_helper.h b/storage/perfschema/table_helper.h index 35039b47ccc8e..bfd15d8af0f3d 100644 --- a/storage/perfschema/table_helper.h +++ b/storage/perfschema/table_helper.h @@ -700,4 +700,3 @@ struct PFS_user_variable_value_row /** @} */ #endif - diff --git a/storage/perfschema/table_md_locks.cc b/storage/perfschema/table_md_locks.cc index e7b76c1276487..54d9162301261 100644 --- a/storage/perfschema/table_md_locks.cc +++ b/storage/perfschema/table_md_locks.cc @@ -177,7 +177,7 @@ int table_metadata_locks::read_row_values(TABLE *table, m_row.m_object.set_nullable_field(f->field_index, f); break; case 3: /* OBJECT_INSTANCE */ - set_field_ulonglong(f, (intptr) m_row.m_identity); + set_field_ulonglong(f, m_row.m_identity); break; case 4: /* LOCK_TYPE */ set_field_mdl_type(f, m_row.m_mdl_type, m_row.m_object.m_object_type == OBJECT_TYPE_BACKUP); @@ -211,4 +211,3 @@ int table_metadata_locks::read_row_values(TABLE *table, return 0; } - diff --git a/storage/perfschema/table_md_locks.h b/storage/perfschema/table_md_locks.h index 62f681435f684..47b8f6f78ae78 100644 --- a/storage/perfschema/table_md_locks.h +++ b/storage/perfschema/table_md_locks.h @@ -43,7 +43,7 @@ struct PFS_metadata_lock; struct row_metadata_lock { /** Column OBJECT_INSTANCE_BEGIN. */ - const void *m_identity; + pfs_identity m_identity; opaque_mdl_type m_mdl_type; opaque_mdl_duration m_mdl_duration; opaque_mdl_status m_mdl_status; diff --git a/storage/perfschema/table_prepared_stmt_instances.cc b/storage/perfschema/table_prepared_stmt_instances.cc index 2db6018e94897..cbbcf4dba6538 100644 --- a/storage/perfschema/table_prepared_stmt_instances.cc +++ b/storage/perfschema/table_prepared_stmt_instances.cc @@ -236,7 +236,7 @@ ::read_row_values(TABLE *table, unsigned char *buf, Field **fields, switch(f->field_index) { case 0: /* OBJECT_INSTANCE_BEGIN */ - set_field_ulonglong(f, (intptr)m_row.m_identity); + set_field_ulonglong(f, m_row.m_identity); break; case 1: /* STATEMENT_ID */ set_field_ulonglong(f, m_row.m_stmt_id); @@ -299,4 +299,3 @@ ::read_row_values(TABLE *table, unsigned char *buf, Field **fields, return 0; } - diff --git a/storage/perfschema/table_prepared_stmt_instances.h b/storage/perfschema/table_prepared_stmt_instances.h index 838deec55a0f3..80ccdd620f65b 100644 --- a/storage/perfschema/table_prepared_stmt_instances.h +++ b/storage/perfschema/table_prepared_stmt_instances.h @@ -43,7 +43,7 @@ struct row_prepared_stmt_instances { /** Column OBJECT_INSTANCE_BEGIN. */ - const void *m_identity; + pfs_identity m_identity; /** Column STMT_ID. */ ulonglong m_stmt_id; diff --git a/storage/perfschema/table_socket_instances.cc b/storage/perfschema/table_socket_instances.cc index 2435d19df3c7b..8d2d8d923353c 100644 --- a/storage/perfschema/table_socket_instances.cc +++ b/storage/perfschema/table_socket_instances.cc @@ -184,7 +184,7 @@ int table_socket_instances::read_row_values(TABLE *table, set_field_varchar_utf8(f, m_row.m_event_name, m_row.m_event_name_length); break; case 1: /* OBJECT_INSTANCE_BEGIN */ - set_field_ulonglong(f, (intptr)m_row.m_identity); + set_field_ulonglong(f, m_row.m_identity); break; case 2: /* THREAD_ID */ if (m_row.m_thread_id_set) diff --git a/storage/perfschema/table_socket_instances.h b/storage/perfschema/table_socket_instances.h index 61cb121afd4f9..e5e0f7ea0a1ca 100644 --- a/storage/perfschema/table_socket_instances.h +++ b/storage/perfschema/table_socket_instances.h @@ -44,7 +44,7 @@ struct row_socket_instances /** Length in bytes of @c m_event_name. */ uint m_event_name_length; /** Column OBJECT_INSTANCE_BEGIN */ - const void *m_identity; + pfs_identity m_identity; /** Column THREAD_ID */ ulonglong m_thread_id; /** True if thread_is is set */ diff --git a/storage/perfschema/table_socket_summary_by_instance.cc b/storage/perfschema/table_socket_summary_by_instance.cc index 5df9edc13b8cd..fd9a038a6f848 100644 --- a/storage/perfschema/table_socket_summary_by_instance.cc +++ b/storage/perfschema/table_socket_summary_by_instance.cc @@ -195,7 +195,7 @@ int table_socket_summary_by_instance::read_row_values(TABLE *table, m_row.m_event_name.set_field(f); break; case 1: /* OBJECT_INSTANCE */ - set_field_ulonglong(f, (intptr)m_row.m_identity); + set_field_ulonglong(f, m_row.m_identity); break; case 2:/* COUNT_STAR */ diff --git a/storage/perfschema/table_socket_summary_by_instance.h b/storage/perfschema/table_socket_summary_by_instance.h index 2f51d8b5e3944..1ed679621fd05 100644 --- a/storage/perfschema/table_socket_summary_by_instance.h +++ b/storage/perfschema/table_socket_summary_by_instance.h @@ -49,7 +49,7 @@ struct row_socket_summary_by_instance PFS_event_name_row m_event_name; /** Column OBJECT_INSTANCE_BEGIN */ - const void *m_identity; + pfs_identity m_identity; /** Columns COUNT_STAR, SUM/MIN/AVG/MAX TIMER and NUMBER_OF_BYTES for each operation. */ PFS_socket_io_stat_row m_io_stat; diff --git a/storage/perfschema/table_sync_instances.cc b/storage/perfschema/table_sync_instances.cc index 497f7fa1854f6..2cf8cc65db4dc 100644 --- a/storage/perfschema/table_sync_instances.cc +++ b/storage/perfschema/table_sync_instances.cc @@ -174,7 +174,7 @@ int table_mutex_instances::read_row_values(TABLE *table, set_field_varchar_utf8(f, m_row.m_name, m_row.m_name_length); break; case 1: /* OBJECT_INSTANCE */ - set_field_ulonglong(f, (intptr) m_row.m_identity); + set_field_ulonglong(f, m_row.m_identity); break; case 2: /* LOCKED_BY_THREAD_ID */ if (m_row.m_locked) @@ -334,7 +334,7 @@ int table_rwlock_instances::read_row_values(TABLE *table, set_field_varchar_utf8(f, m_row.m_name, m_row.m_name_length); break; case 1: /* OBJECT_INSTANCE */ - set_field_ulonglong(f, (intptr) m_row.m_identity); + set_field_ulonglong(f, m_row.m_identity); break; case 2: /* WRITE_LOCKED_BY_THREAD_ID */ if (m_row.m_write_locked) @@ -480,7 +480,7 @@ int table_cond_instances::read_row_values(TABLE *table, set_field_varchar_utf8(f, m_row.m_name, m_row.m_name_length); break; case 1: /* OBJECT_INSTANCE */ - set_field_ulonglong(f, (intptr) m_row.m_identity); + set_field_ulonglong(f, m_row.m_identity); break; default: assert(false); @@ -490,4 +490,3 @@ int table_cond_instances::read_row_values(TABLE *table, return 0; } - diff --git a/storage/perfschema/table_sync_instances.h b/storage/perfschema/table_sync_instances.h index 1a6cf81a173b3..2807b45d7c2c9 100644 --- a/storage/perfschema/table_sync_instances.h +++ b/storage/perfschema/table_sync_instances.h @@ -48,7 +48,7 @@ struct row_mutex_instances /** Length in bytes of @c m_name. */ uint m_name_length; /** Column OBJECT_INSTANCE_BEGIN. */ - const void *m_identity; + pfs_identity m_identity; /** True if column LOCKED_BY_THREAD_ID is not null. */ bool m_locked; /** Column LOCKED_BY_THREAD_ID. */ @@ -104,7 +104,7 @@ struct row_rwlock_instances /** Length in bytes of @c m_name. */ uint m_name_length; /** Column OBJECT_INSTANCE_BEGIN. */ - const void *m_identity; + pfs_identity m_identity; /** True if column WRITE_LOCKED_BY_THREAD_ID is not null. */ bool m_write_locked; /** Column WRITE_LOCKED_BY_THREAD_ID. */ @@ -162,7 +162,7 @@ struct row_cond_instances /** Length in bytes of @c m_name. */ uint m_name_length; /** Column OBJECT_INSTANCE_BEGIN. */ - const void *m_identity; + pfs_identity m_identity; }; /** Table PERFORMANCE_SCHEMA.COND_INSTANCES. */ diff --git a/storage/perfschema/table_table_handles.cc b/storage/perfschema/table_table_handles.cc index 7ea2b0a331667..82b5f9dd6a233 100644 --- a/storage/perfschema/table_table_handles.cc +++ b/storage/perfschema/table_table_handles.cc @@ -196,7 +196,7 @@ int table_table_handles::read_row_values(TABLE *table, m_row.m_object.set_field(f->field_index, f); break; case 3: /* OBJECT_INSTANCE_BEGIN */ - set_field_ulonglong(f, (intptr) m_row.m_identity); + set_field_ulonglong(f, m_row.m_identity); break; case 4: /* OWNER_THREAD_ID */ set_field_ulonglong(f, m_row.m_owner_thread_id); @@ -218,4 +218,3 @@ int table_table_handles::read_row_values(TABLE *table, return 0; } - diff --git a/storage/perfschema/table_table_handles.h b/storage/perfschema/table_table_handles.h index d2330104e5d52..883849d415da7 100644 --- a/storage/perfschema/table_table_handles.h +++ b/storage/perfschema/table_table_handles.h @@ -48,7 +48,7 @@ struct row_table_handles /** Column OBJECT_TYPE, SCHEMA_NAME, OBJECT_NAME. */ PFS_object_row m_object; /** Column OBJECT_INSTANCE_BEGIN. */ - const void *m_identity; + pfs_identity m_identity; /** Column OWNER_THREAD_ID. */ ulonglong m_owner_thread_id; /** Column OWNER_EVENT_ID. */ diff --git a/support-files/CMakeLists.txt b/support-files/CMakeLists.txt index acf6d1912ae08..ac8d381d25ac0 100644 --- a/support-files/CMakeLists.txt +++ b/support-files/CMakeLists.txt @@ -223,11 +223,8 @@ IF(UNIX AND NOT WITHOUT_SERVER) ENDIF() IF((HAVE_SYSTEMD OR INSTALL_SYSTEMD_TMPUSERS) AND INSTALL_SYSTEMD_TMPFILESDIR) - IF(MYSQL_DATADIR STREQUAL INSTALL_RUNDATADIR) - SET(DISABLE_RUNDATADIR "#") - ENDIF() get_filename_component(MYSQL_UNIX_ADDRDIR ${MYSQL_UNIX_ADDR} DIRECTORY) - IF(MYSQL_UNIX_ADDRDIR STREQUAL MYSQL_DATADIR OR MYSQL_UNIX_ADDRDIR STREQUAL INSTALL_RUNDATADIR) + IF(MYSQL_UNIX_ADDRDIR STREQUAL MYSQL_DATADIR OR MYSQL_UNIX_ADDRDIR STREQUAL INSTALL_RUNDIR) SET(DISABLE_MYSQL_UNIX_ADDRDIR "#") ENDIF() diff --git a/support-files/galera.conf.in b/support-files/galera.conf.in index 35fad506d80b8..f3737758c4f85 100644 --- a/support-files/galera.conf.in +++ b/support-files/galera.conf.in @@ -1,32 +1,27 @@ # This is a drop-in for the mariadb.service to # extend its functionality for Galera. # -# ExecStart overrides the base service and the -# other lists, like ExecStartPre, end the previous -# lists. +# ExecStart overrides the base service, the other +# lists, like ExecStartPre, end the previous lists. [Service] -# Use an environment file to pass variable _WSREP_NEW_CLUSTER -EnvironmentFile=-@INSTALL_RUNDATADIR@/wsrep-new-cluster - -# Use an environment file to pass variable _WSREP_START_POSITION -EnvironmentFile=-@INSTALL_RUNDATADIR@/wsrep-start-position - -# Perform automatic wsrep recovery. When server is started without wsrep, -# galera_recovery simply returns an empty string. In any case, however, -# the script is not expected to return with a non-zero status. -# It is always safe to remove @INSTALL_RUNDATADIR@/wsrep-start-position -# environment file. -# Do not panic if galera_recovery script is not available. (MDEV-10538) -ExecStartPre=/bin/sh -c "[ ! -e @bindir@/galera_recovery ] && VAR= || \ - VAR=`@bindir@/galera_recovery`; [ $? -eq 0 ] \ - && echo _WSREP_START_POSITION=$VAR > @INSTALL_RUNDATADIR@/wsrep-start-position || exit 1" +# Use an environment file to pass variable _WSREP_NEW_CLUSTER. +# Must not be writable by mysql user. +EnvironmentFile=-@INSTALL_RUNDIR@/mariadb-wsrep-new-cluster # Reset of ExecStart list from base service ExecStart= # Start main service # _WSREP_NEW_CLUSTER is for the exclusive use of the script galera_new_cluster -ExecStart=@sbindir@/mariadbd $MYSQLD_OPTS $_WSREP_NEW_CLUSTER $_WSREP_START_POSITION +# +# Perform automatic wsrep recovery. When server is started without wsrep, +# galera_recovery simply returns an empty string. In any case, however, it is +# not expected to return a non-zero status. +# Do not panic if galera_recovery script is not available. (MDEV-10538) +ExecStart=/bin/sh -c "set -f; [ ! -e @bindir@/galera_recovery ] && VAR= || \ + VAR=`@bindir@/galera_recovery`; [ $? -eq 0 ] || exit 1; \ + exec @sbindir@/mariadbd $MYSQLD_OPTS $_WSREP_NEW_CLUSTER $VAR" -# Unset _WSREP_START_POSITION/_WSREP_NEW_CLUSTER environment variable. -ExecStartPost=/bin/rm -f @INSTALL_RUNDATADIR@/wsrep-start-position @INSTALL_RUNDATADIR@/wsrep-new-cluster +# have to set the name explicitly now, otherwise it'll be the +# first name from ExecStart +SyslogIdentifier=mariadbd diff --git a/support-files/rpm/server-prein.sh b/support-files/rpm/server-prein.sh index a57dcd348d271..c472483bdd955 100644 --- a/support-files/rpm/server-prein.sh +++ b/support-files/rpm/server-prein.sh @@ -68,6 +68,8 @@ fi # Create a MariaDB user and group. Do not report any problems if it already exists. groupadd -r %{mysqld_group} 2> /dev/null || true -useradd -M -r --home %{mysqldatadir} --shell /sbin/nologin --comment "MariaDB server" --gid %{mysqld_group} %{mysqld_user} 2> /dev/null || true +useradd -M -r --home /nonexistent --shell /sbin/nologin --comment "MariaDB server" --gid %{mysqld_group} %{mysqld_user} 2> /dev/null || true # The user may already exist, make sure it has the proper group nevertheless (BUG#12823) usermod --gid %{mysqld_group} %{mysqld_user} 2> /dev/null || true +# Make sure the home dir is correct too (must not be datadir) +usermod -d /nonexistent %{mysqld_user} 2> /dev/null || true diff --git a/support-files/tmpfiles.conf.in b/support-files/tmpfiles.conf.in index 6a2a28b59a3e8..e41585c08a88d 100644 --- a/support-files/tmpfiles.conf.in +++ b/support-files/tmpfiles.conf.in @@ -1,9 +1,6 @@ # This is the directory where the unix socket # of MariaDB may be created. # -# Under Galera this is where an envfile of -# the Galera start position is stored. -# # Other temporary directories can be created here like: # * tmpdir # * innodb_tmpdir @@ -12,5 +9,4 @@ # etc. # It shouldn't be used for datadir which is why it # may be disabled. -@DISABLE_RUNDATADIR@d @INSTALL_RUNDATADIR@ 0755 @MYSQLD_USER@ @MYSQLD_USER@ - @DISABLE_MYSQL_UNIX_ADDRDIR@d @MYSQL_UNIX_ADDRDIR@ 0755 @MYSQLD_USER@ root -