Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion Project.toml
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,8 @@ Chairmarks = "0ca39b1e-fe0b-4e98-acfc-b1656634c4de"
Harbor = "af79dbb9-1a80-47ad-8928-192a4af69376"
Logging = "56ddb016-857b-54e1-b83d-db4d58db5568"
Printf = "de0858da-6303-5e67-8744-51eddeeeb8d7"
Sockets = "6462fe0b-24de-5631-8697-dd941f90decc"
Test = "8dfed614-e22c-5e08-85e1-65c5234f0b40"

[targets]
test = ["Chairmarks", "Harbor", "Logging", "Printf", "Test"]
test = ["Chairmarks", "Harbor", "Logging", "Printf", "Sockets", "Test"]
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,8 +64,8 @@ client certificates), option files (`option_file`, `read_default_file`), timeout
(`connect_timeout`, `read_timeout`, `write_timeout`), `reconnect`, `multi_statements`,
`init_command`, decoding policies (`zero_dates`, `time_type`), and
memory limits (`max_buffered_bytes`); see the
[documentation](https://mysql.juliadatabases.org/dev/). The transport is TCP or TLS
(Unix sockets, named pipes, and compression are not supported yet and raise clear errors).
[documentation](https://mysql.juliadatabases.org/dev/). The transport is TCP, TLS, or a Unix
socket (named pipes and compression are not supported yet and raise clear errors).

Results use the JuliaData value types: text and BLOB columns are zero-copy
[DataStrings.jl](https://github.com/JuliaData/DataStrings.jl) `DataString`/`DataBytes`
Expand Down
10 changes: 6 additions & 4 deletions docs/src/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,8 @@ using MySQL, DBInterface, Dates # MySQL re-exports Durations.Timestamp
conn = DBInterface.connect(MySQL.Connection, "localhost", "user", "password"; db="mydb")
```

Every host, `"localhost"` included, is dialed over TCP (port 3306 unless `port` is given).
Hosts are dialed over TCP (port 3306 unless `port` is given). To use a local server's Unix
socket, pass its path as `unix_socket` with host `"localhost"`.
TLS is used whenever the server offers it (`ssl_mode=:preferred`); see
[Connection options](@ref) for the full keyword list.

Expand Down Expand Up @@ -136,7 +137,8 @@ raise `ArgumentError` unless their value is `nothing` or `false`.
|---|---|---|
| `db` | `""` | default database (`USE`) |
| `port` | `3306` | TCP port; omitted/`nothing` uses file or opt-in environment defaults; `0` explicitly selects 3306 |
| `protocol` | `:default` | `:tcp`, or the deferred `:socket`/`:pipe` (they raise a clear error) |
| `protocol` | `:default` | `:tcp`, `:socket` (a Unix socket for host `"localhost"`, `/tmp/mysql.sock` without `unix_socket`), or the deferred `:pipe` (it raises a clear error) |
| `unix_socket` | `nothing` | Unix socket path, used instead of TCP when the host is `"localhost"` (not on Windows); TLS is not available over a socket |
| `bind` | `nothing` | local interface/address to connect from |
| `connect_timeout` | `nothing` | positive integer seconds for the whole establishment (dial, TLS, authentication, charset bootstrap) |
| `read_timeout`, `write_timeout` | `nothing` | positive integer seconds per transport read/write; expiry closes the connection |
Expand Down Expand Up @@ -179,7 +181,7 @@ raise `ArgumentError` unless their value is `nothing` or `false`.
| keyword | default | meaning |
|---|---|---|
| `default_auth` | supported server default, else `caching_sha2_password` | plugin to answer the handshake with; an auth switch selects the account's plugin |
| `get_server_public_key` | `false` | fetch the server's RSA key for `caching_sha2`/`sha256` full authentication over plain TCP |
| `get_server_public_key` | `false` | fetch the server's RSA key for `caching_sha2`/`sha256` full authentication without TLS |
| `server_public_key` | `nothing` | path of that key in PEM form |
| `enable_cleartext_plugin` | `false` | allow `mysql_clear_password` (PAM/LDAP accounts); `default_auth="mysql_clear_password"` also enables it |
| `insecure_cleartext_auth` | `false` | allow cleartext authentication without `:verify_identity` |
Expand Down Expand Up @@ -226,7 +228,7 @@ ignored. This is a subset of the [MySQL option-file format](https://dev.mysql.co
Supported file keys are `host`, `user`, `password`, `port`, `database`, `connect-timeout`,
`default-character-set`, `protocol`, `bind-address`, `socket`, `tls-version`, `ssl-mode`,
`ssl-ca`, `ssl-capath`, `ssl-cert`, and `ssl-key`. Underscores can replace hyphens.
`socket` is accepted but unused. `ssl-cipher`, `ssl-crl`, and `ssl-crlpath` raise an error
`socket` gives the Unix socket path for host `localhost`. `ssl-cipher`, `ssl-crl`, and `ssl-crlpath` raise an error
because these settings are unavailable. `compress` also raises an error unless its value
is `0`, `OFF`, or `FALSE` (case-insensitive).

Expand Down
22 changes: 13 additions & 9 deletions docs/src/migration.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,11 +44,16 @@ MySQL.jl 2.0 requires **Julia 1.10 or later**.
`"stmt1; stmt2"` by default (an `if/elseif` bug made 1.x enable it silently).
4. **Enum-valued options**: pass Symbols — `ssl_mode=:required` (was
`MySQL.API.SSL_MODE_REQUIRED`), `protocol=:tcp` (was `MySQL.API.MYSQL_PROTOCOL_TCP`).
5. **Local servers**: every host, `"localhost"` and `""` included, is dialed over TCP (the
Connector/C rule that turned `localhost` into a Unix socket is gone). The socket and
named-pipe transports are not implemented yet: `protocol=:socket`/`:pipe` or
`named_pipe=true` raise a clear error (an explicit `protocol=:tcp` overrides
`named_pipe`). A `unix_socket` path is accepted but unused.
5. **Local servers**: `"localhost"` and `""` are dialed over TCP unless a socket path is
given. With a path (the `unix_socket` keyword, `socket` in an option file, or
`MYSQL_UNIX_PORT` with `read_env=true`), `localhost` connects through that Unix socket,
as with Connector/C. Without a path, Connector/C used `/tmp/mysql.sock`; MySQL.jl uses
TCP. On Windows, `localhost` always uses TCP. `protocol=:socket` uses a socket, at
`/tmp/mysql.sock` when no path is given, and requires host `localhost`. TLS is not
available over a socket: `ssl_mode=:preferred` connects without it, and a mode or CA
option that requires TLS raises an `ArgumentError`. Named pipes are not
implemented yet: `protocol=:pipe` or `named_pipe=true` raise a clear error (an explicit
`protocol=:tcp` overrides `named_pipe`).
TCP access also needs a matching account grant. With `skip_name_resolve`, a
`'user'@'localhost'` grant need not cover `127.0.0.1` or `::1`; see
[MySQL's host-resolution rules](https://dev.mysql.com/doc/refman/8.0/en/server-system-variables.html#sysvar_skip_name_resolve).
Expand Down Expand Up @@ -228,10 +233,9 @@ either `:verify_identity` or `insecure_cleartext_auth=true`.
Documented gaps, planned for later 2.x releases — attempting to use them raises a clear
error rather than misbehaving:

- **Unix sockets and Windows named pipes** (transport is TCP/TLS). Every host, including
`"localhost"`, is dialed over TCP; asking for the local transport explicitly
(`protocol=:socket`/`:pipe`, or `named_pipe=true` without `protocol=:tcp`) raises a clear error, and a `unix_socket`
path (keyword or option file) is accepted but unused until the transport exists.
- **Windows named pipes** and **TLS over a Unix socket**. `protocol=:pipe`, or
`named_pipe=true` without `protocol=:tcp`, raises a clear error, and so does a TLS
requirement on a socket connection.
- **Compression** (`compress=true` is an `ArgumentError`), server cursors /
`COM_STMT_FETCH`, query attributes, `COM_STMT_BULK_EXECUTE`
- MariaDB `client_ed25519` / PARSEC / `dialog` (PAM) authentication (`UnsupportedAuthError`)
Expand Down
29 changes: 16 additions & 13 deletions src/Protocol/auth.jl
Original file line number Diff line number Diff line change
@@ -1,23 +1,25 @@
# Authentication plugins and the authentication exchange.
#
# Per-plugin transport-security rules (TCP only; Unix sockets/named pipes are deferred):
# Per-plugin transport-security rules, as the server applies them:
#
# | plugin step | TLS (any mode) | plain TCP |
# |--------------------------------------|----------------------------------------|----------------------|
# | caching_sha2 full auth (cleartext) | allowed | RSA path (opt-in) |
# | sha256 full auth (cleartext) | allowed | RSA path (opt-in) |
# | mysql_clear_password | :verify_identity only, else opt-in | opt-in only |
# | plugin step | TLS (any mode) | Unix socket | plain TCP |
# |------------------------------------|------------------------------------|-------------------|-------------------|
# | caching_sha2 full auth (cleartext) | allowed | allowed | RSA path (opt-in) |
# | sha256 full auth (cleartext) | allowed | RSA path (opt-in) | RSA path (opt-in) |
# | mysql_clear_password | :verify_identity only, else opt-in | opt-in only | opt-in only |
#
# The RSA path needs `server_public_key` (PEM) or `get_server_public_key=true`; the cleartext
# plugin needs explicit enablement (`enable_cleartext_plugin` or `default_auth`).

"""
AuthPolicy(; secure_transport=false, identity_verified=false, server_public_key=nothing,
get_server_public_key=false, enable_cleartext_plugin=false,
insecure_cleartext_auth=false)
insecure_cleartext_auth=false, local_socket=false)

Connection-level facts and user policy the plugins consult. `secure_transport` is true on
TLS (any mode); `identity_verified` only under `ssl_mode = :verify_identity`.
TLS (any mode); `identity_verified` only under `ssl_mode = :verify_identity`. `local_socket`
is true on a Unix socket, which the server counts as secure for `caching_sha2_password`
only; `sha256_password` still requires TLS for a cleartext password.
"""
struct AuthPolicy
secure_transport::Bool
Expand All @@ -26,11 +28,12 @@ struct AuthPolicy
get_server_public_key::Bool
enable_cleartext_plugin::Bool
insecure_cleartext_auth::Bool
local_socket::Bool
end

function AuthPolicy(; secure_transport::Bool=false, identity_verified::Bool=false, server_public_key::Union{Nothing, AbstractVector{UInt8}, AbstractString}=nothing, get_server_public_key::Bool=false, enable_cleartext_plugin::Bool=false, insecure_cleartext_auth::Bool=false)
function AuthPolicy(; secure_transport::Bool=false, identity_verified::Bool=false, server_public_key::Union{Nothing, AbstractVector{UInt8}, AbstractString}=nothing, get_server_public_key::Bool=false, enable_cleartext_plugin::Bool=false, insecure_cleartext_auth::Bool=false, local_socket::Bool=false)
pem = server_public_key === nothing ? nothing : server_public_key isa AbstractString ? Vector{UInt8}(codeunits(server_public_key)) : Vector{UInt8}(server_public_key)
return AuthPolicy(secure_transport, identity_verified, pem, get_server_public_key, enable_cleartext_plugin, insecure_cleartext_auth)
return AuthPolicy(secure_transport, identity_verified, pem, get_server_public_key, enable_cleartext_plugin, insecure_cleartext_auth, local_socket)
end

abstract type AuthPlugin end
Expand Down Expand Up @@ -177,7 +180,7 @@ cleartext_password(password::AbstractVector{UInt8}) = return nul_terminated_pass

# ---- policy ----

@noinline rsa_unavailable(plugin::String) = return throw(AuthError("$plugin requires a secure connection for full authentication; over plain TCP pass `server_public_key=<PEM path>` or `get_server_public_key=true` to use RSA password exchange, or connect with `ssl_mode=:required`"))
@noinline rsa_unavailable(plugin::String) = return throw(AuthError("$plugin requires a secure connection for full authentication; without TLS pass `server_public_key=<PEM path>` or `get_server_public_key=true` to use RSA password exchange, or connect over TCP with `ssl_mode=:required`"))

function require_cleartext_allowed(policy::AuthPolicy)
policy.enable_cleartext_plugin || throw(AuthError("the server requested mysql_clear_password, which is disabled; pass `enable_cleartext_plugin=true` (or `default_auth=\"mysql_clear_password\"`)"))
Expand Down Expand Up @@ -253,7 +256,7 @@ function step!(::CachingSha2Password, state::AuthState, data::AbstractVector{UIn
end
data[1] == CACHING_SHA2_PERFORM_FULL_AUTH || protocol_error("unexpected caching_sha2_password status byte 0x$(string(data[1], base=16, pad=2))")
state.full_auth = true
policy.secure_transport && return cleartext_password(password)
(policy.secure_transport || policy.local_socket) && return cleartext_password(password)
policy.server_public_key === nothing || return rsa_encrypt_password(password, state.nonce, policy.server_public_key)
if policy.get_server_public_key
state.awaiting_public_key = true
Expand Down Expand Up @@ -281,7 +284,7 @@ function trace_event(state::AuthState, data::AbstractVector{UInt8}, policy::Auth
isempty(data) && return :continue
data[1] == CACHING_SHA2_FAST_AUTH_SUCCESS && return :fast_auth
data[1] == CACHING_SHA2_PERFORM_FULL_AUTH || return :continue
policy.secure_transport && return :full_auth_cleartext
(policy.secure_transport || policy.local_socket) && return :full_auth_cleartext
state.awaiting_public_key && return :rsa_request
return :full_auth_rsa
end
Expand Down
3 changes: 2 additions & 1 deletion src/Protocol/tls.jl
Original file line number Diff line number Diff line change
Expand Up @@ -118,6 +118,7 @@ end

raw_tcp(t::Reseau.TCP.Conn) = return t
raw_tcp(t::FaultTransport) = return t.inner isa Reseau.TCP.Conn ? t.inner : throw(ArgumentError("STARTTLS needs a TCP transport"))
raw_tcp(::Reseau.Unix.Conn) = throw(ArgumentError("STARTTLS needs a TCP transport"))
raw_tcp(::Reseau.TLS.Conn) = return throw(ArgumentError("the session is already on TLS"))

function socket_fd(tcp::Reseau.TCP.Conn)
Expand All @@ -142,7 +143,7 @@ function has_pending_tcp_bytes(tcp::Reseau.TCP.Conn)
end

is_secure_transport(t::Reseau.TLS.Conn) = return true
is_secure_transport(t::Reseau.TCP.Conn) = return false
is_secure_transport(t::Union{Reseau.TCP.Conn, Reseau.Unix.Conn}) = return false
is_secure_transport(t::FaultTransport) = return t.inner isa Reseau.TLS.Conn
is_secure_transport(s::Session) = return is_secure_transport(s.transport)

Expand Down
34 changes: 22 additions & 12 deletions src/Protocol/transport.jl
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
# Transports the session can own. A concrete union keeps the packet hot path an `isa` split
# (no abstract-typed field). Unix sockets / named pipes are deferred, so there is no
# `Sockets` dependency.
# (no abstract-typed field).

"""
FaultTransport(inner; fail_read_at=-1, fail_write_at=-1, read_error, write_error, after_write_error=nothing, discard_writes=false)
Expand All @@ -18,8 +17,8 @@ Test-only transport wrapper that injects faults at byte offsets:
Byte counters are plain integers (a `FaultTransport` is used from one task); `close_count`
is atomic because the reaper's timer task may race an explicit close in tests.
"""
mutable struct FaultTransport{T <: IO} <: IO
inner::T
mutable struct FaultTransport <: IO
inner::Union{IOBuffer, Reseau.TCP.Conn}
read_bytes::Int
write_bytes::Int
fail_read_at::Int
Expand All @@ -32,14 +31,15 @@ mutable struct FaultTransport{T <: IO} <: IO
@atomic close_count::Int
end

function FaultTransport(inner::IO; fail_read_at::Integer=-1, fail_write_at::Integer=-1, read_error::Exception=EOFError(), write_error::Exception=EOFError(), after_write_error::Union{Nothing, Exception}=nothing, discard_writes::Bool=false)
return FaultTransport{typeof(inner)}(inner, 0, 0, Int(fail_read_at), Int(fail_write_at), read_error, write_error, after_write_error, false, discard_writes, 0)
function FaultTransport(inner::Union{IOBuffer, Reseau.TCP.Conn}; fail_read_at::Integer=-1, fail_write_at::Integer=-1, read_error::Exception=EOFError(), write_error::Exception=EOFError(), after_write_error::Union{Nothing, Exception}=nothing, discard_writes::Bool=false)
return FaultTransport(inner, 0, 0, Int(fail_read_at), Int(fail_write_at), read_error, write_error, after_write_error, false, discard_writes, 0)
end

# A closed union of concrete types: the packet hot path stays an `isa` split, `--trim=safe`
# can resolve every transport operation statically, and no transport type (so no `close`
# method) can be defined after the reaper's timer task fixes its world age.
const Transport = Union{Reseau.TCP.Conn, Reseau.TLS.Conn, FaultTransport{IOBuffer}, FaultTransport{Reseau.TCP.Conn}}
# A closed union of four concrete types, Julia's limit for splitting a union into static
# calls: the packet hot path stays an `isa` split, `--trim=safe` can resolve every transport
# operation statically, and no transport type (so no `close` method) can be defined after the
# reaper's timer task fixes its world age.
const Transport = Union{Reseau.TCP.Conn, Reseau.TLS.Conn, Reseau.Unix.Conn, FaultTransport}

function Base.unsafe_read(ft::FaultTransport, ptr::Ptr{UInt8}, nbytes::UInt)
n = Int(nbytes)
Expand Down Expand Up @@ -95,7 +95,7 @@ end
# Whether the packet reader may batch reads through its read buffer (Reseau's `unsafe_read`
# costs one `recv` per call, so per-packet exact reads dominate large scans; §8.9). The
# test-only `FaultTransport` stays byte-exact so fault byte offsets remain deterministic.
supports_buffered_reads(::Union{Reseau.TCP.Conn, Reseau.TLS.Conn}) = return true
supports_buffered_reads(::Union{Reseau.TCP.Conn, Reseau.Unix.Conn, Reseau.TLS.Conn}) = return true
supports_buffered_reads(::FaultTransport) = return false

# A socket error on an open connection (a reset or broken pipe; under TLS the cause of a
Expand All @@ -108,7 +108,7 @@ supports_buffered_reads(::FaultTransport) = return false
end

# Reads 1..n available bytes into `buf[offset:end]` (one transport read); 0 means EOF.
function transport_read_some!(t::Union{Reseau.TCP.Conn, Reseau.TLS.Conn}, buf::Vector{UInt8}, offset::Int, n::Int)
function transport_read_some!(t::Union{Reseau.TCP.Conn, Reseau.Unix.Conn, Reseau.TLS.Conn}, buf::Vector{UInt8}, offset::Int, n::Int)
try
return Base.readbytes!(t, view(buf, offset:lastindex(buf)), n; all=false)
catch err
Expand Down Expand Up @@ -152,6 +152,11 @@ function set_read_deadline!(t::Reseau.TCP.Conn, deadline_ns::Integer)
return nothing
end

function set_read_deadline!(t::Reseau.Unix.Conn, deadline_ns::Integer)
Reseau.Unix.set_read_deadline!(t, deadline_ns)
return nothing
end

function set_read_deadline!(t::Reseau.TLS.Conn, deadline_ns::Integer)
Reseau.TLS.set_read_deadline!(t, deadline_ns)
return nothing
Expand All @@ -167,6 +172,11 @@ function set_write_deadline!(t::Reseau.TCP.Conn, deadline_ns::Integer)
return nothing
end

function set_write_deadline!(t::Reseau.Unix.Conn, deadline_ns::Integer)
Reseau.Unix.set_write_deadline!(t, deadline_ns)
return nothing
end

function set_write_deadline!(t::Reseau.TLS.Conn, deadline_ns::Integer)
Reseau.TLS.set_write_deadline!(t, deadline_ns)
return nothing
Expand Down
Loading
Loading