fix(plugin-mongodb): make MQL exports and the query editor round-trip BSON values

CHANGELOG.md

@@ -12,6 +12,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - SSL settings on mobile for MySQL, PostgreSQL and Redis: a mode picker plus CA, client certificate and client key. Import a PEM file, paste one, or use a PKCS#12 file. (#2083)
 - Legacy UUID Encoding on a MongoDB connection, so binary UUIDs written by the Java, C# or Python drivers read as UUIDs instead of hex. Filters, edits and MQL exports write the same bytes back. (#2086)
 
+### Added
+
+- The MongoDB editor accepts mongosh value constructors in filters and pipelines, so a value copied from the grid pastes straight into a query. Covers `ObjectId`, `ISODate`, `Date`, the `Number*` family, `Timestamp`, `BinData`, `HexData`, `MinKey`, `MaxKey` and the UUID names. (#2086)
+
 ### Changed
 
 - Mobile keeps remote connections open when you switch apps.
@@ -40,6 +44,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ### Fixed
 
+- MQL export keeps a binary value's BSON subtype and writes it as a `BinData(...)` constructor, so running the script inserts the same bytes back. It wrote Extended JSON that mongosh reads as a plain object, and stamped every value as subtype 0. (#2086)
 - MongoDB no longer prints a binary field nested inside a document as a UUID when it is not one, or labels a UUID with the wrong byte order. (#2086)
 - Deleting a MongoDB document with a binary `_id` deletes that document. It used to match on the other fields and could remove the wrong one.
 - TablePro Mobile no longer gets killed by iOS when you leave the app with a DuckDB file open.

CLAUDE.md

@@ -199,7 +199,7 @@ These have caused real bugs when violated:
 
 **The data grid header owns all of its own chrome, so nothing may ask AppKit to paint any of it**: `NSTableHeaderCell` and `NSTableHeaderView` both paint a fixed 28pt band that they centre vertically in whatever frame they are given, a 16pt column divider on `midY` and a 1pt rule at `midY + 13`. The data grid grows its header to 42pt for a column comment, so that band lands mid-cell: the rule crosses the comment's descenders and sits 8pt above the real bottom edge. `SortableHeaderChrome` is therefore the single owner of header geometry and colours, `SortableHeaderCell.draw(withFrame:in:)` never calls `super`, and `SortableHeaderView.draw(_:)` fills the background and rules the bottom edge itself. The trap is that the header view paints a second copy of that same band for `NSTableView.highlightedTableColumn`, driven by *state* rather than by a drawing call, so no cell override can reach it: setting it gives the sorted column a stray divider and a rule no other column has. TablePro already draws the sorted-column affordance itself (bold title, chevron, priority number, with `drawSortIndicator` overridden to nothing), so `highlightedTableColumn` is a redundant second channel and must stay unset. All sorted-column presentation goes through `SortableHeaderView.applySortState(_:schema:)`, which publishes the order natively through `tableView.sortDescriptors` (for accessibility; it paints nothing) and updates the cells. `SortableHeaderRenderingTests` rasterises the header and guards this. This shipped as a rule through the comment line and a stray divider on the sorted column (#2017).
 
-**Decoding a MongoDB binary UUID is a per-column decision, and the column's type name is load-bearing**: BSON binary subtype 3 is the legacy UUID format, and the Java, C# and Python drivers each wrote it with a different byte order with nothing in the stored bytes to say which. `MongoDBUuidCodec` therefore decodes subtype 3 only when the connection names one (`mongoUuidRepresentation`); subtype 4 is unambiguous and always decodes. The choice is made once per column from `BsonDocumentFlattener.columnKinds`' majority vote, never per value, because a decoded cell is `.text` and an undecoded one is `.bytes`, and `CellDisplayFormatter` runs blob formatting over a `.text` cell whenever its column type is BLOB. One UUID decoded inside a column the app still types `BLOB` renders as `0x4c65676163...`. For the same reason `BsonDocumentFlattener.typeName` must return exactly `"BLOB"` for undecoded binary: `ColumnTypeClassifier` maps any name containing `BLOB` to `.blob`, and that classification is the only thing keeping a binary cell out of the inline editor. Once a column does decode, both edit guards (`isBlobType` and `asBytes != nil`) fall together, so every write path must parse the wrapper back to `$binary`: `MongoDBStatementGenerator.jsonValue` and `idValueJson`, `MongoDBQueryBuilder.jsonValue` plus its `=`, `!=` and `IN` arms (a case-insensitive regex can never match a binary field), and `MQLExportHelpers.mqlJsonValue`. An `_id` filter left as wrapper text matches zero documents while the UI reports the save succeeded. (#2086)
+**Decoding a MongoDB binary UUID is a per-column decision, and the column's type name is load-bearing**: BSON binary subtype 3 is the legacy UUID format, and the Java, C# and Python drivers each wrote it with a different byte order with nothing in the stored bytes to say which. `MongoDBUuidCodec` therefore decodes subtype 3 only when the connection names one (`mongoUuidRepresentation`); subtype 4 is unambiguous and always decodes. The choice is made once per column from `BsonDocumentFlattener.columnKinds`' majority vote, never per value, because a decoded cell is `.text` and an undecoded one is `.bytes`, and `CellDisplayFormatter` runs blob formatting over a `.text` cell whenever its column type is BLOB. One UUID decoded inside a column the app still types `BLOB` renders as `0x4c65676163...`. For the same reason `BsonDocumentFlattener.typeName` must keep `BLOB` as the base name for undecoded binary: `ColumnTypeClassifier` splits a type name at the first `(` and looks the base up, so `BLOB` and `BLOB(3)` both classify as `.blob`, and that classification is the only thing keeping a binary cell out of the inline editor. The parenthesised part carries the BSON subtype so MQL export can write it back; `MongoDBUuidCodec.columnTypeName(forSubtype:)` and `binarySubtype(fromColumnTypeName:)` are the only two places that spelling is produced or read, and MQL export is `supportedDatabaseTypeIds = ["MongoDB"]`, so it never sees another driver's `BLOB`. Once a column does decode, both edit guards (`isBlobType` and `asBytes != nil`) fall together, so every write path must parse the wrapper back to `$binary`: `MongoDBStatementGenerator.jsonValue` and `idValueJson`, `MongoDBQueryBuilder.jsonValue` plus its `=`, `!=` and `IN` arms (a case-insensitive regex can never match a binary field), and `MQLExportHelpers.mqlJsonValue`. An `_id` filter left as wrapper text matches zero documents while the UI reports the save succeeded. (#2086)
 
 **A MongoDB update or delete is anchored on `_id` or it does not run**: `generateDelete` used to fall back to a filter built from the remaining columns, which silently dropped every value it could not stringify (all binary) and then `deleteOne`d the first partial match, so a document with a binary `_id` could delete a different document. Both paths now skip with a logged warning instead, matching what `generateUpdate` already did.
 

Plugins/MQLExportPlugin/MQLExportHelpers.swift

@@ -24,6 +24,10 @@ enum MQLExportHelpers {
         return "db.\(escaped)"
     }
 
+    static func mqlBinaryValue(for data: Data, subtype: UInt8) -> String {
+        MongoDBUuidCodec.binaryText(for: MongoDBBinaryValue(data: data, subtype: subtype))
+    }
+
     static func mqlJsonValue(for value: String) -> String {
         if value == "true" || value == "false" {
             return value
@@ -37,8 +41,8 @@ enum MQLExportHelpers {
         if Double(value) != nil, value.contains(".") {
             return value
         }
-        if let binary = MongoDBUuidCodec.extendedJsonFromWrapper(value) {
-            return binary
+        if let binary = MongoDBUuidCodec.parseWrapper(value) {
+            return MongoDBUuidCodec.binaryText(for: binary)
         }
         if (value.hasPrefix("{") && value.hasSuffix("}")) ||
             (value.hasPrefix("[") && value.hasSuffix("]")) {

Plugins/MQLExportPlugin/MQLExportPlugin.swift

@@ -94,6 +94,7 @@ final class MQLExportPlugin: ExportFormatPlugin, SettablePlugin {
 
             if includeData {
                 var columns: [String] = []
+                var columnTypeNames: [String] = []
                 var documentBatch: [String] = []
 
                 let stream = dataSource.streamRows(table: table.name, databaseName: table.databaseName)
@@ -103,6 +104,7 @@ final class MQLExportPlugin: ExportFormatPlugin, SettablePlugin {
                     switch element {
                     case .header(let header):
                         columns = header.columns
+                        columnTypeNames = header.columnTypeNames
                     case .rows(let rows):
                         for row in rows {
                             var fields: [String] = []
@@ -114,7 +116,11 @@ final class MQLExportPlugin: ExportFormatPlugin, SettablePlugin {
                                 case .null:
                                     continue
                                 case .bytes(let data):
-                                    jsonValue = "{\"$binary\": {\"base64\": \"\(data.base64EncodedString())\", \"subType\": \"00\"}}"
+                                    let typeName = colIndex < columnTypeNames.count ? columnTypeNames[colIndex] : ""
+                                    jsonValue = MQLExportHelpers.mqlBinaryValue(
+                                        for: data,
+                                        subtype: MongoDBUuidCodec.binarySubtype(fromColumnTypeName: typeName)
+                                    )
                                 case .text(let value):
                                     jsonValue = MQLExportHelpers.mqlJsonValue(for: value)
                                 }

Plugins/MongoDBDriverPlugin/BsonDocumentFlattener.swift

@@ -9,12 +9,12 @@
 import Foundation
 import TableProPluginKit
 
-enum BsonValueKind {
+enum BsonValueKind: Hashable {
     case double
     case string
     case document
     case array
-    case binary
+    case binary(subtype: UInt8)
     case boolean
     case date
     case null
@@ -113,7 +113,7 @@ struct BsonDocumentFlattener {
         case .double: return "FLOAT"
         case .string, .null: return "VARCHAR"
         case .document, .array: return "JSON"
-        case .binary: return "BLOB"
+        case .binary(let subtype): return MongoDBUuidCodec.columnTypeName(forSubtype: subtype)
         case .boolean: return "BOOLEAN"
         case .date: return "TIMESTAMP"
         case .int32: return "INTEGER"
@@ -290,11 +290,11 @@ struct BsonDocumentFlattener {
             return .date
         case let binary as MongoDBBinaryValue:
             guard MongoDBUuidCodec.isDecodableUuid(binary, representation: representation) else {
-                return .binary
+                return .binary(subtype: binary.subtype)
             }
             return binary.subtype == MongoDBUuidCodec.standardUuidSubtype ? .uuid : .legacyUuid
         case is Data:
-            return .binary
+            return .binary(subtype: 0)
         case is [String: Any]:
             return .document
         case is [Any]:

Plugins/MongoDBDriverPlugin/MongoDBPlugin.swift

@@ -77,9 +77,9 @@ final class MongoDBPlugin: NSObject, TableProPlugin, DriverPlugin {
             label: String(localized: "Legacy UUID Encoding"),
             fieldType: .dropdown(options: [
                 .init(value: "", label: String(localized: "Do Not Decode")),
-                .init(value: "javaLegacy", label: String(localized: "Java")),
-                .init(value: "csharpLegacy", label: String(localized: "C#")),
-                .init(value: "pythonLegacy", label: String(localized: "Python")),
+                .init(value: "javaLegacy", label: "Java"),
+                .init(value: "csharpLegacy", label: "C#"),
+                .init(value: "pythonLegacy", label: "Python"),
             ]),
             section: .advanced
         ),
@@ -141,7 +141,13 @@ final class MongoDBPlugin: NSObject, TableProPlugin, DriverPlugin {
             CompletionEntry(label: ".findOneAndReplace", insertText: ".findOneAndReplace"),
             CompletionEntry(label: ".findOneAndDelete", insertText: ".findOneAndDelete"),
             CompletionEntry(label: ".countDocuments", insertText: ".countDocuments"),
-            CompletionEntry(label: ".createIndex", insertText: ".createIndex")
+            CompletionEntry(label: ".createIndex", insertText: ".createIndex"),
+            CompletionEntry(label: "ObjectId", insertText: "ObjectId"),
+            CompletionEntry(label: "ISODate", insertText: "ISODate"),
+            CompletionEntry(label: "NumberLong", insertText: "NumberLong"),
+            CompletionEntry(label: "NumberDecimal", insertText: "NumberDecimal"),
+            CompletionEntry(label: "BinData", insertText: "BinData"),
+            CompletionEntry(label: "UUID", insertText: "UUID")
         ]
     }
 

Plugins/TableProPluginKit/MongoDBUuidCodec.swift

@@ -44,6 +44,19 @@ public enum MongoDBUuidCodec {
         "BinData(\(binary.subtype), \"\(binary.data.base64EncodedString())\")"
     }
 
+    public static func columnTypeName(forSubtype subtype: UInt8) -> String {
+        subtype == 0 ? binaryColumnTypeName : "\(binaryColumnTypeName)(\(subtype))"
+    }
+
+    public static func binarySubtype(fromColumnTypeName name: String) -> UInt8 {
+        guard name.hasPrefix(binaryColumnTypeName + "("), name.hasSuffix(")") else { return 0 }
+        let start = name.index(name.startIndex, offsetBy: binaryColumnTypeName.count + 1)
+        let digits = name[start ..< name.index(before: name.endIndex)]
+        return UInt8(digits) ?? 0
+    }
+
+    private static let binaryColumnTypeName = "BLOB"
+
     public static func isDecodableUuid(
         _ binary: MongoDBBinaryValue,
         representation: MongoDBUuidRepresentation

Plugins/TableProPluginKit/MongoShellParser.swift

@@ -77,7 +77,8 @@ public struct MongoShellParser {
 
     /// Parse a MongoDB Shell expression into a MongoOperation
     public static func parse(_ input: String) throws -> MongoOperation {
-        let trimmed = input.trimmingCharacters(in: .whitespacesAndNewlines)
+        let trimmed = try MongoShellValueTranslator.translate(input)
+            .trimmingCharacters(in: .whitespacesAndNewlines)
 
         if trimmed.isEmpty {
             throw MongoShellParseError.invalidSyntax("Empty query")