Skip to content

fix: report the right line and column for CRLF and mixed line endings - #642

Open
feiiiiii5 wants to merge 1 commit into
python-poetry:masterfrom
feiiiiii5:c-144-linecol-crlf
Open

feiiiiii5 wants to merge 1 commit into
python-poetry:masterfrom
feiiiiii5:c-144-linecol-crlf

Conversation

@feiiiiii5

Copy link
Copy Markdown

Summary

The line and column reported in a parse error depend on the document's line endings. Source._to_linecol splits the source with splitlines(), which drops the terminators, and then assumes every break costs exactly one character:

cur += len(line) + 1

A \r\n break costs two, so the running offset falls one character behind for every line before the error, and the error lands on the wrong line and the wrong column. exceptions.py documents that a ParseError "references the line and location within the line where the error was encountered", and toml_file.py:36-44 only normalises \r\n when the whole document uses it consistently — so a file with mixed endings read from disk reaches this path today.

The fix lets each split line carry its own terminator (splitlines(keepends=True)) and uses that length, so the offset matches the characters actually consumed. Documents that use only \n come out identical; CRLF and mixed documents now report the position the LF form already did.

Test: pytest tests/test_parser.py -k position fails on main (858186a) — the CRLF case reports UnexpectedCharError("Unexpected character: '@' at line 4 col 0") where the character is on line 3, column 4 — and passes on this branch. The full suite (pytest --ignore=tests/test_toml_tests.py -q) is 420 passed, and ruff check / ruff format --check at the version pinned in .pre-commit-config.yaml (0.16.10) are clean. I also walked every character index of every file under tests/examples and compared the reported position before and after: 6913 positions disagreed for CRLF copies of those documents and 0 disagree after the change, while LF-only documents show 0 differences either way.

Not run: tests/test_toml_tests.py, because the tests/toml-test submodule is empty in this checkout. I read it — it asserts TOMLKitError is raised for invalid documents and compares parsed JSON for valid ones, and does not look at line/column — but that is a reading, not a run.

Agent Drafting Metadata

  • Agent: Qoder (AI coding assistant), autonomous contribution
  • Model: not disclosed by the agent host
  • Notes: diagnosis and patch drafted with the assistant; every number above comes from commands run on this checkout. AI-assisted development. Human review requested before merge.

@feiiiiii5

Copy link
Copy Markdown
Author

The one failing check here is not from this change. Integration / Ubuntu / 3.12 / poetry fails inside Poetry's own suite, not tomlkit's:

FAILED tests/utils/test_helpers.py::test_extractall_sdist_no_symlink_path_traversal_via_directory_symlink
E   Failed: DID NOT RAISE OutsideDestinationError
1 failed, 3309 passed, 34 skipped, 2 xfailed in 56.64s

The same job fails identically on other open PRs that do not touch parsing — run 38073613876 on fix-array-inplace-repeat-202 (2026-10-10 17:54) reports the same test with 1 failed, 3309 passed, and run 37986342804 (2026-10-09) also fails that job. master's own integration run passed on 2026-10-07, so this looks like Poetry's CI moving under the integration job rather than anything in this branch.

Every other check passes: the 3.9-3.13 test matrix on Ubuntu, macOS and Windows, and the five other integration jobs. Failure log: https://github.com/python-poetry/tomlkit/actions/runs/38060239938/job/114236854801

@feiiiiii5

Copy link
Copy Markdown
Author

The Integration / Ubuntu / 3.12 / poetry failure is not from this change. It is poetry's own tests/utils/test_helpers.py::test_extractall_sdist_no_symlink_path_traversal_via_directory_symlink (Failed: DID NOT RAISE OutsideDestinationError), and that job checks out poetry's current main rather than a pinned tag, so it moves independently of tomlkit. Two other open PRs fail it identically with the same totals — run 38073613876 (head 0c09e04) and run 37986342804 (head b480695): 1 failed, 3309 passed, 34 skipped, 2 xfailed.

This branch's own Tests run succeeded on head 911db3d, and the diff is tomlkit/source.py plus tests/test_parser.py only — _to_linecol cannot reach tar extraction.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant