Conversation
There was a problem hiding this comment.
Code Review
This pull request introduces a new internal package, flutter_goldens, to enable Skia Gold testing for packages within the flutter/packages repository. It adds the necessary client logic, environment-aware comparators, and test configurations, with two_dimensional_scrollables serving as the initial implementation example. My review has identified some critical and high-severity issues, including a syntax error that will prevent compilation, incorrect Skia Gold configuration, and fragile logic for naming golden files. I have also noted several medium-severity issues related to code duplication, debugging artifacts, and dependency management that should be addressed to improve maintainability. Overall, this is a significant and valuable addition, and my feedback aims to ensure its correctness and robustness.
|
Note self for tomorrow: Use the SDK path environment variable, this appears more stable cross CI platforms -flutter +packages. |
|
autosubmit label was removed for flutter/packages/10753, because - The status or check suite Linux_web web_dart_unit_test_wasm_shard_2 master has failed. Please fix the issues identified (or deflake) before re-applying this label.
|
…the design migration (#11649) This branches off of #10753 and strips it down to just the skipping comparator. This means when golden file testing is used with this version of flutter_goldens, `matchesGoldenFile` will return true in all environments without executing image comparison. The goal of this PR is to join the material_ui and cupertino_ui mega-migration PR (cc @justinmc ) to disable golden file testing for now in material_ui and cupertino_ui so we can get a clearer signal from CI about any remaining blockers. Full golden file testing support will likely land in #10753 as a separate step given the enormity of these PRs. cupertino_ui and material_ui are set `publish_to: none` here, which is intentional. This allows us to split up these changes, and wait to publish until everything lands and is working as we want it to. ## Pre-Review Checklist **Note**: The Flutter team is currently trialing the use of [Gemini Code Assist for GitHub](https://developers.google.com/gemini-code-assist/docs/review-github-code). Comments from the `gemini-code-assist` bot should not be taken as authoritative feedback from the Flutter team. If you find its comments useful you can update your code accordingly, but if you are unsure or disagree with the feedback, please feel free to wait for a Flutter team member's review for guidance on which automated comments should be addressed. [^1]: Regular contributors who have demonstrated familiarity with the repository guidelines only need to comment if the PR is not auto-exempted by repo tooling.
…the design migration (flutter#11649) This branches off of flutter#10753 and strips it down to just the skipping comparator. This means when golden file testing is used with this version of flutter_goldens, `matchesGoldenFile` will return true in all environments without executing image comparison. The goal of this PR is to join the material_ui and cupertino_ui mega-migration PR (cc @justinmc ) to disable golden file testing for now in material_ui and cupertino_ui so we can get a clearer signal from CI about any remaining blockers. Full golden file testing support will likely land in flutter#10753 as a separate step given the enormity of these PRs. cupertino_ui and material_ui are set `publish_to: none` here, which is intentional. This allows us to split up these changes, and wait to publish until everything lands and is working as we want it to. ## Pre-Review Checklist **Note**: The Flutter team is currently trialing the use of [Gemini Code Assist for GitHub](https://developers.google.com/gemini-code-assist/docs/review-github-code). Comments from the `gemini-code-assist` bot should not be taken as authoritative feedback from the Flutter team. If you find its comments useful you can update your code accordingly, but if you are unsure or disagree with the feedback, please feel free to wait for a Flutter team member's review for guidance on which automated comments should be addressed. [^1]: Regular contributors who have demonstrated familiarity with the repository guidelines only need to comment if the PR is not auto-exempted by repo tooling.
There was a problem hiding this comment.
Code Review
This pull request integrates Skia Gold into the flutter/packages repository's CI and local testing environments. It adds the goldctl dependency to .ci.yaml targets and introduces several golden file comparators along with a SkiaGoldClient to handle authentication, initialization, and image uploads. Feedback on the changes includes fixing an invalid Dart list literal syntax, correcting the path to result-state.json inside the temp directory, adding HTTP status code checks before processing network responses, handling potential missing environment variables, and catching io.IOException instead of specific socket exceptions for more robust error handling.
Adds Skia Gold integration (
flutter_goldens) to theflutter/packagesrepository (targeting theflutter-packagesGold instance) to support golden file testing across packages such asmaterial_uiandcupertino_ui.Temporary Rollout Behavior (
TODO(Piinks)): While theflutter-packagesGold dashboard (https://flutter-packages-gold.skia.org) is being stood up,FlutterPostSubmitFileComparatorandFlutterPreSubmitFileComparatorupload images and results to Skia Gold so the backend can ingest initial data, while logging anySkiaExceptions and returningtrueto avoid blocking CI until the instance frontend is live.Nearly identical to flutter/flutter instance, adapting to unique aspects of CI between repos and naming conventions/directory structure for packages.
Part of flutter/flutter#172933
Pre-Review Checklist
[shared_preferences]pubspec.yamlwith an appropriate new version according to the pub versioning philosophy, or I have commented below to indicate which version change exemption this PR falls under1.CHANGELOG.mdto add a description of the change, following repository CHANGELOG style, or I have commented below to indicate which CHANGELOG exemption this PR falls under1.///).If you need help, consider asking for advice on the #hackers-new channel on Discord.
Note: The Flutter team is currently trialing the use of Gemini Code Assist for GitHub. Comments from the
gemini-code-assistbot should not be taken as authoritative feedback from the Flutter team. If you find its comments useful you can update your code accordingly, but if you are unsure or disagree with the feedback, please feel free to wait for a Flutter team member's review for guidance on which automated comments should be addressed.Footnotes
Regular contributors who have demonstrated familiarity with the repository guidelines only need to comment if the PR is not auto-exempted by repo tooling. ↩ ↩2 ↩3