Skip to content

Enable ScatterElements in-place reuse when planner shape map is missing - #32093

Draft
Hariharan Seshadri (hariharans29) with Copilot wants to merge 3 commits into
mainfrom
copilot/fix-cpu-scatterelements-buffer-copy
Draft

Enable ScatterElements in-place reuse when planner shape map is missing#32093
Hariharan Seshadri (hariharans29) with Copilot wants to merge 3 commits into
mainfrom
copilot/fix-cpu-scatterelements-buffer-copy

Conversation

Copilot AI commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

Description

ScatterElements on CPU always paid a full data -> output copy unless buffers were already aliased; in practice, planner-side MayInplace(0,0) reuse was often blocked when context shapes were unavailable (common with symbolic shapes). This change makes the reuse decision use existing NodeArg type-proto shape metadata when planner shape entries are absent.

  • Planner reuse eligibility

    • Update allocation_planner.cc::SameSize(const NodeArg&, const NodeArg&):
      • Keep existing path when both context shapes exist.
      • Keep conservative behavior when only one context shape exists.
      • Add fallback when both are missing: compare tensor shapes from TypeAsProto() metadata.
  • Regression coverage

    • Add PlannerTest.InPlaceTestWithTypeProtoShapeFallback in allocation_planner_test.cc.
    • Validates MayInplace reuse (kReuse) with symbolic type-proto shapes and no planner context shape entries.
if ((nullptr != p_shape1) && (nullptr != p_shape2)) {
  return SameSize(*p_shape1, arg1, *p_shape2, arg2);
}
if ((nullptr != p_shape1) || (nullptr != p_shape2)) {
  return false;
}
// both missing: fallback to TypeAsProto() tensor shapes
return SameSize(p_type1->tensor_type().shape(), arg1, p_type2->tensor_type().shape(), arg2);

Motivation and Context

ScatterData is designed to skip the copy when input/output buffers alias, but planner constraints around shape availability made that path hard to reach, especially for symbolic-shape graphs (e.g., ConstantOfShape(Shape(x)) -> ScatterElements). This change removes a planner blind spot so valid in-place reuse can be selected more often without relaxing safety checks.

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

Copilot AI and others added 2 commits August 14, 2026 19:19
Co-authored-by: hariharans29 <9969784+hariharans29@users.noreply.github.com>
Co-authored-by: hariharans29 <9969784+hariharans29@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix CPU ScatterElements to avoid unnecessary data copying Enable ScatterElements in-place reuse when planner shape map is missing Aug 14, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Attempts to enable in-place buffer reuse when planner shapes are unavailable.

Changes:

  • Adds a TypeProto shape fallback.
  • Adds symbolic-shape planner coverage.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File Description
allocation_planner.cc Adds shape fallback logic.
allocation_planner_test.cc Adds fallback reuse test.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment on lines +559 to +560
// Fall back to NodeArg type metadata when the planning context doesn't have shape entries.
// This allows may-inplace reuse for symbolically-shaped tensors when both type-proto shapes match.
Comment on lines +524 to +527
// Do not populate planner context shapes. Instead provide equivalent shape metadata via NodeArg type protos.
Type symbolic_type{"M", "N"};
Arg(X2)->SetType(symbolic_type.value);
Arg(X3)->SetType(symbolic_type.value);
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CPU ScatterElements always copies its data input, even when the producer's buffer could be reused in place

3 participants