[mxfp8 moe training] test floor vs rceil in 3d quantization benchmarks

[danielvegamyhre]

Stacked PRs:

• [mxfp8 moe training] update readme with kernel microbenchmarks for dsv3 #3521
• ->[mxfp8 moe training] test floor vs rceil in 3d quantization benchmarks #3507
• [mxfp8 moe training] update benchmark plots to use new kernel #3506
• [mxfp8 moe training] integrate new cuda kernel for blocked layout for groups along K #3505
• [mxfp8 moe training] add CUDA kernel for per-group conversion of scale factors to blocked layout #3504

---

[mxfp8 moe training] test floor vs rceil in 3d quantization benchmarks

input_shape       scaling_mode                  to_mx_us    cuda_2d_us    cuda_3d_us    to_mx_gbps    cuda_2d_gbps    cuda_3d_gbps
----------------  --------------------------  ----------  ------------  ------------  ------------  --------------  --------------
(1, 8192, 5120)   ScaleCalculationMode.FLOOR     117.76         30.816        32.8        1079.65          4125.77         3876.22
(1, 8192, 5120)   ScaleCalculationMode.RCEIL     121.888        30.784        30.752      1043.09          4130.06         4134.36
(2, 8192, 5120)   ScaleCalculationMode.FLOOR     377.856       103.424        59.392       672.954         2458.61         4281.38
(2, 8192, 5120)   ScaleCalculationMode.RCEIL     388.256       101.376        53.472       654.928         2508.28         4755.38
(4, 8192, 5120)   ScaleCalculationMode.FLOOR     747.52        212.16        119.904       680.329         2397.06         4241.39
(4, 8192, 5120)   ScaleCalculationMode.RCEIL     789.536       207.808       101.312       644.124         2447.26         5019.73
(8, 8192, 5120)   ScaleCalculationMode.FLOOR    1504.27        408.608       240.752       676.153         2489.23         4224.76
(8, 8192, 5120)   ScaleCalculationMode.RCEIL    1539.1         465.888       220.16        660.851         2183.18         4619.91
(16, 8192, 5120)  ScaleCalculationMode.FLOOR    3018.75        727.168       486.4         673.867         2797.48         4182.23
(16, 8192, 5120)  ScaleCalculationMode.RCEIL    3083.34        746.512       447.584       659.75          2724.99         4544.93
(64, 8192, 5120)  ScaleCalculationMode.FLOOR   11624          3294.14       1904.61        700.016         2470.13         4272.24
(64, 8192, 5120)  ScaleCalculationMode.RCEIL   12020.9        3178.5        1777.01        676.901         2560            4579.02

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/ao/3507

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

⏳ No Failures, 7 Pending

As of commit 3d4d7b1 with merge base 7035fb7 ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.