You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
#5084 added SME for the "small matrix" SGEMM pathway but needs some small tweaks to connect the M4 cpu target to it
#5011 has a more general SME GEMM kernel but needs fixes for proper SYMM/TRMM support before it can be merged
Based on a SC24 workshop Hello SME, llvm/llvm-project#114987 and llvm/llvm-project#95478 . Apple M4 does not support SVE outside of streaming.
However, concurrent [WIP] #5011 is on top of KERNEL.ARMV8SVE. Result in illegal instruction. Any good ideas to solve that? Is create a new KERNEL.M4SME2 based on KERNEL.ARMV8 a good idea?
Further more, I have made some test on differences between SME1 and SME2 recently. It's quiet different to achieve best performance.
I don‘t known if ACLE could fully utilize these resources.
Yes, M4 only does streaming SVE so you'd need at least some setup code to enter streaming mode and perhaps save some dual-use registers beforehand, or even work in a totally different set of registers than what the existing SVE code uses.
Both #5011 and #5084 introduced an ARMV9SME target for differentiation, it would also be possible to select kernel implementations (either at the KERNEL file level or within individual implementations) based on HAVE_SME or a similiar define. As #5011 is a WIP only concerned with GEMM and related functions, it does not work outside its narrow scope.
The way forward - at least short-term - should be to split out M4 from the general "VORTEX" target into its own designation and enable the SME-based "small gemm" pathway for it. I hope to complete this very soon.
the fast path for small matrix sgemm should be working on M4 with #5222 - but I'm now stuck on an illegal instruction error involving cntd/cntw myself, trying to get dot_kernel_sve working in streaming mode with the __arm_streaming attribute
the fast path for small matrix sgemm should be working on M4 with #5222 - but I'm now stuck on an illegal instruction error involving cntd/cntw myself, trying to get dot_kernel_sve working in streaming mode with the __arm_streaming attribute
Activity
martin-frbg commentedon May 23, 2024
PRs welcome... do you have the hardware to test ?
oscarbg commentedon May 24, 2024
Not yet.. waiting for a mac mini m4..
brada4 commentedon May 24, 2024
martin-frbg commentedon May 24, 2024
try reading that again, it's about SME...
brada4 commentedon May 24, 2024
Mousius commentedon May 25, 2024
You can develop and test using the Fixed Virtual Platform (FVP):
apache/tvm#16755
apache/tvm#16749
brada4 commentedon May 25, 2024
martin-frbg commentedon Aug 16, 2024
some implementation hints there: https://scalable.uni-jena.de/opt/sme/index.html
oscarbg commentedon Nov 22, 2024
Hi,
any news?
have a Mac Mini m4 to test..
martin-frbg commentedon Nov 22, 2024
Bought an M4 mini myself recently but have not gotten around to doing much with it yet.
martin-frbg commentedon Feb 22, 2025
#5084 added SME for the "small matrix" SGEMM pathway but needs some small tweaks to connect the M4 cpu target to it
#5011 has a more general SME GEMM kernel but needs fixes for proper SYMM/TRMM support before it can be merged
ITCJ commentedon Mar 27, 2025
Based on a SC24 workshop Hello SME, llvm/llvm-project#114987 and llvm/llvm-project#95478 . Apple M4 does not support SVE outside of streaming.
However, concurrent [WIP] #5011 is on top of
KERNEL.ARMV8SVE. Result in illegal instruction. Any good ideas to solve that? Is create a newKERNEL.M4SME2based onKERNEL.ARMV8a good idea?ITCJ commentedon Mar 27, 2025
Further more, I have made some test on differences between SME1 and SME2 recently. It's quiet different to achieve best performance.
I don‘t known if ACLE could fully utilize these resources.
martin-frbg commentedon Mar 27, 2025
Yes, M4 only does streaming SVE so you'd need at least some setup code to enter streaming mode and perhaps save some dual-use registers beforehand, or even work in a totally different set of registers than what the existing SVE code uses.
Both #5011 and #5084 introduced an ARMV9SME target for differentiation, it would also be possible to select kernel implementations (either at the KERNEL file level or within individual implementations) based on HAVE_SME or a similiar define. As #5011 is a WIP only concerned with GEMM and related functions, it does not work outside its narrow scope.
The way forward - at least short-term - should be to split out M4 from the general "VORTEX" target into its own designation and enable the SME-based "small gemm" pathway for it. I hope to complete this very soon.
violet73 commentedon Apr 10, 2025
Hi, I would like to ask why I encountered the following error on M4pro:
Is it possible that my compiler does not recognize streaming flags?
Compilation:
clang -g -O0 -march=armv9.2-a+sme+sme2 ./test_sme_acle.cc -o ./test_sme_acle
Clang version:
Homebrew clang version 20.1.2
Target: arm64-apple-darwin24.3.0
Thread model: posix
InstalledDir: /opt/homebrew/Cellar/llvm/20.1.2/bin
ITCJ commentedon Apr 10, 2025
looks like the same problem mentioned above. Try using
disassemble --mixedto show illegal instruction.violet73 commentedon Apr 10, 2025
Thank you for your kindly reply, I disassembled it in lldb and the illegel instruction turns out to be
cntd!That means the streaming flags will make the compiler add some illegal sve instructions that are not in streaming mode.
This is somehow wired. Because I can't manually set the streaming mode before main is called.
So I tried to remove the streaming flags in main and moved the sve code into another function
foowith the local streaming flags.
I also manually placed the invocation statement of
foowithinsmstartandsmstop.After these, the code could finally run normally!
ITCJ commentedon Apr 10, 2025
congrats， I also tried resolve similar issues. I encounter
ADVLduring unit test.加个微信？码发你邮箱了捏。
martin-frbg commentedon Apr 11, 2025
the fast path for small matrix sgemm should be working on M4 with #5222 - but I'm now stuck on an illegal instruction error involving cntd/cntw myself, trying to get dot_kernel_sve working in streaming mode with the __arm_streaming attribute
ITCJ commentedon Apr 14, 2025
Is it a bug of LLVM compiler?
martin-frbg commentedon Mar 22, 2026
Closing as the VORTEXM4 target received SME-based SGEMM/SSYMM/STRMM/SSYRK/SSYR2K in 0.3.31