Arm backend: Add 16A8W FCNode support with BMM dependency fix #13801

Ninja91 · 2025-08-29T06:43:04Z

Stack from ghstack (oldest at bottom):

Add 16A8W quantization support for FCNode operations with BMM dependency fix in ExecutorTorch ARM backend.

This follows the pattern established for linear, mul, sigmoid, tanh, slice, view/transpose, and cat operations, extending int16 support to FCNode operations.

Changes:

Add INT16 dtype validation support in op_bmm.py
Add test_addmm_tensor_16a8w_tosa_INT test function
Enable test_addmm.py in test targets configuration
Fix BMM dependency for FCNode operations

The 16A8W configuration uses 16-bit activations with 8-bit weights, enabling higher precision for activations while maintaining weight efficiency.

Differential Revision: D80512504

cc @digantdesai @freddan80 @per @zingo @oscarandersson8218

Add 16A8W quantization support for FCNode operations with BMM dependency fix in ExecutorTorch ARM backend. This follows the pattern established for linear, mul, sigmoid, tanh, slice, view/transpose, and cat operations, extending int16 support to FCNode operations. Changes: - Add INT16 dtype validation support in op_bmm.py - Add test_addmm_tensor_16a8w_tosa_INT test function - Enable test_addmm.py in test targets configuration - Fix BMM dependency for FCNode operations The 16A8W configuration uses 16-bit activations with 8-bit weights, enabling higher precision for activations while maintaining weight efficiency. Differential Revision: [D80512504](https://our.internmc.facebook.com/intern/diff/D80512504/) [ghstack-poisoned]

pytorch-bot · 2025-08-29T06:43:08Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/13801

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 15 New Failures, 6 Unrelated Failures

As of commit e8418d4 with merge base 6208340 ():

NEW FAILURES - The following jobs have failed:

Lint / lintrunner / linux-job (gh)
>>> Lint for backends/arm/operators/op_bmm.py:
pull / unittest / linux / linux-job (gh)
RuntimeError: Command docker exec -t da8acb6822bb79ddb854e2ec79f028a19fcfd96428bfcb63a3d9a20738fc649c /exec failed with exit code 3
pull / unittest / macos / macos-job (gh)
RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 3
pull / unittest-arm-backend-with-no-fvp (test_pytest_models) / linux-job (gh)
RuntimeError: Command docker exec -t 23d960c8e7b33c3ca918ec585f12195c331fa74b69691755497df16d41b87197 /exec failed with exit code 3
pull / unittest-arm-backend-with-no-fvp (test_pytest_ops) / linux-job (gh)
RuntimeError: Command docker exec -t 4192bb3dc9b59efad78ba43a3fc8d50be6dea8894af4567d610f9ad8702b90c8 /exec failed with exit code 3
pull / unittest-editable / linux / linux-job (gh)
RuntimeError: Command docker exec -t dd6c74874fca665751bcb31260ebb652e6a7838825bb3d8b849333daa8065474 /exec failed with exit code 3
pull / unittest-editable / macos / macos-job (gh)
RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 3
trunk / test-arm-backend (test_pytest_models_ethosu_fvp) / linux-job (gh)
RuntimeError: Command docker exec -t a2200c2dd33966a8ba4b905cd26fb05d07258f6c32e59d213d864b3d2bda06a5 /exec failed with exit code 3
trunk / test-arm-backend (test_pytest_ops_ethosu_fvp) / linux-job (gh)
RuntimeError: Command docker exec -t f1ad60519f99da2ee1a055aa71f78c3ab921c84f1b1275df9f9e5ce8b11126d4 /exec failed with exit code 3
trunk / test-arm-backend (test_smaller_stories_llama) / linux-job (gh)
RuntimeError: Command docker exec -t b5a02937cba0dcf21491cf0ebd0efed1daa5924a74e3c842ccaaf8b6caa4d893 /exec failed with exit code 3
trunk / test-huggingface-transformers-coreml (qwen3-1.7b|xnnpack|--quantize) / macos-job (gh)
RuntimeError: Command bash /Users/runner/work/_temp/exec_script failed with exit code 139
trunk / test-llama-runner-mac (fp32, coreml) / macos-job (gh)
RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 1
trunk / test-models-macos-coreml (mobilebert) / macos-job (gh)
RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 139
trunk / unittest-release / linux / linux-job (gh)
RuntimeError: Command docker exec -t c0a28d70602dbb442d83840619cf3c0a0f202b8bad8837c8270320a299a5e6fe /exec failed with exit code 3
trunk / unittest-release / macos / macos-job (gh)
RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 3

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

pull / test-llama_runner_eager-linux / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 128
trunk / test-models-macos-cpu (mv3, xnnpack-quantization-delegation) / macos-job (gh) (detected as infra flaky with no log or failing log classifier)

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

Apple / build-benchmark-app / macos-job (gh) (trunk failure)
RuntimeError: Command bash /Users/runner/work/_temp/exec_script failed with exit code 1
Apple / build-demo-ios / macos-job (gh) (trunk failure)
RuntimeError: Command bash /Users/runner/work/_temp/exec_script failed with exit code 1
Apple / build-frameworks-ios / macos-job (gh) (trunk failure)
RuntimeError: Command bash /Users/runner/work/_temp/exec_script failed with exit code 1
trunk / test-coreml-delegate / macos-job (gh) (trunk failure)
RuntimeError: Command bash /Users/runner/work/_temp/exec_script failed with exit code 1

This comment was automatically generated by Dr. CI and updates every 15 minutes.

github-actions · 2025-08-29T06:43:38Z

This PR needs a `release notes:` label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

facebook-github-bot · 2025-08-29T06:44:00Z

This pull request was exported from Phabricator. Differential Revision: D80512504

digantdesai · 2025-08-29T20:10:59Z

backends/arm/operators/op_bmm.py

@@ -93,7 +93,8 @@ def define_node(
        if output.dtype == ts.DType.INT8:
            output_qparams = get_output_qparams(node)[0]
            final_output_scale = (
-                input_qparams[0].get_scale_per_tensor() * input_qparams[1].get_scale_per_tensor()  # type: ignore[possibly-undefined]  # pyre-ignore[61]
+                input_qparams[0].get_scale_per_tensor()


digantdesai · 2025-08-29T20:11:24Z

backends/arm/test/ops/test_addmm.py

+
+
+@common.parametrize("test_data", test_data_suite)
+def test_addmm_16a8w_tosa_INT(test_data: input_t1):


ok so this passes but linear doesn't, interesting :)

digantdesai

Please fix mypy before landing. Code dupe issue with the quant helper.

Ninja91 requested a review from digantdesai as a code owner August 29, 2025 06:43

meta-cla bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Aug 29, 2025

facebook-github-bot added the fb-exported label Aug 29, 2025

zingo added ciflow/trunk module: arm Issues related to arm backend partner: arm For backend delegation, kernels, demo, etc. from the 3rd-party partner, Arm labels Aug 29, 2025

zingo changed the title ~~Add 16A8W FCNode support with BMM dependency fix~~ Arm backend: Add 16A8W FCNode support with BMM dependency fix Aug 29, 2025

digantdesai reviewed Aug 29, 2025

View reviewed changes

digantdesai approved these changes Aug 29, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Arm backend: Add 16A8W FCNode support with BMM dependency fix #13801

Arm backend: Add 16A8W FCNode support with BMM dependency fix #13801

Uh oh!

Ninja91 commented Aug 29, 2025 •

edited by pytorch-bot bot

Loading

Uh oh!

pytorch-bot bot commented Aug 29, 2025 •

edited

Loading

Uh oh!

github-actions bot commented Aug 29, 2025

Uh oh!

facebook-github-bot commented Aug 29, 2025

Uh oh!

digantdesai Aug 29, 2025

Uh oh!

digantdesai Aug 29, 2025

Uh oh!

digantdesai left a comment

Uh oh!

Uh oh!



		@common.parametrize("test_data", test_data_suite)
		def test_addmm_16a8w_tosa_INT(test_data: input_t1):

Arm backend: Add 16A8W FCNode support with BMM dependency fix #13801

Are you sure you want to change the base?

Arm backend: Add 16A8W FCNode support with BMM dependency fix #13801

Uh oh!

Conversation

Ninja91 commented Aug 29, 2025 • edited by pytorch-bot bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-bot bot commented Aug 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/13801

❌ 15 New Failures, 6 Unrelated Failures

Uh oh!

github-actions bot commented Aug 29, 2025

This PR needs a release notes: label

Uh oh!

facebook-github-bot commented Aug 29, 2025

Uh oh!

digantdesai Aug 29, 2025

Choose a reason for hiding this comment

Uh oh!

digantdesai Aug 29, 2025

Choose a reason for hiding this comment

Uh oh!

digantdesai left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Ninja91 commented Aug 29, 2025 •

edited by pytorch-bot bot

Loading

pytorch-bot bot commented Aug 29, 2025 •

edited

Loading

This PR needs a `release notes:` label