Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[GPU] Enable f4_e2m1 jit gemm #2442

Merged
merged 7 commits into from
Feb 6, 2025
Merged

[GPU] Enable f4_e2m1 jit gemm #2442

merged 7 commits into from
Feb 6, 2025

Conversation

kealan-barbieri
Copy link
Contributor

@kealan-barbieri kealan-barbieri commented Jan 17, 2025 •
edited
Loading

Description

  • Enable f4_e2m1 in jit::gemm.

Partially covers MFDNN-124711

Checklist

General

  • Do all unit and benchdnn tests (make test and make test_benchdnn_*) pass locally for each commit?
  • Have you formatted the code using clang-format?

@kealan-barbieri kealan-barbieri added the platform:gpu-intel Codeowner: @oneapi-src/onednn-gpu-intel label Jan 17, 2025
@kealan-barbieri kealan-barbieri requested review from a team as code owners January 17, 2025 18:23
@github-actions github-actions bot added the component:tests Codeowner: @oneapi-src/onednn-arch label Jan 17, 2025
petercad
petercad reviewed Jan 17, 2025
View reviewed changes
src/gpu/intel/jit/gemm/generator/pieces/copy_plan.cpp
// cmp (ge) t0:w, y:w, 31
// shr y:uw, 10
// csel (ge) y:fp16, 0x7bff, y:fp16, t0:fp16
// csel (ze) y:fp16, NaN:fp16, y:fp16, t1:fp16
Copy link
Contributor

@petercad petercad Jan 17, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Side note: there's a much faster sequence, though this is OK for now:

shl   t0:ud  x:ub   24
add   t0:ud  t0:ud  1
mov   y:hf   t0:f

@kealan-barbieri kealan-barbieri force-pushed the kealanba/f4_e2m1_gemm branch 3 times, most recently from 4a4aabf to b823c25 Compare January 18, 2025 00:20
@kealan-barbieri
Copy link
Contributor Author

make test
disable test_device_cpu
disable build_cpu_runtime_omp
disable build_cpu_runtime_sycl
disable build_cpu_runtime_tbb
disable benchdnn_all
enable benchdnn_matmul
enable benchdnn_ip

@echeresh
Copy link
Contributor

@kealan-barbieri Do we have f4_e2m1 coverage in benchdnn input files? If missing, can you please add some?

In the long term #2434 should help with that.

@kealan-barbieri
Copy link
Contributor Author

@echeresh there is existing coverage: https://github.com/oneapi-src/oneDNN/blob/main/tests/benchdnn/inputs/matmul/test_matmul_fp4

@kealan-barbieri kealan-barbieri force-pushed the kealanba/f4_e2m1_gemm branch 2 times, most recently from 91cfc9b to 8a4fc5e Compare January 30, 2025 22:04
@github-actions github-actions bot removed the component:tests Codeowner: @oneapi-src/onednn-arch label Jan 30, 2025
@kealan-barbieri kealan-barbieri force-pushed the kealanba/f4_e2m1_gemm branch 2 times, most recently from 6cf6814 to 895871f Compare January 31, 2025 23:03
@kealan-barbieri kealan-barbieri force-pushed the kealanba/f4_e2m1_gemm branch 4 times, most recently from b4fbafd to a21bc13 Compare February 4, 2025 01:42
@kealan-barbieri
Copy link
Contributor Author

make test
disable test_device_cpu
disable build_cpu_runtime_omp
disable build_cpu_runtime_sycl
disable build_cpu_runtime_tbb
disable benchdnn_all
enable benchdnn_matmul
enable benchdnn_ip

@kealan-barbieri
Copy link
Contributor Author

make test
disable test_device_cpu
disable build_cpu_runtime_omp
disable build_cpu_runtime_sycl
disable build_cpu_runtime_tbb
disable benchdnn_all
enable benchdnn_matmul
enable benchdnn_ip

@kealan-barbieri
Copy link
Contributor Author

make test perf-gpu
set primitive=matmul

@kealan-barbieri
Copy link
Contributor Author

make test
disable test_device_cpu
disable build_cpu_runtime_omp
disable build_cpu_runtime_sycl
disable build_cpu_runtime_tbb
disable benchdnn_all
enable benchdnn_matmul
enable benchdnn_ip

@kealan-barbieri kealan-barbieri merged commit 3028a0b into main Feb 6, 2025
15 of 24 checks passed
@kealan-barbieri kealan-barbieri deleted the kealanba/f4_e2m1_gemm branch February 6, 2025 23:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@mgouicem mgouicem mgouicem left review comments

@petercad petercad petercad approved these changes

@atkassen atkassen atkassen left review comments

@dzarukin dzarukin dzarukin approved these changes

Assignees
No one assigned
Labels
platform:gpu-intel Codeowner: @oneapi-src/onednn-gpu-intel
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@kealan-barbieri @echeresh @mgouicem @dzarukin @petercad @atkassen