Skip to content

[CPU] [API CONFORMANCE] Support unsupported property OPTIMAL_BATCH_SIZE in CPU #28948

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 4 commits into
base: master
Choose a base branch
from
Open

[CPU] [API CONFORMANCE] Support unsupported property OPTIMAL_BATCH_SIZE in CPU #28948

wants to merge 4 commits into from
+30 −2

Conversation

@tiger100256-hu
Copy link
Contributor

@tiger100256-hu tiger100256-hu commented Feb 12, 2025 •
edited
Loading

Details:

  • rebase the pr 23882 and continue to finish the code
  • return value 1 for property ov::optimal_batch_size
  • return nullptr for property ov::hint::model
  • when property ov::optimal_batch_size of CPU plugin is enabled , core would apply auto batch on CPU, because this PR only make the property working to pass API CONFORMANCE test, but not really implement the function, so disable apply auto batch on CPU

Tickets:

@github-actions github-actions bot added category: IE Tests OpenVINO Test: plugins and common category: CPU OpenVINO CPU plugin labels Feb 12, 2025
@tiger100256-hu tiger100256-hu mentioned this pull request Feb 12, 2025
Closed
@github-actions github-actions bot added the category: inference OpenVINO Runtime library - Inference label Feb 17, 2025
@tiger100256-hu tiger100256-hu marked this pull request as ready for review February 18, 2025 09:06
@tiger100256-hu tiger100256-hu requested review from a team as code owners February 18, 2025 09:06
karanjakhar and others added 4 commits March 5, 2025 15:06
@github-actions
Copy link
Contributor

github-actions bot commented Apr 5, 2025

This PR will be closed in a week because of 2 weeks of no activity.

@github-actions github-actions bot added the Stale label Apr 5, 2025
@wenjiew wenjiew removed the Stale label Apr 9, 2025
@wenjiew wenjiew added this to the 2025.2 milestone Apr 9, 2025
@github-actions
Copy link
Contributor

This PR will be closed in a week because of 2 weeks of no activity.

@github-actions github-actions bot added the Stale label Apr 24, 2025
@github-actions
Copy link
Contributor

github-actions bot commented May 3, 2025

This PR was closed because it has been stalled for 2 week with no activity.

@github-actions github-actions bot closed this May 3, 2025
@yuxu42 yuxu42 reopened this May 7, 2025
@yuxu42 yuxu42 added no_stale Do not mark as stale and removed Stale labels May 7, 2025
usstq
usstq approved these changes May 7, 2025
View reviewed changes
Copy link
Contributor

@usstq usstq left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, but seems to be a meaningless effort.

FYI:
https://docs.openvino.ai/2025/openvino-workflow/running-inference/inference-devices-and-modes/automatic-batching.html#automatic-batch-size-selection

praasz
praasz reviewed May 9, 2025
View reviewed changes
src/inference/src/dev/core_impl.cpp
auto it = std::find(metrics.begin(), metrics.end(), ov::optimal_batch_size.name());
if (metrics.end() == it)
// not apply auto batch for CPU plugin
if (metrics.end() == it || parsed._deviceName == "CPU") {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There should not be customization for single device.
If auto batch should not be applied here maybe plugin should not return it as supported property and old condition will be enough.

Here should be generic logic not per device.

@praasz praasz modified the milestones: 2025.2, 2025.3 Jun 2, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@praasz praasz praasz left review comments

@usstq usstq usstq approved these changes

@riverlijunjie riverlijunjie Awaiting requested review from riverlijunjie

@songbell songbell Awaiting requested review from songbell

@iefode iefode Awaiting requested review from iefode

@dmitry-gorokhov dmitry-gorokhov Awaiting requested review from dmitry-gorokhov

Assignees

No one assigned

Labels

category: CPU OpenVINO CPU plugin category: IE Tests OpenVINO Test: plugins and common category: inference OpenVINO Runtime library - Inference no_stale Do not mark as stale

Projects

None yet

Milestone

2025.3

Development

Successfully merging this pull request may close these issues.

6 participants

@tiger100256-hu @usstq @praasz @wenjiew @yuxu42 @karanjakhar