feat: added ability to select runtime for playground #3171

bmahabirbu · 2025-06-12T05:45:12Z

What does this PR do?

adds the ability to select the runtime for playground

Screenshot / video of UI

What issues does this PR fix or reference?

fixes #2613

How to test this PR?

feloy · 2025-06-12T07:21:18Z

A few questions:

openvino is not compatible with mac/arm. Should we list it in such platform?
what is the meaning of the all inference runtime?

bmahabirbu · 2025-06-12T07:42:00Z

@feloy good questions! That would be a good idea to remove openvino from mac for now

'ALL' in this case is a placeholder for every runtime provider besides a hardcoded exclusion.

Essentially I wanted a way to list all the runtime providers in the playground creation and this made the most sense to me at the moment. There's most definitely a cleaner approach!

gastoner

I think we are missing none option.
The facebook model has none runtime but the filter for it is not there

axel7083

The runtime select should be seen as a filter, no hardcoded all, it should be clearable, if no value is selected, then all models should be listed.

No need to define what mean "all"

bmahabirbu · 2025-06-17T00:13:48Z

Thanks, @axel7083. That's a good way to handle it! In that case, should the runtime selection dropdown be defaulted to having no value so the user can see all the models available?

bmahabirbu · 2025-06-17T00:15:18Z

@gastoner good catch I assumed "none" in this case was simply to have no selection. I did have a playground with resnet setup but since the model is used for object detection, I didn't get the chat to work. Maybe we can keep it excluded?

axel7083 · 2025-06-17T07:05:58Z

should the runtime selection dropdown be defaulted to having no value so the user can see all the models available?

Yes! By default no filtering

bmahabirbu · 2025-06-17T19:47:50Z

@axel7083 updated!

jeffmaury

Have tested it and got 2 issues:

in the selector the none value is presented but this value is useless because there are no inference servers for those kind of models: they are just there so that recipes can load them independently from an inference server
when the inference runtime from the settings is different from all then it should be the default value of the selector here

bmahabirbu · 2025-06-20T07:27:07Z

@feloy I believe that Mac support for OpenVINO might be coming soon, maybe we can keep it as a selection? Also I was trying to import @podman-desktop/api to the frontend script playground create to use the env.ismac but it seems frontend scripts only have the ability to pull from pd ui-svelte. Is there an easier approach to having ismac usable for frontend?

axel7083

LGTM, tested on Windows, works as expected 👍

bmahabirbu · 2025-06-22T18:13:08Z

@jeffmaury gave me the idea to use the inference registry store, I believ,e to obtain what the user operating system is to filter openvino for mac so will try this out before merge

bmahabirbu · 2025-06-23T23:22:01Z

@jeffmaury did we want to have a defaut model selected when the runtime is all (e.i no filter)?

My current implementation doesnt have the model selected so I had to edit one of the e2e tests

axel7083 · 2025-06-26T07:48:35Z

@jeffmaury did we want to have a defaut model selected when the runtime is all (e.i no filter)?

Would be nice to have this PR merged! So might be a follow-up PR? cc @jeffmaury

axel7083 · 2025-06-26T07:48:47Z

PR checks need to be fixed

bmahabirbu · 2025-06-27T05:33:33Z

I think this is all set to go now! I'm now pulling the registered provider backend list rather than using the enum InferenceType itself. Since openvino isnt registered on mac we shouldnt see it now for that platform

feloy

LGTM, thanks

jeffmaury · 2025-06-27T14:45:58Z

Still not working for me:in the preferred runtime I have set llama-cpp when I got to the playground image, the llama-cpp provider is not selected in the inference runtime selection box and I can see an OpenVINO model in the list of models

bmahabirbu · 2025-06-27T23:36:30Z

I can use the preferred runtime settings instead for runtime select but in the inference runtime dropdown, I dont see OpenVINO. Shouldn't we have it as a runtime backend? We can download the openvino model and create a service for it so we should have it for the playground too!

bmahabirbu · 2025-07-01T20:35:12Z

Now im preselecting the playground runtime based on the recommended config. I removed the logic that allowed me to retrieve the registered runtimes in favor of this approach. Once openvino gets added to the selection I will remove it from my exclude inferencetype array

jeffmaury · 2025-07-02T12:41:54Z

packages/frontend/src/pages/PlaygroundCreate.svelte

+let runtime: InferenceType | undefined = undefined;
+
+// Exlude certain runtimes from selection
+export let exclude: InferenceType[] = [InferenceType.NONE, InferenceType.WHISPER_CPP, InferenceType.OPENVINO];


Why is OpenVINO be excluded ?

We don't want openvino for mac so I exclude it till we have a pr that updates the runtime select in settings with openvino

Signed-off-by: Brian <[email protected]>

config store Signed-off-by: Brian <[email protected]>

Signed-off-by: Brian <[email protected]>

this will show on mac as well Signed-off-by: Brian <[email protected]>

Signed-off-by: Brian <[email protected]>

bmahabirbu requested review from benoitf, jeffmaury and a team as code owners June 12, 2025 05:45

bmahabirbu requested review from feloy and SoniaSandler June 12, 2025 05:45

bmahabirbu force-pushed the playground-runtime branch 2 times, most recently from bc087d7 to bdabfb0 Compare June 12, 2025 06:06

gastoner reviewed Jun 16, 2025

View reviewed changes

axel7083 reviewed Jun 16, 2025

View reviewed changes

bmahabirbu force-pushed the playground-runtime branch from bdabfb0 to 385f589 Compare June 17, 2025 00:32

jeffmaury reviewed Jun 19, 2025

View reviewed changes

bmahabirbu force-pushed the playground-runtime branch 2 times, most recently from 59dad1b to c590339 Compare June 20, 2025 07:24

bmahabirbu force-pushed the playground-runtime branch 2 times, most recently from 802118c to c0281d0 Compare June 20, 2025 08:24

axel7083 approved these changes Jun 20, 2025

View reviewed changes

shipsing mentioned this pull request Jun 23, 2025

Doc: Update documentation to highlight user's ability to select an inference runtime podman-desktop/podman-desktop#12932

Closed

bmahabirbu force-pushed the playground-runtime branch from c0281d0 to 05c82b2 Compare June 23, 2025 17:50

bmahabirbu requested a review from a team as a code owner June 23, 2025 21:48

bmahabirbu force-pushed the playground-runtime branch from 7c808e3 to 9bbb52f Compare June 23, 2025 23:00

bmahabirbu force-pushed the playground-runtime branch from 519e6a5 to 029f1c5 Compare June 25, 2025 07:52

bmahabirbu force-pushed the playground-runtime branch from 029f1c5 to 1e64080 Compare June 25, 2025 08:00

bmahabirbu force-pushed the playground-runtime branch from 834d7dc to 0fd2c7e Compare June 27, 2025 05:20

feloy approved these changes Jun 27, 2025

View reviewed changes

bmahabirbu force-pushed the playground-runtime branch 2 times, most recently from c6d13e4 to 4f69688 Compare July 1, 2025 19:46

bmahabirbu force-pushed the playground-runtime branch from 7c6b709 to 6882d66 Compare July 1, 2025 20:36

jeffmaury reviewed Jul 2, 2025

View reviewed changes

bmahabirbu force-pushed the playground-runtime branch from 34af99e to a6c76b7 Compare July 7, 2025 18:21

bmahabirbu added 12 commits July 8, 2025 11:43

feat: added ability to select runtime for playground

0f7aff8

Signed-off-by: Brian <[email protected]>

fix: updated runtime selection filter

63d25a2

Signed-off-by: Brian <[email protected]>

fix: adjusted tests with new filter change

c6815ed

Signed-off-by: Brian <[email protected]>

fix: removed none and enforced runtime exlusion

18205cc

Signed-off-by: Brian <[email protected]>

chore: edited e2e test to comply with new runtime selection

03dc4a0

Signed-off-by: Brian <[email protected]>

revert: model preset dependent on runtime it should always be preset

cccd2dd

Signed-off-by: Brian <[email protected]>

feat: get the inference provider from a registered store

a35438c

Signed-off-by: Brian <[email protected]>

chore: reworked getting registered providers

8b1632c

Signed-off-by: Brian <[email protected]>

fix: working on refractoring change to use recommended runtime from

a8d7c8c

config store Signed-off-by: Brian <[email protected]>

fix: use config store to preselect runtime fixed tests

eb995df

Signed-off-by: Brian <[email protected]>

fix: added back openvino selection

c1e17cf

this will show on mac as well Signed-off-by: Brian <[email protected]>

fix: added back retrieval of registered providers

949aa9d

Signed-off-by: Brian <[email protected]>

bmahabirbu force-pushed the playground-runtime branch from a6c76b7 to 949aa9d Compare July 8, 2025 15:43

bmahabirbu merged commit b8583e9 into containers:main Jul 9, 2025
7 checks passed

feat: added ability to select runtime for playground #3171

feat: added ability to select runtime for playground #3171

Conversation

bmahabirbu commented Jun 12, 2025

What does this PR do?

Screenshot / video of UI

What issues does this PR fix or reference?

How to test this PR?

Uh oh!

feloy commented Jun 12, 2025

Uh oh!

bmahabirbu commented Jun 12, 2025

Uh oh!

gastoner left a comment

Choose a reason for hiding this comment

Uh oh!

axel7083 left a comment

Choose a reason for hiding this comment

Uh oh!

bmahabirbu commented Jun 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bmahabirbu commented Jun 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

axel7083 commented Jun 17, 2025

Uh oh!

bmahabirbu commented Jun 17, 2025

Uh oh!

jeffmaury left a comment

Choose a reason for hiding this comment

Uh oh!

bmahabirbu commented Jun 20, 2025

Uh oh!

axel7083 left a comment

Choose a reason for hiding this comment

Uh oh!

bmahabirbu commented Jun 22, 2025

Uh oh!

bmahabirbu commented Jun 23, 2025

Uh oh!

axel7083 commented Jun 26, 2025

Uh oh!

axel7083 commented Jun 26, 2025

Uh oh!

bmahabirbu commented Jun 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

feloy left a comment

Choose a reason for hiding this comment

Uh oh!

jeffmaury commented Jun 27, 2025

Uh oh!

bmahabirbu commented Jun 27, 2025

Uh oh!

bmahabirbu commented Jul 1, 2025

Uh oh!

jeffmaury Jul 2, 2025

Choose a reason for hiding this comment

Uh oh!

bmahabirbu Jul 2, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

bmahabirbu commented Jun 17, 2025 •

edited

Loading

bmahabirbu commented Jun 17, 2025 •

edited

Loading

bmahabirbu commented Jun 27, 2025 •

edited

Loading