Using Azure Search / RAG in Azure AI Foundry Pipeline

[ThoSt81]

First, thanks for the great work! I can use my Azure AI just fine and it's working well!

However, since we are using an Azure Search index (RAG) holding documents/information for the AI to use, I cant find a way to add this to the configuration.

Essentially, the following needs to be added to the request body sent to Azure AI:

  "data_sources": [
    {
      "type": "azure_search",
      "parameters": {
        "filter": null,
        "endpoint": "https://xxx.search.windows.net",
        "index_name": "xxx",
        "project_resource_id": "xxx",
        "semantic_configuration": "azureml-default",
        "authentication": {
          "type": "system_assigned_managed_identity",
          "key": null
        },
        "embedding_dependency": null,
        "embeddingEndpoint": "xxx",
        "embeddingKey":"xxx",
        "query_type": "vectorSimpleHybrid",
        "in_scope": false,
        "role_information": "You are an AI assistant.",
        "strictness": 5,
        "top_n_documents": 20,
        "key": "xxx",
        "indexName": "xxx"
      }
    }
  ]

Is there an easy way to add this?

[owndev]

Hello @ThoSt81

I haven't used Azure AI Search yet and can't help with the details.

Theoretically, I could implement additional fields to add a custom body field. But then it would be defined for all configured models. I don't know how helpful this would be.

[ThoSt81]

That would be great! I dont think there is any harm if this additional body field is left empty but it would allow for greater flexibility.

Happy to test as I have a few different configurations to try.

[owndev]

I had Copilot write the extensions. Can you test it and give me some recommendations?

• https://github.com/owndev/Open-WebUI-Functions/blob/56579809c992c0ce934a9f34002d301287a89b50/pipelines/azure/azure_ai_foundry.py

Many new variables have been defined.
Is it better to define only one variable and then define the entire JSON there, or should I do it like Copilot did?

---

#48 and #49

[ThoSt81]

Hm, getting quite a few errors:

Error: Validation error at #/data_sources/0/azure_search/parameters/project_resource_id: Extra inputs are not permitted
Validation error at #/data_sources/0/azure_search/parameters/embeddingEndpoint: Extra inputs are not permitted
Validation error at #/data_sources/0/azure_search/parameters/embeddingKey: Extra inputs are not permitted
Validation error at #/data_sources/0/azure_search/parameters/indexName: Extra inputs are not permitted
Error: Validation error at #/data_sources/0/azure_search/parameters/project_resource_id: Extra inputs are not permitted
Validation error at #/data_sources/0/azure_search/parameters/embeddingEndpoint: Extra inputs are not permitted
Validation error at #/data_sources/0/azure_search/parameters/embeddingKey: Extra inputs are not permitted
Validation error at #/data_sources/0/azure_search/parameters/indexName: Extra inputs are not permitted

I think it's good to have all the additional valves, but it may be easier (for testing) if we can just add a whole section to the default body that you're sending. The full body that I send, e.g. when using Postman or Bruno is:

{
	"data_sources": [
		{
			"type": "azure_search",
			"parameters": {
				"filter": null,
				"endpoint": "https://xxx.search.windows.net",
				"index_name": "myindexname",
				"project_resource_id": "/subscriptions/xx-xx-xx-xx/resourceGroups/my_azure_ai_rg/providers/Microsoft.MachineLearningServices/workspaces/xxx",
				"semantic_configuration": "azureml-default",
				"authentication": {
					"type": "system_assigned_managed_identity",
					"key": null
				},
				"embedding_dependency": null,
				"embeddingEndpoint": "https://xxx.openai.azure.com/openai/deployments/text-embedding-ada-002/embeddings?api-version=2023-05-15",
				"embeddingKey": "xxx",
				"query_type": "vectorSimpleHybrid",
				"in_scope": false,
				"role_information": "You are an AI assistant.",
				"strictness": 5,
				"top_n_documents": 20,
				"key": "xxx",
				"indexName": "myindexname"
			}
		}
	],
	"messages": [
		{
			"role": "user",
			"content": "how are you?"
		}
	],
	"deployment": "gpt-4.1-mini",
	"temperature": 0.2,
	"top_p": 0.8,
	"max_citations": 1,
	"max_tokens": 1600,
	"stop": null,
	"stream": false
}

[owndev]

I have adapted it and now also added a new field Azure Search Raw Data Sources so that you can test the entire JSON.

• https://github.com/owndev/Open-WebUI-Functions/blob/d901dd8fbb6fa85a9f98df5d0a8d81de8bbbe644/pipelines/azure/azure_ai_foundry.py

[ThoSt81]

Great! Thank you for the quick update! Will try this tomorrow and will let you know!

[ThoSt81]

The errors are gone but the settings dont seem to be respected. I will try to troubleshoot further but something important came up and it may take some time before I can test more.