Step-By-Step Tutorial To Integrate OpenAI With Azure Cognitive Search (Vector Search)

Shweta Lodha
13 min readOct 4, 2023

In this article, I’ll explain how you can use Azure Cognitive Search with OpenAI, but before that let’s understand the role of each of these major components.

Azure Cognitive Search

This is an offering from Azure where we can store our data as high-dimensional vectors where each vector has a certain number of dimensions. This search service has a capability to perform both text and vector search, but in this article, I will be focusing only on the vector search capability.


OpenAI provides APIs which can be used to create interactive chatbots and virtual assistants that can carry out conversation in a natural manner. It works with prompts of various sizes based on the selected AI model.

As OpenAI is a prompt-based system, the user is charged based on the number of tokens, which means we have to be careful while sending data to the OpenAI API. We cannot pass huge text file in a single shot, just to get the answer to one question. Rather, it is suggested to pass only the relevant content to the API and to get that relevant content, we are using Azure Cognitive Search.

With this much of background, let’s jump on to the key steps and understand how both of these systems can talk to each other.

Create An Instance Of Cognitive Search

The very first thing we need to do is to create an instance of Cognitive Search on the Azure portal and that can be done by searching for Cognitive Search in the search bar and clicking on Create button. Make sure that you have logged on to Azure portal with an active subscription in order to do this step.

Follow the guided dialog and you will get your instance created. Once an instance is created, you need to make a note of the endpoint and a key, which can be found by clicking on the highlighted boxes below.

Get An OpenAI API Key