Implementing Semantic Search with Gemini API

Introduction

In today's data-driven world, the sheer volume of information available can be overwhelming. Traditional keyword-based search methods often fall short when it comes to understanding the context or intent behind a query. This is where Semantic Search comes into play.

Semantic Search goes beyond simple keyword matching by understanding the meaning behind the words. It leverages advanced algorithms to interpret the context, making it possible to deliver more accurate and relevant search results.

With Cosmocloud's services, integrating Semantic Search into your applications is simple. This tutorial will show you how to use the Gemini API and MongoDB Atlas Vector Search to create vector embeddings and enabling intelligent search that understands user intent and delivers relevant results.

Our Agenda

In this tutorial, we will cover:

• Implementing Semantic Search using Vector Search Index from Cosmocloud and Vector Embeddings generated from Gemini to implement a Semantic Search.

Prerequisites

• Our tutorial assumes that you have already created a project on Cosmocloud, picking the cloud of your choice.

• Connecting your database to your Cosmocloud Project.

Creating Semantic Search

To implement semantic search, you will need to store vector embeddings of your data and query these embeddings using a Vector Search Index.

We will use the following sample data generated by the Gemini chatbot for hotel information -

Hotel Sample Data

[
  {
    "hotelName": "Desert Oasis",
    "city": "Phoenix",
    "rating": 4.4,
    "ammenities": [
      "Spa",
      "Golf course",
      "Free breakfast"
    ],
    "price": 210.5,
    "details": "Desert Oasis lived up to its name. The spa treatments were incredibly relaxing and professionally done. The golf course was well-maintained and enjoyable to play on. The free breakfast had a good variety of choices. The hotel grounds were beautiful and well-kept. We had a very relaxing and enjoyable stay.",
  },
  // ...
]

The complete dataset can be found here: Hotel's Data

We will create the following two APIs:

1. POST /hotels: This endpoint will generate vector embeddings for each record inserted into the database, based on the details field. The vector embeddings will be generated using the Gemini API.

2. GET /hotels/_search: This endpoint will allow you to search for hotels based on a query describing the hotel. The search will use the Vector Search Index created in Cosmocloud Platform.

Creating Hotel DB Model

The first step in Cosmocloud is to build your database models for the entities defined in your system. We'll start by creating our Hotels model.

The name of the model should match the collection name (table name) in your database. Keeping it Hotels would create a collection named Hotels in your MongoDB database.

• Navigate to Application Layer -> DB Models to create new database models in Cosmocloud.

• Click on New Model button.

• Enter Hotels as the model name, provide a description, and click Create.

• Switch to the Schema tab at the top to define the schema for the Hotels model.

Defining the Schema of our Hotels Model

As Cosmocloud is connected to your MongoDB database, we can define a Document Model based schema for our Hotels model.

• Click on JSON to Schema button to quickly generate schema from a sample Hotel record.

• Copy the snippet below as the sample record in our Hotels model.

We will not add _id as it is going to be auto-generated by our database (MongoDB) for us

The detailsEmbeddings field will store the vector embeddings of the details field which will be generated from Gemini API. We'll see this later below.

{
    "hotelName": "Desert Oasis",
    "city": "Phoenix",
    "rating": 4.4,
    "ammenities": [
      "Spa",
      "Golf course",
      "Free breakfast"
    ],
    "price": 210.5,
    "details": "Desert Oasis lived up to its name. The spa treatments were incredibly relaxing and professionally done. The golf course was well-maintained and enjoyable to play on. The free breakfast had a good variety of choices. The hotel grounds were beautiful and well-kept. We had a very relaxing and enjoyable stay.",
    "detailsEmbeddings": [1.23, 1.23]
}

• Click on Save button to instantly generate the schema of your Hotels model as well as save it in the system. If the schema doesn't look as the image given below, do necessary changes manually.

Hotels Schema

Store Secret of Gemini API

After creating your Gemini API key from Google AI Studio, you'll need to store it as a Custom Secret in Cosmocloud.

Creating Subflow to Generate Embeddings

Next, we'll create a Subflow to generate embeddings from a text using the Gemini API. We are creating a Subflow as it will allow us to quickly reuse this flow in multiple APIs such as storing data in the database and when querying it.

1. Head over to Application Layer -> Subflows.

2. Name the subflow as create_embeddings.

3. Set the argument key as text, type as String, and mark it as required by setting the first option to true.

4. Click on Create button to create the new subflow.

create_embeddings Subflow

Here comes the main part, we'll now edit the main flow of generating the embeddings of a text from Gemini. Head over to the Flow tab.

Build JSON Object

• Add a Build JSON Object node, and name it jsonPayload.
• This node will create the request body for the Gemini API. We are going to use $.variables.text which is coming as an argument to our Subflow. The JSON content should look like this:

{
  "model": "models/text-embedding-004",
  "content": {
    "parts": [
      {
        "text": "$.variables.text"
      }
    ]
  }
}

jsonPayload JSON Object

Concat Strings

• Add a Concat Strings node and name it as gemini_url.
• This node will concatenate the base URL https://generativelanguage.googleapis.com/v1beta/models/text-embedding-004:embedContent?key= with the gemini_api_key stored as a secret, forming the full API endpoint. Refer this doc to learn more about the Base URL.

gemini_url - Conact Strings Node

API Call

Add an API Call node to make a POST request to the above gemini_url URL generated, using the jsonPayload as the request body. This call will generate the text embeddings.

Gemini API Call Node

Subflow Response

Add a Subflow Response node at the end to return the embeddings generated by the previous API Call node.

Subflow Response Node

In the end, the flow will look like this:

create_embeddings Final Subflow

Pheww, a lot of work right! Take a moment to relax, you've made great progress! There's still more to come, but the upcoming sections will get even more interesting as you continue to build out your Semantic Search functionality.

Creating POST API

In this section, we'll use the create_embeddings Subflow to generate embeddings for each record added to the database. These embeddings will be stored alongside the Hotels data for future querying.

Start by using the CRUD API's Template to streamline the creation of your API. We'll specifically choose the /hotels POST API and the Create Hotels RB Request Body. This will provide a boilerplate to simplify the creation of the POST API's request body.

Hotels CRUD API Template

Once the template is applied, head over to the Flow Builder for the generated POST /hotels API and start customising the flow.

Execute Subflow

• Add an Execute Subflow node and select create_embeddings under Select Subflow to execute
• Pass the details field from the request body to the subflow to generate the embeddings:

{
  "text": "$.request.body.details"
}

Execute Subflow Node

Update JSON Object

• Add an Update JSON Object node to update the request body JSON with generated embeddings.
• Use $.request.body as the Existing Object Name and update the detailsEmbeddings field in the request body with the embeddings generated by the subflow:

{
  "detailsEmbeddings": "$.<EXECUTE_SUBFLOW_NODE_NAME>.result"
}

Update JSON Object Node

The remaining nodes - Insert One, Build JSON Object and HTTP Response can be left as they are. These nodes will handle inserting the record into the database, building the final JSON response, and returning the HTTP response, respectively.

The final flow should somewhat look like this:

Hotels POST API Final Flow

This setup ensures that each hotel record inserted into the database will have its detailsEmbeddings field populated with vector embeddings, making it ready for efficient semantic search.

Vector Search Index

To enable semantic search capabilities, we'll leverage Cosmocloud's Vector Search Index. This powerful feature will allow you to query data based on the vector embeddings we've generated for each hotel record.

Start by creating a new vector search index named hotels_vector_search. Follow the detailed instructions in this tutorial: Create a Vector Search Index.

Gemini AI models/text-embedding-004 creates an embedding of size 768, hence we will use this value as Number of Dimensions while creating our Vector Search below. You can check Gemini docs for more info.

You’ll need to configure the fields according to the image below to guide your setup:

Hotels Vector Search Index

By properly configuring your vector search index, you'll be set to perform advanced semantic searches on your hotel data. This index will allow you to query records not just by keyword but by the meaning and context of the hotel descriptions, resulting in more accurate and relevant search results.

Defining Search Query Parameters

First, we’ll create a Query Params model for the search query parameters, which will be used in the API to handle incoming queries.

1. Navigate to Application Layer -> Request Models.

2. Click on Create Model and set the Model Name as Search Hotels QP and the Model Type as Query Params.

3. Once created, switch to Schema tab and paste the following JSON inside the widget - JSON to Schema:

{
    "query": "text",
    "limit": 10,
    "offset": 0
}

Don't forget to mark all query, limit and offset as required. (Red dot against the field name)

Search Hotels Query Parms Model

Creating the Search Query API

Next, we’ll create the GET /hotels/_search API to handle the search functionality.

1. Navigate to Application Layer -> APIs.

2. Click on Create API and choose Start from Scratch.

3. Name the API Search Hotels, set the Request method to GET, and set the endpoint to /hotels/_search. Select the Query Params as Search Hotels QP, which we created in the previous step.

4. Click Create to generate the API.

Once the template is applied, head over to the flow builder for the generated GET /hotels/_search API and start customising the flow.

Execute Subflow

• Add an Execute Subflow node and select create_embeddings Subflow under Select Subflow to execute

• Pass the query parameter from the request query params to generate the embeddings:

{
  "text": "$.request.queryParams.query"
}

Execute Subflow Node

Run Aggregation Pipeline

• Below the Execute Subflow node, add a Run Aggregation Pipeline node and select the Database Collection as Hotels. Select the Response Type as Array.
• Edit the Aggregate Query as follows:

[
  {
    "$vectorSearch": {
      "index": "hotels_vector_search",
      "path": "detailsEmbeddings",
      "queryVector": "$.node_9.result",
      "numCandidates": 15,
      "limit": "$.request.queryParams.limit"
    }
  },
  {
    "$facet": {
      "data": [
        {
          "$project": {
            "hotelName": 1,
             "details": 1,
             "score": {
                  "$meta": "vectorSearchScore"
              }
          }
        },
        {"$skip": "$.request.queryParams.offset"},
        {"$limit": "$.request.queryParams.limit"}
      ],
      "count": [
        {"$count": "totalCount"}
      ]
    }
  }
]

Run Aggregation Pipeline Node

HTTP Response

• Add an HTTP Response node at the end of the flow.

• Set the Response value to $.<AGGREGATION_NODE_NAME>.result and the Status Code to 200.

HTTP Response Node

The final flow for the GET /hotels/_search API should resemble the structure below, enabling the API to perform sophisticated semantic searches:

Search Query API Final Flow

Testing APIs

Adding Data

To add hotel records to the database, use the POST /hotels API. We'll be using the Hotel's Data dataset, and each record should be added individually.

Follow the instructions in the Testing Free Tier APIs document for detailed guidance on setting headers and URLs.

Postman Add Hotels

Once the data is added, inspect your database. You'll notice that a detailsEmbeddings field with a 768-dimensional vector added to each record. These embeddings are generated using the Gemini API and are essential for performing semantic searches.

MongoDB Hotels Collection

Searching Data

To search the data, use the GET /hotels/_search API. This API will allow you to perform semantic searches based on the vector embeddings stored in the database.

For example, set the query parameter to stylish city stay, rooftop bar with skyline views, well-equipped fitness center, with limit as 10 and offset as 0. The result will return records that are most relevant to the search query.

Postman Search Hotels

In the aggregation pipeline, we added a score field which represents the similarity between the query input and the records. This score helps in identifying the most relevant records. To understand more about the Vector Search Score refer this documentation.

Conclusion

Congratulations on completing this tutorial 🎉!

You've successfully navigated through the process of integrating semantic search capabilities into your application using Cosmocloud and the Gemini API. From creating a Hotels database model, generating vector embeddings, setting up a vector search index, and building both POST and GET APIs, you now have a powerful system capable of understanding and interpreting user intent at a deeper level.

Thank you for following along, and happy coding!