Search

Add natural language search to your app using AI embeddings

The Modus Collections API provides a robust way to store, retrieve, and search through data using both natural language and vector-based search methods. By leveraging embeddings, developers can enable semantic and similarity-based searches, improving the relevance of search results within their applications.

For example, with natural language similarity, if you search for a product description like ‘sleek red sports car’, the search method returns similar product descriptions such as “luxury sports car in red” or ‘high-speed car with sleek design’.

Understanding key components

Collections: a collection is a structured storage that organizes and stores textual data and associated metadata. Collections enable sophisticated search, retrieval, and classification tasks using vector embeddings.

Search Methods: a search method associated with a collection, defines how to convert collection items into a vector representation and provides indexing parameters.

Vector embeddings: for vector-based search and comparison, Modus converts each item in the collection into a vector representation called embedding. By embedding data, you enable powerful natural language and similarity-based searches.

Modus runtime automatically compute the embeddings, according to your configuration, when you add or update items.

Initializing your collection

Before implementing search, ensure you have defined a collection in the app manifest. In this example, myProducts is the collection used to store product descriptions.

First, we need to populate the collection with items (for example, product descriptions). You can insert individual or multiple items using the upsert and upsertBatch methods, respectively.

Use upsert to insert a product description into the collection. If you don’t specify a key, Modus generates a unique key for you.

Configure your search method

The search capability relies on a search method and embedding function. To configure your search method.

Create an embedding function

An embedding function is any API function that transforms text into vectors that represent their meaning in a high-dimensional space.

Embeddings functions must have the following signature:

Modus computes vectors using embedding models. Here are a few examples:

• all-MiniLM-L6-v2
• OpenAI

Declare the model in the app manifest

model.json

  "models": {
    // model card: https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2
    "minilm": {
      "sourceModel": "sentence-transformers/all-MiniLM-L6-v2", // model name on the provider
      "provider": "hugging-face", // provider for this model
      "connection": "hypermode" // host where the model is running
    }
  }

Create the embedding function using the embedding model:

Declare the search method

With an embedding function in place, declare a search method in the collection properties.

modus.json

  "collections": {
    "myProducts": {
        "searchMethods": {
            "searchMethod1": {
                "embedder": "minilm" // embedding function name
            }
        }
    }
  }

Implement semantic similarity search

With the products stored, you can now search the collection by semantic similarity. The search] API computes an embedding for the provided text, compares it with the embeddings of the items in the collection, and returns the most similar items.

Search result format

The search response is a CollectionSearchResult containing the following fields:

• collection: the name of the collection.
• status: the status of the operation.
• objects: the search result items with their text, distance, and score values.
  • distance: a lower value indicates a closer match between the search query and the item in the collection
  • score: a higher value (closer to 1) represents a better match

{
  "collection": "myProducts",
  "status": "success",
  "objects": [
    {
      "key": "item-key-123",
      "text": "Sample product description",
      "distance": 0.05,
      "score": 0.95
    }
  ]
}

Search for similar Items

When you need to search similar items to a given item, use the searchByVector API. Retrieve the vector associated with the given item by its key, then perform a search using that vector.