Algolia

Suggestion Search

Input

The getSearchSuggestions method in the AlgoliaSearchService class takes a single input parameter:

• query (String): This is the search query entered by the user. The query is used to find relevant suggestions from multiple indices (like query suggestions, product suggestions, department suggestions, etc.).
• Along with multiple indeces, we are setting clickAnalytics to true. This will generate the queryID and added to the search response. You have to pass this queryID when sending search-related events to the Insights API with the clickedObjectIdsAfterSearch or convertedObjectIdsAfterSearch methods. Calculating metrics, such as Click Rate and Conversion Rate, for Analytics and A/B Testing requires this setting.
• ruleContexts (List): contexts can be used to apply rules based on which device is being used to search products. We are assigning app. It indicates that the search requests are coming from the mobile application.

Output

The method returns a Future, which contains the following information:

• query (String): The original search query that was passed as input.
• queryHits (List): A list of suggestion IDs from the suggestionsIndex. These IDs represent the search suggestions directly related to the user's query.
• productsHits (List): A list of product suggestions that match the search query. Each ProductSuggestionHit includes:
  • code (String): Product id or code.
  • name (String): The name of the product.
• categoryHits (List): A list of department or category suggestions. Each Category includes:
  • id (String): Category ID.
  • defaultName (String): The display name of the category.
  • image (String?): The image URL associated with the category.
  • slug (String): The slug.
• redirect (String): A URL or identifier extracted from the redirect results, which can be used to direct the user to a specific page if the query matches a predefined redirect rule.
• querySearchMetaData (AlgoliaSearchMetaData?): Metadata related to the query results, including the index name and query ID.
• productsSearchMetaData (AlgoliaSearchMetaData?): Metadata related to the product results, including the index name and query ID.
• categorySearchMetaData (AlgoliaSearchMetaData?): Metadata related to the category results, including the index name and query ID.

Process

1. Search Queries Creation: The method generates a list of SearchForHits queries targeting different indices (like suggestions, products, departments, and redirects) to cover various types of suggestions.
2. Query Execution: The queries are executed using the Algolia client, which returns snapshots of the results.
3. Results Parsing:
   • Query Suggestions: The first snapshot contains query suggestions IDs.
   • Product Suggestions: The second snapshot is parsed to extract product names and codes.
   • Category Suggestions: The third snapshot is parsed to extract categories based on the name match.
   • Redirects: The fourth snapshot checks if the query should redirect to a specific page.
4. Metadata Extraction: Metadata for query, product, and category suggestions is extracted to track the search context.
5. Return: The method returns a SearchSuggestionHits object with all the parsed data, which can then be used to display search suggestions to the user.

Keyword Search

Input

The searchForProducts method in the AlgoliaSearchService class takes the following input parameters:

• initial (SearchParameters): The initial search parameters used to define the context or scope of the search, such as filters or sort options.
• params (SearchParameters): The dynamic search parameters containing the user's query and any additional filters or sorting preferences.
• page (int): The page number for pagination. This determines which page of results to retrieve.
• pageSize (int): The number of results to return per page.
• inStock (bool, optional): A flag to indicate whether to filter for products that are in stock. Defaults to false.
• defaultBranch (Branch?, optional): The default branch or location context for the search, if applicable.

Output

The method returns a Future, which contains the following information:

• productCodes (List): A list of product codes as objectID from List of Hit object representing the products that match the search criteria.
• facets (Map<String, TSFacet>): A map of facet filters available for further refining the search results. Each TSFacet includes:
  • key (String): The facet's key or name.
  • name (String): The display name of the facet.
  • options (List): The list of available options for this facet.
  • isNumeric (bool): A flag indicating if the facet's values are numeric.
• page (int): The current page number of the search results.
• numberOfPages (int): The total number of pages available for the given search.
• numberOfHits (int): The total number of products that match the search criteria.
• searchMetaData (AlgoliaSearchMetaData?): Metadata related to the search, including the index name and query ID.

Process

1. Initial Filter and Redirect Handling:
   • The method checks if the search is based on a taxonomy (category) and applies the appropriate filter string.
   • If the query has a predefined redirect, it adjusts the search parameters accordingly.
2. Index Selection:
   • The method selects the appropriate Algolia index based on the sorting preference (ascending, descending, relevant).
   • If the bestSeller option is selected, an exception is thrown, as Algolia does not support this sort type.
3. Facet Filters:
   • The method converts the user's selected filters into a list of facet filters to be applied to the search.
4. Search Execution:
   • A SearchForHits object is created with the relevant search parameters, including the query, facet filters, page number, and results per page.
   • The Algolia client executes the search, returning a result set that includes products, facets, and metadata.
5. Result Parsing:
   • Product Codes: The method extracts the product codes from the search results.
   • Facets: The facets returned from the search are organized and ordered according to predefined keys.
   • Metadata: The query ID and other relevant metadata are extracted for tracking purposes.
6. Return:
   • The method returns a ProductSearchHits object containing the product codes, available facets, pagination information, and search metadata. This result is then used to display the search results to the user.

No-Operation Implementations

Input

The _AlgoliaAnalyticsServiceNoOp class provides no-operation (no-op) implementations of the above methods. All methods in this class take the same parameters as their counterparts but do not perform any operations.

Output

All methods in the _AlgoliaAnalyticsServiceNoOp class return Future but do not perform any actions, ensuring that the interface is fulfilled without triggering any analytics events.

Algolia Index

An index is where Algolia stores data for search and discovery, similar to a table in a database. You can structure your indices based on your search and display needs.

Structuring Indices

• Optimized for Search: An index is a collection of records optimized for searching.
• Flattened Data: Flattened data is ideal for searching. Creating several indices mapped to your tables, where each index represents a different kind of entity, is a good approach.
• Hierarchy and Relationships: Maintaining hierarchy and relationships in your records can provide a better search experience.

When to Use Multiple Indices

• Separate Display of Content: Use separate indices to display different types of content separately in your UI.
• Popular Queries and Ranking Strategies: Consider using multiple indices for showcasing popular queries or allowing users to switch between different ranking strategies.

In Toolstation App

• Index Setup: Toolstation sets up indices in the Algolia Dashboard. Each index retrieves different information based on what the app needs.
• Autocomplete Suggestions: For example, the Toolstation App uses various indices to get autocomplete suggestions while users type in the search bar, like TSFR-suggestion. This index is included in the Algolia API request, returning suggested products, categories, and query keywords.
• Regional and Language-Specific Indices: The Toolstation App has separate indices for each region and language, like TSNL_NEW_en_products_query_suggestions for the NL region and English language. The format is consistent across regions and languages, with only the region and language strings changing.

Auto Complete Suggestion Index

• Three Types of Suggestions: Toolstation provides three types of search suggestions when users type keywords into the search bar using Algolia's multi-index API, which allows querying multiple indices in a single request.
• Keyword Highlighting: Algolia highlights typed keywords in search results by surrounding them with <em> tags. For example, if the user types "Dri," results might include <em>Dri</em>ll or <em>Dril</em>l Drivers.
• Indices Used (NL region, English language):
  1. TSNL_NEW_en_products_query_suggestions
  2. TSNL_NEW_en_products
  3. TSNL_NEW_en_categories

Example Auto Complete Request JSON:

{
  "requests": [
    {
      "query": "Dril",
      "ruleContexts": ["app"],
      "clickAnalytics": true,
      "analyticsTags": ["autocomplete"],
      "hitsPerPage": 5,
      "indexName": "TSNL_NEW_en_products_query_suggestions"
    },
    {
      "query": "Dril",
      "ruleContexts": ["app"],
      "clickAnalytics": true,
      "analyticsTags": ["autocomplete"],
      "hitsPerPage": 5,
      "indexName": "TSNL_NEW_en_products"
    },
    {
      "query": "Dril",
      "ruleContexts": ["app"],
      "clickAnalytics": true,
      "analyticsTags": ["autocomplete"],
      "hitsPerPage": 3,
      "indexName": "TSNL_NEW_en_categories"
    }
  ]
}

Note:

1. ruleContexts includes 'app' to identify the requests as coming from the mobile app.
2. clickAnalytics is set to true to get a queryID in the response, necessary for logging Algolia analytics.

Search Index

The search index helps find products based on the query, providing a list of products and their details. Product codes (identified as objectIds) are extracted from the response to fetch product details via the ecom API, which is necessary for displaying products and descriptions on the product listing page.

• Facets: During the API request, we include a list of filters within the facets parameter, instructing Algolia to consider these filter types and return associated products and filters (e.g., list of brands, price ranges).
• Facet Filters: When users search for products, they can refine their results by selecting filters like brand, category (e.g., Power Tools, Hand Tools), or price range. These filters are grouped into lists based on their type. For example, if a user selects filters for brand, category, and price range, these selections are organized into separate lists within the search query request.

Check below example how to include filters of different types and grouped the same type of filters in a list.

Search Index: TSNL_NEW_en_products (NL region, English language)

Example Search Products Request with Facet Filters:

{
  "requests": [
    {
      "query": "drill bits",
      "facetFilters": [
        ["brand:DEWALT"],
        ["price:100-200"],
        ["category:Power Tools", "category:Hand Tools"]
      ],
      "facets": [
        "departments",
        "departmentListing",
        "brand",
        "review",
        "width",
        "widthfeet"
      ],
      "page": 0,
      "ruleContexts": ["search-unaltered", "page-search", "app"],
      "clickAnalytics": true,
      "hitsPerPage": 24,
      "maxValuesPerFacet": 101,
      "indexName": "TSNL_NEW_en_products"
    }
  ]
}

Sort By Price Index

• Sort By: We use different indices for sorting from highest to lowest and lowest to highest. Specifically, one index sorts from highest to lowest, and another sorts from lowest to highest.
• Indices Used (NL region, English language):
  1. TSNL_NEW_en_products_by_price_asc
  2. TSNL_NEW_en_products_by_price_desc

Example Request for Sorting Lowest Price to Highest:

{
  "requests": [
    {
      "query": "drill bits",
      "facetFilters": [],
      "facets": [
        "departments",
        "departmentListing",
        "brand",
        "review",
        "width",
        "widthfeet",
        ...
      ],
      "page": 0,
      "ruleContexts": [
        "search-unaltered",
        "page-search",
        "app"
      ],
      "clickAnalytics": true,
      "hitsPerPage": 24,
      "maxValuesPerFacet": 101,
      "indexName": "TSNL_NEW_en_products_by_price_asc"
    }
  ]
}

Index for sorting from highest to lowest: TSNL_NEW_en_products_by_price_desc

Facets and Facet Filtering

What Are Facets?

Facets are categories used to filter search results by specific attributes, such as brand, size, or color. They help users explore products more effectively by:

• Providing Visibility: Displaying possible values for each attribute along with their availability (count).
• Enhancing Precision: Allowing users to select multiple facet values to refine their search.
• Improving Usability: Enabling dynamic filtering based on user preferences.

On the Product Listing Page (PLP), facets and their counts are displayed in a filter section. Users can interact with these filters to narrow down results. For instance, selecting "Brand: Nike" ensures the displayed results include only Nike products.

Why Send Facets and Filters to Algolia Along with a Search Query?

When a user applies filters, the application needs Algolia to process these inputs alongside the main search query. For example:

• Query: "Shoes"
• Filters: Brand: Nike, Color: Red
• Request to Algolia: Includes both the search term and filters.
• Result: Returns only red Nike shoes.

This mechanism ensures the results are relevant while preserving counts for other facet values.

Algolia Facet Filtering Problem

The Issue

When filters are applied using Algolia's default behavior, some facet values m from the UI. This happens because the filtered result set may not contain all possible facet values. For instance:

1. Query: "Drill"
   • Facets:
     • Brand: a, b, c, d
     • Width: 1.4mm, 1.6mm
2. Filter Applied:
   • Selected Brand: a.
   • Other brands (b, c, d) disappear from the filter options.

This behavior can confuse users, as they might want to refine their selection further or toggle between filters.

Solution: Multi-Index Search

Why Use Multi-Index Search?

Multi-index search resolves this issue by ensuring that all facet values remain visible, even after filters are applied. It does so by running multiple Algolia queries and merging the results.

Steps in Multi-Index Search

1. Generate Filter Combinations
   The selected filters are converted into subsets by excluding individual filter categories. This ensures every filter category is queried independently.
   Example:
   Selected Filters:

   [
     {"key": "brand", "value": "a"},
     {"key": "brand", "value": "b"},
     {"key": "width", "value": "1.4mm"}
   ]
   

   
   Generated Combinations:

   [
     [{"key": "brand", "value": "a"}, {"key": "brand", "value": "b"}],
     [{"key": "width", "value": "1.4mm"}]
   ]
   

2. Run Multiple Queries
   For each combination, an Algolia query is sent:
   • Base Query: Includes all selected filters.
   • Combination Queries: Exclude specific categories to retrieve additional facet values.
3. Merge Results
   Combine facet values from all queries. If a facet value is missing in one query (e.g., due to filters), it is retrieved from another query and added to the final result.
4. Dynamic Facet Reconstruction
   Rebuild facets dynamically to include missing values with a count of 0. This ensures a consistent UI display.

Code Implementation

Generate Filter Combinations

This function creates subsets of selected filters by excluding specific categories:

List<List<SelectedFilter>> generateCombinations(List<SelectedFilter> filters) {
  final combinations = <List<SelectedFilter>>[];
  for (final filter in filters) {
    combinations.add(filters.where((f) => f.key != filter.key).toList());
  }
  return combinations;
}

Send Multiple Algolia Requests

Using the generated combinations, send queries to Algolia:

      final List<SearchForHits> searchQueries = [];

      // Base search query
      searchQueries.add(
        SearchForHits(
          indexName: searchIndex,
          query: searchText,
          facetFilters: formattedFilters,
          facets: tsProvidedFacets,
          maxValuesPerFacet: 101,
          page: page,
          hitsPerPage: pageSize,
          filters: filterString.isNotEmpty ? filterString : null,
          clickAnalytics: true,
          ruleContexts: ['search-unaltered', 'page-search', ...appSearchContexts],
        ),
      );

      // Convert the selected filters into a map for easy comparison.
      final originalFiltersMap = selectedFilters.convertToMap();

      // Iterate over each filter combination
      // & prepare a search query for all filter combination
      for (final filterCombination in filterCombinations) {
        final List<SelectedFilter> selectedFiltersFromCombination =
            filterCombination.expand((f) => f).toList();
        final Map<String, List<String>> combinationFiltersMap =
            selectedFiltersFromCombination.convertToMap();
        final Set<String> missingFilterCategory =
            originalFiltersMap.keys.toSet().difference(combinationFiltersMap.keys.toSet());

        searchQueries.add(
          SearchForHits(
            indexName: searchIndex,
            query: searchText,
            facetFilters: selectedFiltersFromCombination.toFormattedFilterGroups(),
            facets: missingFilterCategory.toList(),
            maxValuesPerFacet: 101,
            page: page,
            hitsPerPage: pageSize,
            filters: filterString.isNotEmpty ? filterString : null,
            ruleContexts: ['search-unaltered', 'page-search', ...appSearchContexts],
          ),
        );
      }

      // Execute the search request.
      final List<SearchResponse> algoliaResponses =
          (await client.searchForHits(requests: searchQueries)).toList();

Merge and Update Facets

Combine facets from all responses:

Future<Map<String, Map<String, int>>> mergeFacetResponses(
  List<SelectedFilter> selectedFilters,
  List<SearchResponse> responses,
) async {
  final Map<String, Map<String, int>> mergedFacets = {};

  for (final response in responses) {
    final facets = response.facets ?? {};

    for (final facetKey in facets.keys) {
      final facetValues = facets[facetKey] ?? {};
      mergedFacets[facetKey] ??= {};

      // Merge new values
      mergedFacets[facetKey]!.addAll(facetValues);

      // Add missing selected values with count 0
      for (final filter in selectedFilters) {
        if (filter.key == facetKey && !mergedFacets[facetKey]!.containsKey(filter.value)) {
          mergedFacets[facetKey]![filter.value] = 0;
        }
      }
    }
  }

  return mergedFacets;
}

Format Selected Filters

Format filters for Algolia's facetFilters parameter:

extension FilterFormatting on List<SelectedFilter> {
  List<List<String>> toFormattedFilterGroups() {
    return fold<Map<String, List<SelectedFilter>>>(
      {},
      (acc, filter) => acc..putIfAbsent(filter.key, () => []).add(filter),
    ).values.map(
      (filters) => filters.map((filter) => filter.format).toList(),
    ).toList();
  }
}