Meilisearch - organizing search in conjunction with Laravel

Hola, Amigos! I am a backend developer on Laravel at a product development agency Amiga . In the article I describe the organization of search through Meilisearch and the nuances of use in conjunction with Laravel.

Out of the box, according to the documentation, Laravel (scout) supports the following options for organizing the search system:

Algolia,

Meilisearch,

Typesense,

MySQL / PostgreSQL.

Algolia

In the Algolia option we have cloud storage. This option has both its pros and cons:

+ fast connection;
+ scaling;

- data is stored on a third-party server (may be critical for some);
- from a certain limit payment is required (free limit of 10k requests per month).

Typesense

Next, let's look at Typesense - a lightweight opensource search engine. From the features announced on the developers’ website:

Typo Tolerance (search with typos).;

Geosearch;

Custom ranking;

Merchandising (you can configure sorting for specified entities);

Vector & Semantic Search (automatically creates embeddings using built-in ML models or OpenAI / PaLM API and performs semantic search or nearest neighbor search);

Synonyms (supports search by synonyms);

Filtering & Faceting (supports faceted search).

MySQL / PostgreSQL

There is another option for organizing a search using standard MySQL / PostgreSQL tools. Suitable for projects that do not require search functionality. One of the advantages is ease of implementation.

Meilisearch

In our practice (for small and medium-sized projects) we use this particular search engine, because... it is quite simple and covers basic needs. The engine itself is written in Rust.

Pros:

lightweight,

multilingual out of the box,

search with typos,

facet search,

supports flexible ranking settings,

supports geosearch.

You can use it directly through the rest API, or use ready-made wrapper .

The developers themselves position Meilisearch specifically as a search engine and do not recommend using it as a separate database.

Features of Meilisearch

Among the features of Meilisearch, it should be noted that the search is carried out only by the beginning of the word ( prefix search ). That is, if we have the words: “car”, “automatic”, “bus”, “material”, and we enter the search query “mat”, then only the word will be presented in the results mat serial, but not the word auto mat .

It should also be noted that the search query cannot consist of more than 10 words. If the request contains more than 10 words, then what goes beyond these limits is ignored.

The key to meilisearch is indexes. An index is a set of documents united by common settings. A document is a record containing a primary key and a set of attributes. Below is an image from the off site. documentation explaining the structure of the document.

We can set search settings at the index level. To set the settings and, in general, to organize interaction with meilisearch, we can use the REST API directly, or use official package . Next we will consider changing the settings through this package.

Let's take a closer look at the settings available for indexes:

dictionary - specifies a list of phrases that meilisearch will perceive as one whole.

Example: $client->index('books')->updateDictionary(['J. R. R.', 'W. E. B.']);

displayedAttributes - specifies the list of attributes returned in search results. By default, when indexing, all attributes are included here automatically. Below is an example for a manual task:

$client->index('products')->updateDisplayedAttributes([
'title',
'description'
]);

faceting — settings for faceted search. The maxValuesPerFacet parameter specifies the maximum number (default 100) of values ​​returned during a faceted search. The second parameter specifies the sorting order of the results for this search.

Example:
$client->index('books')->updateFaceting([
'maxValuesPerFacet' => 2,
'sortFacetValuesBy' => ['*' => 'alpha', 'genres' => 'count']
]);

filterableAttributes - specifies a list of attributes available for filtering. Empty by default. If set, then when searching it will be possible to use these attributes as filters (available operators =, !=, >, >=, <, <=, TO (equivalent to BETWEEN), EXISTS, IN, NOT, AND, or OR).

Example:
$client->index('products')->updateFilterableAttributes(
['id','category_id','name','_geo','city_id']);

pagination - sets pagination settings. Contains an object with only one maxTotalHits field. The documentation does not recommend setting this value to more than 20,000.

Пример:
$client->index('products')->updateSettings([
'pagination' => [
'maxTotalHits' => 10000
]
]);

proximityPrecision — prediction accuracy. Possible options are byWord - more accurate/longer indexing, byAttribute - faster indexing/less accurate search. ByWord by default.

rankingRules - specifies ranking rules. Sets the priority of ranking rules. The default order is as follows:

[
"words" (sorts results by decreasing number of matching query terms),
"typo" (sorts results by increasing number of typos),
"proximity" (sorts results by increasing distance between matching query terms)
),
"attribute" (sorting based on attribute order),
"sort" (sorting based on the specified field to sort on request),
"exactness" (sorts results by similarity of matching words to the query words
)
]

Example of updating ranking rules:

$client->index('products')->updateRankingRules([
'words',
'sort',
'typo',
'proximity',
'attribute',
'exactness'
]);

searchableAttributes - specifies a list of attributes available for search. This list also sets the priority of attributes. You can also set runtime in the request itself via the attribute.

Пример:
$client->index('movies')->search('products', [
'attributesToSearchOn' => ['title']
]);

separatorTokens - specifies a list of separators for tokens.

nonSeparatorTokens - Specifies a list of characters that do not limit the beginning and end of a single token.

sortableAttributes - specifies a list of fields by which sorting is possible.

Example:
$client->index('products')->updateSortableAttributes(['id','name','_geo']);

stopWords is a list of stop words that are ignored during searches.

synonyms — specifies a list of synonym words.

Пример:
$client->index('movies')->updateSynonyms([
'wolverine' => ['xmen', 'logan'],
'logan' => ['wolverine', 'xmen'],
'wow' => ['world of warcraft']
]);

typoTolerance - sets sensitivity to typos. Contains an object with a set of fields:
– enabled - true|false — turns sensitivity to typos on or off.
– minWordSizeForTypos.oneTypo — minimum word size to accept one typo.
– minWordSizeForTypos.twoTypos — the minimum word size to accept two typos.
– disableOnWords — disables for words.
– disableOnAttributes—disables for attributes.

Example:
$client->index('products')->updateTypoTolerance([
'minWordSizeForTypos' => [
'oneTypo' => 4,
'twoTypos' => 10
],
'disableOnAttributes' => [
'title'
]
]);

//disable recording of spelling errors (you can also set a character limit)
$client->index('products')->updateTypoTolerance([
'enabled' => false
]);

Usage in Laravel

Laravel supports the use out of the box Meilisearch. To do this, specify the appropriate settings in env:

SCOUT_DRIVER=meilisearch
MEILISEARCH_HOST=http://meilisearch:7700
MEILISEARCH_KEY=someKey

Settings for indexing can be specified in the scout config file. In the example below we set attributes for sorting and filtering.

'meilisearch' => [
   'host' => env('MEILISEARCH_HOST', 'http://localhost:7700'),
   'key' => env('MEILISEARCH_KEY', null),
   'index-settings' => [
   	Product::class => [
       	'sortableAttributes' => ['name', '_geo'],
       	'filterableAttributes' => ['id', 'name', '_geo', 'category_id'],
   	],
   ],
],

To use search in the model, we use the Searchable trait. And then we can make queries like:

1) Product::search($q)->get() — as a result we get the same Eloquent collection.
2) If the indexes themselves are needed, then we use raw. Product::search($q)->raw() - as a result, we get the original indexes from Meilisearch.
3) If a more complex selection is needed, then we can pass a callback as the second argument to search. For example:

Product::search(
	'Какое-либо название',
	function (SearchIndex $meilisearch, string $query, array $options) {
   	 
    	$options['filter'] = "category_id IN[$ids]";
    	$options['sort'] = ["title:desc"];
    	$options['limit'] = 100;
 
    	return $meilisearch->search($query, $options);
	}
)->get();

In $options we can set filtering, sorting and sampling options.

If fine-tuning or more complex requests are needed, we can directly use the rest API client from package .

For example, if we need to search in several indexes at once, we can do it with one request using the client:

$client->multiSearch([
  	(new SearchQuery())
      	->setIndexUid('products')
      	->setQuery(‘запрос 1’')
      	->setLimit(5),
  	(new SearchQuery())
      	->setIndexUid('categories')
      	->setQuery('запрос 2')
      	->setLimit(5),
  	(new SearchQuery())
      	->setIndexUid('comments')
      	->setQuery('запрос 3')
	]);

Meilisearch can also be used to build filters and faceted search:

$client->index('products')->search('classic', [
  'facets' => ['color', 'size', 'country']
]);

As a result, we will see the number of records for each attribute.

We can also set the weight of index attributes, which in turn will influence the ranking of results. For example, we'd like the title attribute to have significantly more impact than the brand and description. This can be done with the following query.

$client->index('products')->updateSearchableAttributes([
  'title',
  'brand',
  'description'
]);

That is, this query simultaneously updates the list of attributes to be searched and sets their ranking in the search.

A little about geosearch

You can also use geosearch. Below is an example, with filtering, sorting and pagination.

Product::search(
	'Какое-либо название',
	function (SearchIndex $meilisearch, string $query, array $options) {
   	 
    	$options['filter'] = "_geoRadius($lat,$long,$dist) AND id IN[$ids]";
    	$options['sort'] = ["_geoPoint($lat,$long):$dir"];
    	$options['limit'] = $count;
 
    	return $meilisearch->search($query, $options);
	}
)->get();

In the example above, we filter by radius and certain IDs, sorting by distance.

Semantic search

Among the recent features, meilisearch supports semantic search (currently in experimental status) using embeddings for this. To do this, meilisearch can be configured to use models from OpenA, or use models from Hugging Face (in this case, embeddings are calculated locally. The documentation shows the BAAI/bge-base-en-v1.5 model as an example). Below is an example from the documentation:

curl \
  -X PATCH 'http://localhost:7700/indexes/movies/settings' \
  -H 'Content-Type: application/json' \
  --data-binary '{
    "embedders": {
      "default": {
        "source": "huggingFace",
        "model": "BAAI/bge-base-en-v1.5",
        "documentTemplate": "A movie titled '{{doc.title}}' whose description starts with {{doc.overview|truncatewords: 20}}"
      }
    }
  }'

Embedders indicates the name (default), then the source (huggingFace, or OpenAI), the name of the model itself and the template for the embedder (documentTemplate).

Then you can run the request itself. The semanticRatio parameter is responsible for the ratio of regular and semantic search. We specify the previously created default as the embedding model.

curl -X POST -H 'content-type: application/json' \
  'localhost:7700/indexes/products/search' \
  --data-binary '{ 
    "q": "kitchen utensils",
    "hybrid": {
      "semanticRatio": 0.9,
      "embedder": "default"
    }
  }'

By link demo available. In it we can play with the slider responsible for the ratio of regular and semantic search and see the impact on the search result.

In conclusion, it should be noted that meilisearch is well suited as a search engine for small and medium-sized projects. When choosing, you should also not forget about the feature of searching only from the beginning of the word (prefix search), because somewhere it can be critical.

Prepared example project (Laravel 11/ Posgres SQL/ Nuxt3) with implemented meilisearch search. Write in the comments if it was useful!