Using Meilisearch full-text search engine with Rails

Meilisearch is “An open source, blazingly fast and hyper relevant search-engine”. In this article, I’ll show you how to integrate it with Rails app.

Download and install process described here: https://docs.meilisearch.com/guides/advanced_guides/installation.html

There’s also a nice article about installing MeiliSearch in production and running it as a service: https://docs.meilisearch.com/running-production

Milisearch has a gem for Ruby apps, so first, you need to add to your Gemfile:

gem 'meilisearch', require: false

And run bundle install. Next, let’s create a file where we gonna keep our connection configuration. You don’t need a master key unless Meilisearch running in a production environment, so you can leave a key empty for your development env. Don’t forget to add this file to .gitignore

config/meilisearch.yml

host: http://127.0.0.1:7700
key: masterkey

Now we are ready to connect to our Meilisearch client, for example like this:

config = YAML.load_file('config/meilisearch.yml')
client = MeiliSearch::Client.new(config['url'], config['api_key'])

We want to do two basic things with our database objects: to index them and to search in indexes. So, we’ll need two modules: Searchable to mix class method that will allow us to search in the index, and Indexable to mix instance methods to model. Both modules will need methods to connect to a client so it’ll be handy to create the third module, MeiliSearchable, and include it in those two. Let’s start with it:

#/lib/modules/meili_searchable.rbrequire 'meilisearch'

module MeiliSearchable

private

def client
config = YAML.load_file('config/meilisearch.yml')

@client ||= MeiliSearch::Client.new(config['host'], config['key'])
end

def index
@index ||= client.index(index_name)
end
def index_name
if self.class.to_s == 'Class'
self.name.downcase.pluralize
else
self.class.name.downcase.pluralize
end
end
end

In this module, the client method is used to connect to MeiliSearch client, and the index method allows us to receive specific index for model on that we call our methods.The index_name method will return our object class name or our class name either module is included to class or class is extended with the module. For example, the Account class index name will be accounts. And for Account class instance it will be the same.

The Indexable module needs only one public method: add_to_index. We will add this method to after_commit callback in our model because add_documents method of MeiliSearch client updates the document if it already exists (it even has alias add_or_replace_documents). Note that there’s also a method update_documents with alias add_or_update_documents. The difference is that the second method won’t remove existing fields in the document if they were sent empty from this method.

module Indexable
include MeiliSearchable

def add_to_index
index.add_documents([self.to_indexed])
end
end

Note the to_indexed method. This method must be implemented in each indexable class and define how exactly our object will be stored in our index. We’ll return to this later.

Finally, we need a Searchable module to search through index

module Searchable
include MeiliSearchable

def search(text)
index.search(text, limit: 1000)
end
end

Let’s continue with our Account example. First of all, we shall use our modules:

class Account < ActiveRecord::Base
include Indexable
extend Searchable
...

We also need to create a callback to add our accounts to index after create or update

after_commit :add_to_index

And, finally, to implement to_indexed method:

def to_indexed
{ id: "account#{id}", account_id: id, email: email }
end

Note that id must be unique through indexes, so we add the ‘account’ prefix to it.

Update: as for meilisearch 0.16, you don’t need to do this, an index will be created automatically after you push first document! https://blog.meilisearch.com/whats-new-in-v0-16-0/

For now, we can automatically add our accounts to index, and to search through it. We only need to create an index to start it all working. Let’s do it with migration:

rails g migration create_meilisearch_accounts_index

And edit it like this:

require 'meilisearch'

class CreateMeilisearchAccountsIndex < ActiveRecord::Migration
include MeiliSearchable

def up
client.create_index('accounts')
end

def down
client.delete_index('accounts')
end
end

That’s it! Now run the migration and have some fun

rails db:migrate

rails c

Account.create(email: some_new@mail.org)

Account.search(‘some’)
=> {“hits”=>[{“id”=>”account53120", “account_id”=>”53120", “email”=>”some@mail.org”}], “offset”=>0, “limit”=>1000, “nbHits”=>1, “exhaustiveNbHits”=>false, “processingTimeMs”=>2, “query”=>”some”}

If you want to fetch objects from your databes, you can do like this:

ids = Account.search('some')['hits']&.map{|r| r['account_id']}

Account.where(id: ids)

When you have a lot of documents, adding new documents to index might be slow. It’s a good idea to do in asynchronously, for example, with Sidekiq. I won’t show here how to install and configure it, it’s pretty easy to google. After you deal with it, we can create a worker

# app/workers/meili_search_worker.rbclass MeiliSearchWorker
include Sidekiq::Worker

def perform(class_name, id)
class_name.constantize.find(id).add_to_index
end
end

Add this method to our Indexable module

def async_add_to_index
MeiliSearchWorker.perform_async(self.class.name, self.id)
end

and finally change our after_commit Account callback

after_commit :async_add_to_index

For an existing app, we can use a simple rake task to add existing documents

namespace :meilisearch do
require 'meilisearch'

task :fill_indexes => [ :environment ] do
Account.find_in_batches(batch_size: 500) do |group|
add_batch(index, group)
end
end

def index
config = YAML.load_file('config/meilisearch.yml')

@index ||= MeiliSearch::Client.new(config['host'], config['key']).index('comments')
end

def add_batch(index, group)
documents = group.map(&:to_indexed)
begin
index.add_documents(documents)
rescue
puts "Got timeout, sleeping for 2"
sleep 2
retry
end
end
end

Note that we use add_documents method here, that adds multiple documents to the index.

Thank you for reading this article, hope it was helpful! Please leave feedback if you have any questions or suggestions.