Using Meilisearch full-text search engine with Rails

Meilisearch is “An open source, blazingly fast and hyper relevant search-engine”. In this article, I’ll show you how to integrate it with Rails app.

Quickstart

Download and install process described here: https://docs.meilisearch.com/guides/advanced_guides/installation.html

There’s also a nice article about installing MeiliSearch in production and running it as a service: https://docs.meilisearch.com/running-production

Milisearch has a gem for Ruby apps, so first, you need to add to your Gemfile:

gem 'meilisearch', require: false

And run bundle install. Next, let’s create a file where we gonna keep our connection configuration. You don’t need a master key unless Meilisearch running in a production environment, so you can leave a key empty for your development env. Don’t forget to add this file to .gitignore

config/meilisearch.yml

host: http://127.0.0.1:7700
key: masterkey

Now we are ready to connect to our Meilisearch client, for example like this:

config = YAML.load_file('config/meilisearch.yml')
client = MeiliSearch::Client.new(config['url'], config['api_key'])

Creating modules

We want to do two basic things with our database objects: to index them and to search in indexes. So, we’ll need two modules: Searchable to mix class method that will allow us to search in the index, and Indexable to mix instance methods to model. Both modules will need methods to connect to a client so it’ll be handy to create the third module, MeiliSearchable, and include it in those two. Let’s start with it:

#/lib/modules/meili_searchable.rb
require 'meilisearch'

module MeiliSearchable

  private

  def client
    config = YAML.load_file('config/meilisearch.yml')

    @client ||= MeiliSearch::Client.new(config['host'], config['key'])
  end

  def index
    @index ||= client.index(index_name)
  end
  def index_name
    if self.class.to_s == 'Class'
      self.name.downcase.pluralize
    else
      self.class.name.downcase.pluralize
    end
  end
end

In this module, the client method is used to connect to MeiliSearch client, and the index method allows us to receive specific index for model on that we call our methods.The index_name method will return our object class name or our class name either module is included to class or class is extended with the module. For example, the Account class index name will be accounts. And for Account class instance it will be the same.

The Indexable module needs only one public method: add_to_index. We will add this method to after_commit callback in our model because add_documents method of MeiliSearch client updates the document if it already exists (it even has alias add_or_replace_documents). Note that there’s also a method update_documents with alias add_or_update_documents. The difference is that the second method won’t remove existing fields in the document if they were sent empty from this method.

module Indexable
  include MeiliSearchable

  def add_to_index
    index.add_documents([self.to_indexed])
  end
end

Note the to_indexed method. This method must be implemented in each indexable class and define how exactly our object will be stored in our index. We’ll return to this later.

Finally, we need a Searchable module to search through index

module Searchable
  include MeiliSearchable

  def search(text)
    index.search(text, limit: 1000)
  end
end

Let’s continue with our Account example. First of all, we shall use our modules:

class Account < ActiveRecord::Base
  include Indexable
  extend Searchable
  ...

We also need to create a callback to add our accounts to index after create or update

after_commit :add_to_index

And, finally, to implement to_indexed method:

def to_indexed
  { id: "account#{id}", account_id: id, email: email }
end

Note that id must be unique through indexes, so we add the ‘account’ prefix to it.

Creating an index

Update: as for meilisearch 0.16, you don’t need to do this, an index will be created automatically after you push first document! https://blog.meilisearch.com/whats-new-in-v0-16-0/

For now, we can automatically add our accounts to index, and to search through it. We only need to create an index to start it all working. Let’s do it with migration:

rails g migration create_meilisearch_accounts_index

And edit it like this:

require 'meilisearch'

class CreateMeilisearchAccountsIndex < ActiveRecord::Migration
  include MeiliSearchable

  def up
    client.create_index('accounts')
  end

  def down
    client.delete_index('accounts')
  end
end

That’s it! Now run the migration and have some fun

rails db:migrate

rails c

Account.create(email: some_new@mail.org)

Account.search(‘some’)
=> {“hits”=>[{“id”=>”account53120", “account_id”=>”53120", “email”=>”some@mail.org”}], “offset”=>0, “limit”=>1000, “nbHits”=>1, “exhaustiveNbHits”=>false, “processingTimeMs”=>2, “query”=>”some”}

If you want to fetch objects from your databes, you can do like this:

ids = Account.search('some')['hits']&.map{|r| r['account_id']}

Account.where(id: ids)

Further improvements. Sidekiq

When you have a lot of documents, adding new documents to index might be slow. It’s a good idea to do in asynchronously, for example, with Sidekiq. I won’t show here how to install and configure it, it’s pretty easy to google. After you deal with it, we can create a worker

# app/workers/meili_search_worker.rb
class MeiliSearchWorker
  include Sidekiq::Worker

  def perform(class_name, id)
    class_name.constantize.find(id).add_to_index
  end
end

Add this method to our Indexable module

def async_add_to_index
  MeiliSearchWorker.perform_async(self.class.name, self.id)
end

and finally change our after_commit Account callback

after_commit :async_add_to_index

Final. Adding existing documents

For an existing app, we can use a simple rake task to add existing documents

namespace :meilisearch do
  require 'meilisearch'

  task :fill_indexes => [ :environment ] do
    Account.find_in_batches(batch_size: 500) do |group|
      add_batch(index, group)
    end
  end

  def index
    config = YAML.load_file('config/meilisearch.yml')

    @index ||= MeiliSearch::Client.new(config['host'], config['key']).index('comments')
  end

  def add_batch(index, group)
    documents = group.map(&:to_indexed)
    begin
      index.add_documents(documents)
    rescue
      puts "Got timeout, sleeping for 2"
      sleep 2
      retry
    end
  end
end

Note that we use add_documents method here, that adds multiple documents to the index.

Thank you for reading this article, hope it was helpful! Please leave feedback if you have any questions or suggestions.