Skip to main content

Advanced Search, Autocomplete and Suggestions with ElasticSearch and the Searckick gem Discussion

General • Asked by Chris Oliver

Chris, this is awesome!

I was hoping you'd do a Searchkick/ElasticSearch video since you did the Ransack video!

Thanks for all the great videos!



Thanks Chris – another great tutorial!

What about if you want to search through more than 1 items, with a separate MVC.
Say you have Questions and Answers. I tried modifying the search form but it seems
there are lot more areas to look at.

can you share your experience , searching multiple models in one query please , i can't get it to work

@excid3:disqus can you please reply to my questions here:
1) the github link
2) how do i search multiple models( In my case i have a post, comments and tags . i want to search all at once

Hey Aime! Sorry, I've been meaning to get back to you but have been behind.

Source code: https://github.com/excid3/g...

And searching multiple models works a bit differently. For example, you would want to index and search Post, but the fields you would index should include both the comments and tags. This is what I did for indexing tags in the example.

Another way to do it would be to search those separately and combine the results. Facebook does it this way for example.

@excid3:disqus I believe you can also use the newly added `indices_boost` (https://github.com/ankane/s... with multiple indices as per the updated readme:

Search across multiple indices with:

`Searchkick.search "milk", index_name: [Product, Category]`

Boost specific indices with:

`indices_boost: {Category => 2, Product => 1}`

However, I am having difficulty getting this to work properly (it's currently only returning results from two indices when I have 3 listed),

If you ever get a chance to play with this feature with a similar setup could you please let me know whether you get it working properly? Thanks.

Oh very cool. Nice feature to have. If you're having trouble with it, definitely ask on the SearchKick github issues page. They're the ones that know it best and can probably point out the problem right away.


Thanks for your this great video @excid3:disqus . I advise people to use Algolia if they want to use a database search https://blog.algolia.com/fu...


can we please have the github link?


Hello Chris....
If I have many models and I want a common search that works for all the models and gives results from all models in common search result page then how is this possible?

You'll either need to do separate searches and combine the results or index one main model and include the nested models attributes in the index.

I have one more problem on how to use elastic to capistrano deployment? And install elastic to ubuntu 14.04?


Thanks for the tutorial. I tried to install elasticsearch but i get an error. I explained the full error in this post http://goo.gl/cqVFyl in stackoverflow.


Hello Chris, I included elastic-search as you explained in the tutorial... working perfectly on local... but when I put on digitalocean server, they blocked my droplet that network is not secure with process of elastic search and If I get it stop and secure it then will open... How I can secure the elastic search for production server?


I have just finished the One Month class on RAILS and checked into this. (which is obviously a bit more advanced). How does one generate a Questions controller or does that arrive when you bundle instal the search gem?

You can generate a scaffold for it just like you did with the posts in the Rails class. You'll possibly want different fields, but you can create it the same way.


Is anyone else having this problem? http://stackoverflow.com/qu...

I've been trying to install elastic search but ran into some issues with homebrew and java. I recently upgraded to Yosemti and it seems to be the issue. Anyone know of a fix


Any thoughts on how to highlight results?


I am trying to follow along with this but I get this error when I try Article.reindex (equivalent of Question.reindex)

NoMethodError: undefined method `reindex'

In the console, I try: Article.reindex

It returns:
NoMethodError: undefined method `reindex' for Article (call 'Article.connection' to establish a connection):Class

Also, when I try to run the local host, I get this error:

Index missing - run Article.reindex
# GET /articles.json
def index
@articles = Article.search(params[:q])
end

# GET /articles/1

For me:

def index
query = params[:q].presence || "*"
@articles = Article.search(query)
end

gives this error:

MissingIndexError in ArticlesController#index
Index missing - run Article.reindex

My key difference is that:

I use simple form for forms, so i made a new partial to insert in my articles index as:

<%= simple_form_for(@article) do |f| %>

<%=f.text :q, :nil, placeholder: "Search" %>

<% end %>

TOTAL NUMPTY: I DIDN'T INSTALL THE GEM. SORRY!


Thanks for the awesome video! Switched from sphinx to elasticsearch. Much easier!


How would I search a Post model where I wanted only see published post? Can't figure out how the code should look in the controller index.

You can actually just add that into your query. For example, if your published column was a boolean: Post.search('*',where:{published:true})

How would I apply a scope? Like one of these.

scope :draft, ->{where(published_at: nil)}
scope :published, ->{where.not(published_at: nil).where("published_at <= ?", Time.now.in_time_zone("Eastern Time (US & Canada)"))}
scope :scheduled, ->{where.not(published_at: nil).where("published_at >= ?", Time.now.in_time_zone("Eastern Time (US & Canada)"))}
scope :pastevent, ->{where("event_end <= ?", Time.now.in_time_zone("Eastern Time (US & Canada)"))}

I haven't done the example, but have you tried Post.published.search("*")?


And just like that, all my troubles disappear. Thanks man!


How can I add 'will paginate' to the search results... in class PostsController < ApplicationController?
I currently have this;

if query = params[:post_search].presence || "*"
@posts = Post.search(query, suggest: true)

Finally this works:
@posts = Post.search(query, page: params[:page], per_page: 10)


Thanks for the video Chris. I am having issues with the autocomplete part. I have jquery-ui loaded and everything done up to the point where I try to test the libraries are loading (up to 17:32mins into the video. My first autocomplete test does not show anything (was expecting the word TEST would show as in your case). When I inspect the element, I see the following error:

Failed to load resource: the server responded with a status of 404 (Not Found) - https://bakr-fgneba.c9users...
When i click on the link i see the error (Couldn't find Post with 'id'=autocomplete). Looks like something to do with routing.

My route is defined as:

resources :posts do
get :autocomplete
end

Any idea why this could be happening?


Hi Chris, i've tried using Bonsai and SearchBox for deploying to Heroku and both are a nightmare to get working. Do you have any recommendations on deploying to Heroku?

It should just pick up the searchbox ENV variable and work. Depending on how many models you're indexing, you'll need the correct number of shards available (some of the ES options limit shards or other things).


Hi Chris!

I love the videos! I've been stuck on this problem for a while and tried StackOverflow(http://stackoverflow.com/qu... but with no response and I know you're busy but I thought I'd try.

I have forums(the parent model) that have many forum_threads but each forum_thread is unique to that certain forum. I'd like to be able to search each forum (which IS working just fine) as well as each forum_thread within that unique forum(which is just rendering ALL forum_threads rather than the searched thread). Would you be so kind as to take a look at my SO question and see what I'm doing wrong?


Has anyone else had a problem with search_data like this: https://gorails.com/forum/s... ?


Would you still recommend using Searchkick over alternatives like elasticsearch-rails and elasticsearch-ruby?

Absolutely. If you've looked into ES-rails or ES-ruby gems there docs are pretty poor at best last I saw. I'm sure it's getting better and eventually will be on par with Searchkick, but for now they feel behind. Searchkick is basically a nice ruby-ish wrapper so everything feels intuitive whereas ES-rails and stuff came from non-ruby devs you can tell so it's less ruby-ish in style. I still use Searchkick on a daily basis and recommend it.

My guess is over the next year or so the official ES-rails/ruby gems will get all the nice helpers and things to make it smooth to use like Searchkick but until then you can't go wrong with Searchkick.


I'm a new subscriber and new to rails in general and I feel I'm in rails heaven! :)

Glad you love it and we're happy to have you as part of the community! :D


When I use ["Test"] as my rendered Json it works, however when I try to use data from my posts table, it no longer works. I have rechecked syntax and spelling multiple times and am 100% sure its right exactly following this tutorial. Any Ideas? I've posted in stack overflow, no response, attempted dozens of tutorials with different implementation methods, no use. It's really starting to annoy me because it should be quick and easy like your video. I payed and subscribed to your website hoping you could help me. I also posted this question with my code in the communities section.


Hi Chris, I have the following question about installing and running es in production:

In this article, the following is mentioned:
https://www.digitalocean.co...

"This tutorial assumes that your servers are using a VPN like the one described here: How To Use Ansible and Tinc VPN to Secure Your Server Infrastructure."

In this article however there is no mention of the need for a vpn and Tinc:

https://www.digitalocean.co...

This is a bit confusing, is this vpn necessary for basic usage of elasticsearch or not, or private networking only? It seems like a lot of work to set this all up. Or is the approach in the second article sufficient? ( authentication + firewall)

Thanks
Thomas

Great questions. I've only used Elasticsearch in production on Heroku where they take care of the security for you. As for securing your own instance, I don't believe that you need a VPN in order to secure it properly, but you will definitely want the authentication and firewall. There are some other suggestions out there about adding in SSL and some other things as well, for example: https://brudtkuhl.com/secur...


Thanks Chris. Questions:

1. Does Searchkick auto reindex questions after new question is created?

2. If number 1 is no, what's the best way to make it happen? A callback inside the Question model?

3. You did mentioned earlier inside the video where Searchkick has a 'smart feature' where it can rank the result based on the popularity. Does it available out of the box or extra configuration required?

Thanks.

1. It will auto index the new question, but only the new one that is added. It won't touch the existing ones. Searchkick implements callbacks on the models to automatically index them when they change.

3. You'd have to configure this. For example, you have to figure out what "popular" means. Is it views? Comments? etc. Once you figure out what you want, you can then pass in that data into your search_data so Searchkick indexes it and then you can query against it.


Hi Chris, I want to run Elasticsearch on a separate server. I can't seem to connect to it. I can curl it locally fine but not externally. I tried opening the ports and still no go. I suspect I'm not configuring the elasticsearch.yml correctly... Any tips?

These are likely the two main ones that could be misconfigured:

1. Make sure elasticsearch is bound to the public IP address and not just running internally.
2. Check to make sure the port is open on it through the firewall.

It's probably one of these two but could be something else. It's been a while since I setup ES across machines.


Really enjoyed this video and found it super helpful. One question I have relates to autocomplete. We used
fields: [{name: :text_start}, limit: 10]).map(&:name)
to specify where we'd like to get the autocomplete data from, but what if we wanted to do this on two fields of an entity?

As an example, say you have Person 1 (name = John) and Person 2 (name = Rachel)
Say they both have own a dog that is a breed of "Labrador"... I want my autocomplete to work when I start typing there names or if I start typing labr.... it word return labrador as an option in the autocomplete.
I was thinking it could be something like fields: [{name: :text_start, breed: :text_start}, limit: 10]).map(&:name), but then the map part makes no sense...

I think you're close, but you want an array of hashes.

Person.search params[:q], {
fields: [{name: :text_start}, {breed: :text_start}],
limit: 10
}

The searchkick docs also show another method, putting the autocomplete types on the searchkick method in the model and just listing out the fields by name. https://github.com/ankane/s...

Yep that was the solution! Figured it out this afternoon. Since watching this video and then doing a bunch more reading, I've implemented an insanely powerful search system on our platform. What a great resource :)

ElasticSearch + Searchkick is awesome. I need to get GoRails search swapped over to it very soon!


Hi Chris. Watching the episode, I was wondering how i could implement this feature using a single search input box which is part of my navbar.

Indeed, my app goes through different index views (switchable through sidebar items) but the search input stays at the top. My workflow would be to perform individual searches based on the index view i am and not search across all models. Shall I hide show identical search boxes based on the view page?

I think there are ways to be more clever and to leverage only one and redirect the search with "if statements" (model, view or controller name) but great to have people views on this.

Well, your search in the navbar implies that it's a global search, not just for the category you're viewing. You can do searches across the different models and combine the results (maybe show the current category results first). You can see an example of this on Facebook with the global search bar, yet different categories of results (pages, people, businesses, etc).

That would definitely make for a good episode and I've wanted to do this for GoRails sometime soon anyways to search both episodes and the forum.

Hi Chris - that would be an awesome addition to this search tutorial...(a global search using typeahead.js / searchkick that spans across 2 or 3 models). I'm trying to do this in an app of mine and am hitting a lot of snags. Will you be getting to this anytime soon? much appreciated...

I'm using easyAutocomplete for the global search on GoRails. It just makes an AJAX request that queries episodes and forum threads and returns a JSON object of two arrays, one for each and those get displayed accordingly. I'll try to do an episode on this soon.

Just as a heads up I just published this week's episode on this topic. :)

Ah nice! - you da man...I'll watch it tonight. Much appreciated...


Hi Chris. I had 2 follow up questions for you:
1. how can you send the autocomplete selection directly/automatically to the search instead of the video two steps (indeed first you select and then you have to type enter)?
2. in the autocomplete, how do you merge 2 fields. say you look at last_name but you want to display both first_name and last_name concatenated.
have a good day! Stan

1. I'm not quite sure what you're asking on this one. Rephrase and try me again? :P
2. You'd have to index them combined, so you'd make a "name" attribute that was those concatenated and then search on that instead of the separate fields.


Hey man, I can't figure out how to style the suggestions that show up beneath the search bar. Any links or ideas? I keep trying to Google this in different ways but I'm still coming up emptyhanded.


I figured my own problem out, finally. It is a <ul> and each <li> in the dropdown has the ui-menu-item class, and that's really all you need to attach the styling to, plus :hover for some feedback when the user mouses over the list items. Hope this helps someone else!


Login or Create An Account to join the conversation.

Subscribe to the newsletter

Join 24,647+ developers who get early access to new screencasts, articles, guides, updates, and more.

    By clicking this button, you agree to the GoRails Terms of Service and Privacy Policy.

    More of a social being? We're also on Twitter and YouTube.