Global Autocomplete Search

So in this week's episode, we're going to be talking about adding a global autocomplete to your Rails application, something like you see on GoRails, where you can type in a keyword, or a search term or whatever, and it's going to make some AJAX requests to load the first five results of a couple different modules. This is searching episodes and forum threads, or as they are called here: "Community Discussions", those then search the models appropriately, it actually did two searches, one to get the episodes results, and community discussions was a separate search, then, that get's displayed via a library that you use to create the autocomplete functionality, but you can also just hit "Search" because this is a normal text box, and do your real search, and you can get a lot more results than just the first five here.

Every time you type into this box, it's going to fire an AJAX request, the server is going to do some searches, and combine those results into a JSON object that gets returned to the browser, and then your JavaScript library for the autocomplete will take those results, and then display them however you like. I'm using a library called EasyAutocomplete to pull this off, and the feature that we're going to be using here is the categories feature, so if you were to click on an autocomplete textbox, this is going to give you these options, but as you can see here, fruits and vegetables are the main categories, and they're formatted a little bit different than mine, but if you click on any of these, they will change the text there. You can also type something like "pepper", and it will highlight and bold that, and actually do your autocomplete search there. Now this shows you a good example of this, you have a JSON object that you would get back from the server, it has these groupings, so we have fruits and ther results from that are an array, and vegetables have an array as well, and the way that you tell this library how to parse those out separately, is you give a list of categories and you tell it it's going to pull fruits from the fruits property here, and then the header for what you display will be "Fruits" with a couple hyphens on both sides, and the same thing goes for vegetables, and that is exactly what I do in GoRails, it's just a little bit more complex because we're making these clickable so that when you click one, it will take you to the right page on the app, so this is basically what we're going to be doing, this library already supports this out of the box, so that means that we don't have to worry about the Front End work of making sure that our groupings work appropiately, and that these aren't selectable items and so on. That allows us to skip a lot of the front-end work, we just have to implement easy autocomplete, and make sure that we pass in the appropiate options and feed it with the proper JSON to make sure that it all works. So let's dive into building out an example with this, and see how all of that works behind the scenes.

Our example application here is really straightforward, it's Rails 5.1 app, we've got a model for movies, and another model for directors, and those are the two that we're going to be searching, I've seaded in some data already, and I've installed bootstrap just so that we have a navbar so we can pu the search box in there, and make that look pretty.

EasyAutocomplete has some installation instructions, you can download the library, and it comes with two CSS files and a JavaScript file, you want to include your JavaScript, and then optionally you can include the extra themes, or you can use their default theme, we will need to put those in our appliaction, and you also need to make sure that you have jQuery installed, and weather you do that through a CDN or you do that through your own asset pipeline using the jQuery Rails gem, which is probably the best way to go, for convenience of this example, I'm just going to throw in this exampls using the CDN, because I'm a little lazy on that aspect. For this, we want to grab the CSS, and we want to put those inside our application stylesheets folder, so I'm going to go ahead and do that, and move those over here inside of there. And then our jQuery easy autocomplete is going to go in the JavaScripts folder, and that's going to give us the ability to go into our

app/assets/javascripts/application.js

//= require jquery easy-autocomplete

and do the same thing in our stylesheets

app/assets/stylesheets/appliaction.scss

*= require easy-autocomplete
*= require easy-autocomplete.themes 

That will give us access to all of those in our app. Now the first thing I'd like to do after installing some of that JavaScript and CSS is to open up the browser, and then manually test that out, make sure it is loaded, and we can define some options here, and if we grab the first input, and really the only input on the page in this example, easy-autocompletes and we pass in those options

$("input").easyAutocomplete(options)

If all that works correctly, you saw this change a little bit in styling as it got the CSS applied for easy autocomplete, and now if we were to type in any of those options, like "blue" or "red", we will see it automatically creates the autocomplete window and then it highlights the word that you were typing in any of those matches. And you can see if you type "e", you can see that highlighted on four out of the five results. So we do know easy autocomplete is working, and now we need to go work on our search stuff on the back end so we can then pass that into EasyAutocomplete for these options here. So we need a search endpoint that's actually generic to all of the models that we want to search in our case, I'm going to be searching movies and directors, and so it makes sense for us to have some sort of main controller with a search action on it.

config/routes.rb

Rails.application.routes.draw do 
    get :search, controller: :main 
    root to: "main#index"
end

That should give us a "/search" URL that would go to the main search action, and inside of there, we can define in our main controller, which I just created, we can define a search here that will give us the results back that we want. So really we want to render JSON from here all the time, and we want to render from a JSON object and we want movies, and an array back, and we also want directors, and we want to pass in an array back,

apps/controllers/main_controller.rb

def search 
    render json: {movies: [], directors: []}
end 

If this is all wired up correctly, we should be able to go to search, and you can append .json, or we can just do /search, and you should always get that ruby hash back that's converted to JSON. So we want to dinamically populate these results from a database search, or Elasticsearch or whatever kind of search you want to use, but we want to populate these two sets of results with our search results.

If we hop over to our Gemfile, I've installed the ransack gem, and I've already bundle install this, and this is what we're going to be using for basic search, but of course you can use Elasticsearch for this, or searchkick, or pg_search or any of those other search options. All you need to be able to do really, is to take the params, run a query against your data, and get your array of results back, so as long as your search can do that, you're free to use whatever you like to power this, you just need to get those results back so that you can pass them in in that main controller. And main controller will need to query those results, so let's pull that up here.

We need to search both movies and directors here, and if you're familiar how to do that with Ransack, it looks something like this, where we have a query where we set up the query for ransack, and then we ask for the results, and we can optionally ask for distinct results so we don't get duplicates. So what I'm going to do, is change this up a little bit, we are not going to use that cue variable, which is useful for rendering search forms with ransack, we're just going to get the movies back, so we'll say movie, and we'll do:

app/controllers/main_controller.rb

def search 
    @movies = Movie.ransack(params[:a]).result(distinct: true)
    @directors = Director.ransack(params[:a]).result(distinct: true) 

    render json: {movies: [], directors: []}
end 

We have two arrays of ActiveRecord objects, and we need to be able to convert these to the name and URL JSON objects, and there's of course many different ways that you can convert ActiveRecord to JSON, you can do this in a ruby iterations with maps, you can do this with jbuilder or ActiveModel serializers, it doesn't really matter what option you take here, and what I'm going to do, is I'm going to add a jbuilder template for this, just so that we can have our code organized appropriately there. So I'm going to split the windows, and we're going to edit

app/views/main/search.json.jbuilder

json.movies do
  json.array!(@movies) do |movie|
    json.name movie.name
    json.url movie_path(movie)
  end
end

json.directors do
  json.array!(@directors) do |director|
    json.name director.name
    json.url director_path(director)
  end
end

For all of those, we should now be generating the JSON appropriately so that we have the name and the URL for each one of those. And of course, now we can go try that out by adding .json to the URL, and we're going to get our list of movies and directors, and we get all of the movies and directors in the database as our results, and the only other thing that I would say here, is you can add a

app/controllers/main_controller.rb

before_action :force_json, only: :search 

#code 

private 

    def force_json 
        request.format = :json 
    end 

And then that will make sure that your search doesn't have to have that .json in the URL, and you can hit /search and it will always render JSON no matter what that format was like search.html, it will automatically always return you JSON. Now we have to go and correct our movies and director stuff, because if we add in a query term here, like "Cure". "Cure for wellness" should be a filtered result here, but we don't actually see that filtering correctly, so we need to go and address that, and make sure that we have our search stuff working, so that when you pass in q= whatever, we take that and do a proper search across our models.

First off, what we want to do is to limit the results to about five, we don't want ever want to return more than five, otherwise a dropdown would be enormously long, and that wouldn't be very helpful. So we want to only keep maybe you know, three to five results for each one of those they want to display. That will probably be you best results, now with ransack is actually designed for something more complex like searching against multiple columns and different ways of searching. So what we want to do is to take that params q, which is whatever the user types in, and we want to actually match that against the name. And so we can say name_cont: params[:q], so we're searching against the name column, and we're saying we want to match ones that contain that string that came from the URL, so you can use your search anyway you would like, and you can pass in this into elasticsearch or anything like that, as long as you get those results back, and you can limit them to a reasonable number, like three to five, then you can assign them to this, and your JSON jbuilder will automatically take those records, and convert them to the proper column, and the url for each one of those. This is how we're going to do it for Ransack, your implementation maybe a little different, if you're using a different search mechanism, but here we should be able to say something like q=s, and we will only get results that have the letter s in it, but we can even go further than that. So let's do "split", so "Sp", and we'll find movies that have "Sp" in it, which is only Split, and there are no directors that have "Sp" in their name in the list that I have in the database. We now know that our search is working appropriately, which means that we can go connect easy-autocomplete to the search URL and build our JavaScript to make all of this work nice and easily, so once we have that working, all we have to do now is to go and add our options into the way that we call easy-autocomplete when we set up the page.

I'm going to add a new file in app/assets/javascripts/search.js, and this is where we're going to write all of our code now. I'm going to try to write as much of this as I can with regular old JavaScript instead of jQuery even though we have jQuery available, we're going to add an event listener for Turbolinks

document.addEventListener("turbolinks:load", fuction() {

});

Inside of here, we want to grab that element on the page, and in the app/views/layouts/_navbar.html.erb

<form class="form-inline my-2 my-lg-0">
    <input class="form-control mr-sm-2" type="text" placeholder="Search", data-behavior="autocomplete">
</form>

app/assets/javascripts/search.js

document.addEventListener("turbolinks:load", function() {
  $input = $("[data-behavior='autocomplete']")

  var options = {
  }

  $input.easyAutocomplete(options)
});

We're going to pass in the options, so we will set up options here, and this is going to get kind of long because we have to define all of that stuff that we actually want to implement, now the other thing here, is that we have a form tag here that we want to replace with a form_with, and I just pasted here the final form with because it was going to be a lot to watch on camera, and it wasn't very interesting, but the important bits here are that we're pointing this form to the search path, and actually that means that our main search action should probably respond to HTML and not just JSON.

The other thing is that we wanted to be local= true because by default they also submit as AJAX now, by default, and we want the method to be equal to get.

Jumping back to our main controller, this points out something that if our form is going to submit a get request, our search method doesn't actually respond to HTML requests, it only responds to JSON formats.

So what we could do, is that we could actually duplicate the search method and rename one to autocomplete, force the autocomplete to only respond to JSON, and our regular search, instead of doing limit of 5, we can actually call our pagination here or have no limit, and then of course, we need to also rename our view for the main search JSON jbuilder. This would need to be renamed to autocomplete.json.jbuilder, and so that would allow you to separate out your autocomplete and your search, which actually might be a good thing. So for example, if your search ends up searching maybe more models than you would actually autocomplete, then you could add in some other queries down here or change the way that that works for the HTML search, whereas your autocomplete is going to be very fine-grained and kept to only as a single one. This is something that you would have to change, and of course you'd have to change your routes a little bit, so you could add autocomplete up here and so on. But It may be something wise that you might want to do to separate those two out. The other option that you have here, is to actually do a respond_to block, and pass in the formats:

app/controllers/main_controller.rb

respond_to do |format| 
    format.html {}
    format.json {
        @movies = @movies.limit(5)
        @directors = @directors.limit(5)
}
end 

And then you can get rid of the other limit(5), and kind of combine them into the same action. And this time, you wouldn't want to force JSON for that either. This is another way of handling that, it's kind of up to you if those are going to diverge, then you would want to split these out into their own actions, it can be very dedicated to them, but it really comes down to your application.

For example, facebook, the autocomplete is going to search common things like people or groups, but if you actually hit the search and go to the results that actually can list out businesses or events or locations, all that sort of stuff as well, so they have their autocomplete that actually functions separately from a real search, and so you can do the exact same here, but because we only have two models, I'm going to keep them all together, and let me fix this real quick, so that should be directors and movies.

Ok, let's go through these options, there's a lot of them, so we're going to run through these fairly quickly, but you can look at their documentation for how all of this works.

1. getValue This is going to be assigned to name, this is the name of the object in our JSON

2. url This is going to take a function, it will give us our phrase that we typed in, and we have to return a string for /search

3. categories an array of categories. Each category is going to have a list location. You can also add your header option, so you can have a title

4. list We want to be able to handle the clicks on those events, by passing a list option

var options = {
    getValue: "name",
    url: function(phrase) {
      return "/search.json?q=" + phrase;
    },
    categories: [
      {
        listLocation: "movies",
        header: "<strong>Movies</strong>",
      },
      {
        listLocation: "directors",
        header: "<strong>Directors</strong>",
      }
    ],
    list: {
      onChooseEvent: function() {
        var url = $input.getSelectedItemData().url
        $input.val("")
        Turbolinks.visit(url)
      }
    }
}

We will change this to a jQuery selector so that we have access to easy-autocomplete, since this will now be a jQuery object. Since this is a jQuery plugin, we have to do that. But that's also going to give us the ability to grab the getSelectedItemData that the user just clicked on. And in our browser, if everything goes well, we can type in "Sp", and click this, and we'll see that that gives us the URL of movies/2, and the Split shows up here, and in the field, because we clicked on it by default, it's going to insert that, and so what we can do then is that we can use Turbolinks.visit(url), and we can also say $input.val("") to also clear that out so that it feels more like you clicked on a result.

Now if we type "Split", and we click this, we'll be navigated to the movie "Split" in our database. And so we can do this from anywhere, we can have "A cure for wellness" that will take us there. We can also do something like go to a director "Gore Verbinski", and that will take us to director/1, and now we have this awesome autocomplete that's actually navigating us directly to pages on our site, and we don't have to search in our app.

Last but not least, is that if you type in a word and you don't click on the results, and you submit the search form, you'll be taken to that same search url that will have the query in it, just like the search autocomplete url, the JSON version, this will be able to generate your HTML results to give to the user for their search. That is how you implement autocomplete and a search form, so this gives you the functionality of both of those, and if you're interested, you can always split those two out to their own autocomplete actions, and a search action, and handle those a little bit differently in each case.

So I hope this was useful, and I will talk to you in the next episode.