Tracking Metrics with Ahoy and Blazer

What's up guys? This episode we're continuing on the topic of business metrics. In the last episode we used a tool called Segment to track analytics and visits and events that the users took, and send that data over to Segment would go and send it over to Google Analytics or Mixpanel, or Intercom, or SegmentIO or any other service that you might want to connect, and that is great. Except sometimes you might need that data inside of your own application, and you don't need those third party things, and Ahoy is a good solution for that . So we're going to take a look at this, it has a very similar API where you basically will install this and you have Java tracking for visits and events, and all of that stuff will be saved to your own database, and then we're going to be using a tool called Blazer by the same author in order to look at those business metrics, and then generate dashboards and graphs so that we can analyze that data in our own database. Let's dive into an example:

The installation of Ahoy is really simple, you need to add the gem ahoy_matey to your Gemfile and bundle install that, so first we'll do that. While that is running, we also need to add the ahoy gem to the application.js file, and if you're using Rails 5.1, this is going to require jQuery, so it's going to need you to include the jQuery rails gem, or jQuery from like a CDN so that you have that as well. The installation of this gem is really straightforward, we need to add the ahoy_matey gem to our Gemfile and we also need to add the jQuery rails gem. If you're using rails 5.1, because this JavaScript does depend on jQuery. Hopefully that will get factored out, so that it's not a requirement, but currently it is, and that is OK with me, so we'll just go ahead and paste in jQuery rails and the ahoy_matey gem, and then we'll add bundle install to install it. While that is running we will go to our application.js, and we will add the

//= require jquery 
//= require ahoy 

That will give us access to the ahoy library in our JavaScript, so that we can send over events in our JavaScript just like we would do with Segment, and you can also do events server-side like you would do with Segment as well.

There a ton of back-end options to store your data for this, and we're going to use just the standard ActiveRecord version of this, but you have the ability to store all of this stuff in any of these options, including things like Kafka which can integrate with some other things very nicely, so we're just going to do this in our SQL database, but you're free to pass these over to any one of those, and you can build your own with your custom option as well if you want to do that. We're going to run those two commands to install the ActiveRecord models for that, that's going to create a migration to create a "Visits" model, and Ahoy events, and that is where we'll be able to query for the data that it gives us, so we're going to run rake db:migrate so we now have those two models and our application to use, and we can see those if we look at our app models directory, we have:

app/models/visit.rb

class Visit < ActiveRecord::Base 
    has_many :ahoy_events, class_name: "Ahoy::Event"
    belongs_to :user, optional: true 
end 

The app/models/ahoy/event.rb is basically the table that will track events, so the cool part about this is that anytime someone visits your website, it tracks a couple pieces of data automatically, including the traffic source, location, the technology and the utm parameters, so all those things that you would typically see in Google analytics automatically be stored in your own database, and you will have access to that using the current_visit method, and then for the event side of things, we can give a name and properties to those events, so you can track them with JavaScript, you can use the track_all method which is described a little bit more in the ahoy.js repository, and then on the ruby side you can use ahoy.track and give it a name and a hash of options to pass over as properties. Those will automatically track those events, and if you're using devise or anything that provides current user in the server side aspect of things, that will automatically go and attach the relationship to the user for you, and you don't have to set up a single thing, that will be taken care for you automatically. I also want to mention the ahoy.js repository has a bunch of instructions on the various options that you have for tracking events, so you can do manual tracking by writing that code yourself called the track method, you can also have it track all events, so all views and clicks, you can have it track views specifically or clicks, or submits and so on. All of that is available for you to explore as you want, we're just going to do the manual tracking of events in this example, but feel free to explore that on your own. We now have access to ahoy in our application, both server side and client side, so if you want to fiddle with this on the client side, you can type ahoy.track("test"), and then send that over and you will see that it will give you little JSON objects back of when it happened, the name of your event and any extra properties you might have created, and if you go to the server side, we can take a look at that, and we'll see that we got a POST "/ahoy/events" which is automatically handled for us by installing the gem. So it sets up a route for you automatically and starts listening to that, and you don't have to do a thing, it will automatically take care of that for you, and it will track those to your database, which is really neat. We go into our rails console, say visit.last, and that will give us the last visit, and it gives us all this information like user_agent, which user_id it was, the browser, and it will even do geocoding in the background if you enable that to get you the region, the city, the postal code, all kinds of things which is awesome, so that you can have a good idea of where your users are. This is great, and then also we can look at the Ahoy events, and if we grab the last one, we see that we get that test event in our database that was submitted at the same exact time, and that is all stored for us nicely in our database. One example might be that you have a ecommerce site, and you want to track whenever a user attempts to add an item to their court. There may be errors with it or something, but you want to track that the user actually took an action to attempt to add the item to their card, so you want to do that JavaScript side, because server side would only happen with the successful ones, the client-side is probably where you want to catch the intention of that. So we're going to do this in JavaScript. You've got a link here, it's not really going to do anything, but we want to write a little JavaScript so that when you click on this, we track the event with Ahoy and you will be able to drop in the same kind of thing anywhere in your JavaScript, so that when the user takes an action, you can call Ahoy to save that result in your database. This is going to be pretty straightforward, we've got in our app/views/products/show.html.erb a little link to where we have id: "add_to_cart", data: { id: @product.id }, so we can query that in our database and find the matching product for it when we're analyzing this stuff, so with that said, we can go and create a

products.js

document.addEventListener("turbolinks:load", function() {
    document.querySelector("#add_to_cart").addEventListener("click", function(e) {
        console.log(e.target.dataset.id);
    });
});

If we refresh this page and click on "Add to Cart", we see that we get the link here, and that is correct, and so then we should also be able to say dataset.id to grab that id attribute, and if we just grab dataset we get all of those options. This may work, we'll see. If we were to grab all of the data items on there, might be nice so that we can pass all of that over to our event directly, so we'll try this out and see if that works, I'm not actually sure. Sow we'll say

products.js

document.addEventListener("turbolinks:load", function() {
    document.querySelector("#add_to_cart").addEventListener("click", function(e) {
        ahoy.track("Add Item to Cart", e.target.dataset);
        });
});

If we click "Add to Cart" now, we get that sent over, and we get the properties set to the DOM string map, but we will see in our database side if that actually parsed correctly. So let's run

rails console
Ahoy::Event.last

You will see here that our properties includes data or the id. The data id of 1, and if we were to go modify this in the future, and we went to products/show/html.erb, maybe we wanted to include some other things like we wanted to include the product name here, and then if we refresh our page and click "Add to Cart", if we look at our last event, this time it's going to include the name of the product at the time, so generally you don't need to pass that information over, it's useful in some tools like Mixpanel or something, where you don't actually have access to the product name, because it's in your database, so just the id's are not as useful and something like Segment or Mixpanel, but in these cases, this can be fine to leave out the name because you can always join the product's table and your database queries and match the id on that. So these are kind of not as necessary, but you're free to add as many of those things as you want, and by simply passing in the data set, all of that is going to be then transferred over to the Ahoy record in your database, and you can analyze those later. So let's talk about analyzing this stuff, and we're going to be using another gem called blazer for that. I mentioned this previously, but this gives you an awesome way to create charts and dashboards and things like that from your database, and it can even cache these and update them every few minutes and all kinds of good stuff, so let's take a look at implementing this to analyze stuff in our database. The blazer gem is pretty straightforward, you just add the gem, you run rails g blazer:install, and migrate your database. So that's what we're going to do, we're going to go to our gemfile, add blazer, then hop into our console and run bundle to install it, and then let's also run blazer:install and rake db:migrate. While those are running, we can also grab this Mount, which is going to give us the "/blazer" url that we can visit in our routes file, and we'll paste that here at the top. There are some things you can do to make sure that this is authenticated, so that only admins can access to it. For example if you're using devise, you can authenticate this using authenticate_user and then also restricting this to admins only, and that's going to authenticate that properly so that only admins can access this url in your routes file. You can also make it authenticated, and that will make sure that it shows up as a 404 in case someone who is not authenticated attempts to check that url out, so they won't even know that it exists. That is how you might do that with devise, there are other options for other authentication systems. You can go check out the README for more instructions on how to do that, and last but not least, there's a couple other options in the README for the installation that you'll want to check out. Namely those are mostly around checks, which are things that you can have run every 5 minutes or every hour or every day, and you can have those things run, and basically do different queries and give you alerts based upon those things, and they'll notify you if anything went wrong. We can now jump into the blazer dashboard and create new queries, and this is awesome because it allows us to create any kind of SQL query we want, so for example, something really simple is we could count the number of users in the user's table and run that, and it's going to give us a thousand users, because that's how many my feeds created, and we can then save this and say "Users Total count" or something like that and we can create that. But what's really cool is that if you want to create a new query and you give it some special parameters, it can actually create graphs for you. So I'm going to paste in a SQL query here that's going to look a little bit complex but this is actually mostly generated for me from the group-date gem, so if you ever write your own queries in the rails console, you can actually copy paste those into here, so you can fiddle with it in the rails console if you're not super familiar with using SQL directly, and then you can take those results from the logs then you can paste them in here and run something like this where we get a count by day of the number of new users for each day. So you can see our registrations here, these are all simulated, but it shows you exactly how many users were created each day and that is really cool because then we could do things for example like set a target line, and this gem will allow you to pass in a number as target inside. So if we were to paste in something like this, and we added that as another select. Let's change this to a more reasonable number like 50, then we can have our target line automatically on the graph and it's going to show us our target to get 50 users per day, doesn't look like we're hitting that but you can keep track of it easily on here just by adding more attributes to your SQL query, and basically, it looks at what you selected out, and then generates a different type of chart based upon the types of columns you gave it. So if you look at the chart section, it will tell you. Well if you give it two or more columns, and the first one is the timestamp, and any of the other ones are numerical, then it will create a line chart, and same thing if you do three columns and you do a timestamp, a string, and then numeric. Then for example, if you wanted to graph male and female, you can have that automatically generate two lines there for you or as many as you have results for that middle column, so that's really cool. Then, there are also a bunch of other options, we have the column chart, scatter charts, maps, and those will depend upon, if you're doing a map: Passing in latitude, longitude into your SELECT query, and make sure you retrieve those back, and this is also going to depend upon Mapbox, so you're going to need that access token as well set up so they can render out the map for you. So this is really really awesome and just gives you a super simple way of getting new user's sign ups per day for example. Getting a graph of this kind of thing and saving that, and then this can also run these in the background, so that it can cache the results, so you don't hit your database constantly. There's some other really cool features like checks, which you can have that for example, if you ran this SQL query and you were looking for any ratings where the user id is null, this can notify you of that and say: Hey, there's a bug in your code, we've created a rating in your database with the null user id and that should never be the case.

It's really awesome because you can then analyze your database and keep track of it and make sure that that stuff is being validated properly, especially when your systems get a lot more complex, there are things that can slip by like this, so it's awesome to be able to create a check really quickly, just write one line of SQL and then have that automated away for you.

You can also do some other neat things, for example you can go into a new dashboard. Let's just make a user's dashboard that would show us the new user's sign ups per day and the total count, and if we hit "Save" we'll be able to see all of that on the same page, so any time that we might want to add another kind of metric for our users, we can throw it on the user's dashboard, and that's going to give us access to viewing all this information just at a glance, and it's as simple as doing that by creating your queries and then just adding them to a dashboard. This can be very similar to something like a Mixpanel or something like that where you might want to be creating these dashboards and keeping an eye on your data and your database, but this is really great because you can keep that all internal to your own database and you don't have to implement any other third-party tools that might cost you money unless you need to, or you get benefit from that, so small applications can benefit from blazer and Ahoy, and they can basically just store all this stuff internally, and then you can monitor your business and your database and all of that stuff automatically. The last piece is that check basically you can create a query, so if your user's total count ever has no results or whatever, you can have this basically monitoring your queries, and then sending out an email if anything goes wrong, so it's really cool that you can basically go in here and do your business metrics and intelligent stuff that you need to, all for free inside your rails app. It's really really easy to set up.

Last but not leas, I want to show you in the ahoy gem's README there is a section on exploring the data in here, and you can see some example queries for example they try to keep track of your keywords for your visits, the country's referring domains, that sort of thing, and if you were using a chart Chartkick and group_by_date you want to put these graphs manually somewhere you can go ahead and use these, but you can also just drop that stuff into Blazer and you don't have to include Chartkick or group-date yourself really, this stuff is going to be available in Blazer as long as you can write that SQL query yourself, but you can include group-date to help you generate those queries, and then just copy paste, put them into blazer and make sure you have the ordering for your graphs right and you'll be good to go. This is really nice and then you can do all kinds of things like if you associate orders to visits, then you can see what orders by city and all that kind of thing that Amazon would certainly be interested in seeing: Who purchases what, in what city, that stuff is going to be really interesting, and you have the ability as well to create funnels and see basically all the user's who did this action and then this one, and then this one and you can see how that shrunk each step of the way and that is your funnel, and you can graph those out as well in bar charts or whatever and easily see that information there too. The cool part about all of these gems is that they can allow you to do all your business metrics internally in your own rails app so you don't have to pay for some of those external tools until you really need to use them, and a lot of small businesses and applications can benefit from this, and you have access to all of your data in the house, and it's really easy to query and analyze and do all of that stuff, and that's generally been the problem with trying to track it in-house, it's not easy to generate graphs or whatever, but blazer and ahoy and group-date and Chartkick make that all extremely easy so you don't have to worry about a lot of that, so I hope you enjoyed this episode, and I will talk to you in the next one. Peace

Transcript written by Miguel