Skip to main content

34 GoRails Performance - The Techniques I Use

Episode 113 · April 11, 2016

Learn how I design GoRails for speed and performance on all layers of the stack

Turbolinks Performance


Transcripts

Loading...

What's up guys? This episode on GoRails, we're going to talk about the performance on gorails.com. This is a question that came up on the forums recently about, you know, what are the tactics that I use to make gorails.com fast, and by any measure, I don't think GoRails is very fast. It's definitely faster than your average site, but that's definitely not in the 150 ms or less to glass measurement, which is the measurement used for saying: If you're 150 ms or less, generally that's going to appear to human beings instantaneous. GoRails isn't quite that fast but it's pretty fast, so we're going to talk about all of the different things that come into play to make GoRails as fast as it is, lots of improvements to make but we're going to talk about the ones that I've done so far.

First thing is first, let's talk about the server. I use a $20 a month DigitalOcean server with like 2 or 4 Gigs of RAM, I can't remember exactly, and that basically is going to be a foundation for the speed of the server side work. So there's two different things. There is one thing, which is server side work, and number two is the front end work, so the amount of work that has to be done in the browser also very very very much impacts the performance of the website. Definitely the perceived performance, so there's the first request, and that takes some time and you just see a loading thing in your browser, but the rest of that, the painting, the parsing of the CSS, parsing of the JavaScript, downloading of images, there's so much work that goes into the performance of the front end that also needs to be taken care of as well, so server side stuff is one thing, but we'll also talk about front end stuff immediately after that, so from the very foundation I have a DigitalOcean server. It's not very big, but it does support well over 200,000 page views a month and GoRails gets a lot a lot of traffic, so it's doing a very good job of supporting all of that. To do that, I'm using NgineX as the web server, and I'm also using Passenger to serve up the rails application. I've configured that so that I use a correct amount of workers for the RAM and CPU threads that my server might have. So we get the best performance based on the server that I'm actually running on. So all of this exact same stuff applies if you're running on Heroku or on a DigitalOcean or AWS, or Rackspace or any hosting provider. You just need to figure out what's your server running, how many CPUs does it have, how much RAM, and then configure Puma, NgineX, Passenger, all those things based upon that information, and then optimize that accordingly. Take as best advantage as you can of the hardware provided. So going into the rails application layer. The thing that I do the most is I make sure that every single page generates as few queries as possible. Now you might think that's pretty easy, right? So the dashboard like this, you might think that that's pretty simple, but when you're actually building rails applications, it can be pretty hard. So for example, the left side of this page includes all of the recent screencasts, so it displays like five or six recent screencasts, we display 5 or 6 questions from the forum, and then we display the users along with those. That's actually three database queries at a minimum already, so you have to get all of those episodes, we have to get the forum questions to get the user's for that, but because they're logged in, we have to check to see if your records are in the database. The guides are actually a dynamic thing that I built so I can edit those, kind of like a CSM, like a Wordpress type thing, so I can easily edit my guides, so those are dynamic, so that's five. Then, with the notifications, we have to load those, so that's six, and then we have to get the user who generated the notification, so that's seven database queries already just from one single page. It's quite a lot, and it baloons very very quickly once your pages become more and more complicated.

So step number one is to simplify the amount of queries that you do. Keep your pages as simple as you can, that will help improve the performance at the database level. Now, that is even tricky on it's own, because database queries will show up with ActiveRecord saying: Well, this ran and it only took 3 ms. Well, it's not quite the case, because ActiveRecord actually has to take those results, put them in memory, then it has to convert all of that stuff into ActiveRecord objects, then you have to convert that into html and so on. It's a lot more work than the performance numbers of 3 ms of query time show in your logs, so you have to be careful with that, and that means that doing fragment caching comes into play significantly for performance when you're generating html server side. So when you're fragment caching, you want to make sure one of these records like this is cached into rails cache, but you also want to make sure that the rails cache is configured to save either until like redis or memcached, and the reason for that is because with those, you're saving the cached memory, so it's very quick to access, if you use the default cache, actually you have to write to your hard drive SSD, now SSDs are a lot quicker but they´re not still anywhere near as fast to and from memory, so if you can figure your rails cache to point to one of those two external services, you´re going to get a speed improvement for that, which is what I do. Now to take your fragment caching to another level, you can do rush, which is a recent feature of rails that basecamp talks a lot about, and it's really the concept of recent questions on the dashboard here, this is a section of the five most recent questions, we can cache each one of those in a fragment cache, but we can also encompass into one fragment cache, and then bust the larger cache when there's a new question, or one of the old ones gets updated. This is really nifty because it allows you to say: Well we're only going to access the class once for this entire chunk, instead of accessing the cache five times or more for every one of those questions. So that saves a number of network calls between you and redis o you and memcached from five to one, so that's pretty fantastic as well. That is another layer of reducing the number of queries that you're doing in order to get better performance when you're caching even.

Next, after fragment caching, let's talk about assets. Now assets are an important piece, but we're moving from server side optimizations to front end optimizations now. Some of this needs to be configured server side in order to make this work, but any time you're serving up files from your rails application or your own server, you actually need NgineX or whatever webserver you're using that comes before rails, you need that stuff to be able to serve up the files. For example, if I uploaded all of these thumbnails for the videos in the left side, for all the user avatars from my own DigitalOcean server, that would be slow. If all those images came from gorails.com, that would actually cause a bottleneck in the browser when the page begins to load, because once the html is loaded, the image sources are looked at, and if there are on different domains, it will start issuing a request where it does up like 4 requests per domain, so the more requests you actually serve your images from, the better, because your browser can do more stuff in parallel. The other benefit is if you use the CDN, you can actually distribute those images around the world, so you can download files from a server that's closer to you physically in the world, saving a few ms on page load. Now I actually handle the images by the thumbnails for the videos being hosted on Wistia, and I use the avatar's being hosted by Gravatar, so I personally don't have to do any file uploading, no file management, none of that stuff, and I get to keep my site as simple as possible by using images from other services. That saves me a lot of time in development effort, and maintenance if I ever need to change the size of the thumbnails or any of that stuff. This is really handy for me, and definitely recommend you configuring your own file server like Amazon S3, or your own CDN, if you need to upload files yourself. Now that we're squarely into the front end performance. The real key to all of GoRails's performance, at least it's perceived performance in the front end, is turbolinks. Now front end frameworks are really really hot topic right now, everybody wants to learn react because it's cool, and they want to learn Angular and Ember, but I actually think that the secret weapon is turbolinks, and the reason for that is because turbolinks is so small, it's like just a couple files, and only 100 lines or 200 lines of code, it's not large by any means, and it doesn't try to do much, which is the main benefit of turbolinks against anything else. The reason why you want to use a front end library to increase the performance of your site is to make page loads much much faster. Now you used to have to do this with a bunch of AJAX requests, and that was really hard to do with jQuery, and then building all these custom responses server side, and you had to do a lot of duplicate work just to wire everything up, and it wasn't really great, it just wasn't fun to build. So the front end framework started to come out, you see Ember and Angular, and these are coming from a place where like you have dedicated engineers just for the front end, so you're going to go build that stuff, you're going to give us JSON APIs, and you're going to do all this work on top of the server side still needs to be able to render html for that initial page view. Especially because Google and the other web crawlers still aren't very good at indexing JavaScript stuff, so you still generally want to make sure you return html views, and that ends up being a problem, because you have to do twice the amount of work when you use a front end framework.

React comes from a little bit different direction, but it's still being designed to fit in that same place as ember and angular, and turbolinks just goes for a whole different approach, and goes for simplicity above everything else. Now the reason I like the turbolinks stuff so much is because I can go build the site one time, and always return html responses, and then turbolinks gives me more or less free speed improvements on the front end. It's very snappy, everything is being loaded over AJAX because it intercepts all of the links that are clicked, and then I have to only make minor modifications to my JavaScript, in the case of jQuery code, you don't have to do any modifications, aside from making sure to use the jquery-turbolinks library in order to make those compatible, but there are some things like Google Analytics that you want to make sure run every page view, so you have to do some modifications to that code, but for everything else, turbolinks is actually just a very small, easy to maintain, easy to use library to give you performance on the front end. The reason all of these front end frameworks are important is because what they do is save you the time fof downloading, parsing and rendering your CSS and JavaScript every single page view. Because all of that happens the very first time, you were saved from doing that work every single next page view. That's a real benefit. If we click on community here, you'll see that the fonts didn't go back to the default browser fonts, they actually just automatically render with the correct fonts. You didn't see the page clear out and then get repainted. It actually is just: BOOM! New page is done, and the reason for that is because turbolinks, the same with Angular or Ember, they basically take all this stuff, it's already loaded in your browser, and they don't throw it away. They actually take advantage of it and so that makes the browsers appear significantly faster, and it genuinely is significantly faster, but it's all client side time effort that is saved. Nothing server side. The faster that you build server side stuff, the faster you can give it a response, but you still need to make sure that your JavaScript is as lightweight as possible in order to make it that fast. These light frameworks like Ember and Angular, they end up causing you to have several Megabytes of code sometimes, and that's going to be slow because your browser runs on different devices, you run your app on a phone or a tablet or a desktop or a laptop, you're going to get different performance on all of those, and that's kind of frustrating, because you don't actually have any control of where your JavaScript is executed in the browser performance wise. All you have is, well: I hope this gets executed quickly. So the reason turbolinks I think is important is because it's so lightweight, it really offloads as much of the work as possible, server side, and the browser has to do as little work client side, in order to make fast updates. Now the real key between deciding if you should use turbolinks or React or Angular or Ember, is really just like: How big is your team. Most of the time if you're building something with React or Angular, and Ember, you're going to need to spend a whole lot of time in the front end, whereas with turbolinks, you really just need to to the stuff you've always done in jQuery, stuff you're already familiar with, and you can use very very small team to maintain the speed of your application client side, and that's what I really enjoy about this. If you don't have very complicated front end interactions, lik you're not building Slack, and you're not building all those front ends for the channels and the messages and the sidebars that are dynamic, if you're doing that level of interactivity, then you're going to want to use one of those more complex things, but you're also going to need to do the majority of the work on the client side, whereas a site like GoRails is perfect because there's not really that many interactive JavaScript widgets really about the most complicated things, a credit card form, and that's easily handled by jQuery and regular old JavaScript, so turbolinks is probably in my opinion the best solution for the majority of applications out there. You can even pair really really nicely with React, so if you do have a complicated widget, you can actually use turbolinks for the majority of the speed improvements, where you get those mostly free, and then you can stack on React on top of it, and accomplish all of those same goals about building your router on your front end and the server side and do sort of duplicating work, but that's totally personal opinion on front end frameworks, so that is what I use, feel free to use whatever you like, they're all awesome. I think the world is kind of experiencing a lot of different ideas right now, and I don't necessarily that all of these are good ones, because they get away from the reason why we liked rails in the first place that was that it lets one person build an entire start up, an entire business on their own with not that much time investment, which is awesome. So I think we will probably see more turbolinks stuff in the future, just because it's designed for those small teams and the speed of development, so I think it's important, just to touch on that because it is a huge factor in why GoRails is actually fast to browse, and it's almost entirely up to turbolinks client side to make that happen quickly. So yeah, I do my speed improvements server side, but those will just shave off, you know 50 ms on a good day, and then the client side really is improved by turbolinks pretty much on it's own, along with CDN's for the images, so that is the majority of things that make GoRails fast, it comes down to optimizing your server, minimizing your queries, doing proper caching, minimizing the amount of caching queries that you make, assets go on CDNs or separate domains, like your S3 bucket, and then turbolinks as the JavaScript library. That is it, I hope you enjoyed this episode, if you have any questions, want me to dive into any of those pieces deeper, leave those in the comments below and we will tackle those in the next one. Peace.

Transcript written by Miguel

Discussion


Gravatar
Itzik Ben Hutta on

Thanks Chris. The first time my name appears on a video :)

Gravatar
Chris Oliver (169,610 XP) on

Heck yeah man! :D


Gravatar
Petros Kyriakou (30 XP) on

Thanks for episode Chris. But what happens when turbolinks mess up and do not let your javascript code run when you "change" page? because i have had these issues here and there and i had to do something like data-no-turbolink.

Gravatar
Chris Oliver (169,610 XP) on

Hey Pete, this usually happens from one of two things:

1. You're using a third party library (like Segment.io for example) that isn't compatible with Turbolinks events by default. These usually require some tweaking in order to make compatible. It all depends on the library but there are usually ways to do it. You can google for that library + turbolinks and see if anyone else has made it work, or if you can tweak it yourself.

2. Sometimes it can just be a simple setup issue if your JS is running incorrectly. Maybe you need the jQuery.turbolinks adapter, or just to modify your code slightly to allow running on multiple pages.

It all kinda depends on your code and the issues you're experiencing. They can be kinda hard to track down so the easy solution is to turn off turbolinks for those links in the meantime if you can figure it out. I don't think there's anything wrong with that so you can figure it out later.

Gravatar
Petros Kyriakou (30 XP) on

Hey Chris, thanks for getting back to me.

Well mostly the problem is with external libraries to be honest. Guess data-no-turbolink seems easier than trying to find a solution each time. But maybe its high time i do that.

Gravatar
Chris Oliver (169,610 XP) on

I've found that using Segment.io for managing those external libraries helps a lot. It makes that management a lot easier, but I also intentionally use very few third-party Javascript libraries because they can quickly slow down the frontend performance of a site. You might check out Segment and see how it goes.

https://segment.com/docs/li...

Gravatar
Petros Kyriakou (30 XP) on

will do cheers!


Gravatar
Olaoluwa Oluro (10 XP) on

Great insight! Though now I'm really hoping you'll do another video on how to implement and get the best out of Turbolinks. Few resources out there and you did a good job with the marketing :D

Gravatar
Chris Oliver (169,610 XP) on

I definitely will. I also need to learn some around Swift and Android so that I can make some example apps using the Turbolinks adapters. That might mean we'll have a GoRails mobile app at some point. :)

Gravatar
Itzik Ben Hutta on

Why not React Native? It will save you a lot of time. Or you can wait a year or so. Google plans to ditch Java in favor of Swift for Android development.

Gravatar
Chris Oliver (169,610 XP) on

Mostly because I'd rather spend most of my time doing the heavy lifting in Rails rather than JS. If you built your Rails frontend in React already, React native is the way to go. Since I'm using Turbolinks already and because I don't have any complex JS widgets on the frontend to need React, the Turbolinks adapters are the best solution for me. React could just as easily fill the same gaps, just fits well for me.

Gravatar
Itzik Ben Hutta on

We are talking about a mobile app right? How can Rails do the heavy lifting? It will just serve as the API no? So instead of consuming that with two different platforms just use JavaScript for all. Basecamp uses Turbolinks, but there mobile app is also hybrid as far as I know.

Gravatar
Chris Oliver (169,610 XP) on

With Turbolinks on mobile, you get a web view that embeds the Rails site just like you would have in your browser, but you can override link clicks with native code. So all the stuff you see on mobile is just as if you were viewing it in the browser. It's a hybrid app because of the webview, but easily intercepts those things to do native Swift or whatever. Turbolinks would only need the server to return HTML and so you don't really need to build an API.

React Native is somewhat similar in that you're still sharing the same app code with the main website, but you will have to build an API to make React work, and you'll need to do some extra work to serve up the HTML as well.

Gravatar
Itzik Ben Hutta on

I didn't know it's possible to have a mobile app without building an API. That's good to know.

Gravatar
Chris Oliver (169,610 XP) on

Give their readme a look on the new iOS adapter. It might help wrap your head around it a bit. https://github.com/turbolin...

Basically your mobile app just ends up primarily being a WebView (webkit browser full screen) and it let's you set the website. This is similar to things like PhoneGap in the past, except that the code is all just your public website meaning you can update your mobile app at any time by deploying your website again. Pretty slick! I'm not sure if React Native lets you do things like that.

Gravatar
Matias Pan (10 XP) on

https://www.youtube.com/wat...
Here is a video explaining how the guys/girls at basecamp build(the basics of course) their iOS app using turbolinks 5


Gravatar
Jerome . (100 XP) on

Another suggestion is a non-rails issue, but fundamental nonetheless: focus on the database queries themselves. A great suggestion I have held dear over the years is to ask "what are the critical queries an application may have?" (the most frequent one, the most valuable, the most calculation intense...) On that basis, with a knowledge of how databases index, one can often find a data structure that is more efficient than another (conception over optimisation). Large test data sets for different data structures allow to validate design choices. To me, that is the core of the onion...


Gravatar

Gravatar
Thomas Bush (3,100 XP) on

Chris, any chance you could go into detail about how to set up nginx and passenger to properly utilize the resources of your server? I follow your guides when installing so no fine tuning is ever done to my servers. Also would love to see how to set up redis for caching.


Gravatar
Anthony Lee on

Thanks Chris. I would love to see how you make improvements on all the external API calls for getting data from different services.


Gravatar
Francisco Quinones (7,370 XP) on

im in the Video yay lol Chris keep up the good work

Gravatar
Chris Oliver (169,610 XP) on

Whoo! :)


Gravatar
Frank Kumro (10 XP) on

Does your 2GB droplet contain your entire stack (db/cache/nginx/passenger) or did you break out the db/cache?

Gravatar
Chris Oliver (169,610 XP) on

It also contains the DB and cache because neither are that large. I still have 400MB of RAM free right now.


Gravatar
Albert on

I would love to watch a serie , instead of a bunch of randoms episodes, something like laracasts but for rails..... A lot of people will pay for that

Gravatar
Chris Oliver (169,610 XP) on

I've been realizing I need to make that transition soon anyways just because there are way too many screencasts to keep track of at this point. :)

I'll probably be migrating to a bunch of small series over the next few weeks!

Gravatar
Albert on

thanks mate!


Gravatar
Jordano Moscoso on

Question, maybe to clarify some things, if you can, please answer, thanks.
Why did you choose digitalocean over heroku?
PD: Im thinking about changing to digitalocean because in heroku every add-on in production mode is paid, and if you have few your are going to paid 3x times more than having a digitalocean($20). But i not sure if i'll make the right things in digitalocean.

Gravatar
Chris Oliver (169,610 XP) on

I chose DO because I like running my own servers and its significantly cheaper than running on Heroku for personal or small business projects. You'll have to do a lot more work, but it can be fun so long as you make sure you have backups and all that in place.


Login or create an account to join the conversation.