Solutions for high memory usage in rails development or production(heroku)
So i built my web app that allows users to chat with eachother, see who's online, and send notifications all using actioncable and it's using up A LOT of memory it seems. Correct me if i'm wrong, but memory can highly affect your websites speed, cpu usage, ect. Right?
http://image.prntscr.com/image/9cf01fb6787c42cf9a2859f9c398fc43.png
So is there any types of strategies one can do to lower the memory usage down that I can look into? Possibly any videos that Chris has done to improve memory performance?
Does using front end frameworks like Emberjs even help with this type of problem?
Really it doesn't look that bad - you're peaking at around 1.2 gigs of ram. I have apps that can easily eat up many gigs during certain batch operations, and it's fine for those scenarios.
I think what you need to determine is how many visitors you had when you hit the 1.2 gig spike, and is that acceptable use for that load? If you only had 2 visitors, then yeah, that's probably an issue - if you had a couple million - that would probably be incredible to have only reached 1.2 gigs :)
From there you can start analyzing your app. In your main controllers, do you limit your queries where viable? For example, if you just wanted to display users emails in a view then it's much more memory friendly to do
@user_emails = User.all.pluck(:email)
###
<% @user_emails.each do |email| %>
<%= email %>
<% end %>
vs
@user_emails = User.all
###
<% @user_emails.each do |user| %>
<%= user.email %>
<% end %>
This is because when you don't limit your return, you're loading all the information for that object into ram, instead of just the emails. So if you can imagine a table that has a lot of records and/or a lot of columns, that memory footprint can grow quick, and from just one instance!
Check out Optimizing Rails for Memory Usage - I haven't got to read it all but from what I have, it's good stuff. Should help get you headed in the right direction.
Once you've done some basic analysis, you should be able to tell pretty quick if it's really an issue with your app or just time to upgrade! :)
Thanks for all that info, Jacob! I'll get on that article reading.
Also, it seems I got that high memory usage with only 3-5 people on the site (my team testing it), that's what worrisome for me.
I DID use actioncable chat during that time, so I think that's the reason why. Would you happen to know if i'm suppose to run a procfile for my puma to start working on heroku production as shown in Chris's video: https://gorails.com/episodes/deploying-to-heroku-with-puma?autoplay=1 ? Because I didn't, and my chat is still working. but i wonder if i do what he did, if it'll lower my memory or redis usage.
EDIT: This worries me a lot: http://prntscr.com/dzimys
You're welcome!
And yeah, with only 3-5 then that's not a good result, you'd know it if your app needed that much ram for something special with that few users.
I can't speak on actioncable or heroku setup, I haven't done much with either of them, but I'm sure someone else will chime in soon! However, one thing I'd probably do, if it's feasible, is to disable actioncable temporarily and see if it really is linked to actioncable or not. If you check that link (disable actioncable) - just comment the lines where he says remove - should accomplish the same thing for testing purposes. Then look at your memory usage to see if it's directly related to actioncable or something else.
I think Jacob has given some great advice, especially with regards to using pluck
Nothing aginst Heroku, but due to the way they bill, I would be concerned of having spike charges due to things like this.
I don't think you get charged extra from Heroku for having spikes and exceeding your memory quota. I think when you exceed your memory quota you start to use swap memory, which is less than desirable, but I don't think you are charged extra. https://devcenter.heroku.com/articles/ruby-memory-use
Ah - I'll remove my statement to avoid misleading. I remember reading some articles awhile back related to spikes in charges and ever since then I just never looked back at them.
Thanks for the clarification Mark!
@everyone thanks for the responses! I haven't found a 100% solid solution, yet, but i'm working on it!
I hope Chris keeps on coming out with more performance videos!
Late to the party here, but I remember reading somewhere that there's an open issue with ActionCable consuming excessive memory and leaking. This was on 5.0.0.1 but I'm not sure if it's applicable to your situation. Just figured I'd chime in with what I heard around the campfire.
@masud How is your memory profile looking now that you've switched to DO? Also what version of Rails are you on? I'm on DO and Rails 5.0.0.1 and am experiencing memory bloat in a mulitenant app with actioncable. I've isolated this to cable so far by disabling it but was wondering if you're on a newer version of Rails where the memory leak has been patched. I'm on a 1GB DO box and I start swapping/paging at about 50 concurrenct AC websockets.
Just curious if you were seeing similar degredation.
@shakeycode
I'm on rails 5.0 and my REDIS is at least not giving me the Sessions full error anymore, the 20limit on channels from heroku redis is complete bullshit tbh. AFter that, it doesn't seem like I got any errors.
HOWEVER, my rails app on heroku is still giving me a crap load of high Memory usage. I'm 100% convinced it's my fault though. I upgraded my heroku to 2x dyno with 1gb of memory, and my app sometimes hits 214% with 2.2gb.
I'm working on a solution, but not a whole lot of luck. It's hard to even determine what the problem is.
@shakeycode
The high memory is now fixed. I needed to optimize my cache.
Go check your rails logs to see if you see a lot of
CACHE (0.0ms) .....
loading when you go to a specific page or conversation. This was my issue and soon as I fixed it, my CPU+memory usage dropped dramatically and my app got faster.
This was my issue and soon as I fixed it
Are you able to elaborate on how you fixed it so myself and others can benefit from your solution?
@masud I'm not seeing the same caching thing that you did. What did you change to fix your issue so that others know. I think my issue is a leaky ActionCable implementation from rails 5.0.0.1
@shakeycode & @Mark Radford , I did eager loading on all my conversations and messages.
This is a descent guide for it: https://code.tutsplus.com/articles/improving-the-performance-of-your-rails-app-with-eager-loading--cms-25018
You would need to set your eager loading up based on your settings, but it's pretty straightforward tbh.
It's nothing you do on your connection.rb, it has to do with your controller. Basically it's creating TOO many queries.
@masud I think I may have misunderstood I thought you were talking about optimizing your cache, not dealing with N+1 queries. Either way, eager loading is the way to go. Thanks for posting this up.
Chris's new video might be helpful for you: https://gorails.com/episodes/activerecord-explain?autoplay=1