Background Workers with ActiveJob and Sidekiq Discussion
Great video, I deployed my app with capistrano every thing went well except sidekiq, the jobs are not running. I tried debugging but sidekiq command is not working on my production system, could you please help me on this.
I'd love to see a follow up on deploying to heroku!
Thanks Chris, Great vídeo. Quick comment: I think that Activejob is not passing 6 arguments as said (around minute 22) but the argument "6". That should be the user ID. Looking forward to the follow up on deploying to digitalocean!
Chris, I'm having an issue with getting this background email to send on Heroku. Posted a SO question about it. Please take look at it if you get a chance. Thanks! Learning a ton from you. http://stackoverflow.com/qu...
Did you get this working? You'll need to also make sure your background job runner is a second command added to your Procfile. For example, you'd add sidekiq to your Procfile as a background process and that will make sure these deliver_later emails get sent on Heroku.
I'm still working to correct this problem. I'm able to get deliver_later working in development by running redis-server in one terminal, and sidekiq -q mailers in another, but I keep running into "ERROR: ERR max number of clients reached" issues on heroku. It's an absolute beast to configure redis on heroku for a beginner like myself. I'm trying redis to go now and I'm still running into problems.
You may just need to bump down your concurrency number for Sidekiq so it's not using so many connections or upgrade your Redis instance so you have one with more connections. Most of the Heroku things like this have some pretty low limitations on concurrency on the lower end.
I've actually done both of those things. Lowered concurrency settings in sidekiq.yml and increased redis to go plan to micro and I still run into this issue. Is there any way to see how many clients are attempting to load so I can see what concurrency number I need? This is so ridiculous.
Got it working. Not 100% clear how. I followed the instructions here: (http://maso.logdown.com/pos... and mixed it with the sidekiq.rb initializer from the link you recommended. Curious as to how this will impact performance when I launch the app. That's a question for another day though.
@Chris Oliver I followed this episode and it worked perfectly. One thing though I could not find good resource is how to update a job running time. In my case I have a reminder job, that send email to users 24 hours prior to the event. Now lets say the event is cancel/edited how do we update sidekiq to either modified the time or not run the job at all, can you please give me hint?
I'd probably recommend making a cron job for this case instead of scheduling jobs ahead of time. This way you can write a worker that runs every day or so, and checks for events that happen in the next 24 hours and sends out those email reminders.
By doing this you don't have to keep track of which events were canceled or rescheduled, and you're always looking at accurate data when the job runs so you don't have to worry about any of the cancellations or reschedules.
I made an episode on the whenever gem that you can use to build cron jobs that you might like to watch: https://gorails.com/episode...
Is there a more "beginner-like" version of background workers available? I'm particularly interested in automating e-mails, csv downloads + notifying the user per web or per mail. There really is nothing fundamental available online.
Since ActiveJob is just a wrapper for whatever background processor you use, all you have to do is write your code to use ActiveJob. All your emails already support ActiveJob, so to send them in the background you can just say "UserMailer.notification(user).deliver_later" and the deliver later part will create a job for it.
As for everything else like notifications and csv downloads, you'd just write your code in a job and have your controllers start the job. Nothing too fancy.
Is there anything specific I could cover for you that would help wrap your head around it?
Are there any special considerations/configs to take into account in your experience if you plan on using Redis for both?