Skip to main content

Background Workers with ActiveJob and Sidekiq Discussion

General • Asked by Chris Oliver

Fallback

Great video, I deployed my app with capistrano every thing went well except sidekiq, the jobs are not running. I tried debugging but sidekiq command is not working on my production system, could you please help me on this.

Thanks,
Srinivas


Fallback

I'd love to see a follow up on deploying to heroku!

Fallback

That's a really good idea! I'll do that. :)

Fallback

I second that! :-)

Fallback

@disqus_Uh95M6FOYo:disqus @disqus_4UMLsxjaNb:disqus Enjoy! https://gorails.com/episode...


Fallback

Thanks Chris, Great vídeo. Quick comment: I think that Activejob is not passing 6 arguments as said (around minute 22) but the argument "6". That should be the user ID. Looking forward to the follow up on deploying to digitalocean!


Fallback

Chris, I'm having an issue with getting this background email to send on Heroku. Posted a SO question about it. Please take look at it if you get a chance. Thanks! Learning a ton from you. http://stackoverflow.com/qu...

Fallback

Did you get this working? You'll need to also make sure your background job runner is a second command added to your Procfile. For example, you'd add sidekiq to your Procfile as a background process and that will make sure these deliver_later emails get sent on Heroku.

Fallback

I'm still working to correct this problem. I'm able to get deliver_later working in development by running redis-server in one terminal, and sidekiq -q mailers in another, but I keep running into "ERROR: ERR max number of clients reached" issues on heroku. It's an absolute beast to configure redis on heroku for a beginner like myself. I'm trying redis to go now and I'm still running into problems.

Fallback

You may just need to bump down your concurrency number for Sidekiq so it's not using so many connections or upgrade your Redis instance so you have one with more connections. Most of the Heroku things like this have some pretty low limitations on concurrency on the lower end.

Fallback

I've actually done both of those things. Lowered concurrency settings in sidekiq.yml and increased redis to go plan to micro and I still run into this issue. Is there any way to see how many clients are attempting to load so I can see what concurrency number I need? This is so ridiculous.

Fallback
Fallback

Got it working. Not 100% clear how. I followed the instructions here: (http://maso.logdown.com/pos... and mixed it with the sidekiq.rb initializer from the link you recommended. Curious as to how this will impact performance when I launch the app. That's a question for another day though.

Fallback

Thanks for your help!


Fallback

@Chris Oliver I followed this episode and it worked perfectly. One thing though I could not find good resource is how to update a job running time. In my case I have a reminder job, that send email to users 24 hours prior to the event. Now lets say the event is cancel/edited how do we update sidekiq to either modified the time or not run the job at all, can you please give me hint?

Thanks.

Fallback

@excid3:disqus can you please address this when you have a chance?

Fallback

Hey Aime,

I'd probably recommend making a cron job for this case instead of scheduling jobs ahead of time. This way you can write a worker that runs every day or so, and checks for events that happen in the next 24 hours and sends out those email reminders.

By doing this you don't have to keep track of which events were canceled or rescheduled, and you're always looking at accurate data when the job runs so you don't have to worry about any of the cancellations or reschedules.

I made an episode on the whenever gem that you can use to build cron jobs that you might like to watch: https://gorails.com/episode...


Fallback

Is there a more "beginner-like" version of background workers available? I'm particularly interested in automating e-mails, csv downloads + notifying the user per web or per mail. There really is nothing fundamental available online.
Thanks!

Fallback

Since ActiveJob is just a wrapper for whatever background processor you use, all you have to do is write your code to use ActiveJob. All your emails already support ActiveJob, so to send them in the background you can just say "UserMailer.notification(user).deliver_later" and the deliver later part will create a job for it.

As for everything else like notifications and csv downloads, you'd just write your code in a job and have your controllers start the job. Nothing too fancy.

Is there anything specific I could cover for you that would help wrap your head around it?


Fallback

Fallback
Hey Chris, do you think it's fine for an application to use the same Redis instance for Sidekiq and as cache store? I've seen a couple of Mike Perham's comments against it from a couple years back, but it just seems like a much simpler setup to only have Redis instead of Redis + Memcached.
Are there any special considerations/configs to take into account in your experience if you plan on using Redis for both?

Thanks!

Fallback
I've been using Redis for Rails cache + Sidekiq for years now. Absolutely no problems. There's probably some rare situation that could cause trouble, but I prefer the simplicity of managing less services in production and it's worked just fine.
Fallback
Thanks for the reply Chris!

Login or create an account to join the conversation.