Skip to main content

Background Workers with ActiveJob and Sidekiq - GoRails

General • Asked by Chris Oliver

Chris,

I have a website that has a large dataset that can be exported via csv. I followed your video on how to export a csv (solid video) and it was working great until my dataset got too big. The problem is Heroku times out because it takes too long. I'm currently trying to handle this by doing it through a background job, but I'm not exactly sure the best way to do that. I'm sure I need to do something with "send_data" and rather than having it directly send I need to have it send through a job.. but like I said I'm lost ha. Any help would be great.

Thanks in advance.

In that case, you'll typically build the CSV and then store it on S3 in a background job, and then provide the user with a link to download it. You can send them an email once it's uploaded or you can refresh the page periodically with AJAX until the download URL is saved to a record. Really sorta depends, but when things get outside that time window, you'll see lots of apps taking this approach.

Awesome. Makes sense. Thanks Chris.


Login or Create An Account to join the conversation.

Subscribe to the newsletter

Join 24,647+ developers who get early access to new screencasts, articles, guides, updates, and more.

    By clicking this button, you agree to the GoRails Terms of Service and Privacy Policy.

    More of a social being? We're also on Twitter and YouTube.