Skip to main content

Following "Scheduling post" episode with background jobs

General • Asked by Christophe Estanol


Following Chris great post on scheduling I am using delayed_job_active_record to schedule quote from my app to be tweeted and shared on FB.

Here is my quote.rb model with the current logic:

      # == Schema Information
      #
      # Table name: quotes
      #
      #  id           :integer          not null, primary key
      #  content      :text
      #  author       :string
      #  created_at   :datetime         not null
      #  updated_at   :datetime         not null
      #  published_at :datetime
      #  status       :string
      #  facebook     :boolean
      #  twitter      :boolean
      #  user_id      :integer
      #  error        :text
      #

      class Quote < ActiveRecord::Base

        belongs_to :user
        validates :content, presence: true
        validates :author, presence: true
        # validation using validates_timeliness gem
        # validates_datetime :published_at, :on => :create, :on_or_after => Time.zone.now
        # validates_datetime :published_at, :on => :update, :on_or_after => Time.zone.now
        after_save :schedule

        scope :draft,     ->{ where(status: "Draft") }
        scope :published, ->{ where(status: "Published") }
        scope :scheduled, ->{ where(status: "Scheduled") }

        before_validation :clean_up_status

        def clean_up_status
          self.published_at = case status
                              when "Draft"
                                nil
                              when "Published"
                                Time.zone.now
                              else
                                published_at
                              end
          true
        end

        def schedule
          puts "!!!!!!!!!!!!!!!#{self.status}!!!!!!!!!!!!!!!"
          if self.status == "Scheduled"
            begin
                ScheduleJob.set(wait_until: published_at).perform_later(self)
            rescue Exception => e
                self.update_attributes(status: "Scheduling-error", error: e.message)
            end
          else
            publish
          end
        end

        def publish
          unless self.status == "Draft"
            begin
                if self.facebook
                    to_facebook
                end
                if self.twitter
                    to_twitter
                end
              self.update_attributes(status: "Published")
            rescue Exception => e
              self.update_attributes(status: "Publishing-error", error: e.message)
            end
          end
        end

        def to_twitter
          client = Twitter::REST::Client.new do |config|
            config.consumer_key        = ENV['TWITTER_KEY']
            config.consumer_secret     = ENV['TWITTER_SECRET']
            config.access_token        = self.user.twitter.oauth_token
            config.access_token_secret = self.user.twitter.secret
          end
          client.update(self.content)
        end

        def to_facebook
          graph = Koala::Facebook::API.new(self.user.facebook.oauth_token)
          graph.put_connections("me", "feed", message: self.content)
        end
      end

I have tried before_create :schedule and before_update :schedule before before_save and none of them seem to work.

My jobs/schedule_job.rb:

    class ScheduleJob < ActiveJob::Base
     queue_as :default

     def perform(quote)
       quote.publish
     end
    end

With basic config: initializers/delayed_job_config.rb that have also added after the error:

    Delayed::Worker.destroy_failed_jobs = false
    Delayed::Worker.max_attempts = 1

Whenever I schedule a quote delayed_job enter in an infinite loop, eventually publish the quote and keep on runnig in circle.

Here is the log:

    rake jobs:work
    [Worker(host:Christophes-MacBook-Pro.local pid:8962)] Starting job worker
    [Worker(host:Christophes-MacBook-Pro.local pid:8962)] Job ActiveJob::QueueAdapters::DelayedJobAdapter::JobWrapper (id=7) RUNNING
    !!!!!!!!!!!!!!!Publishing-error!!!!!!!!!!!!!!!
    !!!!!!!!!!!!!!!Publishing-error!!!!!!!!!!!!!!!
    !!!!!!!!!!!!!!!Publishing-error!!!!!!!!!!!!!!!
    !!!!!!!!!!!!!!!Publishing-error!!!!!!!!!!!!!!!
    !!!!!!!!!!!!!!!Publishing-error!!!!!!!!!!!!!!!
    !!!!!!!!!!!!!!!Publishing-error!!!!!!!!!!!!!!!
    !!!!!!!!!!!!!!!Publishing-error!!!!!!!!!!!!!!!
    !!!!!!!!!!!!!!!Publishing-error!!!!!!!!!!!!!!!
    !!!!!!!!!!!!!!!Publishing-error!!!!!!!!!!!!!!!
    !!!!!!!!!!!!!!!Publishing-error!!!!!!!!!!!!!!!
    !!!!!!!!!!!!!!!Publishing-error!!!!!!!!!!!!!!!
    !!!!!!!!!!!!!!!Publishing-error!!!!!!!!!!!!!!!
    !!!!!!!!!!!!!!!Publishing-error!!!!!!!!!!!!!!!
    !!!!!!!!!!!!!!!Publishing-error!!!!!!!!!!!!!!!
    !!!!!!!!!!!!!!!Publishing-error!!!!!!!!!!!!!!!
    !!!!!!!!!!!!!!!Publishing-error!!!!!!!!!!!!!!!
    !!!!!!!!!!!!!!!Publishing-error!!!!!!!!!!!!!!!
    !!!!!!!!!!!!!!!Published!!!!!!!!!!!!!!!
    !!!!!!!!!!!!!!!Publishing-error!!!!!!!!!!!!!!!
    !!!!!!!!!!!!!!!Publishing-error!!!!!!!!!!!!!!!
    !!!!!!!!!!!!!!!Publishing-error!!!!!!!!!!!!!!!
    !!!!!!!!!!!!!!!Publishing-error!!!!!!!!!!!!!!!
    !!!!!!!!!!!!!!!Publishing-error!!!!!!!!!!!!!!!
    !!!!!!!!!!!!!!!Publishing-error!!!!!!!!!!!!!!!
    !!!!!!!!!!!!!!!Publishing-error!!!!!!!!!!!!!!!
    !!!!!!!!!!!!!!!Publishing-error!!!!!!!!!!!!!!!
    !!!!!!!!!!!!!!!Publishing-error!!!!!!!!!!!!!!!
    !!!!!!!!!!!!!!!Publishing-error!!!!!!!!!!!!!!!
    !!!!!!!!!!!!!!!Publishing-error!!!!!!!!!!!!!!!
    !!!!!!!!!!!!!!!Publishing-error!!!!!!!!!!!!!!!
    !!!!!!!!!!!!!!!Publishing-error!!!!!!!!!!!!!!!
    !!!!!!!!!!!!!!!Publishing-error!!!!!!!!!!!!!!!
    !!!!!!!!!!!!!!!Publishing-error!!!!!!!!!!!!!!!
    !!!!!!!!!!!!!!!Publishing-error!!!!!!!!!!!!!!!
    !!!!!!!!!!!!!!!Publishing-error!!!!!!!!!!!!!!!
    !!!!!!!!!!!!!!!Publishing-error!!!!!!!!!!!!!!!

Any idea what could I be missing? I am using Puma for dev and prod server.
I also created a question in stackoverflow. http://stackoverflow.com/questions/35206588/delayed-job-infinite-loop
Whenever I manage to make it work I will share a gist so it can be used.

Gravatar Chris Oliver commented on : Mod Staff

Yo! What's happening is basically:

  • Create the new quote
  • After_save fires, attempts to schedule.
  • If that fails, you update the record, causing the after save to run again
  • If it doesn't fail, when the record does get published, you update the record, causing the after_save callback to fire once more
  • This is going to obviously throw an error on Twitter, etc for duplicate posts at some point causing your code to update the record again that there was an error, fire the after_save callback and then attempt to post again.

My suggestion here would be actually to remove the after_save callback and do this explicitly in your controller. You'll save yourself some trouble doing that. You can also set it up so that if it fails to publish, you can note the error, and schedule a job to attempt it again in the future if you would like.

So a couple things:

  1. You're calling after_save which means that if you call update_attributes, it's going to try to publish again. You'll get an infinite loop of this publishing attempt basically. Probably not exactly what you want. This is the primary issue with callbacks.

  2. What's the error message that you're receiving? It looks like you save this to the database, so I'm curious what's causing it to trip up.

Thanks Chris for your suggestion.
You were absolutely right. I removed the callbacks and did it explicitly in the controller and it worked.
I have created a gist with the code I used.
Hopefully someone will recycle it, use it, make it better.
https://gist.github.com/ChrisEstanol/acb0c2883d995d74c498

Gravatar Chris Oliver commented on : Mod Staff

Good work man! It looks like it turned out pretty clean and simple. :D

@Christophe Estanol

Have you manage to solve this issue with the turbolinks caching has mentioned? I have intalled the jquery-turbolinks gem and restart the server as per the comments of another user but the isse still occurs

Login or create an account to join the conversation.