How do I handle the validation of associated objects?

I have a blog applications, in which user has_many blogs, blog has_many notes written by the user. And we have a note_relations(mapped to NoteRelation) table to keep the relation of two notes, if they are connected. And our validation function requires the note_from and note_to has the same blog_id

class NoteRelation < ActiveRecord::Base 
has_one note_from,  :class_name => 'Note'
has_one note_to,  :class_name => 'Note'
def validate_note_relation
   unless note_from.blog_id == note_to.blog_id
     errors.add :note_to_id, :not_same_blog
  end
end

Also, we allow a user to change the blog_id of a note. In this case, if we change one of the note_from's blog_id without changing the note_to's blog_id, then the issue_relation data would be invalid.

How should we handle the validation so that no data will be invalid? Or how can we define the right behavior.