Here's what I ended up doing, for posterity, and also because I'm gonna have to write this documentation again but for real:

Context

Writing a custom API wrapper that will be used in more than one app. Can't explicitly discuss the code itself due to project restrictions, so here's a contrived/structurally similar situation:

We're using the "LemonConfidential" API. It's a custom, private api for one company, and they needs some Rails apps build on top to serve all their Lemon needs.

The API serves up functionality about a couple major resources:

• Farms: the variety of farms the lemons can be at, these records contain name, address, contact info, etc
• Lemons: information about each individual lemon.
• LemonDownload: choose some lemons to download.

All the responses are JSON, and except for creating LemonDownloads, mostly read-only.

Goal

Here's how we'd like to be able to ultimately use our client:

client = LemonConfidential::Client.new(current_user.api_key)

farms = client.farms.all
search = client.lemons.search
lemon = client.lemons.get(lemon_id)

Building it

I'll be using HTTParty. Since we know we're gonna wanna pull this out of just one app eventually, best to keep everything in a namespace. Furthermore, since the API is confidential, every user has their own api key tied to their info so all the requests can be tracked and scoped properly. We'll need to throw that in the header. So to start, we know we're going to need this:

module LemonConfidential
  class Client

    HOSTNAME = Rails.application.secrets.lemons_confidential_hostname
    API_URL = "https://#{HOSTNAME}/api/"

    def initialize(api_key=nil)
      @api_key = api_key
    end

    def headers
      { "Authorization" => @api_key, "Content-Type" => "application/json" }
    end
  end
end

Now, what I was stuck on, is: how do we pass info from this client to different classes for each resource to actually go and make the requests? Inspired by the DO api wrapper linked above:

module LemonConfidential
  class Client

    HOSTNAME = Rails.application.secrets.lemons_confidential_hostname
    API_URL = "https://#{HOSTNAME}/api"

    def initialize(api_key=nil)
      @api_key = api_key
    end

    def headers
      { "Authorization" => @api_key, "Content-Type" => "application/json" }
    end

    def self.resources
      {
        lemons: LemonConfidential::Lemon,
        farms: LemonConfidential::Farm,
        lemon_downloads: LemonConfidential::LemonDownload,
      }
    end

    def method_missing(name, *args, &block)
      if self.class.resources.keys.include?(name)
        resources[name] ||= self.class.resources[name].new(API_URL, headers)
        resources[name]
      else
        super
      end
    end

    def resources
      @resources ||= {}
    end
  end
end

So when we call

client = LemonConfidential::Client.new(current_user.api_key)
search = client.lemons.search("sour")

When we call lemons on our client, it will hit our method_missing and run it against the hash. There, it will find LemonConfidential::Lemon

So, we'll need to build out that class to receive the info, get our hash ready to throw the api_key in the headers of our request.

class LemonConfidential::Lemon
  include HTTParty

  def initialize(api_url, headers)
    @api_url = api_url
    @headers = headers
  end

  def set_options
    options = {}
    options.merge! headers: @headers
    options
  end
end

Of course, now, we have to actually process the request:

class LemonConfidential::Lemon
  include HTTParty

  def initialize(api_url, headers)
    @api_url = api_url
    @headers = headers
  end

  def search(query)
    options = set_options
    options.merge! { query: query } 
    # httpary will append query parameters to the url. {query: {page: 2, q: "sour"} } will produce: ?page=2&q=sour

    # now, time to make the request:
    url = [@api_url, "/lemons/search"]
    self.class.get(url, options)
  end

  def set_options
    options = {}
    options.merge! { headers: @headers }
    options
  end
end

But, as we get through building the resources, it becomes apparent we're duplicating quite a bit:

class LemonConfidential::Farm
  include HTTParty

  def initialize(api_url, headers)
    @api_url = api_url
    @headers = headers
  end

  def search(query)
    options = set_options
    options.merge! { query: query } 

    url = [@api_url, "/farms/search"]
    self.class.get(url, options)
  end

  def set_options
    options = {}
    options.merge! { headers: @headers }
    options
  end
end

So, there's an underlying idea in here. What these all have in common is that they're a "Resource" of the api, and will need similar credentials/setups to make requests. How about we pull that out like:

class LemonConfidential::Resource
  include HTTParty

  def initialize(api_url, headers)
    @api_url = api_url
    @headers = headers
  end

  def set_options
    options = {}
    options.merge! { headers: @headers }
    options
  end
end

and then we can just inherit from that and have our respective Lemon and Farm classes do what they do best - define endpoints, get the right parameters, and make the request:

class LemonConfidential::Farm < LemonConfidential::Resource
  def all
    options = set_options

    url = [@api_url, "/farms"]
    self.class.get(url, options)
  end
  # ex: client.farms.all

  def get(farm_id)
    options = set_options

    url = [@api_url, "/farms/", farm_id]
    self.class.get(url, options)
  end
  # ex: client.farms.get(3)
end

This seems pretty clean, and isolates a lot of configuration to one place that would make changes easy. Of course, adding this layer of abstraction over the raw request-making means if you change your endpoints, you gotta change your methods. But I think in this case, its a pretty good tradeoff.

This feels a lot like a "BattleReport" I used to write about Starcraft games. If you took the time to look in this badly-worded question, I hope you got something out of it!

Little update - this is pretty close to what I'm going for: https://github.com/digitalocean/droplet_kit

and it looks like they deal with it via method_missing

module DropletKit
  class Client

    def self.resources
      {
        actions: ActionResource,
        droplets: DropletResource,
        domains: DomainResource,
        domain_records: DomainRecordResource,
        droplet_actions: DropletActionResource,
        images: ImageResource,
        image_actions: ImageActionResource,
        regions: RegionResource,
        sizes: SizeResource,
        ssh_keys: SSHKeyResource,
        account: AccountResource,
        floating_ips: FloatingIpResource,
        floating_ip_actions: FloatingIpActionResource,
        tags: TagResource
      }
    end

    def method_missing(name, *args, &block)
      if self.class.resources.keys.include?(name)
        resources[name] ||= self.class.resources[name].new(connection: connection)
        resources[name]
      else
        super
      end
    end

    def resources
      @resources ||= {}
    end
  end
end

the confusing part in here for me is the Connection stuff, but that looks like it relates to Faraday...

edit/update: Aha! Yeah. I can make the client responsible for all the info needed to send off the requests. I'll need to figure out how that works with HTTParty, but this is exactly the kind of behavior I'd like to achieve so I consider this question answered, unless anyone has any other thoughts on how to set this up.

I'm building an API wrapper and struggling with how to set it up the way I'd like. Here's an example of what I'd ultimately like this to look like:

ServiceName::Client.new(api_key).record_type.all
# or
ServiceName::Client.new(api_key).record_type.download(args)

Gonna have to swap out that api key a lot based on the user, so can't take care of that by configuring in an initializer or something.

Figure, given the domain, it would be really intuitive just to kind of direct people down specific endpoints as methods like that. Where I'm stuck is, how do I set up the ruby?

module ServiceName # everything will be in this namespace, but gonna ignore it from here on
  class Client
    def initialize(api_key)
       @api_key = api_key
    end
  end
end

# Now...I'd like to be able to call it like this:

ServiceName::Client.new(api_key).private_records.search("query")

# So I'm not sure how this might look:

class Client
  def initialize(api_key)
     @api_key = api_key
  end

  def private_records
    # ?
  end  
end

I could do this, just passing that down through various new classes:

class PrivateRecord
  def initialize(api_client)
    @api_client = api_client
  end

  def search(q)
    # fire off http request
  end
end

class Client
  def private_records
    PrivateRecord.new(api_client)
  end
end

but I'm generally wondering if there's a way I could just have this Client class and then pull in modules intelligently so I just initialize necessary state like the api key one time in one place.

The link in this thread of my thinking stops at something like this:

class Client
  def private_records
    # Refer to an already-included module?
    PrivateRecord
  end
end

# now I can call Client.new(api_key).private_records.search("query"), but how do I also pass in the @api_key ?

# do I have to veer into something like this:

class Client
  def private_records(method, *args)
    PrivateRecord.send(method, api_key, *args)
  end
end

module PrivateRecord
  def search(api_key, *args)
    # do the request
  end
end

Or is there a simpler way? I'm not really sure how to phrase the question, so if you know of any examples of this (esp in the rails codebase) would be mucho appreciated too.

Couple questions and suggestions that might help speed up you getting an answer:

you can wrap code in triple-backticks to make it nice and readable. and you can also specify what language so it highlights syntax nicely. simply do

'''ruby

code here

'''

except instead of triple quotes, use the backtick (like when you press tilde without shift): `

def self.to_csv
  attributes = %w{id product_group quantity product price setup discount order_id}
  CSV.generate(headers: true) do |csv|
    csv << attributes
    all.each do |sale|
      csv << sale.attributes.values_at(*attributes)
    end
   end
end

Second thing is that, your question seems to me more like an excel question than a ruby or rails one. Totally possible it might be super easy to export both csv's in one excel file, but I'm unfamiliar with what that would actually look like. Not at all sure what kind of output you desire so not really sure how to begin helping.

Do you mean than when you open up the excel file, there are two sheets in there with the association already linked up? That quickly veers into, well what spreadsheet programming are you using, how do they store their data (XML?), is there a gem? Perhaps this gem could be helpful to you: https://github.com/zdavatz/spreadsheet

The final thing I'd have to say here is that, I've done quite a bit of ruby/rails/spreadsheet related things. I find they tend to be extremely error-prone and time consuming the more spreadsheet-like things you begin doing in your code. If setting up the file the way you like it is a matter of copy + paste and a couple clicks to set that relationship up, its usually way more efficient just to do that part manually - especially if you don't do it too often.

Could you give some examples of how your data's indexed and the actual terms you're searching with that don't seem to be working, alongside your queries?

One random guess is that maybe you're expecting partial matches, as in "some" to return an article that has the word "sometime" in its title. SearchKick can do that, but you've gotta explicitly tell it to:

class Article < ActiveRecord::Base
  searchkick word_start: [:title]
end

Reindex, then

@search = Article.search(query, where:{status: "Published"}, operator: "or", suggest: true, fields: [:title], match: :word_start)

Not sure how the order and nesting of your arguments might affect your queries in this situation, so maybe worth experimenting with that until you get the behavior you want.

Yeah. So there were several errors, some app-specific that others wouldn't have to deal with, but here's the one that probably matters:

gem 'capistrano', '~> 3.1.0' -> gem 'capistrano', '~> 3.4'

gem 'capistrano-bundler', '~> 1.1.2' -> gem 'capistrano-bundler', '~> 1.1', '>= 1.1.4 '

# Now the issue was, on deploy, a variety of gems weren't being detected/used properly. In my case, 
# it was jwt-something. To resolve this:

delete Gemfile.lock
bundle install

# or 

bundle update

# then run your deploy and you should be ready to roll!

# bundle exec cap staging deploy:migrate didn't seem to be working, had to go run those manually

That worked! Thanks. Now I'm bumping into a different error, but I'll try and handle it from here.

This project is on 4.2.0, but I'll give it a shot

[there was an error on original submission of this post, so I did it again and duplicated it. this one can be deleted!]

Please see this post

Hey, debugging a deployment script I didn't set up and running into an error that I'm not making any headway with via StackOverflow or general Googling.

Here's what it is:

bundle exec cap staging deploy
/Users/chris/.rbenv/versions/2.2.4/lib/ruby/gems/2.2.0/gems/capistrano-3.1.0/lib/capistrano/i18n.rb:4: warning: duplicated key at line 6 ignored: :starting
cap aborted!
NoMethodError: undefined method `on' for main:Object
/Users/chris/.rbenv/versions/2.2.4/bin/cap:23:in `load'
/Users/chris/.rbenv/versions/2.2.4/bin/cap:23:in `<main>'
Tasks: TOP => rbenv:validate
(See full trace by running task with --trace)

here's a version with trace:

bundle exec cap staging deploy --trace
/Users/chris/.rbenv/versions/2.2.4/lib/ruby/gems/2.2.0/gems/capistrano-3.1.0/lib/capistrano/i18n.rb:4: warning: duplicated key at line 6 ignored: :starting
** Invoke staging (first_time)
** Execute staging
** Invoke load:defaults (first_time)
** Execute load:defaults
** Invoke bundler:map_bins (first_time)
** Execute bundler:map_bins
** Invoke deploy:set_rails_env (first_time)
** Execute deploy:set_rails_env
** Invoke deploy:set_linked_dirs (first_time)
** Execute deploy:set_linked_dirs
** Invoke deploy:set_rails_env
** Invoke rbenv:validate (first_time)
** Execute rbenv:validate
cap aborted!
NoMethodError: undefined method `on' for main:Object
/Users/chris/.rbenv/versions/2.2.4/lib/ruby/gems/2.2.0/bundler/gems/rbenv-e056efc0c361/lib/capistrano/tasks/rbenv.rake:3:in `block (2 levels) in <top (required)>'
/Users/chris/.rbenv/versions/2.2.4/lib/ruby/gems/2.2.0/gems/rake-10.5.0/lib/rake/task.rb:240:in `call'
/Users/chris/.rbenv/versions/2.2.4/lib/ruby/gems/2.2.0/gems/rake-10.5.0/lib/rake/task.rb:240:in `block in execute'
/Users/chris/.rbenv/versions/2.2.4/lib/ruby/gems/2.2.0/gems/rake-10.5.0/lib/rake/task.rb:235:in `each'
/Users/chris/.rbenv/versions/2.2.4/lib/ruby/gems/2.2.0/gems/rake-10.5.0/lib/rake/task.rb:235:in `execute'
/Users/chris/.rbenv/versions/2.2.4/lib/ruby/gems/2.2.0/gems/rake-10.5.0/lib/rake/task.rb:179:in `block in invoke_with_call_chain'
/Users/chris/.rbenv/versions/2.2.4/lib/ruby/2.2.0/monitor.rb:211:in `mon_synchronize'
/Users/chris/.rbenv/versions/2.2.4/lib/ruby/gems/2.2.0/gems/rake-10.5.0/lib/rake/task.rb:172:in `invoke_with_call_chain'
/Users/chris/.rbenv/versions/2.2.4/lib/ruby/gems/2.2.0/gems/rake-10.5.0/lib/rake/task.rb:165:in `invoke'
/Users/chris/.rbenv/versions/2.2.4/lib/ruby/gems/2.2.0/gems/capistrano-3.1.0/lib/capistrano/dsl/task_enhancements.rb:12:in `block in after'
/Users/chris/.rbenv/versions/2.2.4/lib/ruby/gems/2.2.0/gems/rake-10.5.0/lib/rake/task.rb:240:in `call'
/Users/chris/.rbenv/versions/2.2.4/lib/ruby/gems/2.2.0/gems/rake-10.5.0/lib/rake/task.rb:240:in `block in execute'
/Users/chris/.rbenv/versions/2.2.4/lib/ruby/gems/2.2.0/gems/rake-10.5.0/lib/rake/task.rb:235:in `each'
/Users/chris/.rbenv/versions/2.2.4/lib/ruby/gems/2.2.0/gems/rake-10.5.0/lib/rake/task.rb:235:in `execute'
/Users/chris/.rbenv/versions/2.2.4/lib/ruby/gems/2.2.0/gems/rake-10.5.0/lib/rake/task.rb:179:in `block in invoke_with_call_chain'
/Users/chris/.rbenv/versions/2.2.4/lib/ruby/2.2.0/monitor.rb:211:in `mon_synchronize'
/Users/chris/.rbenv/versions/2.2.4/lib/ruby/gems/2.2.0/gems/rake-10.5.0/lib/rake/task.rb:172:in `invoke_with_call_chain'
/Users/chris/.rbenv/versions/2.2.4/lib/ruby/gems/2.2.0/gems/rake-10.5.0/lib/rake/task.rb:165:in `invoke'
/Users/chris/.rbenv/versions/2.2.4/lib/ruby/gems/2.2.0/gems/rake-10.5.0/lib/rake/application.rb:150:in `invoke_task'
/Users/chris/.rbenv/versions/2.2.4/lib/ruby/gems/2.2.0/gems/rake-10.5.0/lib/rake/application.rb:106:in `block (2 levels) in top_level'
/Users/chris/.rbenv/versions/2.2.4/lib/ruby/gems/2.2.0/gems/rake-10.5.0/lib/rake/application.rb:106:in `each'
/Users/chris/.rbenv/versions/2.2.4/lib/ruby/gems/2.2.0/gems/rake-10.5.0/lib/rake/application.rb:106:in `block in top_level'
/Users/chris/.rbenv/versions/2.2.4/lib/ruby/gems/2.2.0/gems/rake-10.5.0/lib/rake/application.rb:115:in `run_with_threads'
/Users/chris/.rbenv/versions/2.2.4/lib/ruby/gems/2.2.0/gems/rake-10.5.0/lib/rake/application.rb:100:in `top_level'
/Users/chris/.rbenv/versions/2.2.4/lib/ruby/gems/2.2.0/gems/rake-10.5.0/lib/rake/application.rb:78:in `block in run'
/Users/chris/.rbenv/versions/2.2.4/lib/ruby/gems/2.2.0/gems/rake-10.5.0/lib/rake/application.rb:176:in `standard_exception_handling'
/Users/chris/.rbenv/versions/2.2.4/lib/ruby/gems/2.2.0/gems/rake-10.5.0/lib/rake/application.rb:75:in `run'
/Users/chris/.rbenv/versions/2.2.4/lib/ruby/gems/2.2.0/gems/capistrano-3.1.0/lib/capistrano/application.rb:15:in `run'
/Users/chris/.rbenv/versions/2.2.4/lib/ruby/gems/2.2.0/gems/capistrano-3.1.0/bin/cap:3:in `<top (required)>'
/Users/chris/.rbenv/versions/2.2.4/bin/cap:23:in `load'
/Users/chris/.rbenv/versions/2.2.4/bin/cap:23:in `<main>'
Tasks: TOP => rbenv:validate

I think its referring to this, but not really sure what's happening beyond that point.

Guess what I'm asking is, is there anything cleaner than this:

    valid_photos = Photo.where.not(user_id: User.where(shadowbanned: true).pluck(:id))

I've got:

class User < ActiveRecord::Base
  has_many :photos
end

class Photo < ActiveRecord::Base
  belongs_to :user
end

Each user has a "shadowbanned" attribute (t/f). I'd like to be able to grab all the photos where the users aren't shadowbanned.

How would you recommend I set up either the scope or the association to have Rails do the most work possible?

You fixed that too quickly for me to take a screenshot :(

That was it! I forgot to add the do/end parts to a form tag further up, throwing everything off.

Even when I put the form elsewhere, it's still not rendering the first one correctly. Also, when I try to click the pagination links at the bottom, I get a weird error. I think I'm not actually opening or closing a form properly somewhere

lol

haha. even the text in here is italicized now after you said tag

I’m doing a thing where I’m rendering table rows:

<%= render partial: "choices_table_row", collection: @images, as: :image %>

Then, inside that partial:

<tr id="image_<%= image.id %>">
  <td><%= image.name %></td>
  <td><%= image.file_filename %></td>
  <td><%= image.tag_list %></td>
  <td>
    <%= link_to "#", data: {behavior: "edit-image-button", url: "/artist/images/#{image.id}/form" } do %>
      <i class="bts bt-edit"></i>
    <% end %>
  </td>
  <td>
    <i class="bts bt-plus-circle clickable" data-behavior="create-selection-button">
      <%= form_for([:artist, @gallery, Selection.new], remote: true) do |f| %>
        <%= f.hidden_field :image_id, value: image.id %>
      <% end %>
    </i>
  </td>
</tr>

Then I've got some js running behind the scenes to submit the form inside the icon when said icon is clicked:

class PotentialSelection
  constructor: (item) ->
    @icon = $(item)
    @form = @icon.find('form')
    @setEvents()

  setEvents: =>
    @icon.on "click", @addToCollection

  addToCollection: =>
    console.log "wat"
    @form.submit()

ready = ->
  $("[data-behavior='create-selection-button']").each ->
    new PotentialSelection this
  #$("#current-selections-container").sortable(
    #update: ->
      #$.post $(this).data("order-url"), $(this).sortable('serialize')
  #)

$(document).ready ready
$(document).on 'page:load', ready
$(document).on 'ajax:complete', ready

It all works correctly except for the first one. What renders in the first partial is this:

<i class="bts bt-plus-circle clickable" data-behavior="create-selection-button">
      <input name="utf8" type="hidden" value="✓">
      <input value="4" type="hidden" name="selection[image_id]" id="selection_image_id">
</i>

When all the other ones have the full form inside of them:

<i class="bts bt-plus-circle clickable" data-behavior="create-selection-button">
      <form class="new_selection" id="new_selection" action="/artist/galleries/1/selections" accept-charset="UTF-8" data-remote="true" method="post"><input name="utf8" type="hidden" value="✓">
        <input value="11" type="hidden" name="selection[image_id]" id="selection_image_id">
</form>    </i>

wat

Update:

I was trying to create an array of blocks which contained queries that would lazily evaluate until I realized these are scopes, and they already exist. some of these queries aren't pretty, but for the usage they'll be under thats totally fine.

class Image < ActiveRecord::Base
  scope :unfinished, -> { tagged_with(ActsAsTaggableOn::Tag.all.map(&:to_s), exclude: true).where(year: nil) }
  scope :excluding, -> (ids) { where.not(id: ids) if ids.present? }
  scope :by_tags, -> (tags) { tagged_with(tags) if tags.present? }
  scope :by_year, -> (year) { where(year: year) if year.present? }
  scope :default, -> {order(created_at: :desc).first(8)}
  scope :none_if_all, -> {where(id: nil).where("id IS NOT ?", nil) if all.count == Image.all.count }
end

class SearchImage

  def initialize(params)
    @params = params
    @tags = params[:tags] || ""
    @year = params[:year] || ""

    set_excluded_image_ids if params.has_key?(:collection_id) && !params[:collection_id].empty?
  end

  def search
    if @params[:commit] == "View Uncategorized Images"
      @images = Image.excluding(@excluded_image_ids).unfinished
    elsif @tags.empty? && @year.empty?
      @images = Image.excluding(@excluded_image_ids).default
    else
      @images = Image.excluding(@excluded_image_ids).by_tags(@tags).by_year(@year).none_if_all
    end
  end

  private

  def set_excluded_image_ids
    collection = Collection.find(@params[:collection_id])
    @excluded_image_ids = collection.images.pluck(:id)
  end
end