Jay Killeen

Can you please elaborate on this part `you leave xss/injection etc all wide open so that body may be used to by malicious users to extend reach`? I'd like to understand the security risk a little more.

I found some of this information useful on how to test these polymorphic comment features at https://github.com/thoughtb...

A simple report would be sales where the margin is less than 0. This simply hits one table and I can currently do this using the advanced search feature of Ransack (I have the add filter buttons that Ryan Bates implemented in his Railscast on Ransack).

This is well and good to use the search filters on the existing sales index and show but as soon as it becomes more complex I would like a link that the user goes to and might have different fields etc in the index. I might also like to through a graph on the page.

Another report requires a search form to be filled in to collect certain customer and product groups and then creates a scatterplot of the different customers prices for those products.

In the past I have had this done with a separate controller, view and model for each report but just get concerned as I could have any number of these different reports etc.

My Ruby on Rails enterprise app has a whole range of tables ranging such as customers, products, sales, warehouses, prices etc etc.

I'd like to build a whole range of different reports that users can view that slice and dice the data certain ways. I am imagining there would be a 'search' view where you select your search parameters and then a show page that returns the result.

Is this something you would setup a MVC without a database table for? Also because I could have anywhere up to a hundred of these different reports, would you try to nest the files in your rails app architecture to keep them separated from your core MVC files or would you just not give 2 hoots and let it mix in with the rest and not be OCD about it?

My other alternative is to run a single MVC with conditionals pointing to different views in the controller based on the report selected in the index.

Sometimes I feel deciding these things is the hardest bit...

Cool. I followed this guide http://railsapps.github.io/rails-google-analytics.html and was up and running in 10 minutes.

Looked at the redis and Ahoy one's aswell but wanted something easy to configure.

Out of interest. Google analytics is only traffic information right? (time on site, what page you are on etc) There is no chance of it capturing data that is being sent (ie user details, passwords etc). Seems like a no brainer but want to be certain.

Or ideally do this with Rspec but god I am finding that to be a steep learning curve!

I'd like to get people's thoughts on the best approach.

I'd like to have a search ability where once a selection if made, the next dropdown box populates with only the available options from the database. For example, if I have a product model with three fields 'Group', 'Sub-group' and 'id' and those fields are nested within each other so Group 1 has 10 types and those 10 types have a total of say 100 products. How can I do it so the user can drilldown to a specific product to search on by first selecting a group, and then one of the sub groups and then finally the product.

I think there would need to be some AJAX going on to populate the fields in the background or do I push all the data to the view with Javascript and make updates using JQuery on the DOM change events on the fields?

I'd like to get some thoughts on best ways to implement this type of thing before I go running off and doing it for 15+ models :)

Yeah it is odd. I dived in with pry and saw that the strings are identical. I thought maybe a second loop has run through and truncate customers has been hit again so essentially all is working but I am destroying it all once I am finished! I'll try commenting out the truncation and doing it from PGAdmin and see if it works :)

This is the bash script I use to convert an Cognos tab separated files into straight csv (comma separated).

    awk 'BEGIN{FS="\t";OFS=",";Q="\""}
          {for (i=1;i<=NF;++i)
             if ($i ~ /[",]/)
               $i = Q gensub(/"/,Q Q,"g",$i) Q
          }
          {$1 = $1}
     1' sales_utf8.csv > sales_utf8_awk.csv

I also recommend CSV and try to make sure the CSV file is in UTF-8. as Excel sometimes encodes the files as UTF-16 or other weird things. Also saving an excel file as CSV doesn't always mean it is comma separated. Excel generally defaults to tab separated. To fix all these things I run bash scripts and use tools such as iconv and awk

iconv -f utf-16 -t utf-8 sales.csv > sales_utf8.csv

will convert your utf-16 to utf-8.

Hey Jared, how did you go about importing that much data? In the past I have iterated over csv files but it takes way too long. At the moment I am using a bash script with iconv, awk and other things to manipulate the data so that it can be imported into Rails using the Postgresql COPY command. Works super fast. Now my biggest issue is dealing with the 150MB files by zipping them first. Does this sound like how you have been dealing with it?

I've checked out the videos on importing data to your rails app. Very timely as I am deep into this feature bringing data from all our legacy systems into my rails app.

My issue is I am importing millions of rows of data. I have used the postgresql COPY command to speed this up but it is still taking around 15-20mins to upload the 150MB files to the server (slow connection on the work VPN).

I have raised my question on SO if you can take a look.
http://stackoverflow.com/questions/29225140/using-rubyzip-in-rails-to-import-file-data-to-postgresql-with-copy

The gist of it is refactoring...

THIS

    def process file=nil
      file ||= @file.tempfile
      ActiveRecord::Base.connection.execute('truncate customers')
      conn = ActiveRecord::Base.connection_pool.checkout
      raw  = conn.raw_connection
      raw.exec("COPY customers FROM STDIN WITH (FORMAT CSV, DELIMITER ',',  NULL '')")
      # open up your CSV file looping through line by line and getting the line into a format suitable for pg's COPY...
      data = file.open
      data::gets

      ticker = 0
      counter = 0
      success_counter = 0
      failed_records = []

      data.each_with_index do |line, index|
        raw.put_copy_data line
        counter += 1
      end
      # once all done...
      raw.put_copy_end
      while res = raw.get_result do; end # very important to do this after a copy
      ActiveRecord::Base.connection_pool.checkin(conn)
      return { :item_count => counter, :processed_successfully => counter, :errored_records => failed_records }
    end

INTO

    def process file=nil
      file ||= @file.tempfile
      Zip::File.open(file) do |zip_file|
        zip_file.each do |entry|
          content = entry.get_input_stream
          data = content.read
          ActiveRecord::Base.connection.execute('truncate customers')
          conn = ActiveRecord::Base.connection_pool.checkout
          raw  = conn.raw_connection
          raw.exec("COPY customers FROM STDIN WITH (FORMAT CSV, DELIMITER ',',  NULL '')")
          # open up your CSV file looping through line by line and getting the line into a format suitable for pg's COPY...

          ticker = 0
          counter = 0
          success_counter = 0
          failed_records = []

          data.lines.map(&:chomp).each_with_index do |line, index|
            raw.put_copy_data line
            counter += 1
          end
          # once all done...
          raw.put_copy_end
          while res = raw.get_result do; end # very important to do this after a copy
          ActiveRecord::Base.connection_pool.checkin(conn)
          return { :item_count => counter, :processed_successfully => counter, :errored_records => failed_records }
        end
      end

But for whatever reason when I run it I am getting no data saved to the database.

Also to add to the videos that Chris has made. I chose to create an imports controller with an import module and a class for each model I am importing. It is working really well. It will be super duper if I can figure out how to import it form a Zip file :)

Cheers I'll take a look into that. I did read the part on strong parameters but didn't realise it would be for action based authorisation.

Hey does anyone know how to use Pundit to control a users ability to edit a field. I'd like all users to be able to change simple things like their name but I want admins to be the only one's who can change the security things like 'roles' (as this determines what access they have to other parts of the app).

Please note that if you follow this guide and you are also using devise omniauthable you will not need to create the omniauth.rb file. You can go to devise.rb and find the omniauth configuration lines there (just uncomment the config.omniauth lines and set it like Chris does (I think initially Devise had ENV instead of Rails.application.secrets...

# ==> OmniAuth
# Add a new OmniAuth provider. Check the wiki for more information on setting
# up on your models and hooks.
# config.omniauth :github, 'APP_ID', 'APP_SECRET', scope: 'user,public_repo'
config.omniauth :yammer, Rails.application.secrets.yammer_api_key, Rails.application.secrets.yammer_api_secret

If you did what I did and set the devise.rb as well as the omniauth.rb then you will get two callback requests and trigger a CSRF error. (note I am using Yammer and yammer-omniauth gem instead of Twitter but pretty much all worked the same.

Started GET "/users/auth/yammer/callback?code=keyQ&state=secret" for 127.0.0.1 at 2015-02-05 16:31:15 +1000
I, [2015-02-05T16:31:15.558087 #3900] INFO -- omniauth: (yammer) Callback phase initiated.
I, [2015-02-05T16:31:42.574957 #3900] INFO -- omniauth: (yammer) Callback phase initiated.
E, [2015-02-05T16:31:42.575381 #3900] ERROR -- omniauth: (yammer) Authentication failure! csrf_detected: OmniAuth::Strategies::OAuth2::CallbackError, csrf_detected | CSRF detected
Processing by Users::OmniauthCallbacksController#failure as HTML
Parameters: {"code"=>"key", "state"=>"secret"}
Redirected to http://localhost:3000/sign_in
Completed 302 Found in 2ms (ActiveRecord: 0.0ms)

from the command line `rake secret` and copy into your secrets.yml file for develop and test environments (if you are choosing to gitignore the new file).

Just on the gitignore for secrets.yml. You can't do this after you have already commit secrets.yml. I followed the instructions but had already commited it so gitignore would always see it even if I deleted it etc.

The answer I am now looking for is how to generate a new secrets.yml secret_key_base

Just implemented this in conjunction with Ryan's episode 370 http://railscasts.com/episo.... Have some really cool advanced search features now working on my customer index page.

Now just need to figure out how to rspec this stuff (I find BDD doesn't work in the learning phase).

Hi

I notice this site uses Google Analytics to track usage. Is that the standard when it comes to tracking your rails apps? I know it is generally pretty standard in the case of my websites but for Rails Apps I was thinking it would be cool to have the analytics cooked into the app itself so there isn't a requirement to login to a third party or reduce the risk of having corporate IT ask questions about where our data is being sent to.

I have looked at gems such as Redis Analytics https://github.com/saturnine/redis_analytics which seem interesting (although I do not really understand Redis yet).

Does anyone else have any suggestions on how we can track usage in our apps? I just want things like page views, time on site etc so that when I have my app out in the wild I can report back to management how much the teams are using it so I can put business cases through for further development.

My app is using devise and I have all the controllers locked down for authentication. My only metric at this stage is a count of how many times a user has logged in which is an out of the box devise feature.

Any thoughts would be welcome.

Here is a link to the forum post where I shared links to my Gist code to Bootstrap the Devise views.

https://gorails.com/forum/b...