Robert Adler
Joined
40 Experience
0 Lessons Completed
0 Questions Solved
Activity
Posted in how scrabe data every day
Oh, I don't run GoRails. I'm just another subscriber like you.
I'm sure he'll see this, though. :)
Posted in how scrabe data every day
This isn't efficient nor is it beautiful, but it'll at least get you started. You can use nokogiri as well, but I just used regex as a quick and (emphasis on) dirty solution. Plus I haven't had any caffeine yet today. Here:
require 'mechanize'
page = Mechanize.new.get "http://www.caat.it/it/listino/2015-08-28"
products = page.body.scan(/<tr class="(odd|even).+?">(.+?)<\/tr>/m).map{|thisproduct| thisproduct.last.to_s.scan(/<td data-title="(.+?)" class=".+?" >(.+?)<\/td>/m).map{|key,val| [key,CGI.unescapeHTML(val).gsub(/(<[^>]*>)|\n|\t/s){" "}.strip.chomp]}}
# products.count == 160
# products.first.each{|key,val| puts "#{key} => #{val}"}
# P. Min => 1,10
# P. Pre => 1,15
# P. Max => 1,20
# Specie => ARANCE
# Varietà => VALENCIA LATE
# Calibro => 70-80 (6)
# Cat. => I
# Presentazione => A PIU' STRATI
# Marchio =>
# Origine => SUD AFRICA
# Confezione =>
# Unita misura =>
# Altre =>
# Gruppo => AGRUMI
That should get you started with getting the data. What you do with it after that is up to you. :)
Posted in how scrabe data every day
Err double post. Disregard.
Posted in how scrabe data every day
That's a really open question. Can you give us an example of what you would want to scrape?