All threads / How to scrape the right data in an API built in Rails 5.2.1

Ask A Question

Notifications

You’re not receiving notifications from this thread.

How to scrape the right data in an API built in Rails 5.2.1

Jakub asked in Ruby

Hello,

I'm working on API in Rails 5.2.1.

The API scope is to scrpae flight information from this 2 tables of the cph.dk site:
Departures
Arrivals

I need help to resolve an odd problem I'm facing which is that one:
My scraper service is scrapping the wrong data which means that the data I'm taking is not corresponding with the flights info from the 2 tables. As an example we supposed in departures has a flight at 15:30 to Amsterdam but when I see what the scraper get is like a departure flight is at 20:01 to Berlin which even doesn't exist on departure table so I have issues to understand why is getting wrong data.

What I need is to get the exact same info as are displayed in those 2 tables and update the info I'm scraping every tot minutes to have always the most recent info.

At the moment I stuck with the wrong data and hope someone can take a look in my project and tell me what I'm doing wrong.

My repo:
https://github.com/Jakub41/cph-scraper-api

Thank you for helping

Join the discussion

Want to stay up-to-date with Ruby on Rails?

Join 33,399+ developers who get early access to new tutorials, screencasts, articles, and more.

    We care about the protection of your data. Read our Privacy Policy.

    logo Created with Sketch.

    Ruby on Rails tutorials, guides, and screencasts for web developers learning Ruby, Rails, Javascript, Turbolinks, Stimulus.js, Vue.js, and more. Icons by Icons8

    © 2020 GoRails, LLC. All rights reserved.