Designing our Web Scraper UI with TailwindCSS Discussion

Wow!

This episode is incredibly relevant to me! I manage littlebirdelectronics.com.au, an Australian distributor for brands like Adafruit, SparkFun, Pololu, etc. For years, I've been using Ruby to scrape information from Adafruit and others to update our internal inventory systems.

We share with our customers the details about our local stock levels and how many additional units we can source for them.

Our internal systems, named Koi, are built on Jumpstart (jumpstartrails.com) and we use hatchbox.io for deployment. We've now opened Koi for other e-commerce merchants to use as well.

Recently, we've been developing "mini apps" (see an example) within Koi that execute the scraping code on AWS Lambda. This approach allows us to run code from other merchants in a secure, isolated environment.

One challenge we faced with AWS Lambda was using native binaries required by Nokogiri. Our solution was to use Docker to create a Lambda Layer containing the Nokogiri gem. We then stream the data back to merchants using Hotwire.

Most of the knowledge I've acquired on this journey has come from Gorails!

Thanks Chris for showing me just how small this world is!

Fantastic session, love how it addressed not only CSS styling but database performance. Great refresher for me personally and jammed full of lessons.