All threads / Are there any WORM (write once, read many) data stores that integrate well with Rails?

Ask A Question

Notifications

You’re not receiving notifications from this thread.

Are there any WORM (write once, read many) data stores that integrate well with Rails?

Chris Zempel asked in Databases

I've gotta store a bunch of events, which are snippets of data that contain information about a thing that happened inside the system I'm working on. I'm building out audit trail, which could potentially be used in legal proceedings.

There are a couple predominant things I'll need to be able to achieve with them:

  • Lots of searching, filtering, sorting and slicing by various attributes stored inside
  • csv exporting (wouldn't be a real feature if we didn't need to export to csv!)
  • They'll need to be stored and accessible for a minimum of a year (possibly 3)

And then, not a necessity but would be cool,

  • verify if the data was tampered or not w/ a hash

Currently it's safe to say that, the data model for these events isn't settled.

{
  "name": "some_predefined_event_name",
    "true_tstamp": "some tzinfo value",
    "traits":
     {
       "user_id": 346,
        "investment_id: 256,
        "arbitrary": "json"
     }
}

The best looking approach so far seems to involve longer-term storage on AWS Glacier, as that will tick the compliance boxes, but that leaves the shorter-term needs like searching & filtering, csv generation, unfulfilled.

Or maybe somehow disabling anything but writing and reading in DynamoDB via IAM policies, then taking the monthly chunks of that and moving them over to Glacier.

Is there a better way to go at this?

Join the discussion

Want to stay up-to-date with Ruby on Rails?

Join 33,665+ developers who get early access to new tutorials, screencasts, articles, and more.

    We care about the protection of your data. Read our Privacy Policy.

    logo Created with Sketch.

    Ruby on Rails tutorials, guides, and screencasts for web developers learning Ruby, Rails, Javascript, Turbolinks, Stimulus.js, Vue.js, and more. Icons by Icons8

    © 2020 GoRails, LLC. All rights reserved.