Everyone's Been Bitten Before - Whitespaces/Unwanted Characters in User Input

Hey all!

I've got an interesting question, and I'm sure you're all handling it in different ways. I know there are plenty of ways to tackle this problem, so I was interested in what everyone's doing?

The old issue of removing excess trailing/intermittent/leading whitespace, other characters from copying pasting etc, from user input?

An example:

user.first_name = "John "
user.email = "craig@test.com\n "

etc etc, you all know what I mean.

I see this cleansing/normalisation could happen at a few stages:

Immediately Upon User Form Input - ala Javascript functions to strip as they type/pre-submission
Inside the Controller on Params Submission
Intermediate Model/Attribute Methods (Callbacks, even :o )
Prior to or upon Database Persistence (at the DB Specific level)

*What stage are people doing it at? *

Personally: I feel like the best place for this would be at the model level, as the gatekeeper for the database, but I always try to avoid callbacks wherever possible, just not sure if this would be an appropriate time to use them, or if anyone has a more elegant solution?

I see that there are a few gems out there too , but once again, always trying to minimise the number of dependencies I have in the project and keep the Gemfile lean ;)

It sometimes depends on what you're using the data for. For example, an email address is something you would want to for sure strip extra whitespace on. However, a forum post doesn't really matter. A little extra before or after space isn't necessarily going to matter when it's displayed in HTML as it probably won't even get reflected in the browser because they're not HTML characters.

As part of assigning an attribute, it makes sense to strip characters then before actually assigning the value.

class User < ApplicationRecord
  def email=(value)
      super(value.strip)
  end
end

That said, I imagine that most of the gems will opt to do a before_validation callback for this so they can take care of the cleanup all in one fell swoop without overriding each and every attribute method on the class. An additional benefit of the callback is that it will be done before every save which means that if the data was inserted in a way that skipped the setter, it would still get stripped correctly. A callback is going to guarantee that runs before every save where setters might not.

For example, this gem uses the callback approach: https://github.com/rmm5t/strip_attributes/blob/master/lib/strip_attributes.rb#L8-L10

On the client is fine, but should never be trusted. Server side processing of params should always be done because it's too easy to get around the JS and submit bad data.

There's also things like TRIM in postgres that you could use for this, but it makes sense to clean up whitespace before it hits the db because you'll usually want it done before you run validations in Ruby to verify the data.

I'd probably say the callback approach is the best place for this. All your validations are run by callbacks so you can have data integrity. Stripping whitespace is something you want to run every single save as well for the same reason, so callbacks fit this really well.

And this is a good reminder, callbacks aren't always bad. When people give out the advice to "avoid callbacks" they're saying you shouldn't jump to that when you want to implement business logic. For example, sending an email after registering with a callback isn't great because you can't turn it off. However, stripping whitespace is different because you'll always want that to run and you do want it closely coupled to your data since it's not business logic, it's for data integrity. Same reason why your validations always run on the validation callback, you always want them to run to keep the integrity of your data. 👍

Keep in mind that a gem like strip_attributes is like 100 lines of code and they've already encountered all the obvious gotchas that you will run into if you implement it from scratch. To me, adding a gem that's that short is totally worth it. If something doesn't work you can just dive in and fix it, and if you were to build your own you'd probably end up with a similar amount of code and having to build your own gem to share it between your apps. At that point you're probably reinventing the wheel a bit unnecessarily just to remove one line from your Gemfile. If it were something more complex, I feel like it'd be worth considering but in this case it's almost too simple not to just grab a gem for it and focus on more important things in your app.

Hey Chris, thanks for the awesome and details response, and so quickly too!

Great details consideration there, I think you're right on the money with the two points around callbacks and gems/dependencies. I am just always playing devils advocate when I consider implementing something ;)

Funnily enough, out the probably 10 gems out there for stripping attributes (with widely varied adoption/stars), I also landed on that one strip_attributes!

I've implemented it just before, and it definitely working nicely compared to my own initial attempt at implementing the strip/squish methods on the models.

I think you've also settled the case nicely regarding at the database (eg Postgres) vs Model, since at the database is often too late for things like validations in Ruby/Rails etc.

Thanks again for the detailed and quick answer, onto solving the bigger problems now! ;)