Skip to main content

Subscribe to GoRails to get access to this episode and all other pro episodes, and new awesome content every month.

Subscribe Now
Only $19/month

Unlimited access. Cancel anytime.

20 Direct Uploads to Amazon S3:

Multiple File Uploads with Shrine

Episode 158 · December 14, 2016

Learn how to implement multiple file uploads in your app using Shrine and jQuery File Upload

Gems File Uploading


Transcripts

Subscribe or login to view the transcript for this episode.

Discussion


Fallback

This is fantastic! Exactly what I needed so THANK you! I am having one very strange issue that I wonder if anyone has experience with. "Larger" files uploaded - meaning over a mg end up getting rendered sideways in the browser when referencing uploaded S3 url...small files (in my example 28kb) render correctly. This is driving me pretty crazy

Fallback

I'm not sure if this matters (I know it does with video) but do your photos happen to have been taken at the other orientation originally? I'm not sure if the browser may be reading some metadata and rotating them automatically or not.

Fallback

That must be it. If I upload the pictures as taken from DSLS (portrait) they render wrong. However, if I simply open the file and resave it in photoshop and then re-upload, it renders correctly. Incidentally it opens in photoshop in the "correct" orientation...the only thing I have to do to correct is resave.

Fallback

That makes sense. I noticed Chrome doing this with videos recently so I figured it might be the same. Photoshop is pretty smart about detecting what you intended to shoot as and everything but Chrome can only do the basic orientation stuff that's set.

You should also be able to add a processor into Shrine to have imagemagick correct the orientation on upload as well. There's an "auto_orient" method you can use with MiniMagick to do that automatically.

Fallback

Thanks, Chris! I'll give it a shot.

Fallback

ok this is all working! Everyone take note - these were pics taken with a canon 80d DSLR which obviously does not save exif data correctly. All it took was the following:

require "image_processing/mini_magick"

class ImageUploader < Shrine
include ImageProcessing::MiniMagick

plugin :processing

process(:store) do |io, context|
auto_orient!(io.download)
end
end

Fallback

Great work Scott, and props for coming back and sharing your solution! đź‘Ť


Fallback

ok, so I am experiencing an interesting behavior that I think is important for this approach to scale...any help would be appreciated!

I am using this process to upload high numbers of images so I only think this is relevant when uploading more than 10 or so...I will be uploading hundreds at a time.

I have the following logic in image_uploader.rb to correctly orient the image and resize:

process(:store) do |io, context|
file = io.download
resize_to_limit!(file, nil, 640)
auto_orient!(file)
file
end

This functions correctly. HOWEVER, rails is still working on these files well after the last progress bar has reached 100%. If the user refreshes the page during this time, the "remaining" images will get uploaded without processing - meaning sideways and huge. They still go to amazon but they skip the process method.

So my question is two-fold. Anyone have an idea how to:

1) wrap the progress bar around the "entire" process?
2) either NOT upload the unprocessed files, identify them or, of course, fix the problem entirely!!

Any help or thoughts would be appreciated.

Fallback

I think best behaviour is to send data of each file to the Rails app as soon as it's uploaded to S3.

That gives great user experience, because even if they terminate the upload at some point, the files that were uploaded before stay uploaded. That's why e.g. Flickr upload interface really sucks, because it's all-or-nothing - if there is an error or you have to terminate the multiple file upload midway, nothing stays uploaded.

It is also the most performant, because the server can already start processing each file as soon as it's uploaded to S3, instead of holding off and then sending all files at once.

The shrine-rails-example app already demonstrates this flow, so you can draw your inspiration from there.


Fallback

How should I change the update.js on using the versions plugin? And I can't see the progress bar in Bootstrap 4 and so any solution?

Fallback

JavaScript looks exactly the same if you're using the versions plugin, because direct upload is just another way of caching the file (you're uploading it to temporary storage directly via the client rather than have the server upload it), and processing versions happens on promoting (when cached file is moved to permanent storage).

Fallback

Thank you for your kind answers.


Fallback

Chris-
Thanks a whole bunch for doing this series. Invaluable! Exactly what I needed.
Dave


Fallback

I want to crop uploaded image or cached image before uploading. I tried it on "after_update" callback but it is difficult to reupload the UploadedFile after cropping. Wound you give me a hint for this?

Fallback

You can use the recache plugin, which adds another processing step between caching and promoting, triggered before the record is saved. So you can just download the cached file there, crop it, and upload it to cache storage again.

Fallback

Thank you so much. I'll try and notice you the result.

Fallback

I'm sorry but what does it mean "promote"?

Fallback

This is the term in Shrine for when a cached file is re-uploaded to permanent storage (which happens after the record is saved). Btw, feel free to ask any questions that aren't related to this screencast on the Shrine Google group.

Fallback
Fallback

One more question please...
I would like to get the cropping dimension from user input values and then update the image. In this case, how can I implement using recache plugin. Thank you in advance.

Fallback

See this Google group thread. The question was for cropping on promotion, but you can easily adapt it to have it performed on recache (probably just changing `process(:crop)` to `process(:recache)`).


Fallback

I'm trying to store files in different folders in the same S3 bucket. Let's say I have two models, document and image. How can I store the corresponding files in two folders instead of "store". I was reading the following, but can't find how to get this done: https://twin.github.io/bett.... Any ideas where I can find this information? Thanks!

Fallback

You can choose how you generate the location in your uploader:


class ImageUploader < Shrine
def generate_location(io, context)
"images/#{super}"
end
end

class DocumentUploader < Shrine
def generate_location(io, context)
"documents/#{super}"
end
end

Feel free to post any general Shrine questions on the Shrine Google group ;)

Fallback

oh nice! thank you so much.


Fallback

One small issue I'm running into.. since photos are displayed after uploading with js it's necessary to refresh the page for rails handling to kick in. For example, new images are not linked to their own show pages until a refresh. Also, the max width setting is removed after a refresh, since rails takes over.

Is there any way to handle all display options with rails from the start, so that no page refresh is necessary?


Fallback

Hey Chris, nice work your tutorials on direct upload with Shrine. I've been following
along sticking pretty much to the original code. But now I got stuck when I was
trying to make this work with different image versions.

This is my albums/show.html.haml:

.container
.row
#justified-collection
%p#notice= notice
%p
- @album.photos.each do |photo|
%a{:href => photo.image_url(:size_1000)}
%img{alt: photo.caption , src: photo.image_url(:thumb)}
-if user_signed_in?
.divider
= render partial: "photos/form", locals: {photo: Photo.new}
= link_to 'Edit', edit_album_path(@album)

Since I have to specify now the image version, the original _photo.json.jbuilder
partial won't work anymore. So I modified it like this:

json.extract! photo, :id, :created_at, :updated_at
json.image_url(:size_1000) photo.image[:size_1000].url
json.image_url(:thumb) photo.image[:thumb].url
json.url album_photo_url(@album, photo, format: :json)

But this gives me a syntax error:

SyntaxError (path_to_rails_app/app/views/photos/_photo.json.jbuilder:2: syntax error, unexpected tIDENTIFIER, expecting keyword_end

Any advice for me?


Fallback

Can someone help me understand what this error means?

My jquery uploaded progress bar isn't moving and I'm running this into error: ActionController::RoutingError (No route matches [GET] "/images/upload/cache/presign"):

Fallback

Sounds like you're missing the routes for Shrine. Make sure this is in your routes file. https://github.com/gorails-...


Fallback
Has anyone had any luck getting this working rendering image templates/partial instead of just an image (mentioned as an alternative at around 11.30 in the video)? 

Chris has been kind enough to try and help out but my code differs. I'm unsure where to put the "json.template" section, tried in the show.json.jbuilder and the _attachment.json.jbuilder and how to alter it to suit my attachment model. Got errors and sometimes continuous loops. 

I'd be super grateful for any help or advice: https://twitter.com/aymorgan/status/953563878473924609

Here's some of my code: https://gist.github.com/aymorgan/44a0cb6608b44e560ff5e5429d491642

Login or create an account to join the conversation.