Ask A Question

Notifications

You’re not receiving notifications from this thread.

How to download multiple files from s3 on fly?

mamatharao asked in Rails

Hi,

I am able to download files from s3 as a zip by using below mentioned code.

require 'open-uri'

def download_all_files
folder_path = "#{Rails.root}/public/downloads/"
zipfile_name = "#{Rails.root}/public/archive.zip"

FileUtils.remove_dir(folder_path) if Dir.exist?(folder_path)
FileUtils.remove_entry(zipfile_name) if File.exist?(zipfile_name)
Dir.mkdir("#{Rails.root}/public/downloads")

@model_object.each do |attachment|
open(folder_path + "#{attachment.avatar.file.filename}", 'wb') do |file|
file << open("#{attachment.avatar.url}").read
end
end

input_filenames = Dir.entries(folder_path).select {|f| !File.directory? f}

Zip::File.open(zipfile_name, Zip::File::CREATE) do |zipfile|
input_filenames.each do |attachment|
zipfile.add(attachment,File.join(folder_path,attachment))
end
end

send_file(File.join("#{Rails.root}/public/", 'archive.zip'), :type => 'application/zip', :filename => "#{Time.now.to_date}.zip")

end

But by using this code, that files are storing in my application. Is there any way to download files directly from s3 or is there a way to remove files which are stored in public folder.

Reply

you could use the aws-sdk-s3 gem
and something like this

s3 = Aws::S3::Resource.new
s3.bucket(bucket_name).object(object_key).upload_file(file_path)

I would not store your data in the public folder. if you are running multiple nodes behind a reverse proxy like nginx your client might not be able to get the data

if you upload the file to s3, you could send a signed link to the s3 object with a timed limited access (like 10mins etc)

url = s3.bucket(bucket_name).object(object_key).presigned_url(:get, expires_in: 600)

this will allow the user to download directly off AWS s3 without even needing to use your rails server as a resource.

the other option is to download the aws s3 object to a TempFile then use send_file to send it to the client.. this method hides aws S3 from the client. I generally prefer this method but does add to latency/download speed

Reply

This is a pervasive problem that I still haven't found a solution for.

Reply

Deleting files in a shared folder, if not done correctly. Then you can specify at once to read files from the directory and delete them.

Reply
Join the discussion
Create an account Log in

Want to stay up-to-date with Ruby on Rails?

Join 79,047+ developers who get early access to new tutorials, screencasts, articles, and more.

    We care about the protection of your data. Read our Privacy Policy.

    Screencast tutorials to help you learn Ruby on Rails, Javascript, Hotwire, Turbo, Stimulus.js, PostgreSQL, MySQL, Ubuntu, and more.

    © 2023 GoRails, LLC. All rights reserved.