How to download multiple files from s3 on fly?

mamatharao December 3, 2021 1:53pm

Hi,

I am able to download files from s3 as a zip by using below mentioned code.

require 'open-uri'

def download_all_files
folder_path = "#{Rails.root}/public/downloads/"
zipfile_name = "#{Rails.root}/public/archive.zip"

FileUtils.remove_dir(folder_path) if Dir.exist?(folder_path)
FileUtils.remove_entry(zipfile_name) if File.exist?(zipfile_name)
Dir.mkdir("#{Rails.root}/public/downloads")

@model_object.each do |attachment|
open(folder_path + "#{attachment.avatar.file.filename}", 'wb') do |file|
file << open("#{attachment.avatar.url}").read
end
end

input_filenames = Dir.entries(folder_path).select {|f| !File.directory? f}

Zip::File.open(zipfile_name, Zip::File::CREATE) do |zipfile|
input_filenames.each do |attachment|
zipfile.add(attachment,File.join(folder_path,attachment))
end
end

send_file(File.join("#{Rails.root}/public/", 'archive.zip'), :type => 'application/zip', :filename => "#{Time.now.to_date}.zip")

end

But by using this code, that files are storing in my application. Is there any way to download files directly from s3 or is there a way to remove files which are stored in public folder.

Freibuis September 18, 2023 8:32pm

you could use the aws-sdk-s3 gem
and something like this

s3 = Aws::S3::Resource.new
s3.bucket(bucket_name).object(object_key).upload_file(file_path)

I would not store your data in the public folder. if you are running multiple nodes behind a reverse proxy like nginx your client might not be able to get the data

if you upload the file to s3, you could send a signed link to the s3 object with a timed limited access (like 10mins etc)

url = s3.bucket(bucket_name).object(object_key).presigned_url(:get, expires_in: 600)

this will allow the user to download directly off AWS s3 without even needing to use your rails server as a resource.

the other option is to download the aws s3 object to a TempFile then use send_file to send it to the client.. this method hides aws S3 from the client. I generally prefer this method but does add to latency/download speed