Remote download

[soulviasound]

I'm looking for someone to resolve this php problem:

VM(last verssion) + Joomla 1.5.22 hosted by fatcow

I have problem with download of large files. I already set remote download from Amazon S3 bucket, but I get corrupt files.

I need soon as possible. Name your price, payment via paypal.

[shimi]

How big are the files?

[soulviasound]

2 x 1GB files and 2 x 100MB files. And strange thing, in normal mode (folder ouside of webroot), no buckets3 solution, works the 2 100mb file fine but doesn't work 1GB.

I think it's about serving headers from connectionTools.class.php but it's just an amateur "thinking".

It drive me crazy.

[lowmips]

Are you seeing a timeout issue? 1gb is going to take a long time to download.

[soulviasound]

With Amazon S3 I get a 1kb file, in normal mode, download doesn't start so a timeout is present there.   

[chetanmadaan]

well - i see the class ps_product_files.php downloads the remote file temporarily and then gives it out.

is there a way we can stop this and use a direct download function?

[Stephen Roberts]

I can think of 2 solutions to this problem:

1. Simply start the download through the direct link (the user never sees the link, but they CAN see it, which poses a problem for sellable downloads). Here's how I accomplished this:

In ps_order.php in the download_request() function where it checks if it's a remote file, I added the following line which starts a download for the user directly from Amazon S3.

Code Select Expand

header( 'Location: '.$datei ) ;

The advantage here is that the actual download process is handled by Amazon's servers which is faster and more reliable than most web servers. This takes a load off the web server. The obvious disadvantage here is that the direct link can easily be discovered and distributed and the number of downloads couldn't be tracked.

2. This one is a lot more complicated, but I would assume it's possible to write code to make a copy of the downloadable file in a temp directory on Amazon S3 specifically for that download ID and have the temp file deleted once the download expires. This would require a good bit of coding and could bloat the storage on Amazon S3 if there are a lot of downloads (say you sell 1000 2GB movies, that's 2000GB of storage now!).

I'm still working on this problem because this is VERY important for my project that I have security but also be able to deliver large files to the user without burdening the web server. I'm currently trying to understand if it's possible to deliver the file (chunk by chunk) to the user through the web server without the file ever being downloaded to the web server. If anyone can offer any info on this, I'd appreciate it.

[fandy]

I'm looking for someone to resolve this php problem

[Stephen Roberts]

Fandy, in my last post, I suggested (in option 2) leveraging Amazon's APIs to copy the files to a temporary location and give the user this temp URL. I've actually implemented this and am going live with the site tomorrow. The downloads are handled through VM so the max downloads and download lifetime policies are implemented. Even if the user gets the direct link to the temp file and shares it, the file will be deleted when the download expires, so this should lower the risk of piracy in digital sales. I'm interested in doing some work, so just PM me.

Stephen

[Stephen Roberts]

So after some research I discovered that there is a much better way to handle secure downloads than my previous solution of copying files to a temporary location. I'm sure other people have known about this, but apparently you can create pre-signed URLs that allow users to authenticate to Amazon S3 to access files. These pre-signed URLs also have an expiration date. This seems like a more secure way than my previous solution. I'm thinking about creating an Amazon S3 plugin for VirtueMart.