Why S3-WriteObject Kept the Upload Going When I Control-C’d it

Written by James McDonald

June 6, 2017

I would recommend just using the AWS Cli for windows instead of powershell

Uploading 650MB chunks using S3-WriteObject. Didn’t want to choke link during day so CTRL-C’d to exit the powershell script and found in the morning that the upload had happened any way.

The Amazon docs may have the answer: http://docs.aws.amazon.com/AmazonS3/latest/dev/uploadobjusingmpu.html Note what it says about multi-part uploads

“Once you initiate a multipart upload there is no expiry; you must explicitly complete or abort the multipart upload.” — Amazon Docs

 

And more:

Using multipart upload provides the following advantages:

  • You can upload parts in parallel to improve throughput.
  • Smaller part size minimizes the impact of restarting a failed upload due to a network error.
  • You can upload object parts over time. “Once you initiate a multipart upload there is no expiry; you must explicitly complete or abort the multipart upload.”
  • You can upload an object as you are creating it.

0 Comments

Submit a Comment

Your email address will not be published.

You May Also Like…

ZPL SBPL

Zebra Printer Language http://labelary.com/viewer.html The above website has a page where you can enter ZPL and it...

PHP Iterators

Just came across a Youtube talk "Iterators in PHP" by Jake Smith published in 2014 that steps through the many...

PHP array_map Multiple Arrays

array_map can take multiple arrays. I like how it starts mapping through them starting at the first element of each...