TIL S3 supports conditional writes

Check out the official AWS announcement and the more detailed S3 Conditional requests guide.

Amazon S3 adds support for conditional writes that can check for the existence of an object before creating it. (…) You can perform conditional writes using PutObject or CompleteMultipartUpload API requests in both general purpose and directory buckets.

…

To use conditional writes, you can add the HTTP if-none-match conditional header along with PutObject and CompleteMultipartUpload API requests.

Example:

aws s3api put-object \
  --bucket amzn-s3-demo-bucket \
  --key dir-1/my_images.tar.bz2 \
  --body my_images.tar.bz2 \
  --if-none-match "*"

If the object already exists, the request will fail with a 412 PreconditionFailed error. It may also fail with a 409 Conflict error if the object is deleted before the create request is processed. The creation maybe retried at this point.

Constant Work Patern

A collegue reminded me about this blog post Reliability, constant work, and a good cup of coffee from AWS Builders Library.

My favorite highlight:

This is why many of our most reliable systems use very simple, very dumb, very reliable constant work patterns. Just like coffee urns. These patterns have three key features:

• One, they don’t scale up or slow down with load or stress.
• Two, they don’t have modes, which means they do the same operations in all conditions.
• Three, if they have any variation, it’s to do less work in times of stress so they can perform better when you need them most. There’s that anti-fragility again.

Something that may not be obvious from the paragraph above is that scale up/down referes to vertical scaling and horiontal scaling is OK :) Constant work refers to capacity of the unit of the system, not the number of units:

One reason why we don’t worry about these new customer configurations is that our health checkers and aggregators use a cellular design. We’ve tested how many health checks each cell can sustain, and we always know where each health checking cell is relative to that limit. If the system starts approaching those limits, we add another health checking cell or aggregator cell, whichever is needed.

The cost of overprovisioning is often less than the engineering cost and potential reliability issues of more complex, elastic systems:

Besides being simple and robust, this approach is very cost effective. Storing a file in Amazon S3 and fetching it over and over again in a loop, even from hundreds of machines, costs far less than the engineering time and opportunity cost spent building something more complex.

Of course more dynamic systems have their place:

At the same time, there are cases where the constant work pattern doesn’t fit quite as well. If you’re running a large website that requires 100 web servers at peak, you could choose to always run 100 web servers. This certainly reduces a source of variance in the system, and is in the spirit of the constant work design pattern, but it’s also wasteful. For web servers, scaling elastically can be a better fit because the savings are large. It’s not unusual to require half as many web servers off peak time as during the peak. Because that scaling happens day in and day out, the overall system can still experience the dynamism regularly enough to shake out problems. The savings can be enjoyed by the customer and the planet.

I also like the author’s take on the simple design:

I’ve used the word “simple” several times in this article. The designs I’ve covered, including coffee urns, don’t have a lot of moving parts. That’s a kind of simplicity, but it’s not what I mean. Counting moving parts can be deceptive. A unicycle has fewer moving parts than a bicycle, but it’s much harder to ride. That’s not simpler. A good design has to handle many stresses and faults, and over enough time “survival of the fittest” tends to eliminate designs that have too many or too few moving parts or are not practical.