Benjamin Wood’s Post

“Imagine a hard drive head as a 747 flying over a grassy field at 75 miles per hour. The air gap between the bottom of the plane and the top of the grass is two sheets of paper.” Dive deep with this blog post on storage tech, Amazon S3, and Amazon software practices. I loved the mind bending detail and real world experience. I am feeling lucky to live in a time and place where I can wander my way into this level of free expertise. https://lnkd.in/eNbcyT_J

Interesting design to vend access to S3 Objects using presigned URLs! This post adds the machinery to ensure the application vends the presigned URL only once.

New tricks for the oldest AWS service! Congratulations to AWS SQS for fighting long tail latency with a 17% latency reduction at P90. The architecture diagram is a healthy reminder that SQS has a strong resilience guarantee too.

Resilience requirements must originate from business objectives. Resilience objectives must be cost-informed decisions. What can we do to break the circle? Years ago I was negotiating a big purchase. The seller kept asking if I wanted this feature or that feature, but refused to tell me more than the surface level details before demanding a yes-no answer. I broke the circle by saying “yes”, asking for details, then backtracking my declared decision if the value didn’t add up. This was a terrible (and memorable) customer experience because I felt like I couldn’t make an informed decision without knowing how feature would change the cost or schedule. Resilience conversations between stakeholders can fall into a similar circular trap. Engineers want clarity expressed in quantified requirements like RPO and RTO. Business leaders are prepared to describe mission and cost impact, but do not understand how these requirements impact engineering effort, risk, complexity or cost. This is a challenging stalemate because cost and complexity can change dramatically to small changes to requirements. The opposite is also true; sometimes cost and complexity is neutral even when requirements become more strict. Complexity and performance change dramatically with different technologies. When a requirement is too strict to meet with one approach, it may be orders of magnitude more difficult to meet using different technologies. Example: it generally straightforward to design AWS architectures that are resilient to AZ failure, but more costly and complicated to build multi-Region architectures that can sustain loss of an Region, especially at low RPO and low RTO values. Resilience conversations can avoid frustrating circular requirements by ensuring all stakeholders understand up front the team will refine requirements through an iterative process that may occasionally backtrack. Business leaders should be prepared to explain the general need while engineers should be prepared to provide coarse cost estimates for a spectrum of options. Each stakeholder can iterate and refine to ensure together the business can make risk and cost informed decisions. AWS provides a great conversation starter. As a technical professional, this is a useful reference to build application specific materials to bring to requirements. I would encourage business leaders use this article to dive deep and learn more about the vocabulary and options available. https://lnkd.in/e96zHBqr

This looks like a good baseline or refresher ahead of the exam release. My last ML course was in graduate school just ahead of Dee learning and LLMs starting their dominating decade long run!

Is Amazon S3 Global, Regional, or Zonal? I had a conversation about building resilience to Region failure where this question arose. It is a natural question, especially for people early in their journey to the cloud. S3 is Global. S3 appears as a Global service in the AWS Console and Bucket names must be unique, even across accounts. S3 creates a URL based on the Bucket name and so Bucket names must be unique across accounts and Regions. The deeper truth is that names must be unique across an AWS Partition, but this nuance is not relevant to most people. S3 is Regional. S3’s global nature does not extend to data storage. Individual Buckets are tied to a single Region. The bytes for data stored in the Bucket ultimately resides on AWS-managed servers in multiple Availability Zones all within the same Region. S3 is Zonal. The distribution of data within a Region depends on the type or storage class of Bucket you create. Most storage classes provide Region-level resilience while a few offer lesser durability and availability in exchange for performance or cost, but the details are in the fine print. Infrequent Access One Zone lowers cost by storing data for a specific object in a single Zone, but the service may choose to store data in any of the Zones in the Region. One Zone Express is the newest storage class that pins storage to a specific Zone for higher performance. S3 can be Global by replicating data across Regions. Applications can build cross region resilience using built-in S3 features to replicate data from a source Bucket to a destination Bucket, including destination Buckets in a distant Region. We can use this replication feature as a part of resilience strategy. So, Global, Regional, or Zonal? All of the above. The details matter and AWS lets us choose the storage class that meets our needs.

Planning to migrate, build, or maintain mission critical applications on AWS? If so, you are probably thinking about resilience. Mission critical applications require the highest levels of availability. Your existing on-prem application may use servers, racks, and data centers to isolate faults from different failure scenarios. AWS brings new options. This paper details the Fault Isolation Boundaries AWS bakes into its global infrastructure (Partitions, Regions, and Availability Zones) and further explains how AWS services fit within the infrastructure. Consider this paper as a great place to start your journey into cloud resilience.