Episode 7: S3 Deep Dive - Storage Classes, Versioning & Security | SAA-C03 Exam Prep

Hey everyone, Balu here, welcome back to Tech Talk with Balu, your complete guide to mastering the AWS Solutions Architect Associate Exam. We are on Episode 7 and today we are diving deep into Amazon S3, that simple storage service. This is one of the most fundamental services in AWS and it's absolutely critical for the exam. Now here's why S3 matters. It's one of the main building blocks of AWS. Amazon advertises it as infinitely scaling storage and they're not exaggerating. Thousands of companies use S3 as the backbone of their infrastructure. Think about it. Netflix stores all their video content in S3. NASDAQ stores seven years of financial data worth in S3 glacier. Airbnb hosts millions of property images in S3's everywhere. On the exam, S3 appears in almost every scenario. You need to know when to use which storage class, how to secure buckets, how versioning and replication work and how to optimize costs with life cycle policies. Today we are covering the complete S3 picture. That's what S3 is and how it works. Buckets and objects, fundamentals, all the seven storage classes and when to use each of them. Versioning for data protection, application across regions, cycle policies for cost optimization, security that's bucket policies, encryption and access control. Performance optimization and of course exam traps and real world scenarios. By the end of this episode, you will understand S3 deeply enough to answer any question and design production storage architectures. So let's dive straight in. All right, let's start with the basics. What is Amazon S3? So here's the simple definition. Amazon S3 is object storage, used to files which we call objects in containers which we call buckets. Think of it like having unlimited hard drive space in the cloud. But it's way more than that. S3 is designed for 11 nines of durability. It's 99.9979. Basically that's the person of durability that it offers. If you store 10 million objects in S3, you might lose one object once every 10,000 years. Incredibly reliable. Now, what are the key characteristics you need to know? First, it's infinitely scaling. Start with one file, scale to petabytes. S3 handles it automatically. You never run out of space. Second, even though S3 looks like a global service in the console, it's actually a regional service. Buckets are created in specific regions. When you create a bucket, you choose which region it lives in. So what are the use cases of S3? S3 powers an incredible range of applications. You've got backup and storage. That's companies, backup, database, dumps, file archives. S3 is the default backup destination. Then there is disaster recovery. You store critical data in S3 in a different region if your primary site fails data is safe in S3. The archive is huge, long-term storage for compliance, financial records, medical records, legal documents, store them for years at very low cost. Static website hosting. Host the entire website directly from S3, HTML, CSS, JavaScript, images, serve it all from S3. Data lakes are another big one. Store massive amounts of raw data for analytics. Run big data analysis tools like Athena and EMR directly on S3 data. Then there is application hosting. Store application assets that is images, videos, downloads, and user uploads. Media hosting for videos streaming platforms. Video streaming platforms store content in S3. Cloud front delivers it globally. And then software delivery. That's to distribute software packages, update, and patches. S3 handles the scale. Now let's talk about buckets, the containers. A bucket is like a top-level folder. You create buckets and put objects in them. Here's a critical requirement though. They are globally unique names. Every S3 bucket name must be globally unique across all the AWS accounts worldwide. Not just unique in your region, globally unique across the entire planet. And why is that? Because the bucket names are part of the URL. If I create a bucket called myappdata, nobody else can ever create a bucket with that exact name. Now what are the bucket naming rules? It needs to be three to 63 characters long. It needs to be lowercase letters, numbers, hyphens only. No uppercase or no underscores. It must start with lowercase letter on numbers. Cannot be in an IP address format. And it cannot start with XN hyphen hyphen and cannot end with hyphen S3 alias. Now let me give you some examples. A good name would be my company backups 2024 or prod user uploads or another one would be analytics data lake. Bad names would be my bucket in uppercase or my underscore bucket or an IP192.168.1.1.1. That's also forbidden. Now where do buckets live regionally? When you create a bucket, use specify the region. It could be US East 1, that's not Virginia. US one that's Ireland or AP Southeast one Singapore. Data stays in that region unless you explicitly replicate it elsewhere. This is important for compliance that's GDPR, data residency requirements, etc. Now let's move on to objects, the actual files. An object is a file that could be an image, video, document, back-of-file, anything. Now what's an object key? The key is the full path to the object. Examples could be S3/my bucket/photo.jpg. Now here is an important concept. S3 doesn't actually have folders, there's no directory structure. It's all just keys with slashes in them. The key, user/john/profile.jpg, is just a string. It looks like folders and the console displays it like folders, but internally it's a flat structure. This matters for programming and for understanding how prefix-based permissions work. Now what's the object value? The value is the actual file content, the data. And what about the object size limits? The maximum object size, loud, is 5TB. For files larger than 5GB, you must use multi-part upload. It's recommended to use multi-part upload for files anyway over 100 MB. Now objects also have metadata. These are key value pairs associated with the object, the metadata that's set by S3 and user metadata that's set by you. Examples are like content type, image/jpeg, slash modified date, and the version of the app. You can also use tags after 10 tags are allowed per object. This is normally useful for organization, security policies, and lifecycle rules. An example would be department finance, project migration 2024, and classification confidential. And then finally there is the version ID. If versioning is enabled on the bucket, each object has a version ID. We'll cover this in detail later. For the exam, here is what you need to know. Know that buckets have globally unique names, are regional, and objects have keys. That's the full path, values, which is the data, metadata, and optional tags, and version IDs. Now onto the next section, S3 storage classes. This is one of the most tested topics on the exam. S3 has seven storage classes, each optimized for different use cases. Let me walk through each one when to use it and the cost trade-offs. Let me start with storage class number one, S3 standard, which is the general purpose class. This is the default and the most commonly used. Now what are its characteristics? Its durability is 11.9, like we've discussed before. The availability is 99.99%, that's 53 minutes of downtime per year maximum. It's stored across at least three availability zones. It provides you low latency, that's high throughput, and it can sustain two concurrent facility failures. Now what are the use cases for S3? It's normally used for frequently accessed data, big data analytics, mobile and gaming applications, content distribution, and website assets. Bottom line on when to use it, it's a default choice for active data. In terms of cost, it's got the highest storage cost, but no retrieval fees. Storage class number two is S3 intelligent tiering. This is the set it and forget adoption. So how does it work? S3 automatically moves objects between tiers based on access patterns. There are five tiers. Number one, frequent access tier. That's the default tier for new objects. Then there is infrequent access tier. That's objects not access for 30 days move here automatically. Then there is the archive instant access. That's objects not access for 90 days. Then there is the archive access. That's optional. Objects not access for 90 to 700 plus days move in here. And then finally there is the deep archive access. That's also optional. Objects not access for about 180 to 700 plus days move in here. Now here's the magic. You don't configure anything. S3 monitors access patterns and moves objects automatically. If an object in the infrequent tier is suddenly access frequent again. S3 moves it back to frequent here. In terms of cost, there's a small monthly monitoring fee per object. That's about 0.0025 dollars per thousand objects. There's no retrieval charges for moving between tiers. So what's the bottom line on when to use it? This is normally used when access patterns are unpredictable or change over time. It's perfect for data lakes when you don't know which data will be hot or cold. Now, for an exam scenario, a company has data with unprecedented predictable access patterns. They want to minimize costs without manual intervention, which storage class would they use? The answer would be S3, intelligent tiering. Now let's look at storage class 3. That's S3 standard infrequent access or standard IA. This is normally used for data access less frequently but needs rapid access when required. So what are its characteristics? Durability is 11/9, Availability is 99.9%, it's stored across at least 3 A's. It's lower storage cost than standard and retrieval fee applies. The minimum storage duration is 30 days and the minimum billable object size is 128 KB. What does this mean? Even if you delete an object after 15 days, you're charged for 30 days. Even if your object is 10 KB, you're still charged for 128 KB. So what are the use cases for S3? Disaster recovery backups, long lived data access monthly or quarterly and secondary backups. Bottom line went to use it, data accessed a few times per month not daily. Now onto number 4, which is the one zone IA. This is like the standard IA but stored in only one availability zone. What are its characteristics? Durability is 11/9 within that one A's. Availability is 99.5%, stored in single A's. If that A's is destroyed, data is lost. It's lower cost than standard IA. The minimum storage is 30 days and the minimum billable object size is 128 KB. So what are the use cases for this S3? Secondary backup copies, you have data elsewhere. Data you can easily recreate for thumbnail images that can be regenerated from originals. Bottom line went to use it when you can accept the risk of asifailure and cost savings matter more than redundancy. Now an exam trap. Store critical production data with high availability. Do not use one zone IA. Use standard or standard IA. Store reproducible thumbnails. One zone IA is perfect. So let's look at storage class 5 now. S3 Glacier Instant Retrieval. This is for archive data that's rarely accessed but needs millisecond retrieval when accessed. millisecond retrieval time like standard. Over storage cost and standard IA, but there's a retrieval fee per GB. The minimum storage duration is 90 days. Now what are the use cases for this S3? Medical images access to 1/4 regulatory archives that might be audited or historical records that's rarely accessed. Bottom line went to use it. Data access to 1/4 or less. But when access needs to be immediately available. Number 6 is S3 Glacier Flexible Retrieval which used to just be called S3 Glacier before. This is for archive data where retrieval can wait for hours. You've got three retrieval options. Number 1 is expedited. That's about 1 to 5 minutes. It's cost money. Number 2 is standard. That's about 3 to 5 hours of retrieval time with moderate cost. And the number 3 is bulk which is about 5 to 12 hours which is free. For long term backups, for compliance archives and data that doesn't need immediate access. Bottom line of when to use it is when you have archival data where waiting a few hours for retrieval is totally acceptable. Now onto a real world example. Company policy requires keeping customer records for 7 years. After ear one, they're almost never accessed. Move them to glacier flexible retrieval. If you ever need them, that's for audit or legal request. You can retrieve in about 3 to 5 hours. And finally, let's look at storage class 7. That's S3 Glacier Deep Archive. This is the lowest cost storage in AWS. For long term archives. You've got three retrieval options here. That's standard which takes about 12 hours and then there is bulk which takes about 48 hours. The minimum storage duration here is 180 days. Now in terms of cost, about a dollar per TB per month. So what are its use cases? It's used for regulatory compliance requiring 10 plus ear retention or historical data for analysis where you can wait about 12 to 48 hours for retrieval. And another use case would be for tape replacement for companies moving from physical tapes. A real world example. A stack stores 7 years of financial data in S3 Glacier Deep Archive. They're required to keep it but almost never access it. Cost savings are massive. Bottom line moment to use it. Data access to once or twice a year maximum and retrieval can wait about 12 to 48 hours. So let's look at the comparison between these storage classes. This is the exam decision tree I want you to remember. Think about how often the data is accessed. If it's every day per week, then use S3 standard. If it's every month, then use S3 standard IA. If it's unpredictable, then use S3 intelligent hearing. If it's quarterly, then use S3 Glacier instant retrieval. If it's once a year, it will be S3 Glacier Flexible retrieval. And if it's almost never use S3 Glacier Deep Archive. Then think about if you can lose the data if an easy fails. If you can't, then use multi-AZ classes that will be standard standard IA or Glacier. If it's okay to lose the data and it's reproducible, then you can use S3 one zone IA. The next thing you want to think about is how fast do you need data to be retrieved? If it's milliseconds, then you want to go for standard or intelligent hearing or glacier instant. If it's hours, then go for glacier flexible. If it's okay to wait about 12 to 48 hours, then your answer would be Glacier Deep Archive. Now onto an exam trap. Minimum storage duration. Question. A company stores objects in S3 standard IA and deletes them after 15 days. What happens? The answer is that they are charged for 30 days of storage. That's the minimum, even though they deleted after 15. This applies to standard IA, which is 30 days minimum, one zone IA, again 30 days minimum. Glacier instance that's 90 days minimum. Glacier flexible that's 90 days minimum as well or Glacier Deep Archive that's 180 days minimum. On to an exam trap, minimum object size. Even tiny objects are built at minimum size remember that. Standard IA and one zone IA has got 128 KB minimum. Glacier classes have a 40 KB minimum. A 10 KB file in standard IA is built as 128 KB. Keep that in mind. Now moving on to the next section, versioning. Versioning is a critical S3 feature for data protection. So what exactly is versioning? Versioning keeps multiple versions of an object in the same bucket. So how does this work? You enable versioning at the bucket level, not per object. Once enabled, first upload of file.txt gets version ID V1. Second upload of file.txt gets version ID V2. The third upload gets V3 and so on. All versions are kept. Now why would you use versioning? First, protection against unintended deletes. Someone accidentally deletes an important file. With versioning, the file isn't actually deleted. S3 just adds a delete marker to it. You can restore the file by removing the delete marker. Second, for easy rollback. You upload a new version of the application.js that has a bug. With versioning, you can instantly revert to the previous version. Just restore V2 and make it the current version. And the third case is for compliance. Some regulations require you keeping all the version of the documents. Versioning satisfies this automatically. Now let me explain how delete works when versioning is enabled. With the regular delete, when you delete an object, S3 doesn't actually delete it. It adds a delete marker, as we said, a special version. The object appears deleted, but all the versions still exist. Now to permanently delete, you must specify the version ID. If you delete with the version ID, then remember that specific version is permanently deleted. Other versions still remain. How do you restore a deleted object? To remove the delete marker and the latest version, what you would want to do is to remove the delete marker and the latest version becomes accessible again. A bucket can be in one of the three versioning states. There can be in number one, unversion, that's a default, that is, no versions are kept. Two, versioning enabled, that's all new uploads get version IDs. Number three, versioning suspended. No new versions, but existing versions are kept. An important point to remember is that you cannot completely turn off versioning once enabled. You can only suspend it. Now what about objects uploaded before versioning was enabled. Files uploaded before enabling versioning have version ID as null. And now here's a cost consideration. Every version counts as a separate object for storage costs. That is, if you have a one GB file with 10 versions, you are storing 10 GB total. This is why life cycle policies, which will cover in the next section, are important. You can automatically delete old versions. So for an exam scenario, a company needs to protect against accidental deletions and be able to restore previous versions. What should they do? The answer is to enable estuary versioning on the bucket. An exam trap. This versioning applied retroactively to objects uploaded before it was enabled. You know, the answer is no. Only new uploads after enabling versioning gets the version IDs. Pre-existing objects have version equal to null. Now going on to the next section, replication. Estuary can automatically replicate objects between buckets. Estuary has two types of replication. First is the cross region replication or CRR for short. What it does is replicate from one region to another region example, you can replicate from US East 1 to EU West 1. The second is same region replication or SRR, where you can replicate within the same region but different buckets. Before you can use replication, there are some prerequisite to remember. Number 1, Virgining must be enabled in both the source and destination buckets. Number 2, proper IAM permissions for S3 to replicate must be present. And number 3, buckets can be in different AWS accounts. Let me now explain a bit in detail how replication works. First, it's asynchronous. Replication happens in the background, usually completes within minutes, but there is no SLA. Second, and this is critical. New objects only by default. After you enable replication, only new objects are replicated. Objects that existed before enabling replication are not automatically replicated. To replicate existing objects, you need to use S3 batch replication. That's a separate feature. So what actually gets replicated? New objects they get replicated automatically. Object metadata and tags get replicated. Object ACLs, optionally delete markers. And what is not replicated is deletions with the Virgin IDs. That protects against malicious deletes. What are the use cases for cross region replication? Compliance requirements. Regulations require data to be stored in multiple geographic locations. Lower latency access. Users in Europe and US. Store data in both US East 1 and US 1. Users get low latency access to nearest region. There's a disaster recovery. Primary data in US East 1 is replicated to US 1. If US East 1 region fails, data is safe in Europe. Cross account replication. Company acquired another company scenario. Replicate data to the acquired companies AWs account for their access. Log aggregation. Multiple applications write logs to different buckets. That's replicate all to one central logging bucket for analysis. Lie replication between environments. Procation bucket replicates to test account. Test team has real procation data to work with. And third compliance. Some regulations require storing data in multiple accounts for separation of concerns. Replication chaining. That's an exam trap question. Bucket A replicates to bucket B. Bucket B replicates to bucket C. Does an object created in bucket A? Opure in bucket C. The answer is no. Estery replication does not chain. Remember that. Objects created in A appear in B. But B does not replicate those to see. Only objects created directly in B go to C. Delete marker replication. By default, delete markers are not replicated. You can optionally enable delete marker replication. The many delete an object in the source bucket. Creating a delete marker. That delete marker replicates to the destination. Now for an exam question. A company enables CRR. They upload 100,000 existing objects then enabled replication. Then upload 10,000 new objects. How many objects are in the destination bucket? The answer is 10,000 objects only. Only new objects after enabling replication are replicated. Remember that? The 100,000 existing objects are not. Let's look at now lifecycle policies. Life cycle policies automate transitioning objects between storage classes and deleting old objects. This is critical for cost optimization. So what are life cycle rules? Rules that automatically move or delete objects based on age or other criteria are classified as lifecycle rules. There are two types of actions you can configure. First, the transition actions. That's move objects to a different storage class after X days, for instance. An example would be after 30 days move to standard IA. After 90 days move to glacier flexible retrieval. After 365 days move to glacier deep archive. Second, expiration actions. Delete objects after X days. Examples would be delete access logs after 90 days. Delete old versions after 365 days or delete incomplete multipart uploads after seven days. Let me explain how lifecycle rules actually work. You create a rule that specifies the scope, which objects that is prefix tags, entire bucket, the transitions that's moved to class X after Y days and the expiration. That's delete after Z days, for instance. Estry automatically applies these rules daily. Let me walk you through scenario one, image thumbnail management. Here's the requirement. Your application creates profile photos and thumbnails. Original photos keep forever need immediate access for 60 days, then six hours retrieval is fine. Thumbnails easily recreated only need for 60 days, then delete. And here's the solution. For original photos rule one after 60 days, transition to S3 glacier flexible retrieval. For thumbnails storage class S3 one zone IA, that is cheaper and can lose if AZ fails, rule two would be to expire that's delete after 60 days. Scenario two, regulatory compliance with versioning. A company must recover deleted objects immediately for 30 days. After 30 days recovery within 48 hours is acceptable for up to 365 days. After 365 days data can be permanently deleted. Now here's the solution, enable versioning. Current version stays immediately accessible that's S3 standard, non-current versions meaning previous versions after 30 days, transition to S3 standard IA. After 90 days transition to S3 glacier deep archive after 365 days expire that's delete. This gives 30 day immediate recovery than 48 hour recovery for up to 365 days. Now for an exam trap storage class transitions, you can only transition in one direction. Remember that, you cannot go backwards, you can't move from glacier back to standard automatically. Another one is minimum days before transition, you cannot transition to standard IA or one zone IA until at least 30 days after the object creation. You can't create a rule that says transition to standard one A after one day. S3 won't allow it. Also remember the rule of incomplete multi part upload cleanup. When you upload large files using multi part upload, if the upload fails part way through, those parts set in S3 taking up space and costing money. You want to create a lifecycle rule to automatically delete incomplete multi part uploads after seven days. A quick point about S3 analytics storage class analysis. You don't know which lifecycle rules to create, then what you want to do is enable S3 analytics. It analyzes your access patterns and recommends when to transition objects to standard IA. It reports updates daily takes 24 to 48 hours to start seeing data. Use this to build your initial lifecycle rules. For the exam, here's what you need to know. Life cycle policies are for automatic cost optimization. Transition actions move objects to cheaper storage classes and expiration actions delete objects you don't need. Now let's look at S3 security. Security is massively tested on the exam. S3 has multiple layers of security. S3 has two types of security models. One is user based and the other one is resource based. Let me explain the difference. First, user based security uses IAM policies. IAM policies attach to users, groups, or roles define which API calls they can make to the S3. An example I am policy. Allow Alice to list all the buckets, get objects from my bucket, put object into my bucket. Second, resource based security uses bucket policies. The bucket policies are adjacent policies attached to S3 buckets that define who can access the bucket and what actions they can perform on that particular bucket. An example bucket policy would be allow anyone from the IP address 203.0.103.0/24 to get objects from this bucket. So how is access actually determined? The IAM principle that's user or role can access an S3 object if the IAM policy allows it or the bucket policy allows it and there is no explicit deny anywhere. Remember this we covered this in previous episodes explicit deny always wins over any allow. We did a deep dive of these policies in our IAM episode but let's look at it again real quick. Bucket policies are adjacent documents with these elements. The version is usually 2012, 10, 17, the date. Then you have statements which is an array of statement. Each statement has an effect which is either an allow or deny. The principle defines who the policy applies to whether it's a specific A to blues account whether it's a specific IAM user or if it is everyone or anonymous, which is star. Actions are which S3 API calls S3 colon get object which is download S3 colon put object which is upload. Then there is S3 hyphen delete object there is S3 hyphen list bucket and S3 hyphen as tricks of star which is all actions. The resource field then specifies which bucket and object itself to access. Let's go through a use case number one public read access for website hosting that's make all objects in bucket publicly readable. So the Jason file would look something like this. The effect would be allow the principle would be star. The action would be S3 colon get object and there would be a resource like AR and colon AWS colon S3 colon my website bucket. This allows anyone on the internet to download. objects. This is perfect for static websites. Now let's look at another use case where you want to require an encryption on upload. That is, force all uploads to be encrypted. The JSON would look something like this. The effect would be deny. The principle would be asterisk. That's all. Action would be a street call and would object. There would be a resource to your bucket. And then there would be another feel for conditions where you can specify that you want to enable server side encryption example AES256. Now if you try to upload without encryption the upload is denied. And now to use case 3 cross account access. That is allowing another adbless account to access your bucket. Again, looking at the JSON file it would look like effect allow. The principle would be to your root account. The action would be to get object and put object and then your resource for your bucket. Now let's talk about block public access settings. These are bucket level settings that override bucket policies to prevent accidental public exposure. There are four different settings. Setting number one block new public bucket policies. Number two would be block public access granted through any bucket policies. Number three would be block new public AESILs. And number four would be block public access granted through any AESILs. Best practice here is to lead them on unless you explicitly need public access. Like for example website hosting. These settings exist because of massive data breaches for misconfigured S3 buckets. Companies accidentally made buckets public and millions of records leaked. AWS then created block public access to prevent this. Now for an exam scenario question. A bucket policy allows public access but the bucket still returns 403 code forbidden. What is wrong? The answer is block public access settings are enabled. They override the bucket policy. Now let's look at S3 access control lists or ACLs. It's a legacy feature. AESILs are the old way of controlling access. They are less flexible than bucket policies. AWS recommends not using AESILs use bucket policies instead. You might see AESILs on the exam. Know that they exist but they are not the recommended approach. Another point to note is static website hosting S3 can host static website. That's HTML, CSS or JavaScript. Enable static website hosting on a bucket. Make objects public and you get a UR like HTTP bucket name.S3 the website region. Amazon AWS.com. Now here's a common error. If you get a 403 forbidden the bucket policy doesn't allow public reads. For the exam here is what you need to know. S3 static websites are HTTP only not HTTPS. You need cloud front for HTTPS. Remember that. Now for EC2 instances accessing S3 here's the best practice. Create an IAM role with S3 permissions. Attach the role to the EC2 instance. To not put access keys on the instance we talked about this earlier. Roles provide temporary credentials to rotate automatically. Now let's talk a bit more about encryption. S3 supports encryption address and in transit. There is service ID encryption or what we call SSE. S3 encrypts objects when storing them. There are three options you can choose from. There is the SSE hyphen S3. That's S3 manages encryption keys. This is the default setting and it is free. Then there is the SSE hyphen KMS. AWS KMS manages the key. There is more control but it costs money. Then there is the SSE hyphen C. That's the customer provides the encryption keys that is you have more control over it. And then there is the client sign encryption. You encrypt your data before uploading that to S3. You manage encryption or decryption. For encryption in transit use HTTPS or TLS when uploading or downloading. AWS SDK uses HTTPS by default. So for the exam here is what you need to know. Know that you can enforce encryption with bucket policies that deny uploads without any encryption headers. Okay, onto our next section. It's three performance. S3 is incredibly fast but there are techniques to make it even faster. Let me tell you about S3's baseline performance. S3 automatically scales to high request rates. Prefix in a bucket you can achieve about 3500 put or copy or post or delete request per second or 5500 get or head request per second. Now what is a prefix? The part of the object key before the object name is called the prefix. Example object keys would be my bucket/folder1/sub1/file.jpeg. The prefix is /folder1/sub1. And to look at another one, if your object key would be my bucket/images/photo.jpeg then the prefixes/images that is just before your object name. Each prefix gets 3500 put and 5500 get per second. If you have 10 prefixes you can achieve 35000 put request per second in total and 55000 get request per second total. So an exam tip. Spreading objects across multiple prefixes increases performance. Now let's talk about multi-part upload. For large files over 100 MB you want to use multi-part upload. This is how it works. Number one, divide file into parts that's 5 MB to 5 GB per part. Number two, upload parts in parallel. And number three, S3 assembles parts into complete object. The benefits are it's faster uploads because it's parallel. You can resume for upload fails. That's just re-upload fail parts. It's pause and resume kind of functionality. And this is required for files over 5 GB. There is also S3 transfer acceleration we talked about in another episode which feeds up long distance uploads. How it works is you upload to the nearest AWS edge location. Then the edge location transfers to S3 over AWS private network which is much faster than public internet and then the object appears in the S3 bucket. So here's the use case. A user in Australia is uploading to bucket in US East one without transfer acceleration. You upload over public internet which is very slow. There is about 300 plus milliseconds of latency. Now with transfer acceleration enabled you upload to Sydney edge location which is fast. There's about only 10 millisecond latency. Then the Sydney edge location would transfer that to Virginia over the AWS network which is very fast. In terms of cost there's a small per GB fee. Only charged if transfer acceleration is faster than normal transfer. Another interesting performance feature is the S3 byte range fetches. That's only downloads specific byte ranges of an object. So use case one to speed up downloads would be a large file of one GB instead of downloading as one stream request it in parallel byte ranges. That is you could request bytes 0 to 200 MB then from 200 MB to 400 MB so and so forth. All these requests blocks that you have happen in parallel so that means the download would complete faster. Another use case is to retrieve only partial data. You only need the first one MB of 100 GB file. That's maybe you were trying to read the metadata or the header metadata there. You would request just bytes 0 to 1 MB, download 1 MB instead of the whole 100 GB of that particular file. Another use case would be better resilience. If a download fails part way through you only need to retry the fail by range not the entire file. For the example here's what you need to know. Multi part upload for large files transfer acceleration for long distances by range fetches for partial downloads or parallel downloads. Keep that in mind. Alright finally we're coming to the interesting section exam traps and gotchas. Let's cover common S3 traps. Trapp number one bucket names are globally unique. Question can two accounts have buckets with the same name? We know the answer is no bucket names are globally unique across all AWS accounts worldwide. Trapp number two buckets are regional question is S3 a global service? No even though the console shows S3 globally buckets are created in specific regions data stays in that region unless you explicitly replicated. Trapp number three storage class minimum durations question upload to standard IA delete after 10 days what's the charge? The answer is your charge for 30 days minimum storage even though you deleted that after 10 days remember standard IA and one zone IA 30 days minimum Asia classes would be 90 to 180 days minimum number four glacier retrieval times need to access archive data within five minutes which storage class should I use people think glacier flexible with expedited retrieval that's one to five minutes a better answer would be to use glacier instant retrieval that's millisecond access know the retrieval times in mind standard or standard IA would be milliseconds glacier instant would also be milliseconds glacier flexible expedited would be between one to five minutes glacier flexible standard would be between three to five hours and glacier deep archive would be between 12 to 48 hours. Trapp number five replication only applies to new objects question enable CRR on bucket with 50,000 existing objects how many replicates the answer is zero because remember it only replicates new objects after enabling CRR to replicate existing objects you want to use a three batch replication which is a totally separate feature. Trapp number six replication requires versioning question is can you enable CRR without versioning answer is no both But soz and destination buckets must have versioning enabled. Trap number 7. No replication chaining. Question. Bucket A to bucket B to bucket C. Object in A up your ZnC? Does it? No it doesn't. Replication doesn't chain. A to B works, B to C works. But A to B to C does not work. We discussed earlier. Trap number 8. Block public access overites bucket policy. The question. Policy allows public access. But you're getting 403 errors. Why is that? The answer is block public access settings override bucket policies. Check if block public access is enabled. Trap number 9. Static website is HTTP only. Question. Need HTTPS for S3 static website? How? The answer would be S3 static website hosting in HTTP only for HTTPS for CloudFront in front of S3. Trap number 10. Life cycle transitions are one way. Question is can we move objects to glacier then back to standard automatically? No you can't. Life cycle rules can only transition in one direction. That's towards cheaper storage. You can't automatically move back to more expensive classes. You can manually door restore from glacier to standard, but not with life cycle rules. Trap number 11. Cannot transition to IA classes immediately. Trap number is not allowed. Must wait minimum 30 days before transitioning to standard IA or to one zone IA. And finally Trap 12. Encryption in transit. Question is how to ensure data is encrypted while uploading to S3. Tanswers to always use HTTPS or TLS. AWS SDKs use HTTPS by default. You can enforce with bucket policy that denies non HTTPS requests. Now for some quick examples. Number one default to S3 standard unless the question specifies infrequent access or archival choose S3 standard. Tip number two. Intelligent tearing for unpredictable access. The keywords you will look out for is unpredictable or unknown access patterns or changing access then thing intelligent tearing. Tip number three. One zone IA for reproducible data keywords again you want to look for here is secondary backup can recreate thumbnails for all of that thing one zone IA. Tip number four. Glacier for compliance archives. So the keywords here would be regulatory or seven plus years rarely access data think glacier deep archive. Tip number five. Enable versioning for data protection. The keywords are protect against the leash and rollback accidental delete. Think enable versioning. Tip number six. CRR for disaster recovery. The keywords you look out for disaster recovery multiple regions compliance requirements. Think cross region replication. Tip number seven. Lifestyle policies for cost optimization. The keywords you want to think out for is reduce costs automatic all data think lifecycle policies. Tip number eight. Bucket policies for access controls. The keywords you want to think is public access cross account specific IPs think bucket policy. Tip number nine. I am rolls for easy to keywords easy to access S3. I am role attached to instance not access keys and tip number 10 finally multi part upload for large files keywords. You want to think large files over 100 MB parallel upload think multi part upload. So let me finally leave you with a decision tree for choosing storage classes because it is often confusing think how often it's accessed if it's daily or weekly then go for standard if it's monthly go for standard I. A if it's unpredictable then go for intelligent tearing if it's quarterly then it would be glacier instant if it's early then you want to choose glacier flexible and if it's almost never then you want to choose glacier deep archive. Number two think about whether you can lose the data if the easy fails if you can't then think multi easy class and if you can then that would be one zone I. Then the next thing what you want to think about is how fast you want to retrieve that data if you want to retrieve that data with milliseconds latency then think standard intelligent or glacier instant if it's hours there would be glacier flexible if it's anywhere over 12 hours then you want to go for glacier deep archive. So alright everyone that wraps up our episode 7 on Amazon S3. We covered a ton today as usual we started with S3 fundamentals that's buckets objects and the concept of object storage. We went deep on the seven storage classes and went to use each one of them that is standard for active data intelligent tearing for unpredictable access and glacier for archives. We covered versioning for data production and how delete markers work we explored replication both cross zone region for disaster recovery and same region for log aggregation we walk through lifecycle policies for automatic cost optimization transitioning objects to cheaper storage and expiring all data we dove into S3 security that's bucket policies I am policies block public access and encryption we covered performance optimization with multi part upload transfer acceleration and bite range fetches and we finished with exam traps that trip people on the test day S3 is fundamental to AWS it appears in almost every exam scenario master S3 and your well on your way to passing the exam and building real world solutions. In our next episode episode 8 we are doing something different designing storage architectures will compare EBS EFS and S3 side by side that's when do you use block storage versus file storage versus object storage will walk you through real world use cases so you know exactly which storage service to choose for any scenario. Now if this episode helped you master S3 please do consider to leave a five star rating and to follow my series and of course share this with anyone else studying for the AWS exam. So until next time keep studying keep practicing and I will see you in episode 8 this is Balu signing off bye.