| Storage Class | Key Characteristics |
|---|---|
| S3 Standard | 11 9's durability, 4 9's availability, ≥3 AZs, millisecond latency, no retrieval fees |
| S3 Intelligent-Tiering | Auto moves objects between tiers, no retrieval fees, monitoring fee $0.0025 per 1000 objects |
| S3 Standard-IA | Lower storage cost, retrieval fee, 3 9's availability, ≥3 AZs, 30-day min storage |
| S3 One Zone-IA | Single AZ, 2.5 9's availability, 20% lower cost than Standard-IA, 30-day min storage |
| S3 Glacier Instant Retrieval | Millisecond retrieval, 90-day min storage, lowest cost for instant access archive |
| S3 Glacier Flexible Retrieval | Retrieval: Expedited (1-5 min), Standard (3-5 hrs), Bulk (5-12 hrs), 90-day min storage |
| S3 Glacier Deep Archive | Lowest cost, Standard (12 hrs), Bulk (48 hrs) retrieval, 180-day min storage |
| Type | Details |
|---|---|
| SSE-S3 | S3-managed keys, AES-256, header: x-amz-server-side-encryption: AES256 |
| SSE-KMS | KMS-managed keys, audit trail via CloudTrail, header: x-amz-server-side-encryption: aws:kms |
| SSE-C | Customer-provided keys, HTTPS required, key provided in each request header |
| Client-Side | Encryption done before upload, customer manages entire encryption process |
| DSSE-KMS | Dual-layer encryption, KMS with two layers of encryption, compliance requirements |
| Feature | Details |
|---|---|
| Cross-Region Replication (CRR) | Replicate across regions, compliance, lower latency, disaster recovery |
| Same-Region Replication (SRR) | Log aggregation, live replication between prod/test, data sovereignty |
| Requirements | Versioning enabled on both buckets, IAM permissions, asynchronous replication |
| What Replicates | New objects after replication enabled, encrypted objects (SSE-S3, SSE-KMS with permissions), delete markers (optional) |
| What Doesn't Replicate | Objects before replication enabled, SSE-C encrypted, objects in Glacier/Deep Archive, deletions with version ID |
| Replication Time Control | 15-minute SLA for 99.99% of objects, metrics and notifications included |
| Batch Replication | Replicate existing objects, objects failed initial replication, on-demand replication |
| Feature | Details |
|---|---|
| Request Rate | 3,500 PUT/COPY/POST/DELETE, 5,500 GET/HEAD requests per second per prefix |
| Multipart Upload | Recommended for files >100 MB, required for files >5 GB, parallel uploads, max 10,000 parts |
| S3 Transfer Acceleration | Uses CloudFront edge locations, 50-500% faster long-distance transfers, separate endpoint |
| Byte-Range Fetches | Parallel downloads, retrieve partial object, failure resilience |
| S3 Select | SQL queries to retrieve subset of data, reduces transfer and processing, works on CSV/JSON/Parquet |
| Method | Use Case |
|---|---|
| Bucket Policies | JSON-based, cross-account access, public access, enforce encryption, conditions based on IP/VPC/tags |
| IAM Policies | User/role-based permissions, centralized management, works across all buckets |
| ACLs | Legacy, object-level access, grant basic permissions (READ, WRITE, FULL_CONTROL) |
| Pre-signed URLs | Temporary access, time-limited, inherits creator's permissions, SDK/CLI generated |
| Access Points | Dedicated access points per application, simplified permission management, VPC-restricted access |
| Block Public Access | Account or bucket level, overrides other policies, four settings for blocking public access |
| Feature | Details |
|---|---|
| Compliance Mode | Cannot be overridden by anyone including root, retention period cannot be shortened |
| Governance Mode | Special permissions can override, s3:BypassGovernanceRetention permission required |
| Retention Period | Fixed time period (days/years), protects from deletion and modification |
| Legal Hold | Indefinite protection, no expiration, s3:PutObjectLegalHold permission, independent of retention |
| Requirements | Versioning must be enabled, applied at object version level, cannot be disabled once enabled |
| Feature | Details |
|---|---|
| S3 Inventory | Scheduled reports (daily/weekly), CSV/ORC/Parquet format, metadata about objects, encryption status audit |
| S3 Batch Operations | Bulk operations on billions of objects, invoke Lambda, restore from Glacier, ACL/tag modifications |
| S3 Analytics | Storage class analysis, recommendations for lifecycle policies, CSV export to S3, updated daily |
| S3 Storage Lens | Organization-wide visibility, 29+ metrics, free and advanced tiers, recommendations dashboard |
| Requester Pays | Requester pays transfer costs, bucket owner pays storage, requester must be authenticated |
| Type | Specifications |
|---|---|
| gp3 (General Purpose SSD) | 3,000-16,000 IOPS, 125-1,000 MB/s throughput, 1 GB-16 TB, cost-effective baseline performance |
| gp2 (General Purpose SSD) | 3 IOPS/GB baseline (min 100, max 16,000), burst to 3,000 IOPS, 1 GB-16 TB, throughput 128-250 MB/s |
| io2 Block Express | Up to 256,000 IOPS, 4,000 MB/s throughput, 4 GB-64 TB, 1000:1 IOPS:GB ratio, sub-millisecond latency |
| io2 (Provisioned IOPS SSD) | Up to 64,000 IOPS, 1,000 MB/s throughput, 4 GB-16 TB, 500:1 IOPS:GB ratio, 99.999% durability |
| io1 (Provisioned IOPS SSD) | Up to 64,000 IOPS, 1,000 MB/s throughput, 4 GB-16 TB, 50:1 IOPS:GB ratio, 99.9% durability |
| st1 (Throughput Optimized HDD) | 500 IOPS, 500 MB/s throughput, 125 GB-16 TB, big data/log processing, cannot be boot volume |
| sc1 (Cold HDD) | 250 IOPS, 250 MB/s throughput, 125 GB-16 TB, lowest cost, infrequent access, cannot be boot volume |
| Feature | Details |
|---|---|
| Snapshot Storage | Stored in S3 (not visible), incremental, first snapshot is full copy, regional resource |
| Snapshot Copy | Copy across regions, copy and encrypt, disaster recovery, migration |
| Fast Snapshot Restore | Instant initialization, no latency on first use, charged per DSU-hour per AZ, max 50 FSR snapshots |
| Snapshot Archive | 75% cost reduction, 24-72 hours restore time, minimum 90-day archive period |
| Recycle Bin | Protect against accidental deletion, 1 day to 1 year retention, recover deleted snapshots |
| EBS Direct APIs | Read snapshots directly, create snapshots incrementally, no EC2 instance needed |
| Metric | Details |
|---|---|
| IOPS | Input/Output operations per second, smaller block sizes = more IOPS consumed |
| Throughput | MB/s, larger block sizes = more throughput consumed, independent configuration in gp3 |
| Queue Depth | Number of pending I/O requests, higher queue depth = better performance for high IOPS volumes |
| CloudWatch Metrics | VolumeReadBytes, VolumeWriteBytes, VolumeReadOps, VolumeWriteOps, BurstBalance (gp2/st1/sc1) |
| I/O Credits (gp2) | 5.4 million I/O credits at creation, replenish at baseline rate, depleted during bursts |
| Mode | Details |
|---|---|
| General Purpose | Latency-sensitive workloads (web servers, CMS), lowest latency per operation, default mode |
| Max I/O | Big data, media processing, higher aggregate throughput and IOPS, higher latency per operation, cannot change after creation |
| Mode | Details |
|---|---|
| Bursting | Scales with file system size, 50 MB/s per TB, bursts to 100 MB/s, uses credit system |
| Provisioned | Set throughput independent of storage size, pay for provisioned throughput, can exceed bursting limits |
| Elastic | Auto-scales throughput up/down based on workload, up to 3 GB/s reads, 1 GB/s writes, pay for actual throughput used |
| Class | Details |
|---|---|
| Standard | Frequently accessed files, multi-AZ, highest availability and durability |
| Infrequent Access (IA) | Lower storage cost, retrieval fee, lifecycle policy auto-moves files, up to 92% cost savings |
| One Zone | Single AZ, 47% cost savings over Standard, less availability |
| One Zone-IA | Single AZ + IA pricing, 95% cost savings over Standard, development/backup use cases |
| Feature | Details |
|---|---|
| Lifecycle Management | Auto-move files to IA after N days (7, 14, 30, 60, 90 days), based on last access time |
| Access Points | Application-specific entry points, enforce user identity, root directory per application, simplified access management |
| Mount Targets | One per AZ, multiple EC2 instances share single mount target, requires ENI in VPC subnet |
| Encryption | At rest (KMS), in transit (TLS), must enable at creation for at-rest encryption |
| Backup | Integrates with AWS Backup, point-in-time recovery, incremental backups, cross-region backup |
| Feature | EFS |
|---|---|
| Attachment | Multi-attach across hundreds of instances and AZs, shared file system |
| Availability | Multi-AZ by default, regional service, higher availability |
| Cost | 3x more expensive than gp2, pay-per-use, no capacity planning |
| Use Cases | Content management, web serving, data sharing, WordPress, shared application state |
| Protocol | NFSv4.1, POSIX-compliant, Linux-only |
| Feature | Details |
|---|---|
| Protocol | SMB protocol, Windows-native shared file system, NTFS file system |
| Integration | Active Directory integration, user quotas, access control lists (ACLs) |
| Performance | SSD: 100s MB/s, millions of IOPS, sub-millisecond latency; HDD: 10s MB/s, 100,000s IOPS |
| Storage | SSD (hot data), HDD (warm data), 32 GB to 64 TB capacity |
| Deployment | Single-AZ or Multi-AZ, automatic failover, daily backups to S3 |
| Access | VPC, VPN, Direct Connect, on-premises via DFS Namespaces |
| Deduplication | Data deduplication enabled, saves 50-60% storage costs |
| Feature | Details |
|---|---|
| Use Cases | HPC, machine learning, video processing, financial modeling, genomics |
| Performance | 100s GB/s throughput, millions of IOPS, sub-millisecond latency |
| S3 Integration | Read/write to S3 as file system, output results back to S3 |
| Deployment Types | Scratch (temporary, no replication, 200 MB/s per TB); Persistent (long-term, auto-replicated in AZ, 50/100/200 MB/s per TB) |
| Metadata | Lazy loading from S3, metadata stored on Lustre, instant file visibility |
| File System | POSIX-compliant, Linux-only, parallel distributed file system |
| FSx Type | Best For |
|---|---|
| Windows File Server | Windows applications, Active Directory, SMB, user quotas, NTFS |
| Lustre | HPC, ML, compute-intensive workloads, S3 integration, highest performance |
| NetApp ONTAP | NetApp migration, multi-protocol (NFS/SMB/iSCSI), enterprise features |
| OpenZFS | ZFS migration, NFS protocol, point-in-time cloning, Linux workloads |
| Type | Details |
|---|---|
| File Gateway (S3 File) | NFS/SMB protocol, files stored as objects in S3, local cache for low-latency, integrates with S3 lifecycle/versioning/encryption |
| FSx File Gateway | Native access to FSx for Windows File Server, local cache, Windows-native SMB, Active Directory integration |
| Volume Gateway (Cached) | iSCSI protocol, primary data in S3, frequently accessed data cached locally, low-latency access to hot data |
| Volume Gateway (Stored) | Entire dataset on-premises, asynchronous backup to S3 as EBS snapshots, low-latency access to entire dataset |
| Tape Gateway (VTL) | Virtual Tape Library, iSCSI protocol, backup to S3/Glacier using existing tape software, supports Veeam, Veritas, CommVault |
| Feature | Cached Volumes |
|---|---|
| Storage | Primary data in S3, cache is 1 TB to 32 TB per volume, up to 32 volumes (1 PB total) |
| Snapshots | Point-in-time EBS snapshots stored in S3, incremental backups, can create EBS volumes from snapshots |
| Use Case | Primary data in AWS, on-premises applications need low-latency access to hot data |
| Deployment | Details |
|---|---|
| On-Premises VM | VMware ESXi, Microsoft Hyper-V, Linux KVM, hardware appliance option available |
| Hardware Appliance | Physical device from AWS, pre-configured, 1U/2U rackmount, for locations without virtualization |
| EC2 Instance | Deploy gateway as EC2 instance, disaster recovery, cloud-to-cloud migration |
| Protocol | Details |
|---|---|
| SFTP (SSH File Transfer) | Port 22, secure file transfer over SSH, public key authentication |
| FTPS (FTP with SSL) | Port 21 (control), port 989/990 (data), FTP over TLS/SSL |
| FTP | Port 21, unencrypted, VPC only (not public internet), legacy application support |
| AS2 (Applicability Statement 2) | B2B data exchange, EDI messages, HTTP/HTTPS-based, message encryption and signing |
| Method | Details |
|---|---|
| Service Managed | Store user credentials within Transfer Family, SSH keys stored in service |
| Custom Identity Provider | Integrate with Lambda, authenticate against external systems (LDAP, Active Directory, custom database) |
| AWS Directory Service | Microsoft AD integration, use existing corporate credentials |
| Type | Details |
|---|---|
| Public | Internet-accessible, AWS-provided hostname, cannot whitelist IP addresses |
| VPC (Internal) | Private IP in VPC, access via VPN/Direct Connect, internal applications |
| VPC (Internet-Facing) | Elastic IP attached, fixed IP addresses for whitelisting, internet-accessible via VPC |
| Location Type | Details |
|---|---|
| Sources | On-premises (NFS, SMB), AWS Snowcone, Amazon S3, Amazon EFS, Amazon FSx (Windows, Lustre, NetApp ONTAP, OpenZFS) |
| Destinations | Amazon S3 (all classes including Glacier), Amazon EFS, Amazon FSx (all types) |
| Cross-Region | Transfer between AWS regions, cross-account transfer supported |
| Feature | Details |
|---|---|
| Bandwidth Throttling | Limit bandwidth used by DataSync, prevent network saturation |
| Data Validation | Verify data integrity using checksums, compare source and destination |
| Task Execution | Manual, scheduled, or event-driven, integrates with EventBridge |
| Filtering | Include/exclude patterns, transfer specific files or folders |
| Logging | CloudWatch Logs for transfer details, CloudWatch Metrics for monitoring |
| Device | Storage |
|---|---|
| Snowcone | 8 TB HDD or 14 TB SSD, 2 CPUs, 4 GB RAM, USB-C power or battery, weighs 4.5 lbs |
| Snowball Edge Storage Optimized | 80 TB usable (100 TB raw), 40 vCPUs, 80 GB RAM, 1 TB SSD for block volumes, 10 GbE/25 GbE networking |
| Snowball Edge Compute Optimized | 28 TB NVMe usable (42 TB raw), 104 vCPUs, 416 GB RAM, optional GPU, 100 GbE networking |
| Snowmobile | 100 PB per truck, exabyte-scale transfer, GPS tracking, 24/7 security, climate controlled, requires multiple-week on-site deployment |
| Feature | Details |
|---|---|
| EC2 Instances | Run EC2 instances on Snowball Edge, AMI-based deployment, process data at edge |
| Lambda Functions | Run Lambda at edge, process data locally before transfer to AWS |
| Use Cases | Pre-processing data, machine learning at edge, remote/disconnected locations (ships, mining, military) |
| Clustering | 5-10 Snowball Edge devices can cluster together, increased durability and storage |
| Stage | Details |
|---|---|
| Order | Request device via AWS Console, shipped in days |
| Load Data | Connect device, install OpsHub, copy data, encryption enabled by default |
| Ship Back | E-ink shipping label auto-updates, track via SNS/text/console |
| Import | AWS loads data to S3, device wiped using NIST standards |
| Pricing | On-demand (pay per use) or 1/3-year commitment, data transfer into S3 free, standard S3 pricing applies |
| Requirement | Solution |
|---|---|
| Single-instance boot volume | EBS gp3 (cost-effective) or io2 (performance) |
| Shared access across instances | EBS Multi-Attach (io1/io2) or EFS |
| High IOPS database (>64K IOPS) | EBS io2 Block Express (up to 256K IOPS) or instance store |
| Temporary high-performance storage | Instance store (ephemeral, highest IOPS) |
| Big data sequential I/O | EBS st1 (throughput optimized HDD) |
| Infrequent access, lowest cost | EBS sc1 (cold HDD) |
| Requirement | Solution |
|---|---|
| Linux shared file system | EFS (NFSv4.1, POSIX-compliant) |
| Windows shared file system | FSx for Windows File Server (SMB, Active Directory) |
| High-performance computing | FSx for Lustre (sub-ms latency, millions IOPS) |
| NetApp migration | FSx for NetApp ONTAP (multi-protocol) |
| ZFS migration | FSx for OpenZFS (NFS, snapshots, cloning) |
| On-premises NFS/SMB to S3 | File Gateway (S3 backend with local cache) |
| Requirement | Solution |
|---|---|
| Frequent access | S3 Standard |
| Unknown access patterns | S3 Intelligent-Tiering |
| Infrequent access, millisecond retrieval | S3 Standard-IA or S3 One Zone-IA |
| Archive, instant retrieval | S3 Glacier Instant Retrieval |
| Archive, retrieval in hours | S3 Glacier Flexible Retrieval |
| Long-term archive, lowest cost | S3 Glacier Deep Archive |
| Pattern | Solution |
|---|---|
| On-premises file shares to S3 | File Gateway (NFS/SMB to S3) |
| On-premises block volumes to S3 | Volume Gateway cached/stored volumes |
| On-premises tape backup to S3/Glacier | Tape Gateway (VTL to S3/Glacier) |
| Large-scale data migration | DataSync (scheduled sync) or Snow Family (offline transfer) |
| SFTP/FTPS to S3 | Transfer Family (managed FTP servers) |
| Low-latency access to FSx for Windows | FSx File Gateway |
| Scenario | Recommended Solution |
|---|---|
| Good network, ongoing sync | DataSync (scheduled replication) |
| Limited network, one-time transfer <10>10> | Snowcone |
| Limited network, one-time transfer 10-80 TB | Snowball Edge |
| Limited network, one-time transfer >80 TB | Multiple Snowball Edge or Snowmobile (>10 PB) |
| SFTP/FTP access to S3 | Transfer Family |
| Continuous hybrid access | Storage Gateway (File/Volume/Tape) |