AWS DataSync

What it is

AWS DataSync is a secure, high-speed online transfer service for moving file and object data to, from, and between AWS storage services. It automates the transfer, encrypts the data, and verifies its integrity, so it arrives intact.

Supported locations:

On-premisesIn data centers or offices that you run yourself, rather than in the cloud. Hybrid architectures connect on-premises systems to AWS, for example through AWS Direct Connect or a VPN.Glossary entry: NFS (Network File System)A protocol for sharing files over a network, used mostly by Linux and Unix systems. Amazon EFS, Amazon FSx for OpenZFS, and Amazon FSx for NetApp ONTAP provide NFS file systems.Glossary entry, SMB (Server Message Block)The file-sharing protocol used by Windows. Amazon FSx for Windows File Server and Amazon FSx for NetApp ONTAP provide SMB shares, and Amazon S3 File Gateway can present S3 buckets as SMB shares.Glossary entry, HDFS, and self-managed object storage.
AWS: Amazon S3, Amazon EFS, and Amazon FSx (Windows File Server, Lustre, OpenZFS, and NetApp ONTAP).
Other clouds: for example Google Cloud Storage, Azure Blob Storage, and Azure Files.

Choose DataSync when a question asks to migrate file shares or object data into AWS over the network, replicate data on a schedule (for example to a standby file system), or archive cold data directly into S3 Glacier storage classes.

Concepts

Location: a source or destination, such as an NFS share or an S3 bucket.
Task: what to transfer, from which location to which, and with which options.
Agent: a VM (Virtual machine)A software computer that runs its own operating system on shared physical hardware, kept apart from other VMs by a hypervisor. An EC2 instance is a VM, except bare metal instances.Glossary entry appliance (VMware ESXi, KVM, or Hyper-V) that reads from and writes to your on-premises or other self-managed storage. Transfers between AWS storage services don't need an agent, and neither do transfers from other clouds to S3 in Enhanced mode.
Task modes:
- Basic.
- Enhanced: virtually unlimited numbers of files or objects, with higher performance. Enhanced mode has its own agent type and extra charges. It doesn't support every location: transfers to or from FSx for Windows File Server need Basic mode.
- You choose the mode when you create the task and can't change it later.
Scheduling: run tasks on a schedule for recurring syncs, and pause or check schedules.
Options: filter what to copy, handle metadata, and choose how to verify data (DataSync always checksums during transfer and can verify again afterward).
Metadata: DataSync can preserve file metadata, such as POSIX ownership and permissions, timestamps, and for SMB the owner and NTFS ACL (Access control list)A list attached to a resource that names who may access it and what they may do. VPC subnets have network ACLs; Amazon S3 buckets also support ACLs, but new buckets disable them in favor of policies.Glossary entry (DACLs and SACLs), for example from a Windows file server to FSx for Windows File Server.
Incremental runs: by default, after the first full copy, each run transfers only the data and metadata that changed, which suits a final pass before cutover.

Security, resilience, performance, and cost

Security

Data is encrypted in transit, and stored encrypted at the destination.
For private transfers, create a VPC service endpoint (an interface VPC endpointA private connection from a VPC to an AWS service or another endpoint service without crossing the internet. Gateway endpoints, for Amazon S3 and DynamoDB, are route table targets; interface endpoints are network interfaces that use AWS PrivateLink.Glossary entry for DataSync, through AWS PrivateLink) and activate the agent with it, with your network extended to the VPC over Direct Connect or Site-to-Site VPN (Virtual private network)An encrypted tunnel over a public network, such as the internet, that joins networks or devices as if they were directly connected. AWS Site-to-Site VPN connects networks; AWS Client VPN connects individual users.Glossary entry, so data doesn't cross the public internet. An S3 gateway endpoint doesn't help: gateway endpoints can't be reached from on premises. FIPS (Federal Information Processing Standards)US government standards for computer systems. FIPS 140-3, and the earlier FIPS 140-2, set security requirements for cryptographic modules, such as the hardware security modules behind AWS KMS and AWS CloudHSM.Glossary entry endpoints are also available.
IAM roles control access to S3 and other destinations.

Resilience and performance

DataSync manages retries and parallelism. Use Enhanced mode for very large datasets, and Direct Connect for consistent bandwidth.
By default, a task uses all available bandwidth. On a shared link, set a bandwidth limit on the task; you can change it while a task execution is running, for example lower in business hours and higher at night.

Cost

Pay a flat fee per GB of data copied. Enhanced mode tasks have additional charges. You also pay for the destination storage and any requests.

Monitoring

CloudWatch metrics and logs for tasks; task reports list what was transferred, skipped, or failed verification.

DataSync or another option?

If the requirement is	Choose
Move NFS or SMB shares to S3, EFS, or FSx over the network	AWS DataSync
Recurring, verified copies between storage systems	AWS DataSync scheduled tasks
Windows shares to FSx for Windows File Server, keeping NTFS permissions	AWS DataSync (Basic mode)
Partners send files over SFTP (SSH File Transfer Protocol)A protocol for transferring files over an SSH connection, which encrypts both commands and data. AWS Transfer Family provides managed SFTP servers that store files in Amazon S3 or Amazon EFS.Glossary entry or FTP	AWS Transfer Family
On-premises apps keep using files with cloud storage behind them	AWS Storage Gateway
Data too large for the network (exam scenario)	AWS Snow Family
Migrate a database	AWS DMS

Reverse mindmap: when the question points here

Is it AWS DataSync?

Yes, when the question says
- Migrate file shares to AWS over the network leads to DataSync
- NFS or SMB to S3, EFS, or FSx leads to DataSync with an agent
- Copy between EFS, FSx, and S3 leads to DataSync, no agent
- Recurring sync with verification leads to DataSync scheduled task
- Archive cold data to Glacier leads to DataSync to an S3 Glacier class
- Move data from another cloud leads to DataSync
Make it private and fast
- Avoid the public internet leads to VPC service endpoint over Direct Connect or VPN
- Billions of files leads to Enhanced mode
- Transfer slows the shared link leads to Task bandwidth limit, adjustable while running
- Keep NTFS permissions leads to Copy ownership and ACLs
- Final pass before cutover leads to Run the task again; only changes copy
No, when the question says
- Partners use SFTP leads to Transfer Family
- On-premises apps need ongoing access to cloud files leads to Storage Gateway

Key takeaways

DataSync transfers files and objects online, with encryption and integrity verification.
It supports NFS, SMB, HDFS, object storage, other clouds, S3, EFS, and all FSx file systems.
An agent is needed for on-premises or self-managed storage, not between AWS services.
Use VPC service endpoints over Direct Connect or VPN for private transfers; Enhanced mode for very large datasets.
DataSync preserves metadata such as NTFS permissions, copies only changed data on later runs, and can be ThrottlingLimiting how many requests a client can make in a period. A throttled request is rejected, often with HTTP status 429 or a throttling error, and should be retried later with backoff.Glossary entry with a bandwidth limit.
Pay per GB transferred.