AWS Data Exchange
Subscribing to and sharing third-party data: Marketplace products and data grants, the five data set types, revisions and auto-export, security, and cost.
4 min read · Last reviewed · SAA-C03
Loading your progress
Sign in to bookmark tutorials and track what you have read.
On this page
What it is
AWS Data Exchange lets you find, subscribe to, and use data from other organizations, and share your own data with them, without building delivery infrastructure. There are two ways data changes hands:
- AWS Marketplace data products: providers list data sets in the AWS Marketplace catalog with offers that set the price and terms. You subscribe, and your account is entitled to the data.
- Data grants: any AWS account can send a data grant directly to another account, which accepts it to gain access for a set period.
Publicly available data sets from the Open Data on AWS program can be found and used too, with or without an AWS account.
How the data reaches you
The type of data set decides how you use it:
| Data set type | How a subscriber uses it |
|---|---|
| Files | Export a copy to your own S3 bucket, or download it. |
| API (Application programming interface)A defined way for one program to ask another for data or actions, such as a set of HTTPS requests and responses. Every AWS service is controlled through its API, which the console, the CLI, and the SDKs call for you.Glossary entry | Call the provider's API through endpoints that Data Exchange manages. |
| Amazon Redshift | Query the provider's live tables read-only through a datashare, with no ETL (Extract, transform, and load)Copying data from sources, cleaning and reshaping it, and loading it into a target such as a data warehouse. AWS Glue is the AWS serverless ETL service.Glossary entry to build. |
| Amazon S3 data access | Read the provider's S3 objects in place through an access point, with no copy to manage. |
| AWS Lake Formation (preview) | Query the provider's tagged databases, tables, or columns from your own Lake Formation. |
A data set changes over time through revisions. When a provider publishes a revision, subscribers receive an Amazon EventBridge event, and auto-export can copy each new revision of a Files data set to your S3 bucket automatically.
Security, resilience, performance, and cost
Security
- IAM controls who in your account can subscribe, accept grants, and export data, and the exporting role needs permission to write to the destination bucket.
- Exports to Amazon S3 can be encrypted with SSE (Server-side encryption)Encryption that a service applies when it stores your data and removes when you read it back. Amazon S3 encrypts every new object with SSE-S3 by default; SSE-KMS uses keys in AWS KMS instead.Glossary entry-S3 or SSE-KMS, set in the export job.
- Access ends when a subscription or data grant expires or is revoked; files you already exported stay with you, subject to the data subscription agreement.
- Data marked as containing protected health information (HIPAA (Health Insurance Portability and Accountability Act)A US law that protects health information. AWS customers can process protected health information on HIPAA-eligible services after accepting a Business Associate Addendum (BAA) with AWS, which is available in AWS Artifact.Glossary entry) can be exported only into an account designated as a HIPAA account.
Resilience and performance
- Data sets are Regional, and all data sets in one product or data grant must be in the same AWS RegionA separate geographic area where AWS runs its services, such as us-east-1 in Northern Virginia, made up of several Availability Zones. Most resources live in one Region; you choose Regions for latency, data residency, and the services you need.Glossary entry. Events are emitted in the provider's Region, so create EventBridge rules there or forward the events.
- Redshift datashares and S3 data access avoid copying data at all, so subscribers always read the provider's current version.
Cost
- Subscribers pay the price the provider sets in the offer, upfront or on a schedule, or pay as you go. Normal AWS charges apply for storing and analyzing the data, and for exports across Regions or to signed URLs.
- Providers pay an hourly charge for each active data grant, storage for Files data sets, and AWS Marketplace fees on subscription revenue. For S3 data access, providers can turn on Requester Pays.
Monitoring
- Amazon EventBridge events report new data sets and revisions, revoked revisions, completed or failed auto-export jobs, and provider notifications such as data updates, delays, schema changes, and deprecations. Rules can start a pipeline or send a notification.
- AWS CloudTrail records Data Exchange API calls.
Data Exchange or another way of sharing?
| If the requirement is | Choose |
|---|---|
| Subscribe to data from a third-party provider | AWS Data Exchange |
| License your own data to customers and bill them | AWS Data Exchange with AWS Marketplace |
| Share live warehouse tables with another team or account you work with | Redshift data sharing |
| Share Data lakeA central store for large amounts of structured and unstructured data kept in its original form, usually in Amazon S3, which many tools can analyze. AWS Lake Formation helps build and secure one.Glossary entry tables with fine-grained permissions inside your organization | AWS Lake Formation |
Reverse mindmap: when the question points here
- Yes, when the question says
- Subscribe to third-party data such as market or weather data leads to AWS Data Exchange
- Sell or license data to other AWS customers leads to Data Exchange with AWS Marketplace
- Give another account time-limited access to a data set leads to Data grant
- Use the data efficiently
- Copy new files to S3 automatically when published leads to Auto-export
- Query a provider's tables with no ETL leads to Redshift datashare data set
- Read a provider's S3 data without copying it leads to Amazon S3 data access
- Start a pipeline when new data arrives leads to EventBridge rule on new revisions
- No, when the question says
- Share warehouse tables within your organization leads to Redshift data sharing
- Column or row permissions on your own data lake leads to AWS Lake Formation
Key takeaways
- Data Exchange is for data from (or to) other organizations: Marketplace products or direct data grants.
- The data set type decides delivery: files to S3, APIs, Redshift datashares, S3 data access, or Lake Formation.
- New revisions trigger EventBridge events; auto-export copies them to your bucket.
- Exports can be encrypted with SSE-S3 or SSE-KMS.
Sources
- AWS Data Exchange User Guide: What is AWS Data Exchange?
- AWS Data Exchange User Guide: Data in AWS Data Exchange
- AWS Data Exchange User Guide: Jobs
- AWS Data Exchange User Guide: Exporting assets to an S3 bucket
- AWS Data Exchange User Guide: Using Open Data on AWS data sets
- AWS Data Exchange User Guide: Amazon EventBridge events
- AWS Data Exchange User Guide: Logging API calls with AWS CloudTrail
- AWS Data Exchange pricing