Amazon Redshift
The analytics data warehouse: provisioned and Serverless, loading with COPY and zero-ETL, Spectrum, concurrency scaling, data sharing, and cross-Region snapshots.
7 min read · Last reviewed · SAA-C03
Sign in to bookmark tutorials and track what you have read.
On this page
What it is
Amazon Redshift is a fully managed, petabyte-scale Data warehouseA database designed for analytics, holding cleaned, structured data from many sources, so that reports and queries across years of history run fast. Amazon Redshift is the AWS data warehouse.Glossary entry. You load data from many sources into it and run complex SQL (Structured Query Language)The standard language for querying and changing data in relational databases. Analytics services such as Amazon Athena and Amazon Redshift also use SQL.Glossary entry queries for reporting and business intelligence, with consistent performance as data and users grow. Redshift stores table data by column, which suits analytical queries that read a few columns across many rows.
A data warehouse is built for analytics (OLAP (Online analytical processing)Analyzing large amounts of data with complex queries that summarize many rows, such as sales by country and month. Data warehouses such as Amazon Redshift are built for OLAP.Glossary entry): large scans, aggregations, and joins. It is not the database behind an application's day-to-day transactions (OLTP (Online transaction processing)Handling many small, fast transactions that each read or write a few rows, such as placing orders or updating an account. Relational databases such as Amazon RDS and Amazon Aurora are built for OLTP.Glossary entry); that is the job of Amazon RDS, Amazon Aurora, or Amazon DynamoDB.
Provisioned or Serverless
- Provisioned clusters: you choose the node type and number of nodes, and pay while the cluster runs.
- Redshift Serverless: capacity is provisioned and scaled automatically, and there is no charge while the data warehouse is idle. It suits unpredictable or intermittent workloads.
Getting data in
- COPY from Amazon S3: the standard bulk load. Amazon Data Firehose uses it to deliver streaming data to Redshift.
- Zero-ETL (Extract, transform, and load)Copying data from sources, cleaning and reshaping it, and loading it into a target such as a data warehouse. AWS Glue is the AWS serverless ETL service.Glossary entry integrations: replicate data continuously from Amazon Aurora, Amazon RDS, Amazon DynamoDB, and other sources into Redshift, with no ETL pipeline to build or maintain.
Querying data in Amazon S3: Redshift Spectrum
Redshift Spectrum queries files in Amazon S3 without loading them into Redshift tables, and can join them with data in the warehouse. Several clusters can query the same S3 data without copying it. A common design keeps recent, frequently queried data in Redshift and older data in S3.
Many users at once: concurrency scaling
When concurrency scaling is on, Redshift automatically adds capacity when many queries arrive at once, for both read queries such as dashboards and common write operations. You pay for the extra capacity only while it runs queries.
Sharing data without copying
Data sharing gives other Redshift clusters and Serverless workgroups access to live data, across AWS accounts and AWS RegionA separate geographic area where AWS runs its services, such as us-east-1 in Northern Virginia, made up of several Availability Zones. Most resources live in one Region; you choose Regions for latency, data residency, and the services you need.Glossary entry, without moving or copying it. A common pattern is one cluster for ETL that shares its data with separate clusters for each BI workload, so the workloads do not slow each other down.
Security, resilience, performance, and cost
Security
- Encryption at restEncrypting data where it is stored, such as on disks, in databases, or in backups, so that it cannot be read without the key. On AWS it usually uses keys in AWS KMS.Glossary entry is on by default for the cluster and its snapshots, with an AWS KMS key: an AWS owned key unless you choose a Customer managed keyA KMS key that you create, own, and manage: you control its key policy, rotation, and deletion. AWS managed keys, in contrast, are created and managed for you by an AWS service.Glossary entry. If a customer managed key is disabled, the warehouse becomes inaccessible until the key is restored.
- Encryption in transitEncrypting data while it moves across a network, usually with TLS, so that it cannot be read or changed on the way.Glossary entry: clients connect over TLS (Transport Layer Security)The protocol that encrypts traffic between a client and a server and lets the server prove its identity with a certificate. It replaced SSL, though the older name is still common; HTTPS is HTTP over TLS.Glossary entry. To refuse unencrypted connections, set
require_ssltotruein the cluster's parameter group. - Network: clusters run in your VPC behind Security groupA stateful virtual firewall for network interfaces, such as an EC2 instance's, whose rules can only allow traffic. Return traffic for allowed connections is allowed automatically, and a rule can refer to another security group instead of IP addresses.Glossary entry. Enhanced VPC routing forces
COPYandUNLOADtraffic through the VPC, so you can use VPC endpointA private connection from a VPC to an AWS service or another endpoint service without crossing the internet. Gateway endpoints, for Amazon S3 and DynamoDB, are route table targets; interface endpoints are network interfaces that use AWS PrivateLink.Glossary entry, endpoint policies, and VPC flow logs; without it, that traffic goes through the internet. Redshift-managed VPC endpoints (AWS PrivateLink) let clients in another VPC or AWS account connect privately. - Access: Redshift uses IAM roles to read from Amazon S3 for
COPYand Spectrum, and database permissions for tables. With Lake Formation, Spectrum queries honor column and row permissions on Data lakeA central store for large amounts of structured and unstructured data kept in its original form, usually in Amazon S3, which many tools can analyze. AWS Lake Formation helps build and secure one.Glossary entry tables. - Auditing: audit logging records connections, changes to database users, and, with
enable_user_activity_logging, every query. Logs go to Amazon S3 or CloudWatch Logs; Redshift Serverless sends them to CloudWatch only.
Resilience
- Multi-AZRunning standby or extra copies of a resource in other Availability Zones so that it survives the loss of one. An Amazon RDS Multi-AZ deployment keeps one or two standby copies and fails over to one automatically.Glossary entry deployments (RA3 and RG node types) run compute in two AZ (Availability Zone)An isolated location within an AWS Region, made up of one or more data centers with their own power, cooling, and networking. The AZs in a Region are linked by fast, low-latency connections, so spreading resources across them protects against the failure of one site.Glossary entry behind one endpoint, so the warehouse keeps working if a zone fails. They cost the same hourly rate per node, and AWS raises the SLA (Service level agreement)A provider's commitment to a level of service, such as a monthly uptime percentage, and what customers receive if it is missed, usually service credits.Glossary entry from 99.9% to 99.99%.
- Automated snapshots are incremental. By default Redshift takes one about every eight hours or after every 5 GB per node of changes, and keeps them for one day; you can raise the retention (up to 35 days on RA3 node types).
- Manual snapshots are kept until you delete them, even after the cluster is deleted.
- Cross-Region snapshot copy copies snapshots automatically to another Region, so the cluster can be restored there if the primary Region fails. For a KMS-encrypted cluster, create a snapshot copy grant for a key in the destination Region.
Performance
- Columnar storageStoring data column by column instead of row by row, so that analytical queries read only the columns they need, and similar values compress well. Amazon Redshift and the Parquet and ORC file formats store data this way.Glossary entry, Sort keyThe optional second part of a DynamoDB primary key. It orders the items that share a partition key, so that queries can select a range, such as all orders after a date.Glossary entry, and distribution styles shape query speed; automatic table optimization can choose sort and distribution keys for you.
- RA3 and RG nodes use managed storage, so compute and storage scale independently.
- Concurrency scaling handles bursts of users. In Redshift Serverless, raising the base capacity (in RPUs) helps complex queries and wide tables.
Cost
- Provisioned: pay per node-hour, or buy reserved nodes for one or three years at a discount for steady workloads. Pause a cluster you only need at certain times: while it is paused you pay only for storage, and pause and resume can be scheduled.
- Serverless: pay per RPU-hour, billed per second with a 60-second minimum, and nothing while idle. Maximum RPU-hour limits cap spending.
- Managed storage is billed per GB-month; Spectrum is billed per TB scanned; each cluster earns up to one hour of free concurrency scaling credits per day.
Monitoring
- Amazon CloudWatch metrics for clusters and Serverless workgroups, including a default disk space alarm you can set when you create a cluster.
- Audit logs for connections and queries, and AWS CloudTrail for Redshift API (Application programming interface)A defined way for one program to ask another for data or actions, such as a set of HTTPS requests and responses. Every AWS service is controlled through its API, which the console, the CLI, and the SDKs call for you.Glossary entry calls.
Redshift or Athena?
| If the requirement is | Choose |
|---|---|
| Frequent, complex reports for many users with consistent performance | Redshift |
| Combine data from many sources into one warehouse | Redshift |
| Occasional queries on data that stays in S3 | Amazon Athena |
Reverse mindmap: when the question points here
- Yes, when the question says
- Data warehouse for BI and complex reporting leads to Amazon Redshift
- Unpredictable warehouse usage, no idle cost leads to Redshift Serverless
- Join warehouse tables with files in S3 without loading leads to Redshift Spectrum
- Query spikes from many concurrent users leads to Concurrency scaling
- Give another account or team live data without copying leads to Data sharing
- Analytics on Aurora or DynamoDB data without ETL leads to Zero-ETL integration
- Run it securely
- COPY and UNLOAD traffic must stay in the VPC leads to Enhanced VPC routing
- Reject unencrypted client connections leads to require_ssl parameter
- Clients in another VPC or account connect privately leads to Redshift-managed VPC endpoint
- Record every query for auditors leads to Audit logging with user activity log
- Run it reliably and cheaply
- Keep working if an Availability Zone fails leads to Multi-AZ deployment
- Restore the warehouse in another Region leads to Cross-Region snapshot copy
- Cluster only needed during business hours leads to Scheduled pause and resume
- Steady workload for years leads to Reserved nodes
- No, when the question says
- Occasional SQL on S3, pay per query leads to Athena
- Transactions for an application leads to Amazon RDS or Aurora
- Full-text search or log analytics leads to OpenSearch Service
Key takeaways
- Redshift is the analytics data warehouse; RDS and Aurora handle application transactions.
- Spectrum queries S3 without loading; zero-ETL brings operational data in without pipelines.
- Encrypted by default; enhanced VPC routing keeps COPY and UNLOAD traffic in the VPC;
require_sslenforces encrypted connections. - Multi-AZ survives a zone failure; cross-Region snapshot copy is the answer to "recover in another Region".
- Pause clusters you don't need, reserve nodes for steady use, and cap Serverless spend with RPU-hour limits.
Sources
- Amazon Redshift Management Guide: What is Amazon Redshift?
- Amazon Redshift Database Developer Guide: Columnar storage
- Amazon Redshift Management Guide: Zero-ETL integrations
- Amazon Redshift Database Developer Guide: Amazon Redshift Spectrum
- Amazon Redshift Database Developer Guide: Concurrency scaling
- Amazon Redshift Database Developer Guide: Data sharing
- Amazon Redshift Management Guide: Snapshots and backups
- Amazon Redshift Management Guide: Copying a snapshot to another AWS Region
- Amazon Redshift Management Guide: Database encryption
- Amazon Redshift Management Guide: Configuring security options for connections
- Amazon Redshift Management Guide: Enhanced VPC routing
- Amazon Redshift Management Guide: Redshift-managed VPC endpoints
- Amazon Redshift Management Guide: Database audit logging
- Amazon Redshift Management Guide: Multi-AZ deployment
- Amazon Redshift Management Guide: Provisioned clusters
- Amazon Redshift Management Guide: Pausing and resuming a cluster
- Amazon Redshift Management Guide: Compute capacity for Redshift Serverless
- Amazon Redshift Database Developer Guide: Best practices for designing tables
- Amazon Redshift Management Guide: Logging with CloudTrail
- Amazon Redshift pricing