Amazon OpenSearch Service

What it is

Amazon OpenSearch Service runs OpenSearch clusters for you. OpenSearch is an open-source search and analytics engine for log analytics, full-text search, real-time application monitoring, and clickstream analysis, with OpenSearch Dashboards for visualizing the results. The service also supports legacy Elasticsearch OSS versions up to 7.10.

Choose it when a question is about searching text or exploring logs interactively, not about running SQL (Structured Query Language)The standard language for querying and changing data in relational databases. Analytics services such as Amazon Athena and Amazon Redshift also use SQL.Glossary entry reports over a Data lakeA central store for large amounts of structured and unstructured data kept in its original form, usually in Amazon S3, which many tools can analyze. AWS Lake Formation helps build and secure one.Glossary entry.

Ways to run it

Managed domains: a domain is an OpenSearch cluster. You choose the instance types and the number of data nodes, and the service provisions them and replaces failed nodes. Dedicated master nodes handle cluster management so the data nodes can focus on indexing and search.
OpenSearch Serverless: you create a collection for one workload type, time series (logs), search, or vector search, and the service scales capacity for you. Indexing and search use separate compute, measured in OpenSearch Compute Units (OCUs), and the indexes are stored in Amazon S3.
OpenSearch Ingestion: serverless pipelines, built on the open-source Data Prepper, that collect, transform, and deliver logs, metrics, and traces to domains or collections without running Logstash yourself.

Data also arrives from Amazon Data Firehose, Kinesis Data Streams, CloudWatch Logs, and AWS Lambda functions that index data from Amazon S3 or DynamoDB.

Vector search

k-NN search: add a field of type knn_vector (up to 10,000 dimensions) to an index, and a knn query returns the k nearest vectors. Uses include semantic search, recommendations, image recognition, and fraud detection.
Hybrid search: OpenSearch combines full-text and vector search in one engine, at high query rates with low LatencyThe time between sending a request and receiving the response, usually measured in milliseconds. Distance and processing add to it; caching and serving users from nearby locations reduce it.Glossary entry, which is why it is a common vector store for RAG.
With S3 Vectors: keep large, rarely queried vector sets cheaply in Amazon S3 Vectors and use OpenSearch for the frequent, low-latency queries; a snapshot of an S3 vector index can be exported to OpenSearch Serverless.

Storage tiers for older data

Hot storage, on the data nodes' instance store or EBS volumes, gives the fastest indexing and search for current data.
UltraWarm keeps read-only indexes in Amazon S3 behind a cache. It costs much less per GB, you pay only for the data you store, and it suits logs you query less often.
Cold storage is cheaper still for data you rarely need; you move an index back to UltraWarm to query it.
Index State Management (ISM) policies move indexes between tiers, or delete them, automatically as they age.

Security, resilience, performance, and cost

Security

OpenSearch Service security has three layers:

Network: you choose VPC access or public access when you create the domain, and you can't switch later. A VPC domain is reachable only from inside the VPC (or over a VPN (Virtual private network)An encrypted tunnel over a public network, such as the internet, that joins networks or devices as if they were directly connected. AWS Site-to-Site VPN connects networks; AWS Client VPN connects individual users.Glossary entry or Direct Connect connection) and is protected by Security groupA stateful virtual firewall for network interfaces, such as an EC2 instance's, whose rules can only allow traffic. Return traffic for allowed connections is allowed automatically, and a rule can refer to another security group instead of IP addresses.Glossary entry.
Domain access policy: a Resource-based policyA permissions policy attached to a resource, such as an S3 bucket policy or an SQS queue policy, that says which principals can use it, including principals in other accounts. Identity-based policies, in contrast, attach to users, groups, and roles.Glossary entry that allows or denies requests at the domain endpoint.
Fine-grained access control: roles that grant access at the index, document, and field level, including field masking. It requires HTTPS (HTTP Secure)HTTP sent over a TLS-encrypted connection, so that requests and responses cannot be read or changed on the way, and the server proves its identity with a certificate.Glossary entry, node-to-node encryption, and Encryption at restEncrypting data where it is stored, such as on disks, in databases, or in backups, so that it cannot be read without the key. On AWS it usually uses keys in AWS KMS.Glossary entry, and once turned on it can't be turned off.

Also:

Encryption at rest uses AWS KMS and covers indexes (including UltraWarm), OpenSearch logs, swap files, and automated snapshots. Once on, it can't be turned off. Node-to-node encryption protects traffic inside the cluster with TLS (Transport Layer Security)The protocol that encrypts traffic between a client and a server and lets the server prove its identity with a certificate. It replaced SSL, though the older name is still common; HTTPS is HTTP over TLS.Glossary entry.
OpenSearch Dashboards sign-in can use Amazon Cognito or SAML (Security Assertion Markup Language)An XML-based standard for passing sign-in information from an identity provider, such as Microsoft Entra ID or Okta, to a service. AWS accepts SAML 2.0 for single sign-on, directly in IAM or through IAM Identity Center.Glossary entry with your identity provider.
OpenSearch Serverless encrypts all traffic to its endpoints with TLS 1.2.

Resilience

Multi-AZRunning standby or extra copies of a resource in other Availability Zones so that it survives the loss of one. An Amazon RDS Multi-AZ deployment keeps one or two standby copies and fails over to one automatically.Glossary entry with Standby spreads the domain across three AZ (Availability Zone)An isolated location within an AWS Region, made up of one or more data centers with their own power, cooling, and networking. The AZs in a Region are linked by fast, low-latency connections, so spreading resources across them protects against the failure of one site.Glossary entry, each with a full copy of the data, and keeps one zone's nodes on standby. If infrastructure fails, the standby nodes take over in under a minute. AWS rates it at 99.99% availability, at no extra cost; Multi-AZ without standby is rated at 99.9%.
Use three dedicated master nodes, so the cluster can still elect a master if one zone is lost, and at least one replica for each index.
Automated snapshots are taken every hour and kept for 14 days, at no charge, for recovering the domain. Manual snapshots go to your own S3 bucket, and you can restore them to another domain, for example to migrate.

Performance

Scale up (larger instances) or out (more data nodes) when CPU or JVM memory pressure stays high, and use dedicated master nodes for stable cluster management.
Size shards sensibly and keep older, rarely searched indexes in UltraWarm so hot nodes serve current data.

Cost

Managed domains charge per instance-hour plus EBS storage. RI (Reserved Instance)A billing discount for committing to an instance type in a Region or Availability Zone for one or three years. It is not an instance: running instances that match are billed at the lower rate. AWS now recommends Savings Plans instead.Glossary entry for one or three years discount steady workloads, but apply only to the exact instance type reserved.
Move older data to UltraWarm or cold storage, and let ISM delete indexes you no longer need.
There is no charge for traffic between Availability Zones within a domain, or between UltraWarm nodes and Amazon S3.
OpenSearch Serverless and OpenSearch Ingestion charge for the OCUs they use.

Monitoring

AWS recommends CloudWatch alarms on cluster health (ClusterStatus.red and ClusterStatus.yellow), FreeStorageSpace, JVMMemoryPressure, CPUUtilization, failed automated snapshots, and KMS key errors.
Publish error logs, slow logs (search and indexing), and audit logs to CloudWatch Logs. All are off by default.
AWS CloudTrail records configuration API (Application programming interface)A defined way for one program to ask another for data or actions, such as a set of HTTPS requests and responses. Every AWS service is controlled through its API, which the console, the CLI, and the SDKs call for you.Glossary entry calls.

OpenSearch Service or something else?

If the requirement is	Choose
Full-text search, log analytics, or near real-time dashboards over logs	OpenSearch Service
Semantic or vector search for AI applications	OpenSearch Serverless (vector search collection)
Billions of vectors at the lowest cost, queried now and then	Amazon S3 Vectors
Ad hoc SQL over files in S3	Amazon Athena
A Data warehouseA database designed for analytics, holding cleaned, structured data from many sources, so that reports and queries across years of history run fast. Amazon Redshift is the AWS data warehouse.Glossary entry for business reporting	Amazon Redshift

Reverse mindmap: when the question points here

Is it OpenSearch Service?

Yes, when the question says
- Full-text search for a website or documents leads to OpenSearch Service
- Search and visualize application logs leads to OpenSearch Service and Dashboards
- Vector or semantic search with no cluster to manage leads to OpenSearch Serverless
- Stream logs in with no Logstash to run leads to Firehose or OpenSearch Ingestion
Run it securely
- Domain reachable only from the VPC leads to VPC access, chosen at creation
- Users see only some indexes, documents, or fields leads to Fine-grained access control
- Sign in to Dashboards with the corporate identity provider leads to SAML authentication
- Encrypt indexes and snapshots with KMS leads to Encryption at rest
Run it reliably and cheaply
- Survive an Availability Zone failure leads to Multi-AZ with Standby
- Keep months of logs cheaply leads to UltraWarm and cold storage
- Move or delete old indexes automatically leads to Index State Management
- Steady workload for years leads to Reserved Instances
No, when the question says
- SQL queries over files in S3 leads to Amazon Athena
- Data warehouse for business reporting leads to Amazon Redshift

Key takeaways

OpenSearch Service is for search and log analytics, with Dashboards for exploring results.
Choose VPC access at creation; layer a domain access policy and fine-grained access control on top.
Multi-AZ with Standby across three zones, with three dedicated master nodes, is the production setup.
UltraWarm, cold storage, and ISM keep long log retention affordable.
Serverless collections and OpenSearch Ingestion remove cluster and pipeline management.
k-NN vector search plus full-text search makes OpenSearch a common vector store for RAG; S3 Vectors is the cheaper tier for rarely queried vectors.