Amazon Bedrock for MLA-C02: Study Notes

What it is

Amazon Bedrock is a fully managed service for building generative AI applications with foundation models (FMs). It gives you models from Amazon and from other providers through one API (Application programming interface)A defined way for one program to ask another for data or actions, such as a set of HTTPS requests and responses. Every AWS service is controlled through its API, which the console, the CLI, and the SDKs call for you.Glossary entry, so you can switch models without changing how your application calls them, and there are no servers or GPU (Graphics processing unit)A processor with thousands of small cores that run many calculations in parallel. GPUs speed up machine learning, graphics, and video work; EC2 instance families such as P and G include them.Glossary entry for you to manage. Around the models, Bedrock adds what an application needs: retrieval of your own documents (Knowledge Bases), task automation (agents), safety controls (Guardrails), evaluations, and prompt management.

Choose Bedrock when a question asks you to use, customize, or ground a foundation model with the least operational overhead. Choose Amazon SageMaker AI when you must train or host your own model with full control over the code, framework, and infrastructure. Choose an AI service such as Amazon Comprehend or Amazon Textract when a ready-made API already does the job.

Choosing and calling a model

Model choice (task 2.1): compare models on the task (text, chat, code, images, embeddings), quality on your data, LatencyThe time between sending a request and receiving the response, usually measured in milliseconds. Distance and processing add to it; caching and serving users from nearby locations reduce it.Glossary entry, context size, and price per token. A smaller, cheaper model that passes your evaluation is usually the right answer to a cost question.
Invocation: InvokeModel sends a model-specific request; the Converse API gives one request format across models, including multi-turn messages and tool use. Both have streaming variants for responses that appear token by token.
Embeddings models, such as Amazon Titan Text Embeddings and Cohere Embed, turn text (and some images) into vectors for semantic search and RAG.
Prompt caching lets supported models reuse a repeated prompt prefix, such as a long system prompt or document, which lowers latency and the cost of the cached input tokens.

Inference options

Option	How it works	Choose it for
On-demand	Pay per input and output token; requests are subject to the account's quotas	Variable or unpredictable traffic
Cross-AWS RegionA separate geographic area where AWS runs its services, such as us-east-1 in Northern Virginia, made up of several Availability Zones. Most resources live in one Region; you choose Regions for latency, data residency, and the services you need.Glossary entry inference	An inference profile routes requests to Regions in a geography (such as US or EU) or worldwide (global)	More throughput and resilience to Regional capacity limits; geographic profiles keep processing in the geography
Batch inference	Put many prompts in files in Amazon S3, start a job, and collect the outputs in S3	Large offline jobs that don't need an immediate answer
Provisioned Throughput	Buy model units (MUs), each a fixed number of input and output tokens per minute, billed hourly with no commitment, 1 month, or 6 months	Steady high volume with guaranteed capacity; required to use a customized model

Cross-Region inference has no additional routing cost; the global profiles are priced lower than the geographic ones, and the geographic ones are the choice when data must stay in a region such as the EU. Traffic between Regions stays on the AWS network. Batch inference isn't supported for provisioned models.

Customizing models

Try prompt engineering and RAG first: they change behaviour without training. Customize the model when it must learn a style, format, or domain that prompts can't teach (task 2.2):

Fine-tuning: train on labeled prompt-and-response pairs for a specific task.
Continued pre-training: train on unlabeled domain text, so the model learns its vocabulary.
Distillation: a large "teacher" model generates responses that train a smaller, cheaper "student" model.
Custom Model Import: bring the weights of a supported open model that you customized elsewhere (for example in SageMaker AI) and serve it through Bedrock.

Training data goes in S3, and a customization job writes a custom model that you run through Provisioned Throughput. Watch for overfitting and catastrophic forgetting (the model loses general ability), and evaluate the custom model against the base model before you switch.

Knowledge Bases: managed RAG

A knowledge base connects a model to your documents with Retrieval Augmented Generation (RAG): retrieve relevant passages, then generate an answer grounded in them, with citations.

Data sources: for example Amazon S3, a web crawler, Confluence, SharePoint, or Salesforce. An ingestion job (sync) parses, chunks, and embeds new and changed documents.
Chunking: default (about 300 tokens, keeping sentences whole), fixed-size (tokens per chunk and an overlap percentage), hierarchical (small child chunks for precise matching, replaced by their larger parent chunks for context), semantic (splits where the meaning changes; it uses a model, so it costs more), or no chunking when you split the files yourself.
Embeddings model and vector store: Amazon OpenSearch Serverless, OpenSearch Service managed clusters, Amazon S3 Vectors (low-cost storage for infrequently queried vectors), Amazon Aurora PostgreSQL (pgvector), Amazon Neptune Analytics (GraphRAG), or partner stores such as Pinecone, Redis Enterprise Cloud, and MongoDB Atlas. The embedding dimensions must match the index.
Query: Retrieve returns the passages; RetrieveAndGenerate also generates the answer. Metadata filtering narrows the search (for example to one department's documents), and a reranking model reorders results by relevance.

Refresh a knowledge base by running ingestion when the source changes, for example from an EventBridge schedule or a pipeline (task 3.3).

Agents and Guardrails

Agents break a request into steps, call tools such as APIs and knowledge bases, and keep the conversation's context. Amazon Bedrock Agents, with its action groups (an API schema backed by, for example, a Lambda function), is now Agents Classic and has been closed to new customers since July 30, 2026; existing agents keep working. For new agents, AWS recommends Amazon Bedrock AgentCore, including a managed, configuration-based agent harness. Older exam questions may still describe action groups, so recognize them.

Guardrails apply the same safety policy to prompts and responses, whichever model you use:

Content filters: hate, insults, sexual, violence, misconduct, and prompt attacks, each with a strength you set.
Denied topics and word filters (for example competitor names or profanity).
Sensitive information filters: block or mask PII (Personally identifiable information)Information that can identify a person, alone or combined with other data, such as a name, email address, or passport number. Amazon Macie finds PII in Amazon S3.Glossary entry such as addresses or account numbers, and your own regular expressions.
Contextual grounding checks: catch responses that aren't supported by the retrieved source, or don't answer the question; useful against hallucinations in RAG.
Automated Reasoning checks: validate responses against logical rules that you define.

Attach a guardrail to an inference call by its ID and version, or call the ApplyGuardrail API to check text from any model, including one outside Bedrock.

Evaluate and manage prompts

Evaluations (task 2.3) compare models, prompts, and knowledge bases with automatic metrics, human reviewers, or a model acting as a judge (LLM-as-a-judge), and evaluate RAG retrieval and answers.
Prompt management stores prompts with variables and versions, so applications and tests use a known version and changes can be rolled back.

Security, resilience, performance, and cost

Security

IAM controls who can invoke which models and use which knowledge bases, agents, and guardrails. Applications should use IAM roles.
Bedrock API keys authenticate with a bearer token. Short-term keys last up to 12 hours, inherit the permissions of the identity that created them, and are the ones AWS recommends for production; long-term keys create an IAM user and are meant only for exploration. The bedrock:CallWithBearerToken permission controls whether an identity may use API keys at all.
Data: model providers have no access to the accounts that run their models, so they can't see your prompts, completions, or logs. Traffic uses TLS (Transport Layer Security)The protocol that encrypts traffic between a client and a server and lets the server prove its identity with a certificate. It replaced SSL, though the older name is still common; HTTPS is HTTP over TLS.Glossary entry; encrypt custom models, knowledge base resources, and logs with KMS keys, including Customer managed keyA KMS key that you create, own, and manage: you control its key policy, rotation, and deletion. AWS managed keys, in contrast, are created and managed for you by an AWS service.Glossary entry.
Private access: interface VPC endpointA private connection from a VPC to an AWS service or another endpoint service without crossing the internet. Gateway endpoints, for Amazon S3 and DynamoDB, are route table targets; interface endpoints are network interfaces that use AWS PrivateLink.Glossary entry (AWS PrivateLink) keep calls from a VPC off the internet.
Responsible AI: Guardrails, evaluations for bias and quality, and human review of customization data.

Resilience

Bedrock is a Regional managed service. Cross-Region inference spreads load across Regions; add retries with backoff for ThrottlingLimiting how many requests a client can make in a period. A throttled request is rejected, often with HTTP status 429 or a throttling error, and should be retried later with backoff.Glossary entry and, for critical paths, a fallback model or Region.

Performance

On-demand use is limited by quotas (requests and tokens per minute per model). When requests are throttled, use cross-Region inference, request a quota increase, or buy Provisioned Throughput. Streaming improves perceived latency; a smaller model or prompt caching reduces real latency.

Cost

You pay per input and output token on demand, per model unit hour for Provisioned Throughput, and for customization, storage, and the vector store behind a knowledge base. To pay less: choose the smallest model that meets the quality bar, shorten prompts and cap output tokens, use prompt caching, batch inference for offline work, global cross-Region inference where data may leave the geography, and distillation to move traffic to a smaller model (task 4.2).

Monitoring

CloudWatch metrics: invocations, latency, input and output token counts, throttles, and errors per model.
Model invocation logging (off by default) sends prompts and responses to CloudWatch Logs or S3 for audit and debugging.
CloudTrail records Bedrock API calls, including cross-Region requests (in the source Region).
EventBridge reports state changes of jobs such as batch inference and customization.

Bedrock or something else?

Need	Choose
Use or customize a foundation model through an API, no infrastructure	Amazon Bedrock
Train your own model, or host a model with full control of the container and instance	Amazon SageMaker AI
A task with a ready-made API: OCR, sentiment, transcription, translation	The AI service (Textract, Comprehend, Transcribe, Translate)
Run agents in production with isolated sessions, memory, identity, and observability	Amazon Bedrock AgentCore
Vector search that you manage alongside other search	Amazon OpenSearch Service

Reverse mindmap: when the question points here

Amazon Bedrock: which feature?

Use a foundation model with no servers
- Same request format for many models leads to Converse API
- Answers from company documents, with citations leads to Knowledge Bases (RAG)
- Multi-step tasks that call APIs leads to AgentCore (Agents Classic only for existing agents)
Capacity and cost
- Throttled at peak, data may stay in the EU leads to Geographic cross-Region inference
- Steady volume, guaranteed capacity, custom model leads to Provisioned Throughput
- Millions of prompts overnight leads to Batch inference from S3
- Repeated long prompt prefix leads to Prompt caching
Safety and trust
- Mask PII, block topics or prompt attacks leads to Guardrails
- Answers not supported by the documents leads to Contextual grounding check
- Compare models or prompts, LLM-as-a-judge leads to Evaluations
Not Bedrock
- Full control of training code and instances leads to SageMaker AI
- Extract text from scanned forms leads to Textract

Key takeaways

Bedrock serves foundation models from Amazon and other providers through one API, fully managed.
On-demand for variable traffic, cross-Region inference for throughput (geographic for data residency, global for lower cost), batch for offline jobs, Provisioned Throughput for steady volume and for any customized model.
Prefer prompts and RAG; fine-tune with labeled data, continue pre-training with unlabeled data, distill to a smaller model, or import a model customized elsewhere.
Knowledge Bases manage chunking, embeddings, the vector store, retrieval, and reranking.
Guardrails filter content, denied topics, and PII, check grounding, and work with any model through ApplyGuardrail.
Prefer IAM roles or short-term API keys; use PrivateLink for private access, KMS for encryption, and invocation logging plus CloudWatch token metrics to monitor use and cost.

Sources

Show 17 more sourcesShow fewer sources

Amazon Bedrock