When you type a question into a generative AI assistant, it can feel as if the model reads your words the way you do. It does not. Behind the scenes, text is split into smaller pieces, represented numerically, and processed within a limited working space. Three concepts explain much of this process: tokens, embeddings, and context windows.
These ideas matter when estimating model costs, designing semantic search, building Retrieval-Augmented Generation (RAG), and preparing for the AWS Certified AI Practitioner (AIF-C01) exam.
To place tokens, embeddings, and context windows in the full exam syllabus, continue with the AIF-C01 study guide and its coverage of foundation-model concepts.
Tokens: the units of text a model processes
A token is a piece of text that a model’s tokenizer treats as one unit. A token may be a complete short word, part of a longer word, punctuation, or whitespace. The exact split depends on the tokenizer used by the model.
For example, a tokenizer might split “Unbelievably helpful documentation.” into several sub-word pieces and punctuation. That is an illustration only; another model may split the same sentence differently.
Why sub-word tokens?
A vocabulary containing every possible word, name, spelling variation, and typo would be impractical. Sub-word tokenization lets models reuse smaller text units and handle unfamiliar words without requiring every complete word to exist in the vocabulary.
A rough rule of thumb
For ordinary English prose, people often estimate roughly three-quarters of a word per token, or around 130 tokens for 100 words. Treat this only as a rough estimate. Code, uncommon terminology, punctuation, and other languages can tokenize very differently. For accurate counts, use the specific model’s tokenizer or the token-usage information returned by the service.
Why tokens matter
- Cost: Many model pricing options are based partly on input and output token usage. Pricing varies by model, modality, service tier, and inference option, so check current pricing rather than assuming one universal rate.
- Latency: Longer inputs and longer generated responses can increase processing time.
- Capacity: Models have limits on how much tokenized content they can process in a request.
Remember: Models process tokens, not word counts. Token usage can affect cost, speed, and request limits.
Embeddings: representing meaning as vectors
An embedding is a numerical representation of an item such as a word, sentence, document, or image. For text applications, an embedding model converts text into a vector: an ordered list of numbers.
Embeddings are designed so that items with related meanings can be identified by comparing their vectors. Imagine a map in which concepts have coordinates. “Puppy” and “dog” may be positioned closer than “puppy” and “mortgage.” Real vectors have many dimensions, and similarity depends on the model and the comparison metric; it is not a literal two-dimensional map.
How similarity is measured
- Cosine similarity compares vector direction. In common normalized embedding workflows, values closer to 1 indicate greater directional similarity.
- Euclidean distance measures straight-line distance. Smaller distances indicate closer vectors.
The exam-level idea is more important than the mathematics: an application can compare vectors to find semantically related content.
Semantic search: finding meaning, not just matching words
Traditional keyword search may miss a document if it does not contain the exact search phrase. Semantic search uses embeddings to find content with related meaning.
Example: A library visitor searches for “books about getting over a breakup.” An embedding-based search can retrieve descriptions about heartbreak and emotional recovery even when those descriptions do not use the word “breakup.” The query and catalog descriptions are embedded, and the search system ranks candidates by vector similarity.
Embeddings are also used for recommendations, clustering, classification, duplicate detection, and RAG retrieval.
Vector databases and AWS options
A vector database, or a database with vector-search capabilities, stores embeddings and supports similarity queries. AWS documentation currently describes multiple supported choices for Amazon Bedrock Knowledge Bases, including Amazon OpenSearch Serverless, Amazon OpenSearch Service managed clusters, Amazon Neptune, Amazon Aurora (RDS), and Pinecone, with support depending on the Knowledge Bases configuration and feature. The available choices can change; always check the current documentation before designing a deployment.
Amazon Bedrock Knowledge Bases can coordinate document ingestion, embedding generation, vector storage, retrieval, and response generation as part of a RAG workflow. Some configurations use managed embedding options; others let you select a supported embedding model and vector store.
Generating embeddings with Amazon Bedrock
Amazon Bedrock offers embedding models, including Amazon Titan Text Embeddings V2 and supported models from other providers. Embedding models output vectors; text-generation models produce generated text. These are different tasks, even though both are foundation-model capabilities.
Common exam clue: “Find similar documents,” “semantic search,” “vector representation,” or “retrieve content by meaning” points toward embeddings. “Write a response” or “summarize this text” points toward a generative model.
The context window: the model’s working space
A context window is the maximum amount of tokenized context a model can handle for a request. Depending on the model and interface, this generally includes the supplied instructions, conversation history, retrieved documents, the user’s prompt, and space reserved for the generated response. Check the model documentation for the precise accounting rules and limits.
Think of a desk with a fixed surface area. All the material needed for the task must fit on the desk. If the desk is full, you must remove or shorten something, or use a model with a larger supported context.
A hypothetical token budget
| Component | Tokens |
|---|---|
| System instructions | 500 |
| Conversation history | 2,000 |
| Retrieved documents | 4,500 |
| User question | 200 |
| Input total | 7,200 |
| Room left in an 8,000-token budget | 800 |
If the application needs a 1,500-token answer, the 8,000-token budget is not enough for this example. It could retrieve fewer passages, shorten the conversation history, reduce instructions, or select a model with a larger context limit. The 8,000-token example is hypothetical, not a statement about a current AWS model.
Bigger is not automatically better
- Cost: More input tokens can increase usage charges where pricing is token-based.
- Latency: Processing a long prompt can take longer.
- Relevance: A very large prompt can include distracting or low-value material. Selecting relevant passages can work better than sending everything.
This is one reason RAG is useful. Rather than placing an entire document library into every prompt, a retrieval system selects passages relevant to the current question. See our guide to how Retrieval-Augmented Generation works.
What happens when the context limit is exceeded?
The request may be rejected, or the application may shorten, truncate, or summarize earlier conversation content before sending a new request. A model does not automatically retain a complete record of previous independent requests. If an assistant appears to remember earlier messages, the application may be resending selected history or using a separate memory or retrieval mechanism.
How tokens, embeddings, and context windows work together
Consider a travel-support assistant answering: “Can I change my flight to Lisbon without a fee?”
- The application tokenizes the question for model processing.
- It may create an embedding of the question and search a vector store of travel-policy documents.
- The retrieval system selects relevant passages and returns them to the application.
- The application assembles its instructions, retrieved passages, and question into a prompt that must fit the model’s context budget.
- The model generates a response as a sequence of output tokens.
Tokens are the processing units; embeddings represent meaning for similarity and retrieval; the context window limits how much context can be handled at once.
Related inference settings
| Setting | General effect |
|---|---|
| Maximum output tokens | Caps the length of the generated response. |
| Temperature | Controls sampling variability; higher values generally produce more varied output. |
| Top-p / Top-k | Restricts the candidate tokens considered during sampling. |
Maximum output tokens is a response-length setting. The context window is the overall context capacity, which must accommodate the request and the generated output according to the model’s rules.
Common mistakes and misconceptions
- “A token equals a word.” Not necessarily; tokens are often sub-word pieces.
- “The context window only limits my prompt.” Input and output both consume the model’s available context budget, subject to the model’s documented rules.
- “The model automatically remembers every past conversation.” Continuity usually depends on the application supplying history or another memory mechanism.
- “Embeddings are the model’s answers.” They are numerical representations used for comparison and retrieval, not ordinary human-readable answers.
- “A larger context window makes RAG unnecessary.” Cost, latency, and relevance still make retrieval valuable.
- “All embedding vectors can be compared.” Vectors should generally be compared within compatible embedding spaces; changing embedding models or dimensions commonly requires re-embedding the stored content.
To understand how malicious instructions can exploit the text a model receives, read our guide to prompt injection, jailbreaking, and hallucinations.
Exam-focused insights for AIF-C01
- Connect token usage to token-based cost, latency, and model limits.
- Recognize embedding clues: semantic similarity, vector search, related meaning, and retrieval.
- Remember that the context budget is shared by the request and response as defined by the model.
- Distinguish an embedding model from a text-generation model.
- Know that vector stores support similarity search and that Bedrock Knowledge Bases helps manage RAG workflows.
Key takeaways
- Tokens are the text units processed by language models.
- Embeddings are numerical vectors used to represent content and compare semantic similarity.
- Vector stores keep embeddings available for similarity search.
- The context window limits how much tokenized context a model can handle.
- Relevant retrieval and concise prompts can improve focus and manage token usage.
Original AIF-C01 practice questions
These are original, unofficial educational practice questions written by CloudExamPro. They are not actual AWS exam questions.
Question 1: An 8,000-token context budget receives 7,500 input tokens and needs a 1,500-token response. What is the issue?
Correct answer: A. The combined budget is 9,000 tokens, exceeding the hypothetical 8,000-token limit.
B is incorrect because the scenario does not state a 500-token input limit. C is unrelated; embeddings are not required for this arithmetic. D is incorrect because temperature does not increase context capacity.
Question 2: A store wants “affordable winter coat” to retrieve products described as “budget parka.” Which concept enables this?
Correct answer: B — Embeddings and semantic similarity search. The query and product descriptions can be represented as vectors and compared by meaning. Output-token limits, temperature, and context-window size do not perform semantic retrieval.
Question 3: A company wants to reduce token-based model costs without changing models. What should it do?
Correct answer: B — Shorten prompts and limit unnecessary output length. This reduces input and output token usage. Adding examples increases input tokens; temperature changes sampling, and converting a prompt to an embedding is not a substitute for the text input expected by a generation model.
Question 4: Which statement about tokens is correct?
Correct answer: B. Tokens are often sub-word pieces, so a word may be split into multiple tokens. A token is not always a complete word, tokens are not only used for images, and counts can differ across tokenizers.
Question 5: A team needs vector representations of policy documents for semantic search. Which option is appropriate?
Correct answer: A — An embedding model such as Amazon Titan Text Embeddings in Amazon Bedrock. Embedding models produce vectors. Transcribe converts speech to text, Rekognition analyzes images and video, and Polly converts text to speech.
Frequently asked questions
How many tokens are in a typical page of text?
It varies with formatting, language, content, and tokenizer. The rough English estimate is only a starting point; use the model’s tokenizer or returned usage counts for accuracy.
Is the context window the same as the model’s training data?
No. Training data is used during model training. The context window is the amount of tokenized context the model can handle for a request.
Do embeddings expire or need to be recomputed?
They do not normally expire simply because time passes. If you change the embedding model, model version, or relevant vector configuration, you may need to regenerate vectors so queries and stored content remain compatible.
Can a large context window replace RAG?
Not in every application. RAG can reduce irrelevant input, improve access to changing information, and help control cost and latency.
What is the difference between maximum output tokens and the context window?
Maximum output tokens limits generated response length. The context window is the model’s overall context capacity and must account for the input and output under the model’s rules.
Official AWS references
- AWS Certified AI Practitioner certification and exam guide
- What is Amazon Bedrock?
- What is Generative AI?
- What are embeddings in machine learning?
- Bedrock Knowledge Bases: supported models and Regions
- Turning data into a knowledge base
- Amazon Bedrock pricing
Keep learning
Review the differences between AI, machine learning, deep learning, and generative AI, then explore how RAG works from documents to answers and how AI agents use tools and multi-step workflows.
Keep building your AIF-C01 skills
Check your understanding with our free AIF-C01 diagnostic exam. For deeper scenario-based practice with detailed explanations, explore the AIF-C01 Premium practice exams.
Original Practice Questions — No Dumps. No Shortcuts.