Amazon SageMaker Clarify was designed to help teams examine potential bias in machine-learning data and models and understand why a model produces particular predictions. It supports responsible AI work by producing bias metrics, feature-attribution explanations, and analysis reports.
Bias analysis: before and after training
Pre-training bias analysis examines the input dataset before model training. It can help identify imbalances or representation issues in the data. Post-training bias analysis evaluates model predictions along with the data and labels to identify potential bias introduced or exposed during model development.
Bias metrics are quantitative indicators, not a universal verdict that a model is fair or unfair. Select metrics that match the use case, protected or sensitive attributes, and the consequences of errors. Interpret results with domain expertise.
Model explainability and feature attribution
Clarify uses a model-agnostic feature-attribution approach that includes SHAP (Shapley Additive Explanations). Feature attributions help estimate how input features contribute to a prediction relative to a selected baseline. This can help developers investigate why a particular prediction occurred and identify patterns that deserve further review.
Clarify also supports partial dependence plots (PDPs), which illustrate how changing a feature can affect the model’s predicted outcome on average, subject to the assumptions and limitations of the analysis.
Where Clarify fits in the ML lifecycle
- Before training: analyze data for potential bias.
- After training: analyze model predictions for potential bias and generate feature-attribution explanations.
- During review: inspect reports and investigate features or groups that may contribute to unexpected outcomes.
- In governance: use findings as evidence alongside model documentation, validation, and human review.
Clarify vs. related AWS services
| Service | Primary purpose | Exam clue |
|---|---|---|
| SageMaker Clarify | Bias analysis and model explainability | Why did the model make this prediction? Is there potential bias? |
| SageMaker Model Cards | Document intended use, evaluation, risks, and limitations | Need a governance record or audit documentation |
| SageMaker Model Monitor | Monitor deployed model quality and drift | Detect changes in production data or model behavior |
| Amazon A2I | Human review for selected predictions | Route uncertain outputs to human reviewers |
Exam traps and limitations
- Clarify explains and analyzes; it does not automatically correct bias or retrain a model.
- SHAP feature attribution helps explain contributions to a prediction; it is not a guarantee of causal explanation.
- Pre-training bias analysis focuses on data before training; post-training analysis includes model predictions.
- Model monitoring, model documentation, and human review are separate capabilities.
- Keep the current availability notice in mind: Clarify is no longer open to new customers.
Test yourself
Question 1
A team needs to understand which input features contributed to an individual model prediction and generate feature-attribution explanations. Which SageMaker capability is the closest match?
- SageMaker Clarify
- SageMaker Ground Truth
- Amazon Translate
- Amazon Kendra
Reveal answer
Correct answer: A. Clarify supports model explainability and feature attribution, including SHAP-based methods.
Question 2
A data science team wants to examine potential bias in a dataset before training a model. Which analysis stage is relevant?
- Post-training bias analysis only
- Pre-training bias analysis
- Human-loop orchestration
- Speech synthesis
Reveal answer
Correct answer: B. Pre-training bias analysis examines the dataset before model training.
Key takeaway
Remember Clarify for bias detection and explainability, including pre-training and post-training analysis and SHAP feature attribution. It provides evidence for responsible AI reviews; it does not certify a model as fair or remove bias automatically.
Related AWS topics
SageMaker Model Cards · Amazon SageMaker · Amazon Augmented AI (A2I)
✦ CloudExamPro Premium
Ready for the AWS Certified AI Practitioner (AIF-C01)?
Move beyond memorizing terms. Practice with realistic, scenario-based questions and detailed explanations that help you understand the AWS AI services and concepts behind each answer.
Original practice questions. No dumps. No shortcuts.