Code: S2
Evaluate AI systems using defined criteria, expected outcomes, test cases and user feedback.
Descriptors
Basic
What you learn at this level
Learners define criteria for whether an AI system has accomplished a task.
Learners examine an AI system that recommends new music. In small groups, they define criteria for what would make the system successful and compare the system’s recommendations for users with different interests or interaction histories.
Intermediate
What you learn at this level
Learners assess an AI system performance of a task using defined criteria and feedback from human reviews or benchmark tests.
Learners identify strategies they would use to solve a word puzzle. The teacher presents a sample algorithm for the same task. Using defined criteria (e.g. efficiency, consistency, clarity), learners compare their peers’ strategies to the algorithm, identifying strengths, limitations and trade-offs in each approach.
Advanced
What you learn at this level
Learners design their own evaluation criteria for an AI system, compare performance across different inputs and users and use it to propose improvements.
In pairs, learners draft proposals for an AI tool and share their ideas. Each learner asks their partner a series of questions about their proposal. Next, each partner fills out a model card to identify intended and unintended users, impacts and use cases for the other’s idea. They provide feedback to each other based on the model cards and revise their own tool proposals accordingly.