Teaching AI Fluency

Anthropic

  Open in Claude







                ## What you'll learn

Estimated time: 40 minutes

By the end of this lesson you'll be able to:

Apply outcome, process, and reflection-based assessment strategies to evaluate AI Fluency
Create rubrics that capture the 4D competencies in your specific teaching context

How do we assess the 4Ds?

This video explores three complementary approaches to assessing AI Fluency: outcome-based (focusing on what students produce through AI collaboration), process-based (examining how students work with AI over time), and reflection-based (evaluating metacognitive awareness). The video demonstrates how to apply these three approaches to each of the 4Ds, providing specific examples of what to look for and how to evaluate student development. For Delegation, we assess goal-setting, task breakdown, tool selection; for Description, we evaluate conversation quality and iterative refinement; for Discernment, we look at critical evaluation skills; and for Diligence, we examine ethical decision-making and accountability. The video emphasizes that combining all three assessment approaches provides the most complete picture of student AI Fluency development.

Key takeaways

Effective AI Fluency assessment combines outcome, process, and reflection approaches for a complete picture
Each of the 4Ds requires different types of evidence and assessment strategies
Outcome assessments focus on products created, process assessments examine the journey, and reflection assessments evaluate metacognitive awareness
Observable actions and concrete artifacts are more reliable than assumptions about understanding
Assessment should be a learning opportunity, not just measurement

Exercises

This exercise helps you create a practical rubric for assessing AI Fluency in your specific course context.

Step 1: Establishing Your Assessment Foundation (5 minutes)

Start a new conversation with Claude or continue the conversation from Lesson 1:

Setting up the conversation:

Provide the AI with the transcripts from the two lesson videos.
If starting fresh, share your teaching context and explain you're working on assessment for teaching AI Fluency
If continuing, remind the AI you're now focusing on assessment strategies for your course
Describe a specific assignment or project you're planning where students will use AI
Explain which aspects of AI Fluency are most important for this particular assignment

Identifying your assessment priorities:

Discuss with the AI which of the 4Ds (and sub-competencies) are most relevant to your assignment goals
Explore what success looks like for your students in this specific context
Consider what common challenges or misconceptions you expect them to encounter
Determine what evidence would convince you that students have developed fluency

Step 2: Selecting Assessment Approaches (10 minutes)

Work with the AI to map assessment approaches to competencies:

For each D you're assessing, explore:

Whether outcome-based measures would work (what products would demonstrate competence?)
Whether process-based measures are feasible (can you access and evaluate chat logs?)
Whether reflection-based measures add value (what questions would reveal/promote understanding?)
Which combination of approaches gives you the clearest picture of student development?

Making practical decisions:

Discuss with the AI what's actually assessable given your time and resources
Consider which elements could be peer-assessed or self-assessed
Identify what you need to assess directly versus what students can document
Plan how to balance comprehensive assessment with grading efficiency

Step 3: Creating Observable Indicators (10 minutes)

Develop specific, observable criteria with the AI:

Defining performance levels:

Work with the AI to create three clear levels (such as emerging, developing, proficient)
For each level, describe what you would actually see in student work
Use language specific to your course and assignment rather than generic terms
Include concrete examples from your subject area where helpful

Writing clear indicators:

Transform abstract concepts into observable behaviors
Replace vague terms like "good understanding" with specific actions
Connect indicators directly to your assignment requirements
Ensure each indicator is something you can actually evaluate

Step 4: Formatting Your Rubric (5 minutes)

Create the final rubric document:

Ask the AI to help structure your rubric in a clear, usable format
Review with the AI whether each criterion is clearly distinguishable
Check that the progression between levels is logical and achievable
Verify that the rubric aligns with your stated learning objectives
Consider how you'll communicate this rubric to students
Ensure the language is clear for both you and your students

What's next

In the next lesson, we'll focus on designing assignments that help students both develop and demonstrate AI Fluency.

Feedback

As you progress through the course, we'd love to hear from you about how you are using concepts from the course in your life, work, or classes and any feedback you may have. Share your feedback here.

Acknowledgments and license

Copyright 2025 Rick Dakan, Joseph Feller, and Anthropic. Released under the CC BY-NC-SA 4.0 license. This course is based on The AI Fluency Framework by Dakan and Feller.Supported in part by the Higher Education Authority, Ireland, through the National Forum for the Enhancement of Teaching and Learning.

                        #### Downloads

                        - Delegation assessment matrix.pdf

Description assessment matrix.pdf
Diligence assessment matrix.pdf
Discernment assessment matrix.pdf
TAIF_TRANSCRIPT_04_ASSESS.txt