How to use LLMs for the investigation of claims
We've all gotten into the back and forth. We've all been swayed by the flattery. We've all put our brain on auto-pilot. That's OK.
But this is a session about how to take back control; how with AI to stay above it, not in it.
Not (just) about how we think about what is on our screen, but
how we use AI as a tool to navigate complex information environments.
Stop and get it in. Investigate the evidence. Find better sources. Track it down!
Key Insight: The first step is the hardest part of the journey
Just select the whole claim or upload the whole image
Search was about keywords
LLMs can process claims
Use "evidence-focused" not "answer-focused" follow-ups.
Assemble, weight, and categorize evidence
Don't rush to the answer, explore the space
Explore instead of sleepwalk with an evidence-focused prompt
Categorize, list, produce tables of evidence and sources
Keep a follow-ups file
Choose the follow-up that is most suited to your information need...
Look at sources for support, depth, accuracy
If you see links, scan them. If you don't, ask for them.
Ask for the links you need, whether types of publications, or a link to the original source.
Choose a source-focused follow-up:
Click the links. AI made you the map, now head to your destination.
This is a move we use throughout!
Keep tethered!
Keep a file of follow-ups, or use a text-expander plugin for quick use.
Stop and get it in. Investigate the evidence. Find better sources. Track it down!
Now let's try it together.
Note: "Track it down" is used throughout — we hover over sources, we click, we verify.
You see this come across your feed:
Copy the text below:
Find me a link to the original source of this information. If not available, find me a link to the closest thing to the original source.
Click through the links, and the links in the linked items until you find the study.
Ask what people found and how they got there (what did they click?), then do front of class walkthrough
Copy this text:
Evaluate the evidence for the claim that _____ and provide a table that matches evidence to rebuttals and rates the strength of the evidence
Fill in the blank with the claim you are interested in — memory benefits? Cognitive benefits? Then enter and review the evidence chart.
Once done reviewing, turn and talk to your classmate. Compare the results you got with the results they got and note differences. Then discuss how the different types of evidence impact your confidence that chocolate improves memory in humans and why.
Teacher should ask students what they found compelling, and particularly if there was any information they learned that changed how they thought about the research
Copy one of the following follow-ups:
How does the research that is industry funded compare to the research that is not industry funded when it comes to findings around this category of outcomes around ____?
OR
Is this ____ study industry funded?
Fill in the blank with the claim or study topic and submit.
One of the greatest fears of teachers is that LLMs are "thinking for" students
One of the greatest fears students have? The same!
How do we address it? How do we help our students?
Chatbots were a historical mistake of sorts
These are tools that help you navigate an information environment
The skills you need?
What we teach students: how to ask the right questions, how to interpret sources
This is a fraught time for critical thinking.
This is a golden age for critical thinkers.
By teaching these simple habits, and showing students the power of the right questions, we have a chance to choose the brighter path.
The weird thing? The skills we teach when applied to a tool like this become superpowers.
Critical thinking — or critical doing — may be in jeopardy, but its benefits have never been more visible or immediate.