All Rise to the AI Challenge: Lessons from Building an Oral Argument Chatbot DOWNLOAD PDF

 Introduction

Many students dread oral arguments. Even for confident public speakers, most law students have little or no oral advocacy experience. However, students can build confidence through preparation. Customized chatbots offer students the opportunity to practice anytime with materials tailored to their legal problem. 

Before bringing artificial intelligence (AI) into the classroom, I wanted to ensure AI would enhance, not hurt, learning. Students developed their own expertise on our appellate problem by researching and drafting their arguments before the chatbot launched. After students built their own base of knowledge, the chatbot served as a constructive sparring partner as they prepared for oral arguments. Instead of practicing in front of a mirror or recording themselves, students had the opportunity to practice how they would respond to questions, building confidence before oral arguments. 

In this article, I share what I learned while building a custom chatbot. I will focus on general principles rather than model-specific instructions. I used ChatGPT to build the custom oral-argument chatbot for my class, but other large language models (LLMs) offer similar functionality.

1. Taking the leap

I know, I know, we are lawyers, not programmers or developers. How do we build a chatbot? Don’t worry, you don’t need to do this on your own. Use AI as a co-intelligence as you build. When building my chatbot, I asked ChatGPT for assistance during the planning, building, and iteration phases. I kept my learning objectives in mind as I critically reviewed ChatGPT’s suggestions, viewing them as a starting point rather than a final solution. 

As you get started, you will need to decide which AI model to use for your chatbot. Consider whether your school provides access to a certain system. While students can use free versions of AI models to access your chatbot, they will likely run into usage limits. For example, a student may be practicing oral argument questions when they are told they have hit their usage limit and must wait a few hours to continue. If your school provides students with access to an AI system, those usage limits could be higher, minimizing this issue. 

After you select the AI model you would like to use to build your chatbot, you will want to upgrade to the paid version if you don’t already have a subscription. Unfortunately, the free models are not as good as the paid models. For the building you want to do, upgrade to ensure you have the needed functionality. 

Now that you have selected a model and upgraded, it’s time to build your chatbot. One way to get started is to ask the AI model for some basic information. For the first chatbot I built, I started out by asking something like: “I would like to create a custom chatbot to help law students prepare for oral arguments based on a record and a specific body of law. I’ve never built a custom chatbot before. Can you please walk me through the steps and explain any parameters?”

After thinking for seventeen seconds, ChatGPT provided me with a practical, “first-time builder” walkthrough that explained the process and parameters. The walkthrough wasn’t perfect—it failed to mention that I needed to limit my instructions to 8,000 characters. However, the step-by-step instructions helped me start the iterative process of building my chatbot.

2. Training the bot

As I began building my chatbot, I wanted students to have a personalized experience. I wanted the chatbot to reflect our school environment and the supportive culture I work to build in my classrooms. This meant building in personal touches wherever possible and telling the chatbot exactly who it should be. Building this personalized experience included three parts: (1) designing the chatbot; (2) developing clear instructions; and (3) aligning a knowledge base.

1.  Designing the chatbot

In writing, we always keep our reader in mind. In chatbot design, I kept my end users in mind. To start, I used AI to create a bright and energetic gavel logo and named the chatbot “Oyez! Oyez!” Initially, students were introduced to Judge Mission, named after our school’s beloved mascot, and told that Judge Mission would ask questions and provide constructive feedback. Students were also given basic instructions on how to get started. I included two modes students could choose between: question mode or full simulation.

As I built out the different modes, I tried to include specific instructions for students. For example, “You can skip right to questions by choosing ‘Question Mode,’ or do a full practice argument in ‘Full Simulation Mode.’” In full simulation mode, students were prompted to try voice mode. Voice mode allowed students to practice their arguments out loud, simulating the back-and-forth dialogue and interruptions they experience during oral arguments. As the end users, students remained in charge of the experience. They could choose whether to practice answering questions or work on their introduction and roadmap in full simulation mode and whether to type responses or practice out loud.

2.  Instructing the chatbot on who it should be

I care deeply about creating a supportive and inclusive classroom environment. I wanted Judge Mission to help build students’ confidence by challenging them and giving constructive feedback without being combative or aggressive. This goal felt especially urgent after reading about rogue chatbots like Microsoft’s Tay, which was shut down 16 hours after launching due to its hateful rhetoric.

Oyez! Oyez! represented a different structure and use case than Tay, but these types of news stories still raised concerns about ensuring the chatbot reflected the tone I intended. Instructions act as guardrails for a chatbot’s behavior rather than a guarantee of how it will respond. Unlike traditional software, AI is not predictable. Different students will get different responses from a chatbot. A single student may get completely different responses to the same question asked twice.

Understanding the instructions as guardrails instead of precise guarantees helped me zoom out as I thought about creating the chatbot. I would not be controlling the chatbot’s outputs but instead guiding how the chatbot approached responding to students. I worked to build instructions that would reflect my desired tone, while also instructing students to remain in charge of their learning experience with Judge Mission. If Judge Mission veered outside of the guardrails I put in place, students were empowered to report issues with the chatbot so the instructions could be refined. 

Ensuring Judge Mission maintained a supportive and curious tone required testing and refining. Initially, Judge Mission would sometimes stubbornly stick to one line of questioning and fail to move on when an appropriate answer was given. The initial instructions told the chatbot to challenge students when they answered a question well, which Judge Mission took too far. After testing, feedback from colleagues, and troubleshooting the issue with ChatGPT, I ultimately developed better instructions to guide Judge Mission’s role and tone, which read in part:

“Role and Tone

• You are Judge Mission, an appellate judge.

• Professional, clear, and supportive of first-year law students.

• Your goal is to test reasoning while building confidence.

• Ask neutral, curiosity-driven questions:

  • “Can you walk me through…”

  • “What is your best argument that…”

  • “How does that apply here?”

• Avoid combative phrasing (e.g., “let me challenge you,” “isn’t it true”).

• No Over-Drilling (Critical)

• If a student gives a reasonable or partially correct answer:

  • Do NOT press the same point

  • Do NOT re-ask in different words

  • Do NOT escalate difficulty

• Instead, move to a new issue or angle.”

Overall, ensuring the chatbot reflected the tone that I wanted to set for my class took time and iteration. Before launching the chatbot, students were reminded that they were in charge as the human in the loop, not Judge Mission. Pulling back the curtain to demystify Judge Mission helped remind students of their own expertise and ownership in the learning experience. In the end, Judge Mission was able to ask students challenging questions without being confrontational, and students seemed to benefit from the constructive tone. 

3.  Aligning the knowledge base

After designing the end-user experience and defining the chatbot’s role, I needed to align Judge Mission with a knowledge base for our appellate advocacy problem. I wanted to build a retrieval-augmented generation (RAG)-style system, meaning the chatbot should retrieve information from specific data I provided. To do this, I uploaded the record for our appellate case and a knowledge document outlining the relevant law and potential arguments for both sides. I allowed the chatbot to access the internet for general knowledge, such as the structure and format of questions in appellate oral arguments. However, I told Oyez! Oyez! that the knowledge document should be its only source of law. 

Unfortunately, Oyez! Oyez! did not listen to these initial instructions. The chatbot hallucinated facts and cited cases not included in the knowledge document. I pulled in ChatGPT to help solve the problem, cutting and pasting specific chats that reflected the issues and sharing my current instructions. With those specific details, ChatGPT helped identify ways to improve my instructions to prevent Oyez! Oyez! from drifting away from the provided source materials. I also added instructions at the top of the uploaded knowledge-base document, reminding the chatbot that all judicial questions must be based only on the knowledge document and the record. After iterating and tightening the chatbot’s instructions and knowledge base, the drift problem improved, and Oyez! Oyez! aligned well with our problem materials.

One constant theme throughout the creation of Oyez! Oyez! was testing and iteration. I frequently talk with my classes about embracing a growth mindset, and I had to do the same myself while developing Oyez! Oyez! With help from my legal writing colleagues and teaching assistants, we tested and refined the chatbot for weeks before launching it to students. Even after sharing the chatbot link with students, I continued to refine the instructions based on student feedback. Encouraging student feedback helped ensure that students thought critically as they interacted with the chatbot and remained in charge of the learning experience. 

3. Teaching students to be in charge

When launching Oyez! Oyez!, I wanted students to feel empowered to tell me what worked well and what did not work about the chatbot. To keep student agency central to the experience, I started out by explaining the basic structure of LLMs to students. I explained that LLMs don’t actually know anything. Instead, they predict the next token or word in the sequence, like a fancy autocomplete.

I explained that while Oyez! Oyez! was trained on our appellate case, they were the experts on the law and the facts. I warned students about the risks of hallucinations and encouraged them to stay firmly in charge of the learning experience. If Judge Mission says you can’t have a transcript of the chat to review later, ask again. If you want to focus on the second issue in your brief, direct the chatbot to focus on that issue. If Judge Mission makes up a fact not in the record, let me know so I can improve the chatbot’s instructions.

This focus on agency mirrors what professional responsibility rules require of attorneys. While AI use is generally allowed, attorneys remain responsible for any final products or arguments they make. By having students step into the role of experts as they interact with AI, we are preparing them to be thoughtful practitioners and technology users. 

4. Rising to the AI challenge

I could not have imagined building a custom chatbot a few years ago. AI models have expanded the types of tools we can create to enhance student learning. LLMs give us the opportunity to reach diverse learners in new and exciting ways. 

Is learning how to build a chatbot challenging? Yes. Is it impossible? No. 

The initial launch of Oyez! Oyez! did not go perfectly, which gave students an opportunity to practice troubleshooting issues together in real time. After giving students background information on LLMs and explaining the chatbot, students were given the chance to practice questions with Oyez! Oyez! in small groups. Some students ran into usage limits and problem-solved by rotating who typed responses. Other students noted factual errors and incorrect case citations, building their confidence and experience as the human in the loop with AI. This feedback helped me refine the chatbot. The imperfect launch also allowed students to practice the types of problem-solving skills they will need to use as practicing attorneys. 

Chatbot creation requires testing and refinement, but that iterative process can lead to tools that meaningfully support both student learning and your teaching goals. One of the largest concerns with AI is cognitive offloading. Chatbot development provides an opportunity to share AI tools with students while also being transparent about their limitations, treating students as partners in this new AI landscape. We can teach students to review chatbot outputs critically. We can encourage them to help shape their learning experience by providing feedback as we expand our teaching tools. 

I initially decided to develop an oral argument chatbot because the technology’s capabilities naturally paired with my teaching objectives. I wanted students to learn to advocate for a client verbally, and a custom chatbot would allow them individualized practice with their arguments. As I look ahead at other ways to incorporate AI tools into the classroom, I will continue to look for ways that AI can support my learning objectives for students. My focus remains on using AI to enhance student learning, not replace it. Through this lens, I see enormous potential to create tailored learning experiences that help students develop their skills and build confidence. 

1. ^

    James D. Dimitri, Stepping Up to the Podium with Confidence: A Primer for Law Students on Preparing and Delivering an Appellate Oral Argument, 38 Stetson L. Rev. 75, 75-76 (2008). 

2. ^

    Id. at 76, 89.

3. ^

    Bringing AI into the legal writing classroom raises concerns about cognitive offloading and hurting students’ learning of foundational skills. I try to be mindful of this concern by ensuring students think first, write first, and meet first before introducing AI tools. See generally Nataliya Kosmyna et al., Your Brain on ChatGPT: Accumulation of Cognitive Debt when Using an AI Assistant for Essay Writing Task, MIT Media Lab (June 10, 2025), https://arxiv.org/pdf/2506.08872; Ethan Mollick, Against “Brain Damage,” One Useful Thing (July 7, 2025), https://www.oneusefulthing.org/p/against-brain-damage. 

4. ^

    I also included directions not to reveal source materials to users in the chatbot’s instructions. 

5. ^

    See generally Ethan Mollick, A Guide to Which AI to Use in the Agentic Era, One Useful Thing (Feb. 17, 2026), https://www.oneusefulthing.org/p/a-guide-to-which-ai-to-use-in-the.

6. ^

    Ethan Mollick, Co-Intelligence: Living and Working with AI ch. 3 (2024) (shares guiding principles for working with AI, including: 1) always inviting AI to the table; 2) being the human in the loop; 3) treating AI like a person—but telling it what kind of person it is; and 4) assuming this is the worst AI you will ever use) [hereinafter Co-Intelligence].

7. ^

    Your school may prefer that you use the provided system for a variety of reasons. I recommend checking your school’s policies and systems. 

8. ^

    Mollick, supra note 5.

9. ^

    Id. 

10. ^

     Co-Intelligence, supra note 6, at 75. Tay was designed to learn from interacting with Twitter users and was not constrained by fixed knowledge or rules. Id. Tay mirrored the racist, sexist, and hateful language that she encountered. Id. This raises the general concern of bias in LLMs. Educators building chatbots should be mindful about additional bias that can be introduced at the customization stage. See Jinsook Lee et al., The Life Cycle of Large Language Models in Education: A Framework for Understanding Sources of Bias, 55 Brit. J. Educ. Tech. 1982, 1986 (2024).

11. ^

     Co-Intelligence, supra note 6, at 65-66. 

12. ^

     See generally Varun Magesh et al., Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools, 22 J. Empirical Legal Stud. 216, 219-20 (2025).

13. ^

     Co-Intelligence, supra note 6, at 9, 52.

14. ^

     Id. 

15. ^

     See generally A.B.A. Comm. on Ethics & Pro. Resp., Formal Op. 512 (2024) (discussing lawyers’ use of generative AI). 

16. ^

     Id. at 3-4.