Science of Justice
Our science, your art.
You've got the vision; we've got the data.
Is our science the right fit for your practice? Is the earth round? Let’s find out. We have created a unique suite of machine intelligence solutions that provide you with the best information in your legal cases. We explore insightful results through our proprietary algorithms with experts with decades of experience working with behavioral science issues or collaborating with legal advisors for successful case outcomes.
Science of Justice
Beyond Generic AI: Why Specialized Legal Tools Matter
Use Left/Right to seek, Home/End to jump to start or end. Hold shift to jump forward or backward.
Generic AI tools present serious risks for attorneys including hallucinated legal facts, confidentiality breaches, and strategic failures that can lead to sanctions and case dismissals.
• Large Language Models (LLMs) like ChatGPT create "hallucinations" - confidently stated but completely fabricated legal information including non-existent cases with fake names and citations
• Courts have sanctioned attorneys who submitted AI-generated fake cases, as in Mata v. Avianca and cases involving James Martin Paul
• Using generic AI violates ABA Model Rule 1.1 (duty of competence) when attorneys fail to verify information
• Consumer AI platforms often claim rights to store and reuse input data, violating attorney-client confidentiality under Rule 1.6
• Generic LLMs lack specialized knowledge needed for effective jury selection, missing critical psychographic factors that predict juror decisions
• Specialized legal AI tools offer better alternatives with proper security protocols, contractual data protections, and litigation-specific capabilities
• Attorneys remain fully responsible for verifying all AI outputs regardless of which tools they use
The path forward requires shifting from generic to purpose-built legal technology platforms that incorporate legal rigor, security compliance, and domain-specific expertise while maintaining human oversight of all AI-generated content.
https://scienceofjustice.com/
The AI Integrity Crisis in Litigation
SPEAKER_01If your case strategy and trial prep includes Chat GPT, you might actually be preparing for sanctions instead of success.
SPEAKER_00Aaron Powell That's right.
SPEAKER_01Aaron Powell Today we're breaking down the real risks behind the rise of this generic machine intelligence and litigation. And maybe talk about what smarter attorneys are doing instead.
SPEAKER_00Aaron Powell It's an incredibly high-stakes environment right now, isn't it?
SPEAKER_01Aaron Powell Absolutely.
SPEAKER_00Aaron Powell The availability of tools like large language models, well, it promises immense efficiency. That's the lure.
SPEAKER_01Trevor Burrus Aaron Powell Sure, efficiency is always tempting.
SPEAKER_00Aaron Powell But the legal field, especially for high-stakes plaintiff work, operates under these immutable ethical and professional duties.
SPEAKER_01Aaron Powell Duties that don't just go away because there's new tech.
SPEAKER_00Aaron Powell Exactly. And these generic consumer models, they fundamentally do not respect those duties. They weren't built for it.
SPEAKER_01Aaron Powell Okay. So that's the core tension we need to explore for you, the listener. Our goal here is really to provide an essential understanding, you know, of the speci maybe hidden ethical and strategic dangers. Trevor Burrus, Jr.
SPEAKER_00Hidden is the right word sometimes.
SPEAKER_01Trevor Burrus Right. Dangers that these general purpose LLMs pose when you're dealing with really critical tasks. Things like V Wadir or just managing sensitive client data.
SPEAKER_00Hugely critical tasks.
SPEAKER_01Aaron Powell We probably need to be clear right up front. These generative models, they're trained mainly for conversational fluency, aren't they? For predicting the next word.
SPEAKER_00Aaron Powell That's their core function, prediction.
SPEAKER_01Aaron Powell Not for the absolute competence and frankly the rigorous security that you need in a courtroom setting.
SPEAKER_00Aaron Powell Absolutely not. They can sound supremely confident, though. That's the trap.
SPEAKER_01Aaron Powell Yeah, that confidence. It sounds convincing.
SPEAKER_00Aaron Powell But it's entirely without foundation when it comes to, say, verifiable legal facts or client security protocols. Confidence doesn't equal competence here.
SPEAKER_01Aaron Powell That false confidence. Is that the biggest danger, you think?
SPEAKER_00Aaron Powell It's arguably the greatest danger, yes, because the models synthesize information so convincingly.
SPEAKER_01Right.
SPEAKER_00It leads attorneys, understandably perhaps, to treat them as authoritative experts, like a super fast research assistant.
SPEAKER_01Aaron Powell But they aren't.
SPEAKER_00No. In reality, they are tools. Tools that require a skeptical, verified approach. Always.
SPEAKER_01So treating them as a substitute for, you know, proper legal research and independent strategy.
SPEAKER_00That exposes the attorney and the client to tremendous risk, just massive risk.
SPEAKER_01Okay. So where does that risk start? You mention an integrity crisis. Aaron Powell Yeah.
SPEAKER_00Let's start there. The integrity crisis, it really begins with hallucination.
SPEAKER_01Aaron Powell Okay, let's break that down. The mechanism of failure. What exactly is a generic LLM doing when, say, a plaintiff lawyer asks it to pull up a recent precedent or maybe find a relevant statute? Yeah. It's not accessing some secure, vetted legal database like Westlaw or Lexus, is it?
Hallucination: When AI Fabricates Legal Facts
SPEAKER_00Aaron Powell, not even close. Not the generic ones. At their core, these LLMs are, someone called them highly sophisticated stochastic parrots. Stochastic parrots, meaning meaning they're pattern matching machines. They have been trained on just a massive, often unverified body of human language scraped from the internet. So when you ask it a question, it's calculating the statistically most probable sequence of words that should follow your prompt. It prioritizes linguistic fluency, how convincing it sounds, way over factual verification. Accuracy is secondary to plausibility.
SPEAKER_01And in the legal world, well, precision is everything, isn't it?
SPEAKER_00Paramount.
SPEAKER_01So this capability, this fluency over facts, leads to what people call hallucinations. That's the term, yes. Can you help us understand how severe a legal hallucination is? Is it just like getting a date wrong or it's much worse than a standard factual error?
SPEAKER_00A standard error might be easily spotted, you know, a typo in a date or something. Right. A legal hallucination, though, is it's a confidently stated, often very detailed, and completely fabricated piece of information. We're not just talking about missing a citation or getting a case name slightly wrong. We're talking about completely manufactured case law.
SPEAKER_01You mean cases that don't exist at all?
SPEAKER_00Correct. Including specific made-up case names, docket numbers, even fabricated judges' names, or non-existent statutes, or maybe confident explanations of legal principles based on laws that were repealed years ago or are just flat out outdated.
SPEAKER_01Why does it do that? Why make things up?
SPEAKER_00Because the tool is designed to fill in informational gaps. If it doesn't know the answer from its training data, it tries to construct something that looks like a plausible answer based on the patterns it learned. And for a lawyer who isn't super careful For the untrained legal eye, or even a busy lawyer just trying to save time, that plausibility can be completely deceiving. Dangerously so.
SPEAKER_01And there's another issue, isn't there? The knowledge cutoff date.
SPEAKER_00Oh, yes. That's a huge factor.
SPEAKER_01So if an LLM was trained, say, only on data up to September 2021, how does that practically affect a lawyer looking for a current precedent? Something decided last year.
SPEAKER_00Aaron Powell It creates this massive time capsule problem. It's like asking someone about 2023 who only knows about the world up to late 2021.
SPEAKER_01Aaron Powell So new laws, new cases.
SPEAKER_00Aaron Powell If a state Supreme Court decision, a new federal regulation, maybe a critical legislative change happened after that training cutoff date, the LOM literally does not know it exists. So if you ask it about a legal development from 2023, it might try to predict the answer based on its 2021 data. Aaron Powell Which leads to a hallucination or just completely wrong information that bears no resemblance to the current state of the law.
SPEAKER_01Aaron Powell And for plaintiff attorneys dealing with constantly evolving areas like tort reform, mass torts, shifting rules in different jurisdictions.
SPEAKER_00Trevor Burrus Relying on that kind of temporally limited information is just it's a formula for malpractice. Yeah.
SPEAKER_01Trevor Burrus And this isn't just theoretical risk, right? We've seen this blow up in practice. Trevor Burrus, Jr.
SPEAKER_00Oh, absolutely. We've already seen the consequences play out very publicly. These are immediate real-world examples of the professional danger. Trevor Burrus, Jr.
SPEAKER_01Like the Mata versus Avianta case. That seems to be the one everyone talks about. Trevor Burrus, Jr.
SPEAKER_00That case is, yeah, it's the textbook example. A huge warning sign that every single lawyer should know about.
SPEAKER_01Aaron Powell What happened there exactly?
SPEAKER_00Aaron Ross Powell Well, the attorney filed a brief in federal court, a formal legal document.
SPEAKER_01Okay.
SPEAKER_00And in that brief, he cited about half a dozen cases. Cases that were entirely fabricated by a generic LLM he'd used for research.
SPEAKER_01That just didn't exist.
SPEAKER_00Didn't exist. Pure invention by the AI.
SPEAKER_01How did it come out?
SPEAKER_00Opposing counsel couldn't find the cases, obviously. They alerted the court. The judge investigated.
SPEAKER_01And found out the lawyer relied on Chat GPT or a similar tool.
SPEAKER_00Exactly. The judicial investigation revealed the attorney's reliance on the AI.
SPEAKER_01What made those fake cases look real enough to fool the lawyer? I mean, wouldn't you check?
SPEAKER_00You'd think so, but they weren't just random nonsense. The hallucinations included specific, seemingly legitimate details.
SPEAKER_01Like what?
SPEAKER_00Like volume numbers, page numbers that looked right, court names, even fabricated quotes from these phantom judicial opinions that crucially fit the context of the attorney's argument.
SPEAKER_01So they sounded convincing.
SPEAKER_00Very convincing. Remember, the LLM's job is to produce plausible language. And it did that really well here. It created citations that looked professionally sourced.
SPEAKER_01Leading the attorney to skip the essential step.
SPEAKER_00Of checking the primary source. Going to the actual reporter or database.
SPEAKER_01And the result.
SPEAKER_00Public embarrassment, a judicial finding of bad faith conduct, and severe sanctions. Significant fines imposed by the court for submitting misleading information. A real black eye.
SPEAKER_01But it can get even worse than fines, can't it? There was another case, James Martin Paul.
SPEAKER_00Yes, and the consequences there were truly catastrophic for his practice and his clients.
SPEAKER_01What was different about the Paul case?
SPEAKER_00The critical distinction was the repetition. He was sanctioned not just once, he kept doing it.
SPEAKER_01Kept submitting filings with fake cases.
SPEAKER_00Repeatedly submitted court filings containing hallucinated case law, completely fabricated quotations, even after the court explicitly warned him, and opposing counsel kept pointing out the errors.
SPEAKER_01That sounds deliberate almost.
SPEAKER_00The court certainly viewed it as a deliberate failure of professional duty, not just a mistake.
SPEAKER_01And the outcome.
SPEAKER_00Which means it's essentially a death blow to the client's claim. They can't refile, can't fix it, their case is just gone. And on top of that, he was forced to pay the defense attorney's fees for the trouble he caused.
SPEAKER_01So the courts are not playing around with this?
Real-World Consequences: Sanctions and Case Dismissals
SPEAKER_00No. They've been explicit, ignorance of how the technology works, or just relying on it blindly. That is not a justifiable excuse for professional negligence. Period.
SPEAKER_01This leads us right to the bedrock ethical rule, doesn't it? ABA model rule one point one, the duty of competence. Exactly how do these hallucinations directly violate Rule 1.1?
SPEAKER_00Rule 1.1 mandates that lawyers maintain the requisite knowledge and skill for representation. That's basic.
SPEAKER_01Right.
SPEAKER_00But crucially, the ABA has clarified this includes understanding of the risks and benefits associated with relevant technology. It's part of competence now.
SPEAKER_01So if you use AI.
SPEAKER_00If you employ a machine intelligence tool, you are ethically mandated to understand how it works, at least fundamentally. You need to know it generates outputs based on prediction, not verification.
SPEAKER_01And you have to check everything.
SPEAKER_00The lawyer has an absolute ethical duty to exercise independent verification and rigorous scrutiny of every single output that goes into their work product. Every citation, every legal assertion.
SPEAKER_01Aaron Powell, So you can't just treat an LLM's output like a case citation it gives you as authoritative.
SPEAKER_00Aaron Powell Never. Not without performing your own due diligence. You have to check the actual court records, the primary sources. If you don't, you've committed a clear breach of your duty of competence.
SPEAKER_01Aaron Powell The technology doesn't let you off the hook.
SPEAKER_00Aaron Powell No. If anything, it heightens the responsibility of the human practitioner.
SPEAKER_01Aaron Powell So the bottom line is stark. If the technology is available, lawyers must become competent in its use.
SPEAKER_00Yes.
SPEAKER_01Which means really understanding its limits, its flaws, its dangers. If they don't understand that, they're simply not competent to use it for client work. Trevor Burrus, Jr.
SPEAKER_00Precisely stated. Right. Trevor Burrus And that competence mandate we just discussed, it mostly focuses on the output quality, but there's an equally dangerous ethical failure lurking on the input side where you feed into the machine. And that shifts our focus entirely to a different fundamental issue: client confidentiality.
SPEAKER_01Aaron Powell Okay, let's make that shift. The vulnerability of client data. This feels maybe less obvious than fake case law.
SPEAKER_00Aaron Powell It can be less obvious, but potentially even more damaging in some ways.
SPEAKER_01Aaron Powell So these generic LLMs, designed for mass consumer use, they're built for ease of access, engagement, maybe even fun. Aaron Powell Right.
SPEAKER_00Think chatbots, creative writing aids. Trevor Burrus, Jr.
SPEAKER_01They are emphatically not engineered with the kind of high-level security standards the legal industry demands, are they?
SPEAKER_00Aaron Powell No, not at all. This lack of specialization for legal or other high security fields is a huge red flag. Trevor Burrus, Jr.
SPEAKER_01Especially for plaintiff attorneys who handle incredibly sensitive information.
SPEAKER_00Aaron Powell Exactly. Think about it. You're handling detailed medical records, right? That often requires adherence to high PATA standards.
SPEAKER_01Absolutely. Strict rules there.
SPEAKER_00Aaron Powell And any high-stakes litigation involves strategy memos, deposition transcripts, settlement negotiations, stuff that demands comprehensive data security, things like SOC2 compliance.
SPEAKER_01SOC2. Can you explain that briefly?
Client Confidentiality and Data Security Risks
SPEAKER_00Sure. SOC2 is a framework developed by the AI CPA, the accounting folks, that specifies how organizations should manage customer data based on principles like security, availability, processing integrity, confidentiality, and privacy. Getting SOC2 certified means a third-party auditor has verified your systems and controls.
SPEAKER_01Okay, so it's a rigorous security standard.
SPEAKER_00Very. And generic LLMs, they typically lack these kinds of rigorous controls built in.
SPEAKER_01So if I input sensitive information, say detailed case notes about my client's injuries, maybe protected health information, PHI. Trevor Burrus, Jr.
SPEAKER_00Or confidential settlement strategies, details from a non-disclosure agreement.
SPEAKER_01If I put that into one of these general consumer platforms, what happens?
SPEAKER_00You are effectively compromising that data. You're putting it into an environment not designed to protect it adequately.
SPEAKER_01And the risk isn't just some theoretical hack, is it? It's actually written into the terms of service.
SPEAKER_00Yes. And this is the part that many lawyers, frankly, might skim past or not fully appreciate. It's what I call the terms of service trap.
SPEAKER_01Okay, explain the trap.
SPEAKER_00It's perhaps the most insidious contractual danger. When you use a generic LLM, you click agree to their terms, not yours. You're operating under their rules. And those rules often say most vendors explicitly disclaim any duty of confidentiality regarding the data you input. They essentially say, what you put in here, not our job to keep it secret.
SPEAKER_01Wow. Okay.
SPEAKER_00But it gets worse. More critically, they often reserve the right in those terms to store your input, process it, and here's the kicker, reuse your input data for the purpose of further training and improving their own underlying AI model.
SPEAKER_01Wait, so if I paste, say, a detailed confidential medical summary for my client's file.
SPEAKER_00Or maybe a unique WarDire questionnaire you developed looking for feedback on the structure.
SPEAKER_01If I put that into a generic LLM, that confidential information, my client's private medical details, my firm's proprietary strategy, it could now be used to train a commercial AI product.
SPEAKER_00Exactly. It gets absorbed into their training data.
SPEAKER_01Aaron Powell An AI that potentially anyone could interact with down the line.
SPEAKER_00That's the risk. Your client's confidential information, their vulnerabilities, your case strategy, their protected health information. It's retained by this third-party vendor who owes you no duty of privilege.
SPEAKER_01And it might inadvertently surface later.
SPEAKER_00It could be reflected or influence future outputs for completely different users in ways that are impossible to predict or control.
SPEAKER_01That sounds like an immediate, major ethical violation for any lawyer.
SPEAKER_00It is an egregious violation of fundamental ethical duties.
SPEAKER_01Which specific rule does this hit head on?
SPEAKER_00It directly violates ABA Model Rule 1.6, the cornerstone duty of confidentiality. That rule requires lawyers to make reasonable efforts to prevent the inadvertent or unauthorized disclosure of information relating to the representation of a client.
SPEAKER_01And using a tool that says we might use your data is not reasonable effort.
SPEAKER_00Clearly not. Using an LLM platform that explicitly reserves the right to retain and potentially use client data for its own training purposes is arguably the opposite of making a reasonable effort to prevent disclosure.
SPEAKER_01Any other risks tied to this? NDAs, maybe.
SPEAKER_00Absolutely. It also risks violating specific confidentiality agreements or non-disclosure agreements, NDAs, that many plaintiff firms have with their clients or with third parties, especially in complex commercial torts or sensitive personal injury cases. Feeding that protected data into an unsecured platform could constitute a breach of those agreements too.
SPEAKER_01So the danger isn't just a potential data breach by hackers.
SPEAKER_00No, that's a separate, also valid concern. But this risk is about the LLM company's own legitimate use of your data, as permitted by the terms you agreed to. It's a contractual issue creating an ethical nightmare.
SPEAKER_01It sounds like the due diligence required here is pretty intense.
SPEAKER_00It has to be meticulous. When you're assessing any technology that will handle client data, especially AI, you must look for explicit contractual guarantees from the vendor.
SPEAKER_01Guarantees about what specifically?
SPEAKER_00Guarantees that they will not retain your input data, not store it long term unless necessary for the service you requested, and absolutely not use it to train their models.
SPEAKER_01And that SOC2 compliance you mentioned?
SPEAKER_00That should be a baseline requirement. And ideally, not just SOC two type 1, which is a snapshot in time, but SOC2 type 2. Type 2 certification means an auditor has verified that the security controls are not only designed properly, but have been operating effectively over a sustained period, usually six months or more. It's a much stronger assurance.
SPEAKER_01So if a vendor doesn't offer that or won't guarantee data privacy.
SPEAKER_00It should be a mandatory, non-negotiable red flag. Stop right there.
SPEAKER_01And these checks, they're more rigorous than what you'd do for, say, cloud storage like Dropbox.
SPEAKER_00Far more rigorous. Because a simple storage service isn't actively learning from your documents. An LLM is designed to consume and process the content of the input. The risk profile is fundamentally different.
SPEAKER_01This responsibility, it doesn't just sit with the partner using the tool, does it? It flows down.
SPEAKER_00Absolutely. This brings in another ethical rule. ABA model rule 5.3, the duty to supervise non-lawyer assistance.
SPEAKER_01Rule 5.3, how does that apply to AI?
SPEAKER_00The ABA and state bars have made it pretty clear that this duty extends to supervising the use of technology, including machine intelligence.
SPEAKER_01So the supervising attorney is on the hook.
SPEAKER_00Completely. They are responsible for ensuring that all integrated AI tools, and critically, the way junior staff, paralegal's associates use them, do not result in ethical breaches or security failures.
SPEAKER_01Can you give an example? Sure.
SPEAKER_00Imagine a scenario. A busy paralegal, trying to be efficient, gets asked to summarize a huge batch of confidential medical records for a complex personal injury case. Lots of PHI in there.
SPEAKER_01Okay.
SPEAKER_00They think, hey, this ChatGPT thing is fast. They paste chunks of those records into the generic LLM to get summaries. They might assume it's secure or just not think about it.
SPEAKER_01Aaron Powell, but if that data gets retained or used for training by the LLM vendor.
SPEAKER_00The supervising attorney is the one who bears the professional and ethical liability for that disclosure under Rule 5.3. Even if they didn't know the paralegal used that specific tool.
SPEAKER_01So firms need policies.
Strategic Failures in Jury Selection
SPEAKER_00Yes. Clear, explicit protocols that ban the use of non-secure generic LLMs for any work involving confidential client information. Period. Training is also key.
SPEAKER_01So wrapping up this section on ethics and security, the core message seems to be that generic LLMs operate under a business model, really, of data capture and reuse.
SPEAKER_00Often, yes. Or at least they reserve the right to.
SPEAKER_01And that model is just fundamentally incompatible with the legal profession's absolute requirement for data security and client confidentiality.
SPEAKER_00Diametrically opposed, you could say. Convenience in this context simply cannot be allowed to outweigh the foundational duties of the profession. The risks are just too high.
SPEAKER_01Aaron Powell Okay. We've covered the integrity crisis with hallucinations and the major ethical wall around confidentiality. Let's pivot now to strategic failure.
SPEAKER_00Aaron Powell Right. Because even if you somehow navigated the ethical minefield, these tools often just aren't very good at the actual strategy part of lawyering.
SPEAKER_01Aaron Powell Especially for a plaintiff trial lawyer where the goal is one: securing a favorable jury, presenting a strategy tailored to a specific judge, a specific venue.
SPEAKER_00Aaron Powell Exactly. And this is where generic LLMs fail pretty dramatically, particularly in high-stakes strategic tasks like voir dire jury selection.
SPEAKER_01Aaron Powell Why is that? What's the fundamental mismatch?
SPEAKER_00Well, remember, general LLMs are programmed for generalization. They're masters of the average, the boilerplate, the statistically probable conversational response based on that huge mixed data set they trained on. They simply lack the capacity to weigh subtle strategic implications. They don't understand the hierarchy of legal authorities in your specific jurisdiction versus another.
SPEAKER_01They don't know the local rules or the judge's history.
SPEAKER_00Precisely. And they certainly can't align a case strategy with the nuanced context of a local court venue or a specific client situation or the opposing counsel's known tactics. It's all just text to them.
SPEAKER_01So their output, it focuses on maybe superficial things like grammar, clarity, giving you a general outline.
SPEAKER_00Yeah, it might help you structure an argument or polish prose, but it misses the deep strategic cuts, the nuanced points that actually win cases.
SPEAKER_01Aaron Powell Can you give an example outside of litigation, maybe?
SPEAKER_00Sure. Think about contract review. A generic LLM might review a complex commercial lease and offer, you know, generic suggestions on standard clauses like indemnification or force majeure.
SPEAKER_01Okay. Sounds helpful so far.
SPEAKER_00Aaron Powell But it will likely completely overlook the specific critical negotiation points unique to that deal. Maybe a poorly worded option to renew, or a subtle flaw in the rent escalation clause that could cost your client millions down the road. It misses the context specific risks and opportunities.
SPEAKER_01Aaron Powell Because it doesn't understand the client's business or the market conditions.
SPEAKER_00Exactly. It applies generalized knowledge, and that generalized thinking is absolutely catastrophic when applied to something like jury selection.
SPEAKER_01Where specificity is everything.
SPEAKER_00Everything. A generalized boilerplate LLM approach to var dye is practically useless if your goal is to actually generate a winning plaintiff strategy. It just can't do it effectively.
SPEAKER_01Jury selection. People often say it's where cases are won or lost, especially for plaintiffs. It's the foundation.
SPEAKER_00It really is. An effective war dire requires understanding deep attitudinal markers in potential jurors, not just service level stuff.
SPEAKER_01Aaron Powell Okay, so let's talk about that difference. You hear about demographics, but you also mentioned something deeper psychographics. What's the distinction?
SPEAKER_00Right. Demographics are the what? Age, occupation, maybe where they live, education level. Easy to collect, but not very predictive on their own.
SPEAKER_01Okay.
SPEAKER_00Psychographics are the why. They tell you why a juror might think the way they do. What are their core values? Their underlying beliefs, their personality traits, their motivations, their level of cynicism or trust in institutions.
SPEAKER_01Aaron Powell And that's what predicts how they'll decide.
SPEAKER_00Much more reliably, yes. Plaintiff strategies, successful ones, are built on identifying these key psychographic factors. Things like how much does this juror value personal responsibility versus corporate accountability? How skeptical are they of large companies by default? How much do they trust authority figures?
SPEAKER_01Can you give a concrete example of why this psychographic insight matters so much?
SPEAKER_00Absolutely. There's consistent data showing, for instance, that potential jurors identify it as having a high degree of inherent skepticism toward corporations. They are significantly more likely to favor the plaintiff in a typical civil tort case.
SPEAKER_01How much more likely?
SPEAKER_00Depending on the study and venue, somewhere in the range of 37 to 42 percent more likely.
SPEAKER_01Wow. That's a huge difference.
SPEAKER_00It's a massive strategic advantage. And that factor skepticism towards corporations far outweighs simple demographics, like whether the juror is male or female, or works in retail versus manufacturing.
SPEAKER_01And generic LLMs, they just can't find that.
SPEAKER_00They lack the necessary foundation. They don't have the behavioral science models baked in. They don't have access to proprietary localized venue data that tracks these attitudes over time. They weren't built to identify these predictive psychographic factors.
SPEAKER_01So if a lawyer asks a generic LLM, suggest some questions for Vardyyer in my personal injury case. What kind of output are they likely to get? How does it fall short?
SPEAKER_00It fell short because it lacks segmentation capability. It will likely give you very generic surface level questions. Do you know any of the parties? Have you ever been sued? Stuff like that.
SPEAKER_01Standard questions.
SPEAKER_00Right. The basics. But modern jury analysis, the kind winning firms use, involves layering different data sources to cluster potential jurors into predictive mindsets or archetypes.
SPEAKER_01Mindsets like what?
SPEAKER_00Well, you might interview two potential jurors. On paper, demographically they look similar, both male, middle-aged, maybe work in skilled trades.
SPEAKER_01Okay.
SPEAKER_00But through deeper questioning and analysis, you might find one is, say, a risk-averse pragmatist. This person might distrust plaintiff arguments because they worry large verdicts will just raise insurance costs for everyone, including them.
SPEAKER_01Right, worried about the system.
SPEAKER_00While the other juror, despite similar demographics, might be a fairness-focused empathizer. This person's core motivation might be seeing victims made whole, especially if they perceive corporate negligence or arrogance. They prioritize restoring balance.
SPEAKER_01Two very different jurors, potentially deciding the case differently.
SPEAKER_00Hugely different. And a generic LLM. It cannot reliably differentiate between those two mindsets based on simple demographic data or generic questions. It provides boilerplate output that misses the implicit biases, the underlying values that actually drive decision making in the jury room.
SPEAKER_01So the lawyer using only that is basically flying blind, relying on gut feeling.
SPEAKER_00Aaron Powell Pretty much. Gut feeling, which we know can be riddled with its own biases. It's not a systematic data-driven approach.
SPEAKER_01Aaron Powell You mentioned layering data. The outline mentioned five core metrics for assessing bias. Can we elaborate on those? What are they?
SPEAKER_00Sure. Effective modern jury assessment really uses a layered approach, pulling from multiple angles to get a complete picture. Think of it as five pillars.
SPEAKER_01Okay, pillar one.
SPEAKER_00Pillar one is demographic analysis, the basics age, gender, race, employment, location, foundational but limited. Pillar two. Social media and background review, looking into publicly available digital footprints, news mentions, property records, et cetera, for clues about attitudes or potential conflicts. Generic LOMs might help search this, but interpretation is key. Case-specific questioning. This involves carefully crafted survey questions, often administered before a formal voard dire, if possible, or integrated into the voir dire itself, designed to gauge opinions on specific themes relevant to your case, attitudes towards damages, corporate responsibility, specific industries, etc. Using validated behavioral science instruments, things like implicit association testing or IETs, which can measure unconscious biases related to key concepts like trust in institutions, attitudes towards wealth, or assigning blame. This gets below the surface of conscious answers.
SPEAKER_01And the final pillar.
SPEAKER_00Then the V war deer assessment. This is the observation and scoring of potential jurors' responses and how to answer. What's their body language? Tone of voice. Hesitations. This requires skilled human observation, often aided by specialized scoring systems.
SPEAKER_01That sounds like a tremendous amount of very specialized, behaviorally focused data.
SPEAKER_00It is. And here's the point a generic LLM might be able to help with maybe the first two pillars pulling basic background data. It might even try to generate some boilerplate questions for pillar three, but it absolutely cannot perform or interpret the results of behavioral testing like IETs, Pillar Four. And crucially, it cannot integrate the critical behavioral observations and nonverbal cues from the juror assessment in the courtroom, pillar five.
SPEAKER_01Aaron Powell Which leads us to another major strategic failure you mentioned.
SPEAKER_00Yes, the final and perhaps most profound strategic blind spot, the emotional intelligence gap.
Specialized Legal AI: The Path Forward
SPEAKER_01Tell me more about that gap. Particularly how does it affect evaluating witnesses and jurors during the actual trial process. Aaron Powell Right.
SPEAKER_00Because the courtroom isn't just about facts and law, it's inherently a human environment. It runs on subtle behavioral feedback loops, nonverbal communication, emotional responses.
SPEAKER_01Generic LLMs are.
SPEAKER_00They're text-based models. Fundamentally, they process words. They cannot analyze the nonverbal or the nuanced emotional cues that are absolutely essential for assessing witness credibility or scoring potential juror bias during live questioning. Things like things like micro expressions flickering across a witness's face when they're asked a tough question, changes in vocal pitch or hesitation patterns that might signal deception or uncertainty, posture shifts, averted glances from a potential juror who might be trying to mask a strong bias because they want to avoid conflict or get off the panel. Social desirability bias is huge in Voardier.
SPEAKER_01Aaron Powell And these are exactly the cues that experienced plaintiff attorneys rely on, aren't they?
SPEAKER_00Absolutely. They rely on these behavioral cues when making those crucial split-second decisions. Should I use a peremptory challenge on this juror? Should I push this witness harder on cross-examination or back off? It's reading the room, reading the person.
SPEAKER_01Aaron Powell Behavior modeling is key.
SPEAKER_00It's a key component of systematic trial assessment and strategy. And generic LLMs are completely blind to it.
SPEAKER_01Aaron Powell What about simulating the trial itself? Can they help predict how a jury might deliberate?
SPEAKER_00Aaron Powell No, not the generic ones. They're useless for that too. They cannot realistically model jury deliberation dynamics.
SPEAKER_01Why not?
SPEAKER_00Because deliberation isn't just aggregating individual opinions. It involves groupthink, social pressure, persuasion, conformity. How does a dominant personality influence others? How do certain arguments gain traction or fall flat within that specific group dynamic?
SPEAKER_01Aaron Powell The whole social soup of the jury room.
SPEAKER_00Aaron Powell Exactly. Those subtle emotional cues, the social dynamics, the pressures that ultimately shape a final verdict once the jury is sequestered. They are completely lost on generalized, non specialized models.
SPEAKER_01Aaron Powell So the answers they give are kind of in a vacuum.
SPEAKER_00Aaron Powell Precisely. They offer strategic answers or suggestions in a vacuum, completely ignoring the complex, messy, emotional reality of human. Interaction that defines every trial.
SPEAKER_01Trevor Burrus Okay, so we've laid out these massive liabilities, hallucination risks leading to sanctions, confidentiality breaches violating ethical duties, strategic incompetence, especially in critical areas like war dire.
SPEAKER_00So a pretty damning list.
SPEAKER_01Given all that, we really have to tackle this idea, this myth of the free or low cost tool. Because the appeal is still there, isn't it? It saves time, it's cheap or free. Many attorneys might still be tempted.
SPEAKER_00Of course. The perceived benefit is efficiency and cost savings. But it's often a classic example of being pennywise and pound foolish. Fault economy.
SPEAKER_01So we need to reframe that low cost. What's the true hidden cost of using these generic tools?
SPEAKER_00Aaron Powell The true cost is always born through risk. Always. The actual expense isn't the subscription fee, if there even is one. It comes from the consequences of failure. Such as such as court sanctions and fines, ethical violations that could damage your reputation or even lead to suspension, and profound strategic failures.
SPEAKER_01Aaron Powell Like what kind of strategic failure?
SPEAKER_00Like failing to strike that one dangerous juror who praisons the deliberation. Like relying on discredited evidence because your AI research missed a key ruling, like having a defense motion to dismiss, granted because your brief was billed on fabricated case law.
SPEAKER_01The cost of losing the case for your client?
SPEAKER_00That cost is often incalculable, isn't it? Especially for a plaintiff who may have only one shot at justice. That's the hidden expense.
SPEAKER_01Okay, but I can almost hear a skeptical junior partner listening right now thinking, look, even if I have to double check everything, if using a generic LLM gets me 80% of the way there on a first draft of, say, a motion response, then checking the citations and polishing it is just the last 20%. The efficiency gain is still huge. Isn't this risk being overstated just to cover that last bit of work?
SPEAKER_00That's a fair challenge to raise. It sounds logical on the surface, but I think it misses two really critical points.
SPEAKER_01Okay, what's the first one?
SPEAKER_00First, the human time required to truly independently verify and rigorously scrutinize generic LLM outputs. Yeah. It often completely negates the promised time savings, especially for complex legal work.
SPEAKER_01Aaron Powell Oh so?
SPEAKER_00Because you can't just check the citations, you have to check the underlying logic. Is the reasoning sound? Does this case actually apply to the specific facts and crucially the jurisdiction of my case? Is the interpretation of the law current?
SPEAKER_01It's not just proofreading.
SPEAKER_00No. And think about it. If the LLM output is, say, ninety percent accurate, which might be optimistic for complex tasks, that remaining 10% of errors, they're often the subtle ones. The contextually dependent mistakes, the plausible sounding but legally wrong assertions.
SPEAKER_01And those take longer to fix.
SPEAKER_00Often, yes. It can require more human research time to meticulously unravel and disprove a sophisticated hallucination than it would have taken to just conduct the research properly using reliable sources from the start. The 80% done might be an illusion.
SPEAKER_01Okay, that's a strong point about the time factor. What's the second critical point?
SPEAKER_00The second point is perhaps even more fundamental from an ethical standpoint. Using many generic LLMs fundamentally prevents you from fully fulfilling your duty of confidence. Rule 1.1, because you often cannot verify the reasoning process behind the output.
SPEAKER_01Explain that. Why can't you verify the reasoning?
SPEAKER_00It's the concept of explainability or XAI explainable AI. Many generic LLMs operate as black boxes.
SPEAKER_01Meaning you can't see inside.
SPEAKER_00Right. They can't easily show the attorney the specific source documents they relied on for a particular assertion or the logical steps they took to arrive at a conclusion. They just give you the output.
SPEAKER_01So you're left with just an assertion, a statement.
SPEAKER_00Exactly. And how can a lawyer rigorously vet an assertion, fulfill their duty of independent scrutiny if they cannot examine the underlying data stream or the logic used to generate it? You're being asked to trust the black box.
SPEAKER_01That is profound. If the tool can't explain how it reached its conclusion, the lawyer cannot possibly meet their ethical burden of verification and independent judgment.
SPEAKER_00I believe that's correct. Which circles us back again to the lawyer's inescapable duty of oversight, human oversight.
SPEAKER_01Technology as a support, not a replacement.
SPEAKER_00Precisely. Machine intelligence is meant to support, to augment, maybe to automate certain routine tasks, but it cannot and should not replace human judgment, legal expertise, and ethical responsibility. The lawyer must always remain the final skeptical filter. The buck stops with the human.
SPEAKER_01And the courts seem to agree.
SPEAKER_00The courts have been very clear on this, as we saw in the Mata and Paul cases. And the general consensus emerging in legal tech ethics discussions is interesting. What's the consensus? That the greater long-term danger in the legal tech world might actually be the underutilization of specialized, secure, properly vetted AI, leading to inefficiency and potentially disadvantaging clients. But the most immediate and professionally dangerous threat right now is the over-reliance on non-specialized, unvetted, generic tools like consumer chatbots for substantive legal work.
SPEAKER_01So don't be afraid of all AI, but be very afraid of the wrong AI used wrongly.
SPEAKER_00Exactly. And the burden rests entirely on the professional user, the lawyer to understand the difference, to vet the tools and to know where to draw that line between assistance and abdication of responsibility.
SPEAKER_01The lesson seems clear then. Courts have shown zero sympathy, zero tolerance for lawyers who just blindly rely on AI without doing their own professional verification. The moment you delegate a task, whether to a paralegal or a piece of software, you assume full professional accountability for the quality, the accuracy, and the security of the result. No excuses.
SPEAKER_00Correct. The human element, discretion, judgment, building client relationships, providing nuanced understanding in complex situations, strategic thinking that remains absolutely critical and irreplaceable. Technology is always subordinate to professional duty. Always.
SPEAKER_01Okay. So if these generic LLMs, the readily available ones, present such pervasive risks, hallucination, ethical breaches, strategic incompetence, where should plaintiff trial lawyers actually turn?
SPEAKER_00Yeah, what's the alternative?
SPEAKER_01How can they leverage the power of machine intelligence, which is clearly here stay, but do it safely and effectively? What's the necessary shift in approach?
SPEAKER_00The shift is towards specialization. That's the only viable path forward for high-stakes work. Meaning the solution lies in abandoning the general purpose chat models for legal work and instead moving toward purpose-built, industry-specific platforms. Tools engineered specifically for the demands of high-stakes civil litigation and trial preparation.
SPEAKER_01So tools designed by legal tech companies for lawyers.
SPEAKER_00Exactly. These specialized tools are constructed differently from the ground up. They're built to address the unique requirements of the profession, incorporating legal rigor, security protocols, and specialized data streams that generic models just don't have.
SPEAKER_01Let's contrast these specialized platforms with the generic models, maybe using the key risk areas we discussed. First, security and confidentiality. How do they differ?
SPEAKER_00The differences here are or should be non-negotiable. Purpose-built legal tech platforms worth considering will guarantee things like SOC2 compliance, demonstrating that ongoing operational security. And they should provide explicit, clear, contractual pledges right in the service agreement, that they will not store your input data unnecessarily, not process it for unrelated purposes, and absolutely not train their general models on your confidential client information.
SPEAKER_01So they respect privilege and confidentiality by design?
SPEAKER_00Exactly. Not just relying on the user to somehow avoid inputting sensitive data into a leaky system. That approach is doomed to fail.
SPEAKER_01Okay, so that addresses the rule 1.6, confidentiality, and rule 5.3 supervision compliance concerns we talked about. Huge relief for a firm.
SPEAKER_00It should be, yes.
SPEAKER_01Aaron Powell Now, what about strategic depth? Particularly for those advanced critical tasks like Voir Dyer. How do specialized platforms gain an edge over the generalized approach?
SPEAKER_00Aaron Powell They gain the edge through data and methodology, specifically through relevant data and behavioral science. Okay. Specialized tools designed for trial prep are often built on foundations that integrate deep psychographic modeling, understanding those juror archetypes we discussed.
SPEAKER_01Aaron Powell The why behind juror thinking.
SPEAKER_00Right. They incorporate localized venue data. How have juries in this specific county ruled on similar cases? What are the prevailing attitudes here? They utilize behavioral analytics drawn from actual trial outcomes and jury research.
SPEAKER_01Data that generic tools just can't access.
SPEAKER_00Correct. They can't access it. And even if they could stumble upon some of it, they lack the specialized models to interpret it effectively in a legal strategy context. This is what moves jury selection out of the realm of pure guesswork or relying only on demographics.
SPEAKER_01So instead of getting back a boilerplate list of suggested Vardyer questions from a generic tool, what kind of output does a specialized jury analysis tool provide? What's the deliverable?
SPEAKER_00It should provide an evidence-based roadmap for your jury selection strategy. A road Yes. For example, some advanced specialized platforms utilize simulation-based jury research. This goes way beyond traditional focus groups. Oh, so it uses sophisticated models, often based on data from thousands of real and mock jurors, to simulate how different types of jurors with specific profiles are likely to interact and deliberate as a group.
SPEAKER_01Simulating the jury room dynamics.
SPEAKER_00Exactly. Modeling actual deliberation dynamics, simulating how individual opinions might shift under group pressure, how specific arguments might land with different juror segments, how leader jurors might emerge. It allows counsel to pretest arguments, fine-tune witness order, even analyze how opposing counsel's likely themes might resonate.
SPEAKER_01All based on data-backed insights, not just anecdotes.
SPEAKER_00Correct. It turns jury research from a snapshot into a dynamic simulation.
SPEAKER_01What about that emotional intelligence gap we highlighted earlier? The fact that generic LLMs can't read the room, can't interpret live courtroom behavior. Do specialized tools help there?
SPEAKER_00Some specialized platforms are integrating technology designed to help bridge that gap, or at least provide better data for human interpretation.
SPEAKER_01In what way?
SPEAKER_00How does that work? These might use advanced behavioral science principles, sometimes even incorporating video analysis of practice sessions, to look for subtle facial expression cues, analyze vocal tone and pitch variations for stress or deception indicators, or even assess posture changes. The goal is to help coach witnesses more effectively before they ever step into the actual courtroom. Give lawyers objective data points to consider.
SPEAKER_01So it's about unifying all these different data streams.
SPEAKER_00Yes. The best specialized systems aim to unify complex data streams, the demographics, the psychographic mindsets derived from surveys or analysis, any available social media bias indicators, maybe even real-time responses during voardire, if the system allows for live input into a singular, continuously updated scoring engine or dashboard for each potential juror.
SPEAKER_01It sounds like these specialized systems provide a much more integrated, intelligent picture, refining the whole process from initial juror profiling right through to making that final challenge decision.
SPEAKER_00That's exactly the goal. Transforming Voardire from a highly subjective, often gut-driven exercise into something approaching strategic precision, grounded in data and behavioral science.
SPEAKER_01And ensuring the questions asked are designed to actually reveal those deep predispositions.
SPEAKER_00Right. Maximizing the likelihood of seating a favorable panel, or at least identifying and removing the most dangerous jurors. The strategic output is designed not just to inform, but to directly support the lawyer's critical decision-making process in a way that fully respects professional standards of competence and confidentiality.
SPEAKER_01So ultimately, the shift required for plaintiff trial lawyers isn't about abandoning technology.
SPEAKER_00Not at all.
SPEAKER_01It's about shifting from using a widely accessible generic tool that's designed primarily to sound persuasive.
SPEAKER_00To investing in and learning how to properly use a secured, purpose-built platform, one that's specifically engineered to help maximize the chances of winning a high-stakes civil trial within ethical boundaries. It really is the difference between using, say, a general internet search engine for complex medical research versus using specialized peer-reviewed databases and diagnostic equipment. You have to choose intelligence, specialization, and security over mere convenience and that potentially misleading generic confidence. The stakes demanded.
SPEAKER_01This has been an incredibly detailed breakdown. Really appreciate it. The dangers are clear, but so is the path forward. Hopefully, yes. So to synthesize the central message for you, our listener, the risks of just jumping onto generic LLMs for your courtroom preparations. They extend far beyond just getting some facts wrong or finding fake cases. They touch on fundamental failures in data security, core ethical obligations, and strategic depth, especially in that absolutely critical domain of jury selection.
SPEAKER_00You can't afford to get voir dire wrong.
SPEAKER_01Right. The high-stakes nature of plaintiff trial preparation simply requires specialized tools, tools that operate under the required standards of competence, confidentiality, and strategic relevance.
SPEAKER_00And always, always remember: the moment you introduce any technology, AI or otherwise, into the service you provide your client, you accept full professional accountability for its output and its impact.
SPEAKER_01You own the result.
SPEAKER_00You own the results. You absolutely must understand its limitations, assume the burden of rigorous verification, and ensure its security. Otherwise, any perceived efficiency game is simply not worth the potentially career-ending risk.
SPEAKER_01So maybe the takeaway is this start asking better questions, not just of your witnesses or potential jurors, but of your tools as well. What are their limitations? What are their security guarantees? How are they built?
SPEAKER_00Ask the hard questions before you rely on them.
SPEAKER_01Because the future of trial work isn't just going digital, it's becoming intelligent, it needs to be secure, and it has to be specific to the task at hand.
Podcasts we love
Check out these other fine podcasts recommended by us, not an algorithm.
