Science of Justice

Beyond Generic AI: Why Specialized Legal Tools Matter

• Jury Analyst • Season 1 • Episode 25

Share episode

Use Left/Right to seek, Home/End to jump to start or end. Hold shift to jump forward or backward.

0:00 | 45:17

Generic AI tools present serious risks for attorneys including hallucinated legal facts, confidentiality breaches, and strategic failures that can lead to sanctions and case dismissals.

• Large Language Models (LLMs) like ChatGPT create "hallucinations" - confidently stated but completely fabricated legal information including non-existent cases with fake names and citations
• Courts have sanctioned attorneys who submitted AI-generated fake cases, as in Mata v. Avianca and cases involving James Martin Paul
• Using generic AI violates ABA Model Rule 1.1 (duty of competence) when attorneys fail to verify information
• Consumer AI platforms often claim rights to store and reuse input data, violating attorney-client confidentiality under Rule 1.6
• Generic LLMs lack specialized knowledge needed for effective jury selection, missing critical psychographic factors that predict juror decisions
• Specialized legal AI tools offer better alternatives with proper security protocols, contractual data protections, and litigation-specific capabilities
• Attorneys remain fully responsible for verifying all AI outputs regardless of which tools they use

The path forward requires shifting from generic to purpose-built legal technology platforms that incorporate legal rigor, security compliance, and domain-specific expertise while maintaining human oversight of all AI-generated content.


Send a text


https://scienceofjustice.com/

SPEAKER_01:

If your case strategy and trial prep includes Chat GPT, you might actually be preparing for sanctions instead of success.

SPEAKER_00:

Aaron Powell That's right.

SPEAKER_01:

Aaron Powell Today we're breaking down the real risks behind the rise of this generic machine intelligence and litigation. And maybe talk about what smarter attorneys are doing instead.

SPEAKER_00:

Aaron Powell It's an incredibly high-stakes environment right now, isn't it?

SPEAKER_01:

Aaron Powell Absolutely.

SPEAKER_00:

Aaron Powell The availability of tools like large language models, well, it promises immense efficiency. That's the lure.

SPEAKER_01:

Trevor Burrus Aaron Powell Sure, efficiency is always tempting.

SPEAKER_00:

Aaron Powell But the legal field, especially for high-stakes plaintiff work, operates under these immutable ethical and professional duties.

SPEAKER_01:

Aaron Powell Duties that don't just go away because there's new tech.

SPEAKER_00:

Aaron Powell Exactly. And these generic consumer models, they fundamentally do not respect those duties. They weren't built for it.

SPEAKER_01:

Aaron Powell Okay. So that's the core tension we need to explore for you, the listener. Our goal here is really to provide an essential understanding, you know, of the speci maybe hidden ethical and strategic dangers. Trevor Burrus, Jr.

SPEAKER_00:

Hidden is the right word sometimes.

SPEAKER_01:

Trevor Burrus Right. Dangers that these general purpose LLMs pose when you're dealing with really critical tasks. Things like V Wadir or just managing sensitive client data.

SPEAKER_00:

Hugely critical tasks.

SPEAKER_01:

Aaron Powell We probably need to be clear right up front. These generative models, they're trained mainly for conversational fluency, aren't they? For predicting the next word.

SPEAKER_00:

Aaron Powell That's their core function, prediction.

SPEAKER_01:

Aaron Powell Not for the absolute competence and frankly the rigorous security that you need in a courtroom setting.

SPEAKER_00:

Aaron Powell Absolutely not. They can sound supremely confident, though. That's the trap.

SPEAKER_01:

Aaron Powell Yeah, that confidence. It sounds convincing.

SPEAKER_00:

Aaron Powell But it's entirely without foundation when it comes to, say, verifiable legal facts or client security protocols. Confidence doesn't equal competence here.

SPEAKER_01:

Aaron Powell That false confidence. Is that the biggest danger, you think?

SPEAKER_00:

Aaron Powell It's arguably the greatest danger, yes, because the models synthesize information so convincingly.

SPEAKER_01:

Right.

SPEAKER_00:

It leads attorneys, understandably perhaps, to treat them as authoritative experts, like a super fast research assistant.

SPEAKER_01:

Aaron Powell But they aren't.

SPEAKER_00:

No. In reality, they are tools. Tools that require a skeptical, verified approach. Always.

SPEAKER_01:

So treating them as a substitute for, you know, proper legal research and independent strategy.

SPEAKER_00:

That exposes the attorney and the client to tremendous risk, just massive risk.

SPEAKER_01:

Okay. So where does that risk start? You mention an integrity crisis. Aaron Powell Yeah.

SPEAKER_00:

Let's start there. The integrity crisis, it really begins with hallucination.

SPEAKER_01:

Aaron Powell Okay, let's break that down. The mechanism of failure. What exactly is a generic LLM doing when, say, a plaintiff lawyer asks it to pull up a recent precedent or maybe find a relevant statute? Yeah. It's not accessing some secure, vetted legal database like Westlaw or Lexus, is it?

SPEAKER_00:

Aaron Powell, not even close. Not the generic ones. At their core, these LLMs are, someone called them highly sophisticated stochastic parrots. Stochastic parrots, meaning meaning they're pattern matching machines. They have been trained on just a massive, often unverified body of human language scraped from the internet. So when you ask it a question, it's calculating the statistically most probable sequence of words that should follow your prompt. It prioritizes linguistic fluency, how convincing it sounds, way over factual verification. Accuracy is secondary to plausibility.

SPEAKER_01:

And in the legal world, well, precision is everything, isn't it?

SPEAKER_00:

Paramount.

SPEAKER_01:

So this capability, this fluency over facts, leads to what people call hallucinations. That's the term, yes. Can you help us understand how severe a legal hallucination is? Is it just like getting a date wrong or it's much worse than a standard factual error?

SPEAKER_00:

A standard error might be easily spotted, you know, a typo in a date or something. Right. A legal hallucination, though, is it's a confidently stated, often very detailed, and completely fabricated piece of information. We're not just talking about missing a citation or getting a case name slightly wrong. We're talking about completely manufactured case law.

SPEAKER_01:

You mean cases that don't exist at all?

SPEAKER_00:

Correct. Including specific made-up case names, docket numbers, even fabricated judges' names, or non-existent statutes, or maybe confident explanations of legal principles based on laws that were repealed years ago or are just flat out outdated.

SPEAKER_01:

Why does it do that? Why make things up?

SPEAKER_00:

Because the tool is designed to fill in informational gaps. If it doesn't know the answer from its training data, it tries to construct something that looks like a plausible answer based on the patterns it learned. And for a lawyer who isn't super careful For the untrained legal eye, or even a busy lawyer just trying to save time, that plausibility can be completely deceiving. Dangerously so.

SPEAKER_01:

And there's another issue, isn't there? The knowledge cutoff date.

SPEAKER_00:

Oh, yes. That's a huge factor.

SPEAKER_01:

So if an LLM was trained, say, only on data up to September 2021, how does that practically affect a lawyer looking for a current precedent? Something decided last year.

SPEAKER_00:

Aaron Powell It creates this massive time capsule problem. It's like asking someone about 2023 who only knows about the world up to late 2021.

SPEAKER_01:

Aaron Powell So new laws, new cases.

SPEAKER_00:

Aaron Powell If a state Supreme Court decision, a new federal regulation, maybe a critical legislative change happened after that training cutoff date, the LOM literally does not know it exists. So if you ask it about a legal development from 2023, it might try to predict the answer based on its 2021 data. Aaron Powell Which leads to a hallucination or just completely wrong information that bears no resemblance to the current state of the law.

SPEAKER_01:

Aaron Powell And for plaintiff attorneys dealing with constantly evolving areas like tort reform, mass torts, shifting rules in different jurisdictions.

SPEAKER_00:

Trevor Burrus Relying on that kind of temporally limited information is just it's a formula for malpractice. Yeah.

SPEAKER_01:

Trevor Burrus And this isn't just theoretical risk, right? We've seen this blow up in practice. Trevor Burrus, Jr.

SPEAKER_00:

Oh, absolutely. We've already seen the consequences play out very publicly. These are immediate real-world examples of the professional danger. Trevor Burrus, Jr.

SPEAKER_01:

Like the Mata versus Avianta case. That seems to be the one everyone talks about. Trevor Burrus, Jr.

SPEAKER_00:

That case is, yeah, it's the textbook example. A huge warning sign that every single lawyer should know about.

SPEAKER_01:

Aaron Powell What happened there exactly?

SPEAKER_00:

Aaron Ross Powell Well, the attorney filed a brief in federal court, a formal legal document.

SPEAKER_01:

Okay.

SPEAKER_00:

And in that brief, he cited about half a dozen cases. Cases that were entirely fabricated by a generic LLM he'd used for research.

SPEAKER_01:

That just didn't exist.

SPEAKER_00:

Didn't exist. Pure invention by the AI.

SPEAKER_01:

How did it come out?

SPEAKER_00:

Opposing counsel couldn't find the cases, obviously. They alerted the court. The judge investigated.

SPEAKER_01:

And found out the lawyer relied on Chat GPT or a similar tool.

SPEAKER_00:

Exactly. The judicial investigation revealed the attorney's reliance on the AI.

SPEAKER_01:

What made those fake cases look real enough to fool the lawyer? I mean, wouldn't you check?

SPEAKER_00:

You'd think so, but they weren't just random nonsense. The hallucinations included specific, seemingly legitimate details.

SPEAKER_01:

Like what?

SPEAKER_00:

Like volume numbers, page numbers that looked right, court names, even fabricated quotes from these phantom judicial opinions that crucially fit the context of the attorney's argument.

SPEAKER_01:

So they sounded convincing.

SPEAKER_00:

Very convincing. Remember, the LLM's job is to produce plausible language. And it did that really well here. It created citations that looked professionally sourced.

SPEAKER_01:

Leading the attorney to skip the essential step.

SPEAKER_00:

Of checking the primary source. Going to the actual reporter or database.

SPEAKER_01:

And the result.

SPEAKER_00:

Public embarrassment, a judicial finding of bad faith conduct, and severe sanctions. Significant fines imposed by the court for submitting misleading information. A real black eye.

SPEAKER_01:

But it can get even worse than fines, can't it? There was another case, James Martin Paul.

SPEAKER_00:

Yes, and the consequences there were truly catastrophic for his practice and his clients.

SPEAKER_01:

What was different about the Paul case?

SPEAKER_00:

The critical distinction was the repetition. He was sanctioned not just once, he kept doing it.

SPEAKER_01:

Kept submitting filings with fake cases.

SPEAKER_00:

Repeatedly submitted court filings containing hallucinated case law, completely fabricated quotations, even after the court explicitly warned him, and opposing counsel kept pointing out the errors.

SPEAKER_01:

That sounds deliberate almost.

SPEAKER_00:

The court certainly viewed it as a deliberate failure of professional duty, not just a mistake.

SPEAKER_01:

And the outcome.

SPEAKER_00:

Which means it's essentially a death blow to the client's claim. They can't refile, can't fix it, their case is just gone. And on top of that, he was forced to pay the defense attorney's fees for the trouble he caused.

SPEAKER_01:

So the courts are not playing around with this?

SPEAKER_00:

No. They've been explicit, ignorance of how the technology works, or just relying on it blindly. That is not a justifiable excuse for professional negligence. Period.

SPEAKER_01:

This leads us right to the bedrock ethical rule, doesn't it? ABA model rule one point one, the duty of competence. Exactly how do these hallucinations directly violate Rule 1.1?

SPEAKER_00:

Rule 1.1 mandates that lawyers maintain the requisite knowledge and skill for representation. That's basic.

SPEAKER_01:

Right.

SPEAKER_00:

But crucially, the ABA has clarified this includes understanding of the risks and benefits associated with relevant technology. It's part of competence now.

SPEAKER_01:

So if you use AI.

SPEAKER_00:

If you employ a machine intelligence tool, you are ethically mandated to understand how it works, at least fundamentally. You need to know it generates outputs based on prediction, not verification.

SPEAKER_01:

And you have to check everything.

SPEAKER_00:

The lawyer has an absolute ethical duty to exercise independent verification and rigorous scrutiny of every single output that goes into their work product. Every citation, every legal assertion.

SPEAKER_01:

Aaron Powell, So you can't just treat an LLM's output like a case citation it gives you as authoritative.

SPEAKER_00:

Aaron Powell Never. Not without performing your own due diligence. You have to check the actual court records, the primary sources. If you don't, you've committed a clear breach of your duty of competence.

SPEAKER_01:

Aaron Powell The technology doesn't let you off the hook.

SPEAKER_00:

Aaron Powell No. If anything, it heightens the responsibility of the human practitioner.

SPEAKER_01:

Aaron Powell So the bottom line is stark. If the technology is available, lawyers must become competent in its use.

SPEAKER_00:

Yes.

SPEAKER_01:

Which means really understanding its limits, its flaws, its dangers. If they don't understand that, they're simply not competent to use it for client work. Trevor Burrus, Jr.

SPEAKER_00:

Precisely stated. Right. Trevor Burrus And that competence mandate we just discussed, it mostly focuses on the output quality, but there's an equally dangerous ethical failure lurking on the input side where you feed into the machine. And that shifts our focus entirely to a different fundamental issue: client confidentiality.

SPEAKER_01:

Aaron Powell Okay, let's make that shift. The vulnerability of client data. This feels maybe less obvious than fake case law.

SPEAKER_00:

Aaron Powell It can be less obvious, but potentially even more damaging in some ways.

SPEAKER_01:

Aaron Powell So these generic LLMs, designed for mass consumer use, they're built for ease of access, engagement, maybe even fun. Aaron Powell Right.

SPEAKER_00:

Think chatbots, creative writing aids. Trevor Burrus, Jr.

SPEAKER_01:

They are emphatically not engineered with the kind of high-level security standards the legal industry demands, are they?

SPEAKER_00:

Aaron Powell No, not at all. This lack of specialization for legal or other high security fields is a huge red flag. Trevor Burrus, Jr.

SPEAKER_01:

Especially for plaintiff attorneys who handle incredibly sensitive information.

SPEAKER_00:

Aaron Powell Exactly. Think about it. You're handling detailed medical records, right? That often requires adherence to high PATA standards.

SPEAKER_01:

Absolutely. Strict rules there.

SPEAKER_00:

Aaron Powell And any high-stakes litigation involves strategy memos, deposition transcripts, settlement negotiations, stuff that demands comprehensive data security, things like SOC2 compliance.

SPEAKER_01:

SOC2. Can you explain that briefly?

SPEAKER_00:

Sure. SOC2 is a framework developed by the AI CPA, the accounting folks, that specifies how organizations should manage customer data based on principles like security, availability, processing integrity, confidentiality, and privacy. Getting SOC2 certified means a third-party auditor has verified your systems and controls.

SPEAKER_01:

Okay, so it's a rigorous security standard.

SPEAKER_00:

Very. And generic LLMs, they typically lack these kinds of rigorous controls built in.

SPEAKER_01:

So if I input sensitive information, say detailed case notes about my client's injuries, maybe protected health information, PHI. Trevor Burrus, Jr.

SPEAKER_00:

Or confidential settlement strategies, details from a non-disclosure agreement.

SPEAKER_01:

If I put that into one of these general consumer platforms, what happens?

SPEAKER_00:

You are effectively compromising that data. You're putting it into an environment not designed to protect it adequately.

SPEAKER_01:

And the risk isn't just some theoretical hack, is it? It's actually written into the terms of service.

SPEAKER_00:

Yes. And this is the part that many lawyers, frankly, might skim past or not fully appreciate. It's what I call the terms of service trap.

SPEAKER_01:

Okay, explain the trap.

SPEAKER_00:

It's perhaps the most insidious contractual danger. When you use a generic LLM, you click agree to their terms, not yours. You're operating under their rules. And those rules often say most vendors explicitly disclaim any duty of confidentiality regarding the data you input. They essentially say, what you put in here, not our job to keep it secret.

SPEAKER_01:

Wow. Okay.

SPEAKER_00:

But it gets worse. More critically, they often reserve the right in those terms to store your input, process it, and here's the kicker, reuse your input data for the purpose of further training and improving their own underlying AI model.

SPEAKER_01:

Wait, so if I paste, say, a detailed confidential medical summary for my client's file.

SPEAKER_00:

Or maybe a unique WarDire questionnaire you developed looking for feedback on the structure.

SPEAKER_01:

If I put that into a generic LLM, that confidential information, my client's private medical details, my firm's proprietary strategy, it could now be used to train a commercial AI product.

SPEAKER_00:

Exactly. It gets absorbed into their training data.

SPEAKER_01:

Aaron Powell An AI that potentially anyone could interact with down the line.

SPEAKER_00:

That's the risk. Your client's confidential information, their vulnerabilities, your case strategy, their protected health information. It's retained by this third-party vendor who owes you no duty of privilege.

SPEAKER_01:

And it might inadvertently surface later.

SPEAKER_00:

It could be reflected or influence future outputs for completely different users in ways that are impossible to predict or control.

SPEAKER_01:

That sounds like an immediate, major ethical violation for any lawyer.

SPEAKER_00:

It is an egregious violation of fundamental ethical duties.

SPEAKER_01:

Which specific rule does this hit head on?

SPEAKER_00:

It directly violates ABA Model Rule 1.6, the cornerstone duty of confidentiality. That rule requires lawyers to make reasonable efforts to prevent the inadvertent or unauthorized disclosure of information relating to the representation of a client.

SPEAKER_01:

And using a tool that says we might use your data is not reasonable effort.

SPEAKER_00:

Clearly not. Using an LLM platform that explicitly reserves the right to retain and potentially use client data for its own training purposes is arguably the opposite of making a reasonable effort to prevent disclosure.

SPEAKER_01:

Any other risks tied to this? NDAs, maybe.

SPEAKER_00:

Absolutely. It also risks violating specific confidentiality agreements or non-disclosure agreements, NDAs, that many plaintiff firms have with their clients or with third parties, especially in complex commercial torts or sensitive personal injury cases. Feeding that protected data into an unsecured platform could constitute a breach of those agreements too.

SPEAKER_01:

So the danger isn't just a potential data breach by hackers.

SPEAKER_00:

No, that's a separate, also valid concern. But this risk is about the LLM company's own legitimate use of your data, as permitted by the terms you agreed to. It's a contractual issue creating an ethical nightmare.

SPEAKER_01:

It sounds like the due diligence required here is pretty intense.

SPEAKER_00:

It has to be meticulous. When you're assessing any technology that will handle client data, especially AI, you must look for explicit contractual guarantees from the vendor.

SPEAKER_01:

Guarantees about what specifically?

SPEAKER_00:

Guarantees that they will not retain your input data, not store it long term unless necessary for the service you requested, and absolutely not use it to train their models.

SPEAKER_01:

And that SOC2 compliance you mentioned?

SPEAKER_00:

That should be a baseline requirement. And ideally, not just SOC two type 1, which is a snapshot in time, but SOC2 type 2. Type 2 certification means an auditor has verified that the security controls are not only designed properly, but have been operating effectively over a sustained period, usually six months or more. It's a much stronger assurance.

SPEAKER_01:

So if a vendor doesn't offer that or won't guarantee data privacy.

SPEAKER_00:

It should be a mandatory, non-negotiable red flag. Stop right there.

SPEAKER_01:

And these checks, they're more rigorous than what you'd do for, say, cloud storage like Dropbox.

SPEAKER_00:

Far more rigorous. Because a simple storage service isn't actively learning from your documents. An LLM is designed to consume and process the content of the input. The risk profile is fundamentally different.

SPEAKER_01:

This responsibility, it doesn't just sit with the partner using the tool, does it? It flows down.

SPEAKER_00:

Absolutely. This brings in another ethical rule. ABA model rule 5.3, the duty to supervise non-lawyer assistance.

SPEAKER_01:

Rule 5.3, how does that apply to AI?

SPEAKER_00:

The ABA and state bars have made it pretty clear that this duty extends to supervising the use of technology, including machine intelligence.

SPEAKER_01:

So the supervising attorney is on the hook.

SPEAKER_00:

Completely. They are responsible for ensuring that all integrated AI tools, and critically, the way junior staff, paralegal's associates use them, do not result in ethical breaches or security failures.

SPEAKER_01:

Can you give an example? Sure.

SPEAKER_00:

Imagine a scenario. A busy paralegal, trying to be efficient, gets asked to summarize a huge batch of confidential medical records for a complex personal injury case. Lots of PHI in there.

SPEAKER_01:

Okay.

SPEAKER_00:

They think, hey, this ChatGPT thing is fast. They paste chunks of those records into the generic LLM to get summaries. They might assume it's secure or just not think about it.

SPEAKER_01:

Aaron Powell, but if that data gets retained or used for training by the LLM vendor.

SPEAKER_00:

The supervising attorney is the one who bears the professional and ethical liability for that disclosure under Rule 5.3. Even if they didn't know the paralegal used that specific tool.

SPEAKER_01:

So firms need policies.

SPEAKER_00:

Yes. Clear, explicit protocols that ban the use of non-secure generic LLMs for any work involving confidential client information. Period. Training is also key.

SPEAKER_01:

So wrapping up this section on ethics and security, the core message seems to be that generic LLMs operate under a business model, really, of data capture and reuse.

SPEAKER_00:

Often, yes. Or at least they reserve the right to.

SPEAKER_01:

And that model is just fundamentally incompatible with the legal profession's absolute requirement for data security and client confidentiality.

SPEAKER_00:

Diametrically opposed, you could say. Convenience in this context simply cannot be allowed to outweigh the foundational duties of the profession. The risks are just too high.

SPEAKER_01:

Aaron Powell Okay. We've covered the integrity crisis with hallucinations and the major ethical wall around confidentiality. Let's pivot now to strategic failure.

SPEAKER_00:

Aaron Powell Right. Because even if you somehow navigated the ethical minefield, these tools often just aren't very good at the actual strategy part of lawyering.

SPEAKER_01:

Aaron Powell Especially for a plaintiff trial lawyer where the goal is one: securing a favorable jury, presenting a strategy tailored to a specific judge, a specific venue.

SPEAKER_00:

Aaron Powell Exactly. And this is where generic LLMs fail pretty dramatically, particularly in high-stakes strategic tasks like voir dire jury selection.

SPEAKER_01:

Aaron Powell Why is that? What's the fundamental mismatch?

SPEAKER_00:

Well, remember, general LLMs are programmed for generalization. They're masters of the average, the boilerplate, the statistically probable conversational response based on that huge mixed data set they trained on. They simply lack the capacity to weigh subtle strategic implications. They don't understand the hierarchy of legal authorities in your specific jurisdiction versus another.

SPEAKER_01:

They don't know the local rules or the judge's history.

SPEAKER_00:

Precisely. And they certainly can't align a case strategy with the nuanced context of a local court venue or a specific client situation or the opposing counsel's known tactics. It's all just text to them.

SPEAKER_01:

So their output, it focuses on maybe superficial things like grammar, clarity, giving you a general outline.

SPEAKER_00:

Yeah, it might help you structure an argument or polish prose, but it misses the deep strategic cuts, the nuanced points that actually win cases.

SPEAKER_01:

Aaron Powell Can you give an example outside of litigation, maybe?

SPEAKER_00:

Sure. Think about contract review. A generic LLM might review a complex commercial lease and offer, you know, generic suggestions on standard clauses like indemnification or force majeure.

SPEAKER_01:

Okay. Sounds helpful so far.

SPEAKER_00:

Aaron Powell But it will likely completely overlook the specific critical negotiation points unique to that deal. Maybe a poorly worded option to renew, or a subtle flaw in the rent escalation clause that could cost your client millions down the road. It misses the context specific risks and opportunities.

SPEAKER_01:

Aaron Powell Because it doesn't understand the client's business or the market conditions.

SPEAKER_00:

Exactly. It applies generalized knowledge, and that generalized thinking is absolutely catastrophic when applied to something like jury selection.

SPEAKER_01:

Where specificity is everything.

SPEAKER_00:

Everything. A generalized boilerplate LLM approach to var dye is practically useless if your goal is to actually generate a winning plaintiff strategy. It just can't do it effectively.

SPEAKER_01:

Jury selection. People often say it's where cases are won or lost, especially for plaintiffs. It's the foundation.

SPEAKER_00:

It really is. An effective war dire requires understanding deep attitudinal markers in potential jurors, not just service level stuff.

SPEAKER_01:

Aaron Powell Okay, so let's talk about that difference. You hear about demographics, but you also mentioned something deeper psychographics. What's the distinction?

SPEAKER_00:

Right. Demographics are the what? Age, occupation, maybe where they live, education level. Easy to collect, but not very predictive on their own.

SPEAKER_01:

Okay.

SPEAKER_00:

Psychographics are the why. They tell you why a juror might think the way they do. What are their core values? Their underlying beliefs, their personality traits, their motivations, their level of cynicism or trust in institutions.

SPEAKER_01:

Aaron Powell And that's what predicts how they'll decide.

SPEAKER_00:

Much more reliably, yes. Plaintiff strategies, successful ones, are built on identifying these key psychographic factors. Things like how much does this juror value personal responsibility versus corporate accountability? How skeptical are they of large companies by default? How much do they trust authority figures?

SPEAKER_01:

Can you give a concrete example of why this psychographic insight matters so much?

SPEAKER_00:

Absolutely. There's consistent data showing, for instance, that potential jurors identify it as having a high degree of inherent skepticism toward corporations. They are significantly more likely to favor the plaintiff in a typical civil tort case.

SPEAKER_01:

How much more likely?

SPEAKER_00:

Depending on the study and venue, somewhere in the range of 37 to 42 percent more likely.

SPEAKER_01:

Wow. That's a huge difference.

SPEAKER_00:

It's a massive strategic advantage. And that factor skepticism towards corporations far outweighs simple demographics, like whether the juror is male or female, or works in retail versus manufacturing.

SPEAKER_01:

And generic LLMs, they just can't find that.

SPEAKER_00:

They lack the necessary foundation. They don't have the behavioral science models baked in. They don't have access to proprietary localized venue data that tracks these attitudes over time. They weren't built to identify these predictive psychographic factors.

SPEAKER_01:

So if a lawyer asks a generic LLM, suggest some questions for Vardyyer in my personal injury case. What kind of output are they likely to get? How does it fall short?

SPEAKER_00:

It fell short because it lacks segmentation capability. It will likely give you very generic surface level questions. Do you know any of the parties? Have you ever been sued? Stuff like that.

SPEAKER_01:

Standard questions.

SPEAKER_00:

Right. The basics. But modern jury analysis, the kind winning firms use, involves layering different data sources to cluster potential jurors into predictive mindsets or archetypes.

SPEAKER_01:

Mindsets like what?

SPEAKER_00:

Well, you might interview two potential jurors. On paper, demographically they look similar, both male, middle-aged, maybe work in skilled trades.

SPEAKER_01:

Okay.

SPEAKER_00:

But through deeper questioning and analysis, you might find one is, say, a risk-averse pragmatist. This person might distrust plaintiff arguments because they worry large verdicts will just raise insurance costs for everyone, including them.

SPEAKER_01:

Right, worried about the system.

SPEAKER_00:

While the other juror, despite similar demographics, might be a fairness-focused empathizer. This person's core motivation might be seeing victims made whole, especially if they perceive corporate negligence or arrogance. They prioritize restoring balance.

SPEAKER_01:

Two very different jurors, potentially deciding the case differently.

SPEAKER_00:

Hugely different. And a generic LLM. It cannot reliably differentiate between those two mindsets based on simple demographic data or generic questions. It provides boilerplate output that misses the implicit biases, the underlying values that actually drive decision making in the jury room.

SPEAKER_01:

So the lawyer using only that is basically flying blind, relying on gut feeling.

SPEAKER_00:

Aaron Powell Pretty much. Gut feeling, which we know can be riddled with its own biases. It's not a systematic data-driven approach.

SPEAKER_01:

Aaron Powell You mentioned layering data. The outline mentioned five core metrics for assessing bias. Can we elaborate on those? What are they?

SPEAKER_00:

Sure. Effective modern jury assessment really uses a layered approach, pulling from multiple angles to get a complete picture. Think of it as five pillars.

SPEAKER_01:

Okay, pillar one.

SPEAKER_00:

Pillar one is demographic analysis, the basics age, gender, race, employment, location, foundational but limited. Pillar two. Social media and background review, looking into publicly available digital footprints, news mentions, property records, et cetera, for clues about attitudes or potential conflicts. Generic LOMs might help search this, but interpretation is key. Case-specific questioning. This involves carefully crafted survey questions, often administered before a formal voard dire, if possible, or integrated into the voir dire itself, designed to gauge opinions on specific themes relevant to your case, attitudes towards damages, corporate responsibility, specific industries, etc. Using validated behavioral science instruments, things like implicit association testing or IETs, which can measure unconscious biases related to key concepts like trust in institutions, attitudes towards wealth, or assigning blame. This gets below the surface of conscious answers.

SPEAKER_01:

And the final pillar.

SPEAKER_00:

Then the V war deer assessment. This is the observation and scoring of potential jurors' responses and how to answer. What's their body language? Tone of voice. Hesitations. This requires skilled human observation, often aided by specialized scoring systems.

SPEAKER_01:

That sounds like a tremendous amount of very specialized, behaviorally focused data.

SPEAKER_00:

It is. And here's the point a generic LLM might be able to help with maybe the first two pillars pulling basic background data. It might even try to generate some boilerplate questions for pillar three, but it absolutely cannot perform or interpret the results of behavioral testing like IETs, Pillar Four. And crucially, it cannot integrate the critical behavioral observations and nonverbal cues from the juror assessment in the courtroom, pillar five.

SPEAKER_01:

Aaron Powell Which leads us to another major strategic failure you mentioned.

SPEAKER_00:

Yes, the final and perhaps most profound strategic blind spot, the emotional intelligence gap.

SPEAKER_01:

Tell me more about that gap. Particularly how does it affect evaluating witnesses and jurors during the actual trial process. Aaron Powell Right.

SPEAKER_00:

Because the courtroom isn't just about facts and law, it's inherently a human environment. It runs on subtle behavioral feedback loops, nonverbal communication, emotional responses.

SPEAKER_01:

Generic LLMs are.

SPEAKER_00:

They're text-based models. Fundamentally, they process words. They cannot analyze the nonverbal or the nuanced emotional cues that are absolutely essential for assessing witness credibility or scoring potential juror bias during live questioning. Things like things like micro expressions flickering across a witness's face when they're asked a tough question, changes in vocal pitch or hesitation patterns that might signal deception or uncertainty, posture shifts, averted glances from a potential juror who might be trying to mask a strong bias because they want to avoid conflict or get off the panel. Social desirability bias is huge in Voardier.

SPEAKER_01:

Aaron Powell And these are exactly the cues that experienced plaintiff attorneys rely on, aren't they?

SPEAKER_00:

Absolutely. They rely on these behavioral cues when making those crucial split-second decisions. Should I use a peremptory challenge on this juror? Should I push this witness harder on cross-examination or back off? It's reading the room, reading the person.

SPEAKER_01:

Aaron Powell Behavior modeling is key.

SPEAKER_00:

It's a key component of systematic trial assessment and strategy. And generic LLMs are completely blind to it.

SPEAKER_01:

Aaron Powell What about simulating the trial itself? Can they help predict how a jury might deliberate?

SPEAKER_00:

Aaron Powell No, not the generic ones. They're useless for that too. They cannot realistically model jury deliberation dynamics.

SPEAKER_01:

Why not?

SPEAKER_00:

Because deliberation isn't just aggregating individual opinions. It involves groupthink, social pressure, persuasion, conformity. How does a dominant personality influence others? How do certain arguments gain traction or fall flat within that specific group dynamic?

SPEAKER_01:

Aaron Powell The whole social soup of the jury room.

SPEAKER_00:

Aaron Powell Exactly. Those subtle emotional cues, the social dynamics, the pressures that ultimately shape a final verdict once the jury is sequestered. They are completely lost on generalized, non specialized models.

SPEAKER_01:

Aaron Powell So the answers they give are kind of in a vacuum.

SPEAKER_00:

Aaron Powell Precisely. They offer strategic answers or suggestions in a vacuum, completely ignoring the complex, messy, emotional reality of human. Interaction that defines every trial.

SPEAKER_01:

Trevor Burrus Okay, so we've laid out these massive liabilities, hallucination risks leading to sanctions, confidentiality breaches violating ethical duties, strategic incompetence, especially in critical areas like war dire.

SPEAKER_00:

So a pretty damning list.

SPEAKER_01:

Given all that, we really have to tackle this idea, this myth of the free or low cost tool. Because the appeal is still there, isn't it? It saves time, it's cheap or free. Many attorneys might still be tempted.

SPEAKER_00:

Of course. The perceived benefit is efficiency and cost savings. But it's often a classic example of being pennywise and pound foolish. Fault economy.

SPEAKER_01:

So we need to reframe that low cost. What's the true hidden cost of using these generic tools?

SPEAKER_00:

Aaron Powell The true cost is always born through risk. Always. The actual expense isn't the subscription fee, if there even is one. It comes from the consequences of failure. Such as such as court sanctions and fines, ethical violations that could damage your reputation or even lead to suspension, and profound strategic failures.

SPEAKER_01:

Aaron Powell Like what kind of strategic failure?

SPEAKER_00:

Like failing to strike that one dangerous juror who praisons the deliberation. Like relying on discredited evidence because your AI research missed a key ruling, like having a defense motion to dismiss, granted because your brief was billed on fabricated case law.

SPEAKER_01:

The cost of losing the case for your client?

SPEAKER_00:

That cost is often incalculable, isn't it? Especially for a plaintiff who may have only one shot at justice. That's the hidden expense.

SPEAKER_01:

Okay, but I can almost hear a skeptical junior partner listening right now thinking, look, even if I have to double check everything, if using a generic LLM gets me 80% of the way there on a first draft of, say, a motion response, then checking the citations and polishing it is just the last 20%. The efficiency gain is still huge. Isn't this risk being overstated just to cover that last bit of work?

SPEAKER_00:

That's a fair challenge to raise. It sounds logical on the surface, but I think it misses two really critical points.

SPEAKER_01:

Okay, what's the first one?

SPEAKER_00:

First, the human time required to truly independently verify and rigorously scrutinize generic LLM outputs. Yeah. It often completely negates the promised time savings, especially for complex legal work.

SPEAKER_01:

Aaron Powell Oh so?

SPEAKER_00:

Because you can't just check the citations, you have to check the underlying logic. Is the reasoning sound? Does this case actually apply to the specific facts and crucially the jurisdiction of my case? Is the interpretation of the law current?

SPEAKER_01:

It's not just proofreading.

SPEAKER_00:

No. And think about it. If the LLM output is, say, ninety percent accurate, which might be optimistic for complex tasks, that remaining 10% of errors, they're often the subtle ones. The contextually dependent mistakes, the plausible sounding but legally wrong assertions.

SPEAKER_01:

And those take longer to fix.

SPEAKER_00:

Often, yes. It can require more human research time to meticulously unravel and disprove a sophisticated hallucination than it would have taken to just conduct the research properly using reliable sources from the start. The 80% done might be an illusion.

SPEAKER_01:

Okay, that's a strong point about the time factor. What's the second critical point?

SPEAKER_00:

The second point is perhaps even more fundamental from an ethical standpoint. Using many generic LLMs fundamentally prevents you from fully fulfilling your duty of confidence. Rule 1.1, because you often cannot verify the reasoning process behind the output.

SPEAKER_01:

Explain that. Why can't you verify the reasoning?

SPEAKER_00:

It's the concept of explainability or XAI explainable AI. Many generic LLMs operate as black boxes.

SPEAKER_01:

Meaning you can't see inside.

SPEAKER_00:

Right. They can't easily show the attorney the specific source documents they relied on for a particular assertion or the logical steps they took to arrive at a conclusion. They just give you the output.

SPEAKER_01:

So you're left with just an assertion, a statement.

SPEAKER_00:

Exactly. And how can a lawyer rigorously vet an assertion, fulfill their duty of independent scrutiny if they cannot examine the underlying data stream or the logic used to generate it? You're being asked to trust the black box.

SPEAKER_01:

That is profound. If the tool can't explain how it reached its conclusion, the lawyer cannot possibly meet their ethical burden of verification and independent judgment.

SPEAKER_00:

I believe that's correct. Which circles us back again to the lawyer's inescapable duty of oversight, human oversight.

SPEAKER_01:

Technology as a support, not a replacement.

SPEAKER_00:

Precisely. Machine intelligence is meant to support, to augment, maybe to automate certain routine tasks, but it cannot and should not replace human judgment, legal expertise, and ethical responsibility. The lawyer must always remain the final skeptical filter. The buck stops with the human.

SPEAKER_01:

And the courts seem to agree.

SPEAKER_00:

The courts have been very clear on this, as we saw in the Mata and Paul cases. And the general consensus emerging in legal tech ethics discussions is interesting. What's the consensus? That the greater long-term danger in the legal tech world might actually be the underutilization of specialized, secure, properly vetted AI, leading to inefficiency and potentially disadvantaging clients. But the most immediate and professionally dangerous threat right now is the over-reliance on non-specialized, unvetted, generic tools like consumer chatbots for substantive legal work.

SPEAKER_01:

So don't be afraid of all AI, but be very afraid of the wrong AI used wrongly.

SPEAKER_00:

Exactly. And the burden rests entirely on the professional user, the lawyer to understand the difference, to vet the tools and to know where to draw that line between assistance and abdication of responsibility.

SPEAKER_01:

The lesson seems clear then. Courts have shown zero sympathy, zero tolerance for lawyers who just blindly rely on AI without doing their own professional verification. The moment you delegate a task, whether to a paralegal or a piece of software, you assume full professional accountability for the quality, the accuracy, and the security of the result. No excuses.

SPEAKER_00:

Correct. The human element, discretion, judgment, building client relationships, providing nuanced understanding in complex situations, strategic thinking that remains absolutely critical and irreplaceable. Technology is always subordinate to professional duty. Always.

SPEAKER_01:

Okay. So if these generic LLMs, the readily available ones, present such pervasive risks, hallucination, ethical breaches, strategic incompetence, where should plaintiff trial lawyers actually turn?

SPEAKER_00:

Yeah, what's the alternative?

SPEAKER_01:

How can they leverage the power of machine intelligence, which is clearly here stay, but do it safely and effectively? What's the necessary shift in approach?

SPEAKER_00:

The shift is towards specialization. That's the only viable path forward for high-stakes work. Meaning the solution lies in abandoning the general purpose chat models for legal work and instead moving toward purpose-built, industry-specific platforms. Tools engineered specifically for the demands of high-stakes civil litigation and trial preparation.

SPEAKER_01:

So tools designed by legal tech companies for lawyers.

SPEAKER_00:

Exactly. These specialized tools are constructed differently from the ground up. They're built to address the unique requirements of the profession, incorporating legal rigor, security protocols, and specialized data streams that generic models just don't have.

SPEAKER_01:

Let's contrast these specialized platforms with the generic models, maybe using the key risk areas we discussed. First, security and confidentiality. How do they differ?

SPEAKER_00:

The differences here are or should be non-negotiable. Purpose-built legal tech platforms worth considering will guarantee things like SOC2 compliance, demonstrating that ongoing operational security. And they should provide explicit, clear, contractual pledges right in the service agreement, that they will not store your input data unnecessarily, not process it for unrelated purposes, and absolutely not train their general models on your confidential client information.

SPEAKER_01:

So they respect privilege and confidentiality by design?

SPEAKER_00:

Exactly. Not just relying on the user to somehow avoid inputting sensitive data into a leaky system. That approach is doomed to fail.

SPEAKER_01:

Okay, so that addresses the rule 1.6, confidentiality, and rule 5.3 supervision compliance concerns we talked about. Huge relief for a firm.

SPEAKER_00:

It should be, yes.

SPEAKER_01:

Aaron Powell Now, what about strategic depth? Particularly for those advanced critical tasks like Voir Dyer. How do specialized platforms gain an edge over the generalized approach?

SPEAKER_00:

Aaron Powell They gain the edge through data and methodology, specifically through relevant data and behavioral science. Okay. Specialized tools designed for trial prep are often built on foundations that integrate deep psychographic modeling, understanding those juror archetypes we discussed.

SPEAKER_01:

Aaron Powell The why behind juror thinking.

SPEAKER_00:

Right. They incorporate localized venue data. How have juries in this specific county ruled on similar cases? What are the prevailing attitudes here? They utilize behavioral analytics drawn from actual trial outcomes and jury research.

SPEAKER_01:

Data that generic tools just can't access.

SPEAKER_00:

Correct. They can't access it. And even if they could stumble upon some of it, they lack the specialized models to interpret it effectively in a legal strategy context. This is what moves jury selection out of the realm of pure guesswork or relying only on demographics.

SPEAKER_01:

So instead of getting back a boilerplate list of suggested Vardyer questions from a generic tool, what kind of output does a specialized jury analysis tool provide? What's the deliverable?

SPEAKER_00:

It should provide an evidence-based roadmap for your jury selection strategy. A road Yes. For example, some advanced specialized platforms utilize simulation-based jury research. This goes way beyond traditional focus groups. Oh, so it uses sophisticated models, often based on data from thousands of real and mock jurors, to simulate how different types of jurors with specific profiles are likely to interact and deliberate as a group.

SPEAKER_01:

Simulating the jury room dynamics.

SPEAKER_00:

Exactly. Modeling actual deliberation dynamics, simulating how individual opinions might shift under group pressure, how specific arguments might land with different juror segments, how leader jurors might emerge. It allows counsel to pretest arguments, fine-tune witness order, even analyze how opposing counsel's likely themes might resonate.

SPEAKER_01:

All based on data-backed insights, not just anecdotes.

SPEAKER_00:

Correct. It turns jury research from a snapshot into a dynamic simulation.

SPEAKER_01:

What about that emotional intelligence gap we highlighted earlier? The fact that generic LLMs can't read the room, can't interpret live courtroom behavior. Do specialized tools help there?

SPEAKER_00:

Some specialized platforms are integrating technology designed to help bridge that gap, or at least provide better data for human interpretation.

SPEAKER_01:

In what way?

SPEAKER_00:

How does that work? These might use advanced behavioral science principles, sometimes even incorporating video analysis of practice sessions, to look for subtle facial expression cues, analyze vocal tone and pitch variations for stress or deception indicators, or even assess posture changes. The goal is to help coach witnesses more effectively before they ever step into the actual courtroom. Give lawyers objective data points to consider.

SPEAKER_01:

So it's about unifying all these different data streams.

SPEAKER_00:

Yes. The best specialized systems aim to unify complex data streams, the demographics, the psychographic mindsets derived from surveys or analysis, any available social media bias indicators, maybe even real-time responses during voardire, if the system allows for live input into a singular, continuously updated scoring engine or dashboard for each potential juror.

SPEAKER_01:

It sounds like these specialized systems provide a much more integrated, intelligent picture, refining the whole process from initial juror profiling right through to making that final challenge decision.

SPEAKER_00:

That's exactly the goal. Transforming Voardire from a highly subjective, often gut-driven exercise into something approaching strategic precision, grounded in data and behavioral science.

SPEAKER_01:

And ensuring the questions asked are designed to actually reveal those deep predispositions.

SPEAKER_00:

Right. Maximizing the likelihood of seating a favorable panel, or at least identifying and removing the most dangerous jurors. The strategic output is designed not just to inform, but to directly support the lawyer's critical decision-making process in a way that fully respects professional standards of competence and confidentiality.

SPEAKER_01:

So ultimately, the shift required for plaintiff trial lawyers isn't about abandoning technology.

SPEAKER_00:

Not at all.

SPEAKER_01:

It's about shifting from using a widely accessible generic tool that's designed primarily to sound persuasive.

SPEAKER_00:

To investing in and learning how to properly use a secured, purpose-built platform, one that's specifically engineered to help maximize the chances of winning a high-stakes civil trial within ethical boundaries. It really is the difference between using, say, a general internet search engine for complex medical research versus using specialized peer-reviewed databases and diagnostic equipment. You have to choose intelligence, specialization, and security over mere convenience and that potentially misleading generic confidence. The stakes demanded.

SPEAKER_01:

This has been an incredibly detailed breakdown. Really appreciate it. The dangers are clear, but so is the path forward. Hopefully, yes. So to synthesize the central message for you, our listener, the risks of just jumping onto generic LLMs for your courtroom preparations. They extend far beyond just getting some facts wrong or finding fake cases. They touch on fundamental failures in data security, core ethical obligations, and strategic depth, especially in that absolutely critical domain of jury selection.

SPEAKER_00:

You can't afford to get voir dire wrong.

SPEAKER_01:

Right. The high-stakes nature of plaintiff trial preparation simply requires specialized tools, tools that operate under the required standards of competence, confidentiality, and strategic relevance.

SPEAKER_00:

And always, always remember: the moment you introduce any technology, AI or otherwise, into the service you provide your client, you accept full professional accountability for its output and its impact.

SPEAKER_01:

You own the result.

SPEAKER_00:

You own the results. You absolutely must understand its limitations, assume the burden of rigorous verification, and ensure its security. Otherwise, any perceived efficiency game is simply not worth the potentially career-ending risk.

SPEAKER_01:

So maybe the takeaway is this start asking better questions, not just of your witnesses or potential jurors, but of your tools as well. What are their limitations? What are their security guarantees? How are they built?

SPEAKER_00:

Ask the hard questions before you rely on them.

SPEAKER_01:

Because the future of trial work isn't just going digital, it's becoming intelligent, it needs to be secure, and it has to be specific to the task at hand.

Head Shot

Bekah

Host
Head Shot

Chase

Host

Podcasts we love

Check out these other fine podcasts recommended by us, not an algorithm.