Unauthorized Practice of Law: Where AI Blurs the Line 

Unauthorized practice of law (UPL) is predominantly regulated at the state level in the U.S., and there is no single national definition of the “practice of law.” Jurisdictions vary in how they draw the line between permissible legal information (such as legal education, forms, procedural guidance) and impermissible legal advice (including tailored recommendations, representation, or drafting that implies the application of professional legal judgment). 

Illinois-specific anchor: the Attorney Act 

Illinois’ Attorney Act states: “No person shall be permitted to practice as an attorney or counselor at law within this State” without a license and also restricts receiving compensation for legal services and holding oneself out as providing legal services when unlicensed. Nippon Life’s theory (as described in the reporting) tests how that framework applies when the “assistant” is not a person at all, but an AI system deployed by a technology company and made available to the public to generate legal-like outputs or guidance. 

“Non-human actors” aren’t new to UPL enforcement. 

While an LLM is not a human, the market-facing actor remains a technology vendor, and UPL disputes have long included technology-enabled or software-based delivery models. 

LegalZoom is a canonical example: it litigated with the North Carolina State Bar over whether its interactive document-generation service constituted the unauthorized practice of law, culminating in extensive proceedings and a consent-based resolution framework in North Carolina. The key point is not the outcome; it is that regulators and courts have already treated technology platforms delivering legal assistance or automated document services as a potential unauthorized practice of law. 

DoNotPay is another modern reference point. The FTC finalized an order requiring DoNotPay to stop deceptive “AI lawyer/robot lawyer” claims and noted (among other things) allegations that it failed to test whether its “AI lawyer” performed like a human lawyer and did not retain attorneys to test the quality and accuracy of law-related features. Even outside the traditional unauthorized-practice-of-law doctrine, consumer protection law and deceptive marketing enforcement are a real pathway when legal-like claims are marketed to the public. 

Regulatory reform is already reshaping the gate. 

Two state-led experiments underscored that “who can deliver legal help” is not static: 

Arizona’s Alternative Business Structures (ABS) framework clearly intends to expand access to legal services by allowing entity structures that include nonlawyers with economic interests or decision-making authority, subject to Arizona Supreme Court rules. Utah’s legal regulatory sandbox, a program launched by the Utah Supreme Court and housed within the Utah State Bar, was designed to test whether changing practice regulation can increase access without increasing consumer harm. A scholarly review of Arizona and Utah’s “entity regulation” reforms, five years after their implementation, highlights them as meaningful departures from traditional UPL and ownership constraints in the legal profession’s regulatory structure. 

This matters for Nippon Life v. OpenAI because it suggests a likely future: courts and regulators won’t just ask whether AI crossed a line, they may also ask what a safer, licensed, and supervised regulatory lane for AI-enabled legal assistance should look like. 

The Bigger Story: AI Is Collapsing Procedural Gatekeeping in Law 

To understand why this dispute matters, start with the justice gap: LSC reports that low-income Americans received no or inadequate legal help for 92% of substantial civil legal problems, and that these problems are widespread. Courts and bar organizations have recognized that large numbers of people show up without lawyers, often because they cannot afford or access counsel.  

This “justice gap” is a big driver behind the rapid growth of AI legal tools designed to help individuals understand legal processes without hiring an attorney. As a result, courts, policymakers, and legal technology developers are increasingly exploring tools that help individuals understand legal procedures, prepare documents, and navigate court systems without traditional legal representation. 

“Gatekeeping” in law is not just about elite credentialing, but also about navigating a complex procedural system: deadlines, forms, service rules, evidentiary architecture, and motion practice. Self-represented litigants frequentlystruggle with process rules and court language, practical barriers that can determine case outcomes regardless of underlying merits. 

Why Generative AI Is Changing Self-Representation in the Legal System 

This transformation is rooted in how AI reshapes the sphere of self-representation. Several aspects of AI’s capabilities make these changes possible: 

First is search and comprehension: conversational interfaces can translate legal instructions into plain language and provide contextual explanations (although these outputs still require careful verification).  

Second is drafting: Large language models can produce plausible first drafts of legal motions, demand letters, discovery responses, and procedural filings.  

Third is iteration speed: a motivated pro se litigant can generate, refine, and file at volumes that were previously impractical. 

Importantly, the access-to-justice ecosystem has been building toward this for years via non-generative tools. A2J Author, an established document assembly platform used by courts and legal aid organizations, has helped millions of people generate millions of legal documents, illustrating that structured self-help can scale when built with guardrails and constrained outputs. 

The key takeaway from ‘gatekeeping collapse’ is not that lawyers become irrelevant, but that their value shifts. Lawyers will increasingly focus on strategy, professional judgment, ethical obligations, negotiation, and evidentiary rigor as courts adapt to the growing volume of AI-powered procedural work. 

The Hard Counterpoint: Access vs. Abuse 

The critical takeaway from Nippon Life’s allegations is that AI can lower procedural barriers for all litigants, enabling both greater access and potential abuse. Expect this tension to direct future policy paths, with some jurisdictions tightening restrictions and others formalizing supervised or regulated legal-help pathways. 

A near-term signal is legislative activity: New York Senate Bill S7263 would prohibit proprietors of AI chatbots from permitting “substantive responses” that would constitute unauthorized practice (or unauthorized use of a professional title) if made by a natural person. Whether or not it passes, it reflects the direction of travel: regulators are moving from ethics guidance and professional rules toward explicit statutory liability frameworks. 

The Technical Risks That Turn Legal AI from Helpful to Harmful 

The same properties that make LLMs powerful (fluent generation, implicit pattern matching) also create failure modes that are unacceptable within legal contexts. 

Hallucinations and fabricated citations 

Courts have already sanctioned lawyers who filed AI-generated citations to nonexistent cases. Mata v. Avianca is the best-known example: the sanctions order explains that fabricated citations are not “existing law” and that Federal Rule of Civil Procedure 11 requires attorneys to verify factual and legal assertions before filing. 

Empirical research shows that even specialized legal AI systems marketed as citation-grounded tools are not hallucination-free. A preregistered evaluation by Magesh et al. found that Lexis+ AI and Thomson Reuters AI-assisted tools hallucinated at rates of approximately 17%-33% in their study, despite lower rates than those of general-purpose chatbots. The implication for buyers: “legal-specialized” does not mean “legally reliable by default.” 

Long-context failure and truncation risk 

Litigation and claims work commonly involves extensive records. Even when models can accept long contexts, they may fail to reliably use information in the middle. The “Lost in the Middle” research shows that performance can degrade considerably depending on where relevant information appears in long inputs—often worse near the beginning or end, and better in the middle. 

In practical terms, that means “upload all the correspondence + ask for a conclusion” can yield confident answers that silently ignore critical facts, a pathway from “helpful tool” to potentially misleading legal output. This is especially dangerous in litigation workflows where critical facts may be buried inside thousands of pages of medical records, deposition transcripts, or claim files. 

Data privacy and confidentiality 

Uploading legal correspondence, claim files, or medical records to consumer AI tools can expose privileged or regulated information. OpenAI’s policy materials state that, for consumer services, OpenAI may use your content for model training, with opt-out options; by contrast, business/enterprise offerings emphasize that models are not trained on business data by default. 

For legal work, the compliance question is rarely “is there an opt-out?” The question is: Is the default contractual framework safe for privileged, confidential, or regulated data? And can you prove governance (including access logs, usage controls, and auditability) later? 

A Risk Management Lens is Converging on GenAI. 

NIST’s AI Risk Management Framework (AI RMF 1.0) and its Generative AI profile provide organizations with an organized approach to mapping, measuring, and managing AI risk across the lifecycle, a useful context for compliance officers evaluating AI deployments in regulated or legally sensitive environments. 

Designing Defensible AI for Regulated Legal Workflows 

The legal question in Nippon Life v. OpenAI (“information vs. advice”) ultimately becomes a product question: what does your system allow, what does it discourage, and what does it prove? 

Recommended guardrails 

A defensible design pattern is to treat AI as an evidence intelligence and drafting assistant, never as an autonomous advisor: 

• Use role-aware experiences: professional workspaces should operate under different permissions, safeguards, and governance rules than public consumer interfaces. 

• Make outputs verifiable: require source citations, document links, and traceability to underlying evidence rather than generating unsupported conclusions. 

• Introduce friction for high-risk actions: requests for case-specific advice or filings should trigger disclaimers or “licensed professional review required” workflows. 

• Maintain audit trails: regulated legal teams require logs, version history, and governance controls for defensibility. 

ABA Formal Opinion 512 reinforces why this matters: lawyers must consider duties of competence, confidentiality, communication, supervision, and meritorious claims when using generative AI tools. Those professional obligations do not disappear simply because “the assistant” is software. 

Flowchart: a guardrail “legal AI” product loop