In brief
The court’s October survey of 360 judges found that AI hallucinations—a term for AI‑produced misinformation—are rife in Korean courts. Eight in ten judges encountered nonexistent or false legal precedent citations, and the majority say the justice system needs formal measures to prevent the spread of such fake references.
In a startling revelation, a survey conducted by the Supreme Court of South Korea found that eight out of every ten judges have faced cases where artificial intelligence (AI) generated, nonexistent or false legal precedents were cited in court. The study, covering 360 judges from Oct. 6‑16, points to the growing problem of “AI hallucination” and urges the creation of safeguards to protect the integrity of Korean jurisprudence.
According to Kyodo News, the survey asked judges involved in civil, family and administrative matters to report on half a dozen scenarios in which AI hallucinations surfaced. 196 judges—80.7 percent of the survey—reported seeing citations of non‑existent legal precedents, while 159 judges, or 65.4 percent, observed cases involving false precedent citations. When the question shifted to other legal documents, 41 judges, or 16.9 percent, had encountered citations of fabricated laws and regulations, and 10 judges, or 4.1 percent, were confronted with false or non‑existent theses and publications.
Surprisingly, the survey also revealed that many judges themselves had repeated the mistake. 164 judges, representing 45.6 percent of respondents, admitted to citing false precedents in their rulings. In addition, 136 judges—57.6 percent of the total—indicated that attorneys representing parties on either side of a dispute had cited false statements, further amplifying the spread of fabricated legal evidence.
Nearly 92 percent of the judges who answered the survey agreed that institutional measures are required to curb the use of AI‑generated fake precedents. Kyodo News noted that the most frequently suggested remedies include the establishment of legal grounds for imposing fines on parties who introduce false information, the creation of verification requirements for precedent citations, and the formulation of disciplinary measures against lawyers who repeatedly rely on fabricated data.
In a statement, an official of the National Court Administration cautioned that citing AI‑generated false or non‑existent precedents has the potential to delay court proceedings, highlighting the urgency of crafting effective countermeasures anchored in the survey’s results. National Court Administration officials also emphasized the need to develop verification protocols that can quickly identify hallucinated references before they influence judicial outcomes.
Experts point to the phenomenon as an inevitable consequence of the rapid adoption of generative AI models in the legal sector. Without an institutional framework, the court system stands at risk of allowing misinformation to shape legal precedent and judgment. The fact that a majority of judges and a sizable proportion of lawyers are practicing with “AI hallucination” in the mix raises questions about the bright‑line jurisdiction that should be reserved for humans in matters of legal truth and verification.
Open questions emerge from the findings. How can the court balance the efficiency gains from AI tools with the risk of misinformation? Who bears responsibility when a fabricated precedent emerges from an attorney’s prompt to a language model? And what role should a centralized fact‑checking body play in pre‑judicial research?
While this is the first systematic survey of its kind in South Korea, it echoes similar concerns globally, where legal professionals are increasingly turning to AI for drafting briefs and researching case law. The Korean experience could serve as a blueprint for other jurisdictions navigating the clash between AI innovation and the need for verifiable legal truth.
As the Supreme Court moves forward with drafting policy or rule‑making in response to the survey, the judiciary will be closely watching the development of verification tools and the legal frameworks that will determine whether the mishaps reported this week become isolated incidents or a province of modern Korean law.
