In the landmark case of Pooja Ramesh Singh v. Jammu and Kashmir Bank Ltd. [1], the Supreme Court of India set aside orders passed by both the NCLT and NCLAT based on the fake AI-generated legal precedents. Declaring a strict zero-tolerance policy for both the Bar and the Bench, the Apex Court affirmed that citing, referring to, or relying on hallucinated and fabricated case laws completely vitiates the sanctity of judicial decision-making and decisions based on them must be set aside.
Facts of the Case
Pan India Utilities Distribution Company Ltd. (PIUDCL) availed loan facilities from Jammu and Kashmir Bank Ltd., with its group affiliate, Essel Infraprojects Ltd. (EIL), acting as the corporate guarantor. Following severe financial defaults by PIUDCL, Jammu and Kashmir Bank Ltd. invoked the guarantee and initiated Section 7 insolvency proceedings under the IBC against EIL. The NCLT admitted the insolvency petition, and the NCLAT subsequently affirmed the decision on appeal. However, both tribunals supported their rulings by relying on six Supreme Court decisions referenced in the NCLT’s original order.
Before the Supreme Court, Appellant Pooja Ramesh Singh (a suspended director of EIL) revealed through Senior Advocate Madhavi Divan that all six cited precedents were entirely fake, non-existent, or contained fabricated text generated by AI. Jammu and Kashmir Bank Ltd. confirmed through an affidavit that its lawyers had never cited these cases, proving that the NCLT generated the hallucinated precedents through its own internal research while the NCLAT failed to detect the error. Consequently, the Supreme Court set aside both tribunal orders and remanded the matter, establishing a strict zero-tolerance policy against the reliance on unverified AI outputs in judicial decision-making.
Decision of the Court
While pronouncing its decision, the Supreme Court highlighted an increased integration of AI in modern life, be it routine or intellectual work. And, while AI has acquired the capability to better human effort, a critical vulnerability exists i.e. Large Language Models frequently generate hallucinated and non-existent output when prompted. As legal practitioners, judges, and administrative officers increasingly adopt AI efficiency tools, the Bench emphasized the imperative for absolute human oversight (human-in-the-loop) in the legal ecosystem for ensuring the sanctity of decision-making process by the judges.
The Court established the following binding principles governing use of AI:
- Mandatory Verification of AI output & Zero Tolerance Policy: Courts must enforce an absolute zero-tolerance approach against producing, citing, or using unverified AI-generated precedents.
- Advocate Misconduct: Citing fake or hallucinated precedents without prior verification constitutes professional misconduct by Advocates who must exercise caution while researching with AI tools. The Court directed the Bar Council of India (BCI) to constitute a dedicated committee to formulate guiding principles and prescribe formal disciplinary sanctions for such violations.
- Decisions based on fake precedents are void: A judge relying on hallucinated material commits a serious judicial lapse. Consequently, any judgment tainted by fake citations is deemed “no decision in the eyes of the law” and must be set aside entirely. Even if an iota of fake material enters the decision-making process, it invalidates the ruling regardless of whether it had a direct or indirect bearing on the final determination of the case.
- Permissible Assistance vs. Impermissible Delegation: The Bench clarified that this ruling does not prohibit the rightful, assistive use of AI and Machine Learning tools to improve administrative efficiency, such as transcribing of oral arguments in constitutional matters, translation of judgments in other languages, or conducting verified legal research. However, there has to be a total and absolute control over the use of AI in adjudication of a case, with a human in the loop at every stage.
- Implementation of Policy & Enforceable Regulations: The Bench stressed that AI risks cannot be managed solely through judicial rulings, but require comprehensive public policy, enforceable rules, and active Bar-Bench coordination. In this context, reference was made to the Draft Regulations for the Use of Artificial Intelligence in Courts, 2026 published by the Supreme Court’s AI Committee under Article 145 which represents India’s first major milestone in regulating AI within the legal ecosystem.
Analysis & Conclusion
The ruling in Pooja Ramesh Singh marks a definitive line in the sand for legal technology in India. While AI tools offer unprecedented speed and operational support, they can never replace independent judicial reasoning. Adjudication is a sovereign function of the State [2] that cannot be delegated to automated systems prone to inadvertent errors. While AI can assist in streamlining administrative workflows and improving efficiency, true decision-making requires evaluating unique factual contexts, applying constitutional values, exercising human empathy, and adapting legal principles to evolving societal needs. Ultimate judicial accountability must always rest with human judges rather than automated tools.
Practice Questions
Q1. “The duty of an advocate to the Court is of paramount nature. Citing hallucinated precedents under the guise of AI efficiency threatens the doctrine of Stare Decisis.” Discuss the above statement in light of recent judicial pronouncements.
Q2. “The Supreme Court of India has drawn a strict line between using Artificial Intelligence for court assistance and delegating case decisions to automated systems, making human control mandatory at every stage.” Discuss this statement in light of recent judicial pronouncements and legal developments, explaining why deciding legal disputes remains an irreplaceable human function and analyzing the legal consequences when courts or counsel rely on unverified AI outputs.
- Pooja Ramesh Singh v. Jammu & Kashmir Bank Ltd. & Anr., 2026 SCC OnLine SC 1258.
- All India Judges Association v. Union of India, (1992) 1 SCC 119.