ChatGPT is not a paralegal: the professional implications for lawyers in using ChatGPT
16.2.2024
Open AI announced the release of ChatGPT-4 in late 2022, billing it as a revolutionary next step in 'artificial intelligence'. Many people around the world got stuck into finding out what this new tool could do.
Then, in May 2023, there was news that an attorney in New York, USA had:
Used ChatGPT-4 for legal research.
Referred to the 'cases' that ChatGPT-4 'cited' for him without checking to see if they (a) existed or (b) stood for the principles for which he was using them.
When this was raised with him, put on an affidavit that included the 'text' of most of the 'cases' he had cited – where that text was also produced by ChatGPT-4.
He ultimately admitted to this conduct, pleading ignorance about how the technology operates. He, his supervising attorney and their firm were fined in a sanction imposed by the Court.
More recently, also in New York, Michael Cohen’s lawyer wrote a letter to the Court seeking to terminate his supervised release (which would essentially be a reduction in his sentence) – and that letter contained three citations of cases which turned out not to exist. It turned out that the references had been provided by Cohen and not checked by his attorney.
(The attorney says that he believed the references had been provided by a second attorney who was also working on the case, but that attorney says she was not involved until after the letter was sent, and there has been no reporting on any findings made by the New York Court in this respect.)
In any event, as Joe Patrice at Above The Law observed, it is ‘stunning’ that lawyers in New York had not learnt from the earlier case. Patrice also points out that these are just the cases where the conduct has been discovered and reported. It is almost certain that it is happening in many more cases, and that it is happening here in Australia, too.
So, what ethical issues arise for a lawyer using ChatGPT-4, or a similar tool? How would the Australian system respond to conduct of this kind?
These questions have public law significance because lawyers are officers of the Court, and owe their primary duties to the Court. Their work enables the Court to carry out its functions and to maintain its Constitutionally-required 'institutional integrity'.
What is ChatGPT-4?
ChatGPT is a large language model, also known as a neural network. It is one of a category of models called ‘generative AI’, because they are able to generate content. ChatGPT does this through being ‘trained’ on a very large amount of written content. This training and the algorithms with which the model is built allow ChatGPT to respond to a question or prompt, essentially as a very sophisticated predictive text tool.
ChatGPT-4 is the latest generation of this particular tool.
However, ChatGPT is not intelligent – that is, it does not think in the way we use that word. Although the responses it provides to prompts often appear sensible at face value, those responses frequently contain 'hallucinations', or factual errors, and sometimes logical errors. Law is not the only field in which it has been found to provide citations to works that do not exist. ChatGPT generally fails 'to produce academic texts and articles with authentic and reliable information.'
The New York cases suggest that much the same can be said about its production of legal texts.
What is the issue with what the New York attorneys did?
The major problems in both of the New York cases lie in the failures of the lawyers involved to understand the tool they were using or apply their own thinking to the results it produced.
In the May 2023 case, first, because the attorney did not understand what ChatGPT was doing, he did not recognise that it could produce hallucinations – and that it had in fact produced hallucinations in the text he used. Secondly, a (not very) critical reading of that text would have revealed it for the nonsense it was, and it seems he failed to read critically, or think critically about, that text. As a result, he provided legal texts to the Court which were simply incorrect, both in his own briefs (which in Australia we would call written submissions) and in the supposed cases he provided.
There are potentially two broad breaches of his obligations here: a failure in his obligation to assist the Court (potentially including a failure in his duty of candour), and a failure in his obligation to his client (who certainly did not benefit from the attorney’s work). The first of these breaches has the potential to undermine the relationship of trust between advocates and the Court, and thus have an impact on the capacity of a Court to carry out its constitutional functions, and so is the focus of this post.
Something similar can be said about the Cohen case, although there, the attorney’s first failure seems to have been in not checking the material provided to him, whether by his client or another attorney.
What are the relevant obligations in Australia?
New South Wales is one of the Australian states which has adopted the Legal Profession Uniform Law (Uniform Law) and the various uniform Rules associated with it. The analysis set out below is based on the version of the Uniform Law and Rules enacted in NSW, as well as the caselaw which is relevant to the understanding of the NSW legislation and rules.
Lawyers in Australia are officers of the Court. The kinds of obligations which arise out of this status are embodied in rules such as rule 23 of the Legal Profession Uniform Conduct (Barristers) Rules 2015:
‘A barrister has an overriding duty to the court to act with independence in the interests of the administration of justice.’
Barristers are also required to ‘not deceive or knowingly or recklessly mislead the Court’ (rule 24) and to take steps to correct any misleading statement (rule 25). Rules 3 and 19 of the Legal Profession Uniform Law Australian Solicitors’ Conduct Rules 2015 are of a similar effect.
That the Court can have trust in those appearing before it is fundamental to our system, and forms the basis of certain privileges held by advocates: Re B [1981] 2 NSWLR 372 at 381-382 (as cited, for example, in Morrissey v NSW Bar Association [2006] NSWSC 323 at [36]).
In Australia, there is a Constitutional 'necessity to ensure the integrity of the judicial process and the integrity of the courts', both Federal and State: see, for example, Kable v DPP (NSW) (1996) 189 CLR 51 at 104 (Gaudron J). That 'institutional integrity' means that a Court has certain 'defining or essential characteristics', including procedural fairness and the giving of reasons: see, for example, Wainohu v NSW (2011) 243 CLR 181 at [44] (French CJ and Kiefel J). These tasks are made easier – and, in some cases, even possible – by the work of the advocates appearing before the Court and the fact that judges can trust them. As such, that trust, and the privileges that advocates hold, help the Courts to run more smoothly and efficiently – and to maintain the characteristics that are part of their Constitutionally-required 'institutional integrity'.
The obligations of advocates are much the same in any Australian jurisdiction, although the sources and specific expression of the obligations might be different in jurisdictions which have not adopted the Uniform Law.
A breach of these obligations can amount to 'unsatisfactory professional conduct' (if the breach 'falls short of the standard of competence and diligence that a member of the public is entitled to expect of a reasonably competent lawyer': Uniform Law, s 296) or the more serious 'professional misconduct' (if the failure to meet the standard is 'substantial or consistent': Uniform Law, s 297(1)(a)).
The Uniform Rules can provide a guide to what the 'standard of competence and diligence' is, whether by analogy or through a finding that a breach of the Uniform Rules amounts to unsatisfactory professional conduct and/or professional misconduct (eg Uniform Law, s 298(b)).
Would the New York conduct breach Australian obligations?
On the basis of the publicly available facts, the attorney’s conduct in the May 2023 case would appear to breach the relevant obligations, had a legal practitioner engaged in the same conduct in NSW.
In relation to the obligations to the Court, the conduct was misleading. And, even accepting the attorney’s evidence that he did not understand that ChatGPT might produce hallucinations, it was arguably reckless for him to have proceeded without checking the citations, for example, by searching for them in a reputable database.
The 'cases' which he attached to his affidavit are plainly nonsensical. Failing to have read them with a critical eye would reinforce any finding of recklessness.
On the available material, it would be open to the NSW Civil and Administrative Tribunal (NCAT, the forum in which such proceedings would be heard in NSW) to be satisfied that he had breached rule 24 of the Barristers’ Rules or rule 19.1 of the Solicitors’ Rules, as appropriate. NCAT might also be satisfied that he had breached rule 25 or rule 19.2 respectively, in failing to correct the misleading statements, although he did ultimately come clean once he finally understood the situation.
There might also be a separate finding that he had breached his overriding duty to the Court (rule 23 of the Barristers’ Rules or rule 3 of the Solicitors’ Rules), although this might not add much to any other breaches which were found to have occurred.
A similar result is likely to apply in other Australian jurisdictions.
For conduct more like that in the Cohen case, the fact that the attorney placed reliance on his client’s work without checking it might be considered to fall short of the relevant standard, even if NCAT accepted that he believed it had come from another attorney. But the fact that the explanation was given much more quickly – for example, there was no attempt to provide the Court with the text of the 'cases' that had been cited – would likely mean that any breach was seen as less serious than in the May 2023 case.
Would there be a finding of unsatisfactory professional conduct or professional misconduct in relation to the obligations to the Court?
The need for judges to be able to ‘have confidence in the legal profession’ is one of the ‘four interrelated interests’ identified by Spigelman CJ in NSW Bar Association v Cummins (2001) 52 NSWLR 279 (at [20]) as giving rise to the significant public interest in maintaining the integrity of the profession.
This means that any breaches of a lawyer’s obligations to the Court are taken very seriously, especially where there is any misleading conduct.
In these circumstances, whether seen through the lens of a breach of the Rules or simply because it is a relatively clear failure to reach the relevant standard of conduct required by the Uniform Law, it is likely that conduct of the kind seen in the May 2023 case would result in a finding of unsatisfactory professional conduct.
It would also be open for NCAT to find that the conduct demonstrated a 'substantial or consistent failure to reach or maintain' the relevant standard, and so amounted to professional misconduct.
In Council of the Law Society of NSW v Webb [2013] NSWCA 423 at [22], Meagher JA explained that a 'substantial' failure to reach or maintain the relevant standard 'directs attention to the nature and consequences of the failure, which may be sufficiently serious to raise questions as to the lawyer's competence and diligence', whereas a 'consistent' failure refers 'to ongoing or persisting acts of failing on different occasions to reach or maintain the required standard.'
It would be arguable that there has been a 'substantial' failure to reach or maintain the relevant standard because of the 'nature and consequences' of the lawyer’s conduct. That is, if the relevant standard of competence and diligence includes the use of research tools, critical thinking and knowledge of the law:
the nature of the failure involved quite a large departure from what might reasonably be expected; and
the consequences of the failure include that the lawyer ended up misleading the Court, and caused injury to the proper administration of justice, including greater impact of Court resources and increased legal costs.
In some cases, where conduct includes a breach of the Uniform Rules, an argument based on this professional misconduct test is stronger, because the Rules can be seen to reflect a clear expression of the standard required of legal practitioners. That can see a breach characterised as more serious in nature, because the lawyer should have known what they needed to do.
That argument might add little to the overall characterisation of the conduct relating to misleading the Court. The Rules which would be relied on in this respect state a general standard (ie that lawyers are not to mislead the Court) rather than a specific standard (eg in relation to legal research or the appropriate use of tools by lawyers). So, in this case, the fact that there had been a breach of the Rules would likely take the argument no further than the analysis set out above.
A finding that there was a 'substantial' failure would be enough to justify a conclusion that there was professional misconduct, but there might also be an argument about whether there had been a 'consistent' failure. This really would be arguable either way. On the one hand, the lawyer continuing to use ChatGPT (and not recognising its hallucinations) even after his opponent raised the question of whether the cited 'cases' really existed could be seen as sufficiently 'persistent' as to be relevantly 'consistent'. On the other hand, the conduct could be viewed as a whole, and as a single instance, and if that approach were adopted, it would likely not be relevantly 'consistent'.
In all the circumstances, there is a reasonable likelihood that NCAT would make a finding of professional misconduct.
A finding of professional misconduct on the basis of the above reasoning would not likely be mitigated by an acceptance of the lawyer’s evidence that they did not intend to mislead the Court.
For conduct more similar to the Cohen case, a finding of unsatisfactory professional conduct seems more likely, given the more limited nature of the breach in question.
What would happen to the lawyer involved?
This is a case where the sanctions imposed on a lawyer would very much depend on how the lawyer responded to the complaint and how they dealt with the process before NCAT.
Again, assuming that they made similar admissions to the attorney in the May 2023 case, and particularly if they made those admissions from an early stage, it is very unlikely that they would be struck off for this conduct alone.
Rather, a sanction of a similar level to what was seen in the May 2023 case would be much more likely – perhaps a reprimand or caution (these are both public statements that the lawyer has done the wrong thing), maybe a fine or some educational requirements, and possibly some combination of those sanctions.
In a case more similar to the Cohen case, any sanctions are likely to be of a lower order than in the May 2023 case.
So, can lawyers use ChatGPT?
Lawyers should be cautious in their use of ChatGPT. As with any other tool, as a minimum, lawyers should inform themselves as to how ChatGPT works, at least in general terms, and what its limitations are before using it. This will allow lawyers to determine whether it is an appropriate tool for their practice generally or for a specific task – and to assess whether the text it produces allows the lawyer to meet their own professional obligations. These matters will ensure that the lawyer meets the relevant standard of competence and diligence and complies with their obligations to the Court.
Failing to do so could lead to a finding of unsatisfactory professional conduct or professional misconduct, and to sanctions being imposed. These are very serious matters for a legal practitioner, although it is not likely that a single breach would lead to being struck off.
Brenda Tronson is a senior lecturer at UNSW Law & Justice and a barrister at Level 22 Chambers in Sydney.
Suggested citation: Brenda Tronson, ‘ChatGPT is not a paralegal: the professional implications for lawyers in using ChatGPT’ (16 February 2024) <https://www.auspublaw.org/blog/2024/2/chatgpt-is-not-a-paralegal-the-professional-implications-for-lawyers-in-using-chatgpt>