Analysis: Are law firms right to ban ChatGPT?

Are law firms and legal departments right to ban ChatGPT? YES.

Are better alternatives available? ABSOLUTELY.

THE PROBLEM:

ChatGPT is an extremely powerful and versatile productivity tool, but it is a consumer product that does not meet the industry’s needs for:

1. Compliance with confidentiality, data privacy, cybersecurity and other policies and regulatory obligations;

2. Reliability and ability to provide verifiable source citations;

3. Integration with relevant domain-specific, company-specific, and matter-specific sources of knowledge; or

4. Guardrails against intentional or unintentional misuse.

However, banning ChatGPT can only be a partial and very short-term solution because the competitive pressure to increase productivity is high and circumventing bans is just too easy and too tempting for individual employees.

And ChatGPT is just the tip of the iceberg.  Thousands of companies, including Microsoft and Google, are rolling out new generative AI tools and integrations for every conceivable use so quickly that, within a matter of months, AI tools are likely to impact every profession and become as ubiquitous and essential as mobile devices and the internet.

THE SOLUTIONS:

Fortunately, it is already possible to access GPT, the language model behind ChatGPT (as well as other similarly powerful models), without compromising security, confidentiality or data privacy.  For example:

• Any company can easily deploy their own chatbot by obtaining and using a secret "API Key", in which case OpenAI will NOT store your chat history (except for a 30-day period for abuse prevention purposes). 

• If the commitments of OpenAI are not sufficient – or if there are specific data residency requirements - companies are able to use the Microsoft Azure OpenAI service in Western Europe and other locations, in which case no data is ever sent to OpenAI.  Microsoft offers different encryption options for sensitive data and also allows companies to opt out of the 30-day data retention for abuse purposes.  The service also includes guardrails in the form of content filters.

• If a cloud-based provider is not acceptable, firms can opt for a self-hosted solution, although they would almost certainly have to settle for a smaller language model than the latest version of GPT.  However, in many cases a self-hosted solution that has been trained (fine-tuned) to perform a relevant subset of tasks could be as good or even better than GPT.

• Finally, a number of companies are building generative AI products -- such as "Harvey" deployed by the firm Allen & Overy -- specifically for the legal industry or related use cases. These products typically use one of the approaches described above and can include a range of additional useful features, making them an attractive proposition for firms interested in a packaged solution.

While the comments above are focused on GPT and ChatGPT, similar approaches can be deployed using competing language models such as Google’s Palm2 (and soon, Gemini), Anthropic’s Claude, and many others.

Furthermore, it is also possible to enhance existing language models with company-specific data and other features that dramatically reduce hallucinations, allow for reliable citations and make the models far more useful in practice.  

Historically, machine learning efforts have focused on training custom models for specific tasks, but, with the advent of massive foundation models like GPT, better results can often be achieved much more easily through techniques such as finetuning and retrieval-augmented generation (RAG).  The latter technique is also effective not only in teaching the language model about specific information or set of tasks, but also allows source information to be provided as part of an answer.

- 

To learn more about generative AI and get practical advice on deploying AI in your company, join Arbi.City and stay tuned for our upcoming event series!

The series is suitable for both non-technical and technical professionals and will look beyond the headlines to zero in on issues of particular relevance to the legal sector, such as AI policies and regulation, responsible AI, and deployment of AI tools by law firms.