GPT-4 AI model successfully passes Bar Exam

March 23, 2023 Robert Abruzzese

Text from the ChatGPT page of the OpenAI website is shown in this photo, in New York, Thursday, Feb. 2, 2023. AP Photo/Richard Drew

Share this:

Previous attempts by the artificial intelligence language model, ChatGPT-3.5, failed to pass the multiple-choice section of the bar exam. However, a newer model of GPT has succeeded at passing the multiple choice, the essay portion, and it scored within the top 10% of test takers.

The new GPT-4 was the latest AI model from OpenAI that took the exam. OpenAI claims that GPT-4 demonstrates human-level performance across various professional and academic benchmarks.

For this recent test, legal scholars and scientists Daniel Martin Katz and Michael Bommarito partnered with the legal AI company Casetext. Casetext recently launched the AI legal assistant CoCounsel, which is powered by GPT-4.

“GPT-4 significantly surpasses the capabilities of previous language models,” Pablo Arredondo, co-founder and chief innovation officer for Casetext, said. “Its ability to not only generate text but also interpret it signifies the dawn of a new era in the practice of law.”

In their earlier paper, GPT Takes the Bar Exam, Katz and Bommarito recounted their experience testing GPT’s late 2022 version on the bar exam, which failed to pass any section.

In an upcoming paper, they will discuss how GPT-4 successfully passed the multiple-choice section and both written components, outperforming all previous models and exceeding the average score of actual bar exam candidates.

Casetext advertises its CoCounsel product for seven core functions for attorneys: Database research, document review, document summaries, and it claims it can assess contracts for policy compliance. It can also allegedly extract data from contracts, compose legal research memos and prepare for depositions.