Trying Out GPT4All

I tried out GPT4All. It is an app that can run an LLM on your desktop. Some of the models it can use allow the output to be used for commercial purposes. The only downside was it is not very fast, and makes my CPU run hot. The project provides installers for Linux, Apple, and Windows (or, as I like to call it: Win-Don’t).

I found out about it while looking into alternatives to ChatGPT and OpenAI. There are a few videos about it on Matthew Berman’s channel, including an interview with the CEO of Nomic AI, the company that started the GPT4All project. I think it is great that people are making alternatives to ChatGPT and OpenAI.

I first downloaded it a couple of weeks ago. The installer itself worked fine, but I had to change the permissions to get it to run: chmod 755 I got an error when I tried to run the program: bin/chat: error while loading shared libraries: cannot open shared object file: No such file or directory. I installed the library as root: apt install libxcb-cursor0. After that it worked fine.

I downloaded the installer again today and re-installed to see if anything was different. I might have still gotten the libxcb-cursor error if I had not installed it earlier. Now the program lets you choose a model when you first run it. Before the model file had to be in the same directory as the executable, and if there was more than one model file it would pick the default. When I tried to run it the first time, I could download a couple of models, but I could not get past that screen. I had to exit and restart the app to actually use the models. There is a list of the models available here (historical commit here). The output of the default model can be used in commercial applications. Most of the other models are based on LLaMA, and commercial use of the output is prohibited.

The new version also gives you the option of sending feedback on the responses back to the GPT4All project.

The model files are from 3.5GB to about 7.5 GB. I have downloaded several model files over the past few weeks, and I have used up as much hard drive space in two weeks as I did in five years on my recently deceased laptop.

While the application is idle it does not use a lot of resources, but when it is generating output the CPU usage and temperature both spike. I do not know if I will be running this on a regular basis. My Meerkat has an Intel i5-10210U CPU, 4.20 GHz, 6 MB Cache, 4 Cores and 8 Threads with 64 GB of memory. The GPU is Intel, so I do not think I will be running a lot of Pytorch.

When I was using the version of the software from two weeks ago and using the ggml-gpt4all-j-v1.2-jazzy model file, I asked it to write some sample sales emails for companies in different industries. It kept writing emails asking for sample sales email instead of an actual sales email. I tried the new ggml-gpt4all-j-v1.3-groovy model (“Current best commercially licensable model based on GPT-J and trained by Nomic AI on the latest curated GPT4All dataset”), and it did better, but the results were pretty similar for each one. They all started with “Exciting news!” and offered a “wide range of” something. I read “Exciting news!” so many times, I started hearing the voice of Professor Farnsworth in my head. Here is the output from the prompt “Write an email from a consulting company whose software developers use Emacs”:

Subject: Exciting news! Our software developers are now using Emacs.

Dear [Recipient’s Name],
I hope this email finds you well. My name is [Your Name] and I am the owner of [Company Name]. We are excited to announce that our software developers are now using Emacs as their primary text editor.

As a company that specializes in software development, we understand the importance of having the best tools for the job. That’s why we offer a wide range of software development tools, including Emacs. Our developers are trained to use the best tools available, and we are always looking for new clients to hire.

If you are interested in learning more about our services, please don’t hesitate to contact us. We would love to discuss your project and see how we can help you achieve your goals.

Thank you for considering [Company Name]. We look forward to hearing from you soon.

I have to agree, Emacs is the best tool for just about any job.

I tried the same prompt with the same model on the new version, and I got this:

Subject: Help with Emacs – Our Developers Need Assistance

Dear [Name],

I hope this email finds you well. We are writing to inquire about your experience using the popular text editor, Emacs. As we understand that you work as a developer for our esteemed client, we would like to offer some assistance in resolving any issues you may be facing while working on their projects.

We have noticed that many of our clients often struggle with certain aspects of using Emacs, such as navigating through files efficiently, finding specific lines of code, or even understanding basic syntax. These challenges can lead to frustration when trying to accomplish tasks within the program.

To help alleviate these frustrations, we would love to invite you to participate in a webinar where we will provide step-by-step instructions on how to effectively navigate and utilize the features of Emacs. This webinar will cover topics such as file navigation, search functions, and programming language support.

If you are interested in participating, please let us know by responding to this email. If not, no need to worry, we completely understand if you prefer to continue using another tool. However, we believe that our webinar could greatly benefit you and your team members who also work on projects involving Emacs.

Thank you for considering our invitation. We look forward to hearing back from you soon.

Best regards,
[Your Name]

Not as positive. I selected the option to regenerate output, and it was just one sentence that did not even mention Emacs.

I tried the prompt with the model ggml-gpt4all-l13b-snoozy, which is “Current best non-commercially licensable model based on Llama 13b and trained by Nomic AI on the latest curated GPT4All dataset”, and I got this:

Subject: Emacs Software Development at [Consulting Company]

Dear [Client],

I hope this email finds you well. My name is [Your Name], and I am the Director of Software Development at [Consulting Company]. I am writing to express our excitement about the opportunity to work with you on your upcoming project.

At [Consulting Company], we pride ourselves on our use of Emacs as our primary text editor. Our developers have found that Emacs provides a powerful and flexible platform for software development, and we believe that it will be an asset to your project as well.

We understand that your project is of the utmost importance, and we are committed to delivering a high-quality solution that meets your needs. Our team of experienced developers will work closely with your team to ensure that your project is completed on time and within budget.

If you have any questions or concerns, please do not hesitate to reach out to us. We look forward to working with you and delivering a successful project.

Best regards,

[Your Name]
Director of Software Development
[Consulting Company]

I agree that “Emacs provides a powerful and flexible platform for software development”, and that it is as asset to any project.

Some of the responses interpreted the prompt to produce what I want (“We are amazing and Emacs is amazing and we can make amazing software for you with Emacs”) and one was offering to teach developers how to use Emacs. I tried a couple of more times, and again the results were different with the same prompt.

There is a project that provides a command line interface to the models called LlamaGPTJ-chat, which I might get to in a few more days. I have tried a couple of other CLI programs, and they also use a LOT of CPU.

Michael Finney wrote about his experience with GPT4All here. Per his article, GPT4All will soon have an API that can be called and run on a local server. I joined the GPT4All discord, but I have not spent too much time there.

I do not know if I will be using this too much since it puts a lot of stress on my system. I will see if anyone is running models on the cloud and find out how much it costs. But I support anyone who will give the world alternatives to OpenAI.

You’re welcome.

I am not using an image of someone writing because while this post mentions Emacs, it is not about Emacs.

Image from Madrid Skylitzes, a 12th century manuscript of Synopsis of Histories by Greek historian John Skylitzes; manuscript housed at National Library of Spain; manuscript information here. Image from World Document Library, licensed under CC BY-NC-SA 4.0.

1 thought on “Trying Out GPT4All”

Comments are closed.