Build an AI driven App with OpenAI, LangChain4j, and Spring Boot with React

2023-11-30 Nick DalalelisAI

As I began my journey to learn about AI and Large Language Models (LLMs), I stumbled across the Java library LangChain4j. I dug in and started learning concepts like prompt engineering, embeddings, and vector stores. I was inspired to take a hands on approach and develop an AI driven application.

In this blog post, I’ll walk you through the steps to build an AI driven application with OpenAI, LangChain4j and Spring Boot with React . I’ll also share some of the lessons I learned along the way. If you would like to follow along, the code can be found in my GitHub repository nickdala/piggy-bank-langchain4j.

What do you need to follow along

We’ll use GitHub CodeSpaces to run the application. GitHub CodeSpaces is a cloud-based development environment that runs in your browser. It allows you to develop entirely in the cloud. To follow along, you will need:

You can also use your own development environment to run the application. Below are the requirements for your development environment according to Hilla.

Java 17 or later
Node 18 or later
An IDE that supports Java and TypeScript — IntelliJ Ultimate and VS Code are good choices

Generate an OpenAI API Key

You will need to create an OpenAI API key to run the application. You can create a secret API key here.

Click on the API keys link.
Click on the Create new secret key button.
Give the secret key a name.
Click on the Create secret key button.
Copy the API key.

What are we going to build

The sample application is called Piggy Bank. This is a simple Spring Boot and React application that simulates a bank.

Piggy Bank is written in Java using Hilla. The Hilla web framework was chosen because it brings together the power of Spring Boot and React. It allows you to build a full stack web application with a single codebase.

To keep things simple, the application covers the following use cases.

A Bank Associate can see a list of customers
A Bank Associate can click on a customer to see their accounts
Accounts can either be Opened or Closed

Finally, we’ll integrate OpenAI’s GPT-3 API to allow a customer to close their account through an AI driven chatbot.

Build & Deploy

The easiest way to build the sample is to use GitHub CodeSpaces. You can also use dev container to build the sample locally or use your own development environment.

To build the sample using CodeSpaces, follow the steps below.

Navigate to the GitHub repository nickdala/piggy-bank-langchain4j
Click on the Code button.
Click on the Codespaces tab.
Click on the Create codespace on main button.

This will create a GitHub Codespace and clone the repository. Once the Codespace is ready, you will see the following.

Next, we need to set the OpenAI API key in the application.properties file. To do this, navigate to src/main/resources/application.properties and set the property langchain4j.chat-model.openai.api-key.

Finally, we can build and run the application. Run the following command in the Codespaces terminal.

./mvnw

This will build and run the application. A new browser window will launch automatically with the Piggy Bank application.

Here is the UI of what the application looks like.

Close the account with an AI driven chatbot

To close an account, click on the Chat menu. This will launch an AI driven chatbot. The chatbot will ask you a series of questions to close the account.

Here’s a sample conversation with the chatbot.

Provide the chatbot with the id of the account you want to close. Once the account is closed, verify this in the UI.

Pretty cool, right?

Getting started with LangChain4j

Lets now take a look at how we incorporate LangChain4j into our application.

Maven Dependency

The first step is to add the LangChain4j dependency to your pom.xml file of your Spring Boot application.

<dependency>
  <groupId>dev.langchain4j</groupId>
  <artifactId>langchain4j-spring-boot-starter</artifactId>
  <version>0.24.0</version>
</dependency>

Include OpenAI API Key in application.properties

langchain4j.chat-model.openai.api-key=<your-openai-api-key>

How does the application interact with OpenAI

Now that we have the application up and running, let’s take a look at how it all works. We’ll dive into the following topics.

Large Language Models (LLMs)
Prompt Engineering
LangChain4j Tools

What are Large Language Models (LLMs)

Large Language Models (LLMs) are a new class of AI models that are trained on large amounts of text. The models are designed to understand and produce human-like text. In the Piggy Bank application, the AI powered chatbot produces personalized text to close an account. But how does the LLM produce the text? Let’s take a look at the process.

Prompt Engineering

The system message helps set the behavior of the LLM. A properly crafted system message will help the LLM produce the desired response.

Let’s take a look at the system message that we use to close an account. Within the interface CustomerSupportAgent, we have the following @SystemMessage.

public interface CustomerSupportAgent {
    @SystemMessage({
        """
            You are a customer support agent of a bank named 'Piggy Bank Assistant'.
            Ask the customer how you can help them. The only thing you can assist
            the customer with is closing an account.

            If the customer wants to close their account, you MUST first offer them a deal to stay.
            If they still want to close their account, just close it and provide a polite response.
            If you don't know how to help the customer, ask your supervisor for help.
        """
    })
    String chat(String message);
}

Let’s take a closer look at the system message used by the chatbot.

The first sentence sets the context of the conversation. The context is that the customer is talking to a customer support agent of a bank named Piggy Bank Assistant.
The expectation is that the customer can only close their account.
The support agent will offer a deal to stay before closing the account.
Once the account is closed, the support agent will respond politely.

This is more of a craft than a science. It takes some trial and error to get the system message right. The system message is the key to getting the desired response from the LLM. As an exercise, try changing the prompt and see how the chatbot responds.

LangChain4j Agents & Tools

Agents are used to tell the LLM what actions to take. In chains, a sequence of actions is hardcoded. Agents, on the other hand, utilize a language model as a reasoning engine to determine which actions to take and in which order. Tools are used to help the LLM perform the actions. They are methods an agent can use to perform an action.

Within the class AccountTools, we have the following @Tool annotation.

public class AccountTools {
    @Tool("Close account by id")
    public boolean closeAccount(long accountId) {
        accountService.closeAccount(accountId);
        return true;
    }

    @Tool("Finds the customer by id")
    public CustomerRecord findCustomerById(long customerId) {
        return customerService.findCustomertById(customerId);
    }
}

The @Tool annotation is used to define the tool for the chatbot. It’s important to provide a proper description of the tool. This will help the LLM determine which tool to use.

Cleanup

Make sure to delete the GitHub Codespace when you are done. To do this, click on the Codespaces tab and click on the Delete button.

Takeaways

OpenAI and LangChain4j make it easier to build AI driven applications. We used LangChain4j Agents and Tools to aid the LLM in performing the desired actions. We also used Prompt Engineering to help the LLM produce the desired response. Putting it all together, we integrated a chatbot into our application capable of closing an account.

Nick Dalalelis

Cloud Software Engineer