On this page

Langchain4j

1 Introduction

This module provides integration between Micronaut and Langchain4j.

Note
This module is regarded as experimental and subject to change since the underlying technology (AI) is volatile and subject to change.

Various modules are provided that allow automatically configuring common Langchain4j types like ChatModel, ImageModel etc. Refer to the sections below for the supported Langchain4j extensions.

2 Quick Start

Add the following annotation processor dependency:

annotationProcessor("io.micronaut.langchain4j:micronaut-langchain4j-processor")

Then the core module:

implementation("io.micronaut.langchain4j:micronaut-langchain4j-core")

You are now ready to configure one of the Chat Language Models, for the quick start we will use Ollama:

implementation("io.micronaut.langchain4j:micronaut-langchain4j-ollama")

To test the integration add the test resources integration to your Maven build or Gradle build.

testResourcesService("io.micronaut.langchain4j:micronaut-langchain4j-ollama-testresource")

Add the necessary configuration to configure the model name you want to use:

Configuring the Model Name
langchain4j.ollama.model-name=orca-mini

3 AI Service

You can also define new AI services:

Defining @AiService interfaces

You can now inject the @AiService definition into any Micronaut component including tests:

Calling @AiService definitions
package example.micronaut.aiservice;

import static org.junit.jupiter.api.Assertions.assertNotNull;

import dev.langchain4j.model.chat.ChatModel;
import io.micronaut.langchain4j.testutils.OllamaTestPropertyProvider;
import io.micronaut.test.extensions.junit5.annotation.MicronautTest;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.TestInstance;
import org.testcontainers.junit.jupiter.Testcontainers;

@Testcontainers(disabledWithoutDocker = true)
@MicronautTest(startApplication = false)
@TestInstance(TestInstance.Lifecycle.PER_CLASS)
class AiServiceTest implements OllamaTestPropertyProvider {
    @Test
    void testAiService(Friend friend, ChatModel languageModel) {
        String result = friend.chat("Hello");

        assertNotNull(result);
        assertNotNull(languageModel);
    }
}

4 Tools

Tools allow AI models to request specific actions that extends beyond their built-in capabilities.

A model would have no way to know the last date when the PRIVACY document was updated. Hence, it is a good candidate for a tool. <3> The dates are harcoded for the purpose of this example, but they could have been retrieved from a database or external API.

You can supply the tools to use to an @AiService:

package example.micronaut.aiservice.tools;

import io.micronaut.langchain4j.annotation.AiService;

@AiService(tools = LegalDocumentTools.class)
public interface CompanyBot {
    String ask(String question);
}

5 Chat Language Models

The following modules provide integration with Langchain4j Language Models.

Each module configures one or more ChatLanguageModel beans, making them available for dependency injection based on configuration.

5.1 ChatModel Example

This example, asks a chat model to generate the list of the top 3 albums of a Jazz musician.

package example.micronaut;

public record Musician(String name, String albums) {
}

You can configure the chat model via configuration.

For example, you may want to configure OpenAI in the main classpath:

src/main/resources/application.properties
micronaut.application.name=micronaut-guide
langchain4j.open-ai.chat-model.log-requests=true
langchain4j.open-ai.chat-model.log-responses=true
langchain4j.open-ai.chat-model.timeout=60s
langchain4j.open-ai.chat-model.temperature=0.3
langchain4j.open-ai.chat-model.model-name=gpt-4.1

And a local SLM (Small Language Model) such as Ollama in the test classpath:

src/test/resources/application-test.properties
langchain4j.open-ai.enabled=false
langchain4j.ollama.model-name=tinyllama
langchain4j.ollama.chat-model.timeout=5m
langchain4j.ollama.chat-model.log-requests=true
langchain4j.ollama.chat-model.log-responses=true

Moreover, you can also register a bean of type BeanCreatedEventListener to configure the Chat Model builder programmatically if configuration is not enough.

package example.micronaut;

import dev.langchain4j.model.ollama.OllamaChatModel;
import io.micronaut.context.event.BeanCreatedEvent;
import io.micronaut.context.event.BeanCreatedEventListener;
import io.micronaut.core.annotation.NonNull;
import jakarta.inject.Singleton;

@Singleton
class OllamaChatModelBuilderListener
    implements BeanCreatedEventListener<OllamaChatModel.OllamaChatModelBuilder> {
    @Override
    public OllamaChatModel.OllamaChatModelBuilder onCreated(
        @NonNull BeanCreatedEvent<OllamaChatModel.OllamaChatModelBuilder> event) {
        OllamaChatModel.OllamaChatModelBuilder builder = event.getBean();
        builder.temperature(0.0);
        return builder;
    }
}

5.2 Chat Memory

Models are stateless by design. Chat memory serves as container for previous messages, helping you maintain context in a conversation, but the model itself is not aware of this memory; it relies on you to include the relevant messages in each request for coherent and contextually relevant responses.

Langchain4J provides an API ChatMemory to help you manage chat memory. You can provide your own implementation or use one of the provided implementations.

The default implementation of ChatMemory, dev.langchain4j.store.memory.chat.InMemoryChatMemoryStore, stores ChatMessage instances in memory.

To use the Redis implementation dev.langchain4j.community.store.memory.chat.redis.RedisChatMemoryStore, add the following dependency:

implementation("io.micronaut.langchain4j:micronaut-langchain4j-store-redis")

To use the Neo4J implementation dev.langchain4j.community.store.memory.chat.neo4j.Neo4jChatMemoryStore, add the following dependency:

implementation("io.micronaut.langchain4j:micronaut-langchain4j-store-neo4j")

To use the Cassandra implementation dev.langchain4j.store.memory.chat.cassandra.CassandraChatMemoryStore, add the following dependency:

implementation("io.micronaut.langchain4j:micronaut-langchain4j-cassandra")

The following example shows how to use the ChatMemory:

package example.micronaut;

import dev.langchain4j.data.message.AiMessage;
import dev.langchain4j.data.message.UserMessage;
import dev.langchain4j.memory.ChatMemory;
import dev.langchain4j.memory.chat.MessageWindowChatMemory;
import dev.langchain4j.model.chat.ChatModel;
import dev.langchain4j.model.chat.response.ChatResponse;
import jakarta.inject.Singleton;
import java.util.Map;
import java.util.UUID;
import java.util.concurrent.ConcurrentHashMap;

@Singleton
public class AssistantWithMemory {
    private final Map<String, ChatMemory> conversations = new ConcurrentHashMap<>();
    private final ChatModel model;
    private final MessageWindowChatMemory.Builder messageWindowChatMemoryBuilder;

    public AssistantWithMemory(MessageWindowChatMemory.Builder messageWindowChatMemoryBuilder,
                               ChatModel model) {
        this.messageWindowChatMemoryBuilder = messageWindowChatMemoryBuilder;
        this.model = model;
    }

    public MemoryIdAndResponse chat(String conversationId, String message) {
        ChatMemory chatMemory = conversations.get(conversationId);
        if (chatMemory == null) {
            throw new IllegalArgumentException("Unknown conversation: " + conversationId);
        }
        chatMemory.add(UserMessage.from(message));
        ChatResponse chatResponse = model.chat(chatMemory.messages());
        AiMessage answer = chatResponse.aiMessage();
        chatMemory.add(answer);
        return new MemoryIdAndResponse(conversationId, answer.text());
    }

    public MemoryIdAndResponse chat(String message) {
        String conversationId = startConversation();
        return chat(conversationId, message);
    }

    private String startConversation() {
        String memoryId = generateChatMemoryId();
        ChatMemory chatMemory = generateChatMemory(memoryId);
        conversations.putIfAbsent(memoryId, chatMemory);
        return memoryId;
    }

    private String generateChatMemoryId() {
        return UUID.randomUUID().toString();
    }

    private ChatMemory generateChatMemory(String memoryId) {
        return messageWindowChatMemoryBuilder
            .id(memoryId)
            .build();
    }
}

You could invoke the previous class as illustrated in the following test:

@Test
void chatWithMemory(AssistantWithMemory assistant) {
    MemoryIdAndResponse johnConversation = assistant.chat("Let me introduce myself. My name is John");
    String johnConversationId = johnConversation.memoryId();
    assertNotNull(johnConversationId);
    MemoryIdAndResponse aegonConversation = assistant.chat("Let me introduce myself. My name is Dan");
    String aegonConversationId = aegonConversation.memoryId();
    assertNotNull(aegonConversationId);
    MemoryIdAndResponse answer = assistant.chat(johnConversationId, "What's my name?");
    assertTrue(answer.response().toLowerCase().contains("john"), answer.response());
    answer = assistant.chat(aegonConversationId, "What's my name?");
    assertTrue(answer.response().toLowerCase().contains("dan"), answer.response());
}

5.3 Anthropic

Add the following dependency:

implementation("io.micronaut.langchain4j:micronaut-langchain4j-anthropic")

Then add the necessary configuration.

Example Configuration
langchain4j.anthropic.api-key=YOUR_KEY

5.4 Azure

Add the following dependency:

implementation("io.micronaut.langchain4j:micronaut-langchain4j-azure")

Then add the necessary configuration.

Example Configuration
langchain4j.azure-open-ai.api-key=YOUR_KEY
langchain4j.azure-open-ai.endpoint=YOUR_ENDPOINT

You will additionally need to define a bean of type TokenCredentials.

One way to do this is to include the Azure SDK module.

5.5 Bedrock

Add the following dependency:

implementation("io.micronaut.langchain4j:micronaut-langchain4j-bedrock")

Then add the necessary configuration.

Example Configuration
langchain4j.bedrock-llama.api-key=YOUR_KEY

You will additionally need to define a bean of type AwsCredentialsProvider.

One way to do this is to include the AWS SDK module.

5.6 HuggingFace

Add the following dependency:

implementation("io.micronaut.langchain4j:micronaut-langchain4j-hugging-face")

Then add the necessary configuration.

Example Configuration
langchain4j.hugging-face.access-token=YOUR_ACCESS_TOKEN

5.7 MistralAi

Add the following dependency:

implementation("io.micronaut.langchain4j:micronaut-langchain4j-mistralai")

Then add the necessary configuration.

Example Configuration
langchain4j.mistral-ai.api-key=YOUR_KEY

5.8 Ollama

Add the following dependency:

implementation("io.micronaut.langchain4j:micronaut-langchain4j-ollama")

Then add the necessary configuration.

Example Configuration
langchain4j.ollama.base-url=YOUR_URL

5.9 Oracle Cloud GenAI

Add the following dependency:

implementation("io.micronaut.langchain4j:micronaut-langchain4j-oci-genai")

Setup a supported OCI authentication method.

Then add the necessary configuration to configure a chat model.

Example Configuration
langchain4j.oci-gen-ai.chat-model.model-name=orca-mini
langchain4j.oci-gen-ai.compartment-id=your-compartment

5.10 OpenAi

Add the following dependency:

implementation("io.micronaut.langchain4j:micronaut-langchain4j-openai")

Then add the necessary configuration.

Example Configuration
langchain4j.open-ai.api-key=YOUR_KEY

5.11 Google AI Gemini

Add the following dependency:

implementation("io.micronaut.langchain4j:micronaut-langchain4j-googleai-gemini")

Then add the necessary configuration.

Example Configuration
langchain4j.google-ai-gemini.api-key=YOUR_API_KEY

5.12 VertexAi

Add the following dependency:

implementation("io.micronaut.langchain4j:micronaut-langchain4j-vertexai")

Then add the necessary configuration.

Example Configuration
langchain4j.vertex-ai.endpoint=YOUR_ENDPOINT
langchain4j.vertex-ai.model-name=YOUR_MODEL
langchain4j.vertex-ai.project=YOUR_PROJECT
langchain4j.vertex-ai.location=YOUR_LOCATION
langchain4j.vertex-ai.publisher=YOUR_PUBLISHER

5.13 VertexAi Gemini

Add the following dependency:

implementation("io.micronaut.langchain4j:micronaut-langchain4j-vertexai-gemini")

Then add the necessary configuration.

Example Configuration
langchain4j.vertex-ai-gemini.model-name=YOUR_MODEL
langchain4j.vertex-ai-gemini.project=YOUR_PROJECT
langchain4j.vertex-ai-gemini.location=YOUR_LOCATION

6 Embedding Stores

6.1 Elastic Search

Add the following dependency:

implementation("io.micronaut.langchain4j:micronaut-langchain4j-store-elasticsearch")
Example Configuration
elasticsearch.httpHosts=http://localhost:9200,http://127.0.0.2:9200
langchain4j.elasticsearch.embedding-stores.default.dimension=384

6.2 MongoDB

Add the following dependency:

implementation("io.micronaut.langchain4j:micronaut-langchain4j-store-mongodb-atlas")
Configuring a MongoDB server
mongodb.servers.default.uri: mongodb://username:password@localhost:27017/databaseName
Example Configuration
langchain4j.mongodb-atlas.embedding-stores.default.database-name=testdb
langchain4j.mongodb-atlas.embedding-stores.default.collection-name=testcol
langchain4j.mongodb-atlas.embedding-stores.default.index-name=testindex

6.3 Neo4j

Add the following dependency:

implementation("io.micronaut.langchain4j:micronaut-langchain4j-store-neo4j")
Example Configuration
neo4j.uri=bolt://localhost
langchain4j.neo4j.embedding-stores.default.dimension=384

6.4 Oracle

Add the following dependency:

implementation("io.micronaut.langchain4j:micronaut-langchain4j-store-oracle")

Then add one of the supported JDBC connection pools, for example Hikari:

runtimeOnly("io.micronaut.sql:micronaut-jdbc-hikari")
Example Configuration
datasources.default.dialect=oracle
langchain4j.oracle.embedding-stores.default.table=test
langchain4j.oracle.embedding-stores.default.table.create-option=create_if_not_exists

6.5 Open Search

Add the following dependency:

implementation("io.micronaut.langchain4j:micronaut-langchain4j-store-opensearch")
Example Configuration
micronaut.opensearch.rest-client.http-hosts=http://localhost:9200,http://127.0.0.2:9200
langchain4j.opensearch.embedding-stores.default.dimension=384

6.6 PGVector

Add the following dependency:

implementation("io.micronaut.langchain4j:micronaut-langchain4j-store-pgvector")

Then add one of the supported JDBC connection pools, for example Hikari:

runtimeOnly("io.micronaut.sql:micronaut-jdbc-hikari")
Example Configuration
datasources.default.dialect=postgres
langchain4j.pgvector.embedding-stores.default.table=mytable
langchain4j.pgvector.embedding-stores.default.dimension=384
test-resources.containers.postgres.image-name=pgvector/pgvector:pg16

6.7 Redis

Add the following dependency:

implementation("io.micronaut.langchain4j:micronaut-langchain4j-store-redis")
Example Configuration
langchain4j.redis.embedding-store.host=localhost
langchain4j.redis.embedding-store.port=6379
langchain4j.redis.embedding-stores.default.dimension=384

6.8 Qdrant

Add the following dependency:

implementation("io.micronaut.langchain4j:micronaut-langchain4j-store-qdrant")

To use Testcontainers & Test Resources add the following dependency:

testResourcesService("io.micronaut.langchain4j:micronaut-langchain4j-qdrant-testresource")
Example Configuration
langchain4j.qdrant.embedding-store.host=localhost
langchain4j.qdrant.embedding-store.port=6334
langchain4j.qdrant.embedding-store.collection-name=mycollection

7 Repository

You can find the source code of this project in this repository:

8 Release History

For this project, you can find a list of releases (with release notes) here: