Spring AI now supports NVIDIA's Large Language Model API, offering integration with a wide range of models. By leveraging NVIDIA's OpenAI-compatible API, Spring AI allows developers to use NVIDIA's LLMs through the familiar Spring AI API. We'll explore how to configure and use the Spring AI OpenAI chat client to connect with NVIDIA LLM API. The demo application code is available in the nvidia-llm GitHub repository. The SpringAI / NVIDIA integration documentation. Prerequisite Create NVIDIA account with sufficient credits. Select your preferred LLM model from NVIDIA's offerings. Like the meta/llama-3.1-70b-instruct in the screenshot below. From the model's page, obtain the API key for your chosen model. Dependencies To get started, add the Spring AI OpenAI starter to your project. For Maven, add this to your pom.xml: <dependency> <groupId>org.springframework.ai</groupId> <artifactId>spring-ai-openai-spring-boot-starter</artifactId> </dependency> For Gradle, add this to your build.gradle: gradleCopydependencies { implementation 'org.springframework.ai:spring-ai-openai-spring-boot-starter' } Ensure you've added the Spring Milestone and Snapshot repositories and add the Spring AI BOM. Configuring Spring AI To use NVIDIA LLM API with Spring AI, we need to configure the OpenAI client to point to the NVIDIA LLM API endpoint and use NVIDIA-specific models. Add the following environment variables to your project: export SPRING_AI_OPENAI_API_KEY=<NVIDIA_API_KEY> export…Read More