Google Vertex AI support for LLM Playground and Evaluations incl. Gemini models
Hassieb PakzadLangfuse now supports Google Vertex AI incl. Gemini models for LLM Playground and Evaluations.
We’re excited that Langfuse now supports access to Google Vertex AI and the Gemini model family for both the LLM Playground and Evaluations. This integration allows users to leverage Google Vertex AI platform’s powerful language models directly within the Langfuse ecosystem. Whether you’re prototyping in the Playground or conducting LLM-as-a-judge evaluations, access to Gemini and other models on the Google Vertex AI platform enhances your toolkit for building and optimizing AI-powered features.
Key Features
-
LLM Playground Integration: You can now use the Gemini model familiy and other Google Vertex AI models in the Langfuse LLM Playground. This enables quick experimentation and testing with various Vertex AI platform models.
-
Evaluations Support: Google Vertex AI models can be utilized in Langfuse Evaluations, allowing for comprehensive assessment and comparison of model performance.
-
Easy Setup: Adding your GCP service account JSON key is straightforward in the project settings, making it simple to get started with these new capabilities.
How to Get Started
To begin accessing Google Vertex AI models in Langfuse (assuming you have already activated the Vertex API in your GCP project settings):
- Create a new service account and assign the
Vertex AI Userrole to it in the GCP IAM settings - Create a JSON key for the newly created service account
- Navigate to the LLM API Keys section in your Langfuse settings.
- Add a new LLM API Key for the adapter
vertex-ai. Paste your service account’s JSON key into the secret key field. API keys are stored encrypted on our servers. - You’re all set! You can now select Google Vertex AI models in the LLM Playground and Evaluations.