TruLens ❤️ Snowflake OSS
When Snowflake acquired the TruEra AI Observability platform, we committed to keeping TruLens open source. We're not only keeping that promise; we're emphasizing it. Our goal remains to support LLM app developers in creating trustworthy generative AI applications.
In the weeks since the acquisition, we have already added ecosystem-friendly enhancements including:
- Parallelization of groundedness feedback evaluation
- Support for LangChain > 0.2x retrievers
- Context filtering guardrails
- Query optimizations for the TruLens dashboard resulting in 4 to 32x benchmarked speedup
We plan to continue making enhancements and improvements that benefit the community at large, whether on Snowflake or not.
Snowflake is increasingly focusing on open source projects that provide value to the AI ecosystem. In addition to sharing this continued commitment to growing TruLens, we also want to share some of the ways that OSS Snowflake projects can enable TruLens users to build better LLM apps in open source. Snowflake maintains a stable of open source projects that complement TruLens, notably the Streamlit app framework, the Modin library and Arctic foundation models.
TruLens ❤️ Streamlit
Streamlit makes it easier than ever for gen AI builders to deploy an easy-to-use user interface for all things AI. Streamlit has been the foundation of the TruLens UI since its inception. By working alongside the Streamlit team, TruLens will more quickly benefit from new Streamlit features. In future releases, we also plan to make it even easier to add TruLens components to the applications you are already building in Streamlit.
TruLens ❤️ Arctic LLMs
Snowflake recently released the Arctic family of foundation models in open source. We have made them available for powering TruLens feedback functions with just two lines of code.
from trulens_eval.feedback.provider import Cortex
provider = Cortex(model_engine="snowflake-arctic")You can see the full quickstart available on Github: TruLens + Arctic Quickstart
These models can immediately be plugged into TruLens feedback functions while offering a favorable trade-off for performance and latency compared to other OSS models.
Below you can see our first benchmark result of Arctic Instruct as the base model for our groundedness benchmark using the SummEval data set. Open Source Arctic Instruct offers comparable accuracy and latency to other SOTA models.
Groundedness Task Performance by Base Model
You can read more about TruLens benchmarks here.
Now, as we embark on this endeavor, we're excited to continue developing TruLens-Eval in open source, in a way that benefits the whole ecosystem.
