Braintrust

We are very excited Braintrust was featured in the inaugural Future 50! We are thankful for the recognition and can’t wait to continue supporting amazing AI teams.

We are excited to announce Braintrust is now SOC 2 Type II certified! We have supported enterprise customers from day 1, and achieving SOC 2 compliance is further validation of how seriously our team takes governance, risk, and compliance. https://lnkd.in/gtcYAYQn

🚀 Ready to see how ClickHouse is transforming AI? Braintrust, since 2023, has been helping AI leaders like Zapier and Notion build, test, and improve their products with lightning speed. Thanks to ClickHouse's high performance and real-time data processing, Braintrust provides instant insights and seamless data analysis, enabling faster, data-driven decisions. ⚡ From real-time data replication to efficient query execution, ClickHouse ensures Braintrust can handle vast amounts of data with minimal latency. This integration allows AI companies to iterate quickly and maintain a competitive edge. Discover how ClickHouse powers Braintrust’s innovative AI solutions! https://lnkd.in/ejmDms7C

Braintrust now supports role-based access control (RBAC)! This was not a trivial undertaking -- we have customers that are running in highly regulated industries with thousands of users, dealing with sensitive data. We took a step back and realized we should build RBAC the Right way. Braintrust now has first-class supports for: * Permissions / Custom roles * Users / Custom groups * Scope (org, project, object) Everything can be granularly set at the experiment, project, org level, both in the UI and through the API. This flexible architecture means you can, e.g. easily spin up an API key that has write-only permissions to a project, and use it in a UI. Docs: https://lnkd.in/gzEPDtP8 Although RBAC is powerful/flexible, we also put a lot of work into making it easy to use. E.g. this invite dialog makes it really easy to assign users to a common set of permissions when you invite them. RBAC is now live for all users! Give it a go and let us know what you think :)

If you want to build and ship great AI products, you need evaluations that you can trust. Constructing reliable evals means: 1. Taking your understanding of what good responses look like and codifying it 2. Gathering a set of test examples that is both broad and representative of real-world usage In the blog post below, we cover 3 ways to improve your evals. Check it out!

Honored to be included among giants here. Although we're early, we are fortunate to work with almost every company at the leading edge of AI. Also, we're hiring :)

Another new cookbook - this one covers how to build a Text2SQL app from scratch without any frameworks or data. This is a great template for how to build an app without relying on a bunch of crazy dependencies or having a perfect golden dataset.

New cookbook on how to use the fantastic RAGAS framework in Braintrust! Among other things, the Braintrust implementation: * Available in both TS and Python * Uses function calling (which substantially boosts performance) * Is fully debuggable Check it out: https://lnkd.in/gYVmAqTT

It’s been great working with Braintrust. If you’re building with LLMs, I highly recommend using them for evals.

Over the past year, it's been an absolute joy getting to know Mike Knoop and build Braintrust with our friends at Zapier. We worked together on a blog post that captures their workflow. If you want to build a world-class AI product, this is for you! https://lnkd.in/gYM-Qv8y