Members-Only
Recent Talks & Demos are for members only
You must be an AI Tinkerers active member to view these talks and demos.
Poyro: LLM Testing for Web Apps
This talk covers practical methods for testing LLMs in Node-based web applications, addressing challenges like hallucinations, API errors, and non-determinism.
I want to talk about testing LLMs in production web applications. There are a lot of ways to integrate your LLMs into apps: REST API requests, SDKs, etc. but not a lot of easy-to-use tooling to make testing straightforward and high-signal. I’m collaborating with my MIT roommate on a unit testing library - Poyro - that makes it dead-simple to test LLMs that work as part of Node-based web applications (e.g. Next.js, Express, etc.). I wanted to share how we’re thinking about testing and how other web app engineers might as well.
Poyro extends Vitest, unit testing LLM outputs using a local LLM.
A reserved `.dev` domain placeholder awaiting technical content deployment.