PromptTools

Reading Time: 1 minute

Last updated on August 7, 2024

Sander Schulhoff

PromptTools is an open-source library for experimenting with, testing, and evaluating prompts, LLMs, and vector databases.

Features

PromptTools offers a playground for comparing prompts and models in three modes:

Instruction
Prompt template
Model comparison

In addition to the playground, PromptTools offers and SDK for writing tests and evaluation functions to experiment with and evaluate prompts at scale.

You can check out a tutorial video here.

Sander Schulhoff

Sander Schulhoff is the Founder of Learn Prompting and an ML Researcher at the University of Maryland. He created the first open-source Prompt Engineering guide, reaching 3M+ people and teaching them to use tools like ChatGPT. Sander also led a team behind Prompt Report, the most comprehensive study of prompting ever done, co-authored with researchers from the University of Maryland, OpenAI, Microsoft, Google, Princeton, Stanford, and other leading institutions. This 76-page survey analyzed 1,500+ academic papers and covered 200+ prompting techniques.