All prompt tools
Every tool in the directory. Filter by category or search by name.
AdalFlow
A library for building LLM applications that also auto-tunes them, optimizing prompt text and few-shot demonstrations against a training set. It aims to make prompt tuning as routine as model training.
CrewAI
A Python framework for role-playing agent crews, where each agent is defined by a role, goal and backstory that become its prompt. Tasks are assigned to agents and executed sequentially or hierarchically.
DeepEval
A pytest-style evaluation framework for LLM output with ready-made metrics for hallucination, answer relevancy, faithfulness and task completion. Tests run locally and can be pushed to the Confident AI platform.
DSPy
A Python framework that replaces hand-written prompt strings with declarative modules and signatures, then compiles them against your data. Optimizers search for the wording and few-shot examples that score best on a metric you define.
EvoPrompt
The official implementation of a paper that applies evolutionary algorithms to prompt search, using a language model to perform crossover and mutation on candidate prompts. Populations are scored on a dev set each generation.
garak
A vulnerability scanner for language models that probes a target with hundreds of attack prompts covering jailbreaks, prompt injection, data leakage and toxic output, then reports which probes succeeded.
GEPA
An optimizer that improves prompts and other text artifacts through reflective evolution, reading execution traces and natural-language feedback to propose edits. It integrates with DSPy and other pipelines.
Guidance
A templating language for steering language models token by token, interleaving generation with control flow, regex and grammar constraints. Output structure is enforced during decoding rather than checked afterwards.
Haystack
deepset's orchestration framework for retrieval and agent pipelines built from composable components. Prompt builders are pipeline nodes, so prompt text is versioned with the rest of the pipeline definition.
Inspect AI
An evaluation framework from the UK AI Security Institute with built-in components for prompt engineering, tool use, multi-turn dialogue and model-graded scoring, plus a log viewer for inspecting runs.
Instructor
A library that maps model responses onto Pydantic models so a prompt returns validated typed objects instead of raw text. Failed validations are automatically retried with the error fed back into the prompt.
LangChain
One of the most widely used frameworks for LLM applications, providing prompt templates, chat message abstractions, tool calling and agent loops. It connects to the LangSmith platform for tracing and prompt management.
Letta
A platform for stateful agents, formerly MemGPT, where the agent manages its own context window through editable memory blocks. The prompt is assembled from persistent memory rather than rebuilt each turn.
LlamaIndex
A data framework for connecting documents and databases to language models, with a prompt template layer that can be overridden per query engine. It is commonly used to build retrieval-augmented prompt pipelines.
LLM CLI
Simon Willison's command-line tool and Python library for running prompts against many models. Prompts, system prompts and responses are logged to a local SQLite database, and saved templates can be reused.
Marvin
A Python toolkit from Prefect for building AI features as ordinary functions, where the prompt is generated from type hints and docstrings. It covers classification, extraction and generation tasks without manual prompt strings.
OPRO
Reference code from the Google DeepMind paper that uses a language model as the optimizer for its own instructions. Previously scored prompts are fed back as context so the model proposes better ones.
Outlines
A structured generation library that forces model output to match a JSON schema, regex or context-free grammar. It works with local and hosted models and is used to make prompt output parseable by default.
PromptBench
A Microsoft research framework for evaluating language models with a focus on prompt robustness, including adversarial prompt attacks and prompt engineering method comparisons. The repository is archived and kept for reference.
Promptfoo
A command-line and library test runner for prompts, models and RAG pipelines. Test cases are declared in YAML, run side by side across providers, and the project also performs red-team scans for prompt injection and jailbreaks.
PromptHero
A search engine and community for AI image and video prompts, where generated images are published with the prompt that produced them. It also sells a subscription for on-site generation and premium prompts.
PromptLayer
A prompt management platform where non-engineers can edit prompts in a visual registry while engineers pull versions at runtime. It also logs requests and runs evaluation pipelines against prompt versions.
prompts.chat (Awesome ChatGPT Prompts)
The community prompt collection formerly known as Awesome ChatGPT Prompts, where each entry casts the model in a role such as translator, interviewer or terminal. It is one of the most starred prompt repositories on GitHub and has a companion website.
PromptWizard
A Microsoft framework that iteratively refines both prompt instructions and in-context examples using model-generated critique. It targets task-specific prompt optimization with a modest number of API calls.
SAMMO
A Microsoft research library for structure-aware prompt optimization. Prompts are represented as programs whose components can be mutated and searched against multiple objectives such as accuracy and cost.
Semantic Kernel
Microsoft's SDK for embedding model calls into C#, Python and Java applications. Prompts live in versioned template files with typed parameters and are invoked as callable functions alongside native code.
SGLang
A serving framework and domain-specific language for programs that make many structured model calls. Its runtime reuses shared prefixes across calls, which speeds up branching and multi-step prompt programs.
TextGrad
A research library that treats natural-language critique as a gradient, backpropagating textual feedback through a pipeline to improve prompts and other text variables. It mirrors the autograd style familiar from PyTorch.
TruLens
An instrumentation and evaluation library that wraps an application, records each call and scores it with feedback functions such as groundedness and relevance. Results are browsable in a local dashboard.
Verbalized Sampling
A reference implementation of a training-free prompting strategy that asks a model for a distribution of responses with probabilities, which the authors report reduces mode collapse in creative tasks.
ell
A language model programming library that treats prompts as versioned Python functions. A local store records every prompt version and invocation so changes can be diffed and compared in a studio UI.
Magentic
A small Python library that turns a decorated function signature and docstring into a prompt, returning the declared return type. It supports streaming, function calling and chat-based prompt chains.
LMQL
A query language from ETH Zurich that blends Python control flow with prompt text and declarative constraints. Constraints are compiled into decoding rules so the model is guided while it generates.
Mirascope
A Python library that keeps prompts as decorated functions with normal editor tooling rather than opaque chains. It supports response models, prompt templates and a provider-agnostic call interface.
Pydantic AI
An agent framework from the Pydantic team that applies model validation to prompt inputs and outputs. It supports typed dependencies, streaming structured responses and instrumentation via Logfire.
Ragas
A library of reference-free metrics for retrieval-augmented pipelines, scoring faithfulness, context precision and answer relevance. It also generates synthetic test sets from your own documents.
Questions
Frequently asked questions
Which model do these tools support?
Most work with any major model API. The tool's site lists supported providers.