ToolBench (ToolLLM)

ToolBench is the instruction-tuning dataset behind ToolLLM/ToolLLaMA, built to teach open-source LLMs to use thousands of real-world APIs. It collects 16,464 REST APIs across 3,451 tools from RapidAPI, curates single-tool (G1), intra-category multi-tool (G2), and intra-collection multi-tool (G3) instructions, and annotates each with a solution path using a novel depth-first search based decision tree (DFSDT) that records the model's reasoning, tool calls, and tool execution results. All data is automatically generated with OpenAI's ChatGPT (gpt-3.5-turbo-16k, enhanced with function calling) and then filtered by the authors. Each solution-path record stores the OpenAI-style function schemas of the available tools, the full message trace of assistant function_call turns and function responses, a finish_type (give_answer vs give_up), and a final answer. Correctness during model evaluation is established with ToolEval, an automatic ChatGPT-based evaluator that reports Pass Rate and Win Rate. The corpus contains 469,585 real API calls and averages 4.0 reasoning traces per instance.