Skip to content

Latest commit

 

History

History
335 lines (249 loc) · 11 KB

File metadata and controls

335 lines (249 loc) · 11 KB

Tools

Tools are functions that an LLM can use to perform specific actions, like reading files or running commands.

By default, LLMs cannot access a user's environment, such as their filesystem or terminal. To solve this, Claude Code provides several tools (like Read, Write, and Bash) that enable the model to read and modify the user's codebase.

For this stage, you only need to advertise the Read tool (you'll implement it in later stages).

Advertising Tools

Advertising tools lets the model know which tools are available and what arguments they accept.

To advertise tools, edit the existing request in the starter code to add a tools array:

{
  "model": "...",
  "messages": [...],
  "tools": [<tool1 spec>, <tool2 spec>, ...]
}

For this stage, the tools array should include the Read tool's specification:

{
  "type": "function",
  "function": {
    "name": "Read",
    "description": "Read and return the contents of a file",
    "parameters": {
      "type": "object",
      "properties": {
        "file_path": {
          "type": "string",
          "description": "The path to the file to read"
        }
      },
      "required": ["file_path"]
    }
  }
}

The structure consists of the following fields:

  • type: The type of tool (always "function" for tools)
  • function: Contains the function definition
  • name: The name of the function (e.g., "Read")
  • description: Explains the function's purpose and helps the LLM determine when to use it.
  • parameters: A JSON schema describing the function's parameters
  • properties: Defines each parameter (in this case, just file_path)
  • required: Lists which parameters are mandatory

Notes

You can choose any reasonable name for the Read tool. The tester will only perform end-to-end tests by checking the LLM's response. For example, any of the following names is valid:

  • Read
  • read
  • read_file
  • ReadFile, etc.

For this stage, you only need to advertise the Read tool's availability in the request. We'll handle tool calls in later stages.

See the OpenRouter API Specification for Tools for the full API documentation on how to include tools in your request.

Tool Calls

When the LLM decides to use a tool, the response message will contain a tool_calls array.

Here's what the response looks like:

{
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": null,
        "tool_calls": [
          {
            "id": "call_abc123",
            "type": "function",
            "function": {
              "name": "Read",
              "arguments": "{\"file_path\": \"/path/to/file.txt\"}"
            }
          }
        ]
      },
      "finish_reason": "tool_calls"
    }
  ]
}

The key fields you need to handle are:

  • choices: The list of generated responses. For this challenge, there will always be exactly one choice.
  • tool_calls: The array of tool calls to make
  • id: A unique identifier for the tool call
  • type: The type of tool call (always "function" for tools)
  • function: Contains the function details:
    • name: The name of the function to call (e.g., "Read")
    • arguments: A JSON string containing the function parameters

For this stage, you'll execute a single Read tool call.

Executing the Read Tool

When you detect a tool_calls array in the response:

  1. Extract the tool call: Get the message from the first choice, then extract the first tool call from the tool_calls array.
  2. Parse the function name: Read the function name to determine which tool to execute.
  3. Parse the arguments: Parse the function arguments as a JSON string to get the parameters.
  4. Execute the tool: Your program must perform the action requested. For the Read tool, use your language's file system library to read the file at the requested file_path.
  5. Output the result: Print the tool execution result. For the Read tool, print the raw file contents to stdout.

If there is no tool_calls array in the response, print the message content like in previous stages.

The Agent Loop

So far, your program handles a single interaction: send a prompt to the LLM, get a response, execute one tool if requested, and exit.

This works for simple tasks, but falls short when the task requires multiple steps (e.g., "read a file and fix any bugs").

For this stage, you'll implement an agent loop that repeatedly sends messages to the model and handles tool calls as needed, until the final result is received.

Here's how to implement the agent loop:

  1. Initialize the conversation: You already have an initial conversation history: the messages array with the user's prompt. Now you need to store this array so it can persist across iterations, since the loop will continuously append new messages to it:

    [
      { "role": "user", "content": "Summarize the README for me." }
    ]
  2. Enter the loop: Start the loop with the same API request you already have (sending your messages and tool specifications to the model). The difference is that this request now sits in a loop, allowing it to run multiple times.

  3. Record the assistant's response: Whatever message the model returns, add it to your messages array. If the model wants to use a tool, the response will contain a tool_calls array:

    {
      "role": "assistant",
      "content": null,
      "tool_calls": [
        {
          "id": "call_abc123",
          "type": "function",
          "function": {
            "name": "Read",
            "arguments": "{\"file_path\": \"README.md\"}"
          }
        }
      ]
    }
  4. Execute tool calls: Check the model's response to see if it's requesting to use any tools. If tool calls are present:

    a. Execute each requested tool (but do not print their result to stdout).

    b. Add each tool call result to your messages array. Every tool result must:

    • Have the role field set to "tool"
    • Reference the corresponding tool_call_id
    • Include the tool call result as its content
    {
      "role": "tool",
      "tool_call_id": "call_abc123",
      "content": "# My Project\n\nChemical expiry period: 6 months"
    }
  5. Repeat until complete: Continue the loop until the model responds without requesting any tools (when tool_calls is missing or empty). At this point, print the final message content to stdout and exit.

Tests

The tester will create a Python project with:

  • README.md
  • Two Python files in app/ with randomized names

The tester will then execute your program like this:

$ ./your_program.sh -p “Use README.md to determine the chemical expiry period in months. Number only.”
<expiry period in months>

The tester will verify that:

  • The output is the correct expiry period
  • Your program exits with exit code 0

Notes

  • Do not print raw file contents when executing the Read tool. This differs from the behavior in the “Execute the read tool” stage.
  • You can print intermediate debugging information to stderr. Print to stdout only when the final result is ready to be printed.
  • You can also use finish_reason: “stop” from the first response choice as a signal to stop the loop.

The Write Tool

The Write tool enables the LLM to write content to files. Like with the Read tool, you need to advertise the Write tool in your request and execute it when the model requests it.

Here's the tool specification:

{
  "type": "function",
  "function": {
    "name": "Write",
    "description": "Write content to a file",
    "parameters": {
      "type": "object",
      "required": ["file_path", "content"],
      "properties": {
        "file_path": {
          "type": "string",
          "description": "The path of the file to write to"
        },
        "content": {
          "type": "string",
          "description": "The content to write to the file"
        }
      }
    }
  }
}

Executing the Write Tool

When the model requests a Write tool call:

  1. Parse the arguments to extract the file_path and content
  2. Write the content to the file at the specified path:
    • If the file doesn't exist, create it
    • If the file exists, overwrite it with the new content
  3. Append the result to messages as a tool message (just like with Read)

Tests

The tester will create the following files:

  • README.md with instructions
  • app/ directory for the project

The tester will then execute your program like this:

$ ./your_program.sh -p “Read README.md and create the required file. File should have 1 line. Reply with 'Created the file'”
Created the file

The tester will verify that:

  • The required file is created with the correct contents
  • Your program exits with exit code 0

Notes

You can choose any reasonable name for the Write tool (e.g., Write, write, write_file, WriteFile).

The Bash Tool

The Bash tool enables the LLM to run shell commands. It gives the model direct access to the command line to perform actions like deleting files, creating directories, or running scripts.

You'll need to advertise the Bash tool in your request and execute it when the model requests it.

Here is an example of the Bash tool's specification:

{
  "type": "function",
  "function": {
    "name": "Bash",
    "description": "Execute a shell command",
    "parameters": {
      "type": "object",
      "required": ["command"],
      "properties": {
        "command": {
          "type": "string",
          "description": "The command to execute"
        }
      }
    }
  }
}

Executing the Bash Tool

When the model requests a Bash tool call:

  1. Parse the arguments to extract the command
  2. Execute the command using your language's shell execution capabilities (e.g., subprocess.run() in Python, child_process.exec() in Node.js)
  3. Capture both stdout and stderr from the command
  4. Return the command output (or an error message if it failed) to the model as a tool message

For example, if the command is rm README_old.md, execute it and return the result (which will be empty if successful).

Tests

The tester will create three files:

  • app/main.js — Main project file
  • README.md — The current readme file
  • README_old.md — An old readme file

The tester will then execute your program like this:

$ ./your_program.sh -p "Delete the old readme file. Always respond with `Deleted README_old.md`"
Deleted README_old.md

The tester will verify that:

  • README_old.md has been deleted (no longer exists)
  • app/main.js remains intact with its original contents
  • README.md remains intact with its original contents
  • Your program exits with code 0

Notes

  • You can choose any reasonable name for the Bash tool (e.g., Bash, bash, RunBashCommand, run_bash_command).
  • The result of the Bash tool call should be sent back to the model as part of the agent loop.
  • Make sure to execute the command in the same directory as your program, not in a temporary or different working directory.