Seven questions to ask before hiring an AI agent builder

1. What exact task will it own?

Write down the inputs, outputs, and acceptable exceptions. “Automate sales” is not a testable scope. “Prepare a draft account brief from these approved sources” is.

2. What can it access?

List the accounts, records, and tools the system can use. Give it only the permissions required for the task. Define how access is revoked when the engagement ends.

3. Which actions require approval?

Agree on the boundary between preparing a draft and taking an action. Sending messages, moving money, deleting records, and changing access deserve explicit decisions before development starts.

4. How will success be measured?

Use representative tasks, include difficult cases, and record a baseline. Ask for the model and system version, the number of runs, and the review process behind any reported success rate.

5. How does it fail?

A useful evaluation includes missed cases, incorrect actions, timeouts, and recovery behavior. Ask for an example failure trace, with confidential information removed.

6. Who maintains it?

Establish ownership of source code, documentation, account billing, and monitoring. Agree on the process for retesting after a model, API, or business rule changes.

7. Can someone else reproduce the result?

Ask for setup instructions and a repeatable evaluation. If the data cannot be shared, request an explanation of the limitation and a safe demonstration with representative data.

Explore Proof of Work, browse AI experts, or read our editorial methodology.

For hands-on agent operations, visit AI Agent Insights.