I’ve written that enterprise AI needs people who can turn business understanding into something a system can use. The same applies to vendors trying to earn an agent’s recommendation. Someone has to select representative customer tasks, identify meaningful failures, and decide which limitations must be made explicit. AI can help generate and run the tests; it doesn’t make the choice of tests commercially or technically neutral.
One useful discipline would be to include cases where the product shouldn’t win. Most vendors are loath to tell the world not to use their product, but when done well, it’s a great way to earn trust with developers (and their employers). For example, in the early years of MongoDB, cofounder Dwight Merriman insisted on having a page that identified where the database was excellent, but also where it wasn’t. Those use cases have changed as the product has evolved, but it’s good practice for any vendor or open source project. For example, does the agent recognize an unsupported deployment requirement, or warn about a cost that matters at the customer’s scale? A vendor that measures only how often its name appears may end up burning trust if effective answer engine optimization (AEO) directs its tech to be built into applications where it’s likely to fail.
The developer also has work to do here. For example, stating the constraints before asking for a recommendation and asking what evidence would favor a different choice. Where the decision matters, examine alternatives before the agent has built an application around one. These practices don’t guarantee independence, but they give the developer a better opportunity to exercise it.
influencing-the-agents-infoworld