Visibility of agent schemas

One of my main questions is quite simple, but very difficult:  How do we know that an agent is doing (or programmed to do) what we think it is supposed to be doing?  In other words, how to give greater visibility to the "agent schema" that is implemented by the agent.

 

This question of course begs the question of having a formal "agent schema" to begin with, as contrasted with imagining a schema and then translating that vision into hard code that supposedly implements that imagined schema.

 

My view is that the greatest danger of software agents is that they and their users get trapped
in the "code swamp", where nobody can make sense of what the agent might do in situations that are not precisely as envisioned in "the lab".  Throw in evolutionary programming, and it's a true nightmare (ala Michael Crichton's "Prey").

 

-- Jack Krupansky