Walk through and share the AI adoption journey at a leading software development organization.

* AI accelerates scenario discovery and drafting; it does not replace review of architecture, security, data fixtures or assertion rigor.

* The AI assisted DevOps work turned into a repeatable deployment.

* AI in testing is not writing more tests; it's questioning whether the right tests exist at all.

Check back soon!

*** Rick Cavallaro, President of RJCadvisors & publisher of QAMeditations.com presented October's “Confirming What You Meant — And What You Never Meant: The Oracle Problem in Agentic Testing.”

We had a good online attendance, including several folks who last attended when we were in person at Sun/Oracle. George Nealon, Matt Young, and Xander Lien from Functionize backed up Rick with info about their test Studio tool that Rick demoed.

Extra thanks to Rick for helping me recap and delve further into his extensive presentation. Please see the recording under the Calendar at www.SQGNE.org.

A test oracle is how you tell whether a test result is correct. At its simplest, this means actual results match expected results. Attention lately has focused mainly on the challenge of defining AI’s expected results, since repeating an AI prompt can yield results that are not exactly the same as for prior prompts but still are acceptable. That’s not what Rick addressed.

Rather, the oracle problem, which too seldom is recognized let alone appreciated, is that correct (actual=expected) test results alone do not mean the tested software actually produced intended/needed value. Stated somewhat differently, tests typically confirm that the software works as designed but don’t address whether the design is appropriate to satisfy business needs.

Simply using AI to define tests generally doesn’t address, and often may exacerbate and mask, the oracle problem. Moreover, another common AI focus is self-healing, which detects and adjusts for when objects on the screen change locations; but successfully restoring execution does not necessarily establish that the original business intent has been preserved.

Rick agreed that a starting point is identifying what I call “REAL Business Requirements,” business deliverable _whats_ that provide value when satisfied by a product, system, software, or procedure way _how_ to satisfy the _whats_. But there’s more: We must still establish that the testing system correctly interprets those requirements, translates them into appropriate test conditions and expected results, evaluates observed behavior against those expectations, and preserves their meaning as the automation evolves.

Rick used Functionize Studio in his presentation, not as a solution to the oracle problem, but to illustrate how far agentic testing capabilities have advanced generating and executing tests using natural-language instructions and expected results; and why, as these tools gain autonomy, experienced QA practitioners, working alongside people who understand the business and its requirements, become more important rather than less.

Please see Calendar for a recording of October's presentation.