Skip to main content
Question

AI Evaluations stuck "Active" when Agent triggers Human-in-the-Loop

  • September 13, 2026
  • 0 replies
  • 9 views

Hi all,

I'm working on my AI Automation Engineer certification use case submission and have run into an issue evaluating my Listing Intelligence Agent. My evaluations get stuck in "Active" status and never complete, 5 are currently stuck (ebay v5, ebay v4, Ebay v3, Ebay v2, TRT Ebay v1).

I’ve ruled out: Not a resource issue, other users' evaluations complete fine at the same time. Not a data issue, rebuilt dataset clean, same inputs that succeed via direct "Run." Tried both Evaluate automatically and manually, both hang identically. All underlying tool/bot calls show Completed in Activity history, even during the stuck runs.

Suspected cause: I noticed the evaluation stopped progressing right at the section of my Agent's flow that should trigger Human-in-the-Loop (a Human Review Form). All of my agents/bots up to that point show 100% Completed, but nothing happens after that. In a separate evaluation run, I also saw it stop during a bot step that should have triggered Human-in-the-Loop for a CAPTCHA, that evaluation just stopped but still shows as Active rather than completing or surfacing the CAPTCHA task. When I run this same Agent directly via "Run" (not "Evaluate"), it correctly opens the Human Review request and I can respond to it normally, but during "Evaluate," no interactive task ever surfaces for me to respond to, and the evaluation just sits at "Active" forever. 

Question: Has anyone seen evaluations hang specifically on Human-in-the-Loop? Is there a way to respond to a pending Human Input step inside an evaluation, or a setting I'm missing?

Thanks!