Welcome to the October Product Club Recap! This month our focus was on the A360v.37 Vision Fallback enhancement to Generative Recorder and seamless agent interoperability powered by the Process Reasoning Engine (PRE), led by Rajendra Vijay, Director of Product Management at Automation Anywhere.

HOSTS

Lu Hunnicutt, Community Manager
Rajendra Vijay, Director of Product Management
Prakash, Director of Engineering

TOPIC

In this session, we delved into the new Vision Fallback feature and how agent interoperability can enhance automation processes:

• Rajendra introduced the Vision Fallback enhancement, showcasing its impact on automations.
• We explored how agent interoperability powered by PRE eliminates costly integrations and enables seamless collaboration.
• The session highlighted how AI models like OCR, Object Detection, and Vision-Language Models (VLMs) enable self-healing automation.

VISION FALLBACK: THE ENHANCEMENT

Vision Fallback is a powerful enhancement to Generative Recorder, achieving a 60% improvement in fallback efficacy. It uses an ensemble of AI models to ensure high fallback accuracy on complex UIs, enabling real-time optimization and maintaining automation as applications evolve.

Key Features:

• AI Ensemble for Precision: Achieves high fallback accuracy using AI models.
• Real-Time Optimization: Validates predictions and optimizes fallback accuracy.
• Enterprise-Grade Security: Provides unified access with governance and policy enforcement.

AGENT INTEROPERABILITY: SEAMLESS COLLABORATION

Agent interoperability, powered by PRE, allows agents, systems, and tools to communicate fluidly, reducing integration complexity and cost. It offers intelligent intent-to-automation mapping, ensuring context-aware execution.

Key Benefits:

• Seamless Collaboration: Enables fluid communication without custom integrations.
• Intelligent Mapping: Maps agent requests to the right process at the right time.
• Future-Proof Automation: Overcomes API-based integration limits, ensuring adaptability.

DEMO HIGHLIGHTS

• The Vision Fallback Demo showcased how the Generative Recorder adapts to application changes, preventing automation failures.
• The Agent Interoperability Demo demonstrated how third-party AI agents can perform actions within the Automation Anywhere ecosystem, enhancing agent capabilities.

PATHFINDER ANNOUNCEMENTS

Pathfinder Summit is taking place November 18th and 19th! Join us for a 48-hour nonstop virtual conference with dynamic tracks on strategic leadership, operational governance, and more. Register today to gain insights from global experts and connect with peers.

Preview Participation: Interested in participating in the preview of our new features? Nominate yourself and be part of the innovation journey.
 

SESSION Q&A

Our session concluded with a Q&A segment, addressing audience queries about Vision Fallback and agent interoperability.

Question: Which version of CR has this been published in?
Answer: It's available in all of the cloud control rooms. It is available with the Automator AI license.

Question: What is in case there is no unique property of any object, how does the recorder behave?
Answer: If we see that the nature of change is manageable, based on the latest state of the application, if generative recorder can reliably identify the same UI element, only then will it perform the resiliency. If not, it would rather give an error message.

Question: Sometimes we don't get direct element ID but a DOM X path as div1/div2/group1 and it fails for any change in form in website. Will it help for such DOM X path as well?
Answer: If the change is manageable, the generative recorder, which combines deterministic insights with visual insights, can perform resiliency. If not, it gives an error message to avoid incorrect actions.

Question: Is this healing will work with non-object based elements example legacy desktop application?
Answer: As of now, it's only supported for web applications.

Question: Unexpected pop-up in web apps, can be handled?
Answer: Yes, unexpected popups can be handled using features like popup handler and other new features for dealing with popups.

Question: Is there any separate license for "Generative Recorder"?
Answer: Generative Recorder is part of Automator AI, so you will need the Automator AI license.

Question: How are you calling AA task from Microsoft Co-Pilot Studio?
Answer: On the Microsoft Co-Pilot side, an MCP client is hosted. You configure the control room URL followed by /MCP, which is our MCP server, to run automations.

Question: How will AA/Bot manage payment part if it books ticket end to end and share ticket directly?
Answer: For sensitive transactions, it's recommended to put humans in the loop or have approval steps. You can configure workflows to manage these securely.

Question: Does the new mapping REPLACE the older one? So the next execution will already be with the new mapping?
Answer: The generative recorder prevents failures but does not automatically update automations. Users are notified to make updates manually following their processes.

Question: How do we handle this message for unattended automations?
Answer: In unattended deployments, you will not see popups. Instead, you will receive notifications through system alerts and audit logs.

Question: How big is the difference between cloud vs local sanitization? I imagine it is milliseconds vs 1-5 seconds? Anything beyond that?
Answer: Based on initial testing, the difference could be 30 seconds plus, as cloud sanitization uses powerful infrastructure for faster performance.

Question: Is it mostly for Web applications? Or does the fallback also work for Citrix / virtualized apps?
Answer: As of now, it's only for web applications where the technology type is HTML.

Question: The demo is done using a bot creator license. If it runs through an unattended bot runner, will it automatically use the chosen fallback?
Answer: Yes, it will use the chosen fallback automatically in unattended deployments.

Question: I understand that we will see the popup in the Dev or Attended run mode and NOT in the Prod or Unattended run mode. Will we get any notification from the AA360 platform when the Bot will encounter this scenario in production?
Answer: Yes, you will get notifications through system alerts and audit logs in production environments.

Question: Is this tool intended for developers doing regression testing or is it more to notify a business user using a bot like a co-pilot?
Answer: It is more about preventing automation failures and helping both developers and citizen developers manage dynamic properties and recover from failures.

Question: Can this Bot specific Gen AI fallback settings be exported to different envs like PROD? If Yes do we have option to turn ON/OFF in PROD?
Answer: Yes, you can export settings to different environments and have the option to turn them on or off in PROD with role-based access control.

Question: Would there be an option to upgrade (bulk) the existing capture with Gen Recorder?
Answer: No, generative recorder supports only net new actions. Old actions do not have the extra validation layer required.

Question: How do we implement this updation of element properties without Developer involvement in PROD?
Answer: Automatic updates to properties are not done to ensure processes and approvals are followed. Notifications are provided for manual updates.

Question: The failure popup will impact the unattended Bot runs. Is there an option to disable when moving to Prod?
Answer: In production, the popup will not be shown at all, so no option to disable is needed.

Question: So it will alert the user and then stop the automation?
Answer: It will not stop the automation; it will continue and help the automation recover while notifying the user.

Question: Is there documentation on how to interact with 3rd party agents? If so, can you share?
Answer: Documentation is available to preview customers. Once you nominate and confirm participation, you can access it.

Question: When co-pilot is calling the task, where will these tasks get executed?
Answer: Tasks will be executed in the Automation Anywhere ecosystem using the MCP server configuration.

Question: For agent-to-agent connectivity, would it require embedding AA Co-Pilot within a third-party application to enable agent-to-agent communication?
Answer: Agent-to-agent communication is facilitated through protocols like MCP and A2A.
 

We look forward to seeing you at the next Product Club!