How to Automate Dynamic Applications with Generative Recorder in Automation Success Platform

In the Agile world, we often see frequent changes in web applications. Building resilient automations and adapting to ever changing business applications/environments can be a challenging task. Dynamic properties and UI element updates often lead to automation failures with traditional recorders. In response to this, Automation Anywhere introduces the Generative Recorder, a cutting-edge feature powered by generative AI designed to enhance the reliability of web automation.

Introducing the Generative Recorder:

Automation Anywhere's Generative Recorder emerges as a solution to the challenges posed by dynamic web pages. It leverages Generative AI and Native fallbacks, offering a reliable automation process even in the face of unpredictable changes.

Flow :

When Capture action/command Fails, based on the Generative Recorder selection i.e "Native Fallback", "Generative AI Based Fallback" it will prevent the failure with above selection.

Generative Recorder offers two fallbacks i.e Generative AI-based & Native fallbacks, lets deep dive into them

Native Fallback

Consider web applications that frequently change their UI element properties, the native capture command may fail due to the varying XPath values with each page refresh. In this scenario, you can enable Native Fallback.
If this option is enabled during the capture of the UI element, behind the scenes it captures additional information about the target UI Element i.e during the design time it captures the multiple object properties/Xpath of the business application. So, if the automation does not work, it will leverage the additional information captured and use it for fallback for successful bot run.

To enable this fallback, follow below steps

• Go to Bot editor screen
• Click on Kebab menu
• Navigate to Advanced settings
• Go to Package settings
• Select Native fallback option

Note :
This option is only visible when you have at least one capture command in your bot editor. Currently, it's a bot-level setting.
 

Generative AI-based fallback

Consider web applications where target UI elements are not detected due to page load or entire properties have changed in the live application compared to design time. In such cases, you can leverage Generative AI-based fallback.

If this option is enabled, behind the scenes it makes an API call to the Generative AI model (Microsoft Azure Open AI) with the live application details, like source code (excluding the business sensitive information) to get reliable DOMXPath property for successful automation.

To enable this fallback, follow these steps:

• Go to the Bot editor screen.
• Click on the Kebab menu.
• Navigate to Advanced settings > Package settings.
• Select the Generative AI-based fallback option.
• Configure the Fallback timeout according to your requirements.

With this option, you can specify the number of seconds to wait until we receive a response from Generative AI. This is particularly useful for scenarios where if Generative AI cannot find the target element within a specified time frame, the automation should fail rather than waiting for a longer period of time.

Note : This option is only visible when you have at least one capture command in your bot editor. Currently, it's a bot-level setting.

License 🪪: Users should purchase the "Automator AI for Cloud" SKU to utilize the Generative Recorder.

Building Automation :

We'll explore this using the Resiliency Challenge site. Although we can capture all the elements of this website using the traditional Capture command of the Recorder package, it's important to note that on click of Add button some of the fields position changes i.e UI element properties are getting changed every time on click of Add Button. These elements have dynamic attributes, potentially causing Recorder actions to fail.

Using the Chrome inspect element, we will explore the Company Name UI element's properties and behavior upon clicking the Add button. In Image-1, the element has an id value of 'ZGqxmy' and contains multiple divs and spans. In Image-2, the element has an id value of 'companyName' [Static value] and is located in the first div.

Image-1

Image-2

Let's try to capture the “Company Name” UI element from the Resiliency Challenge page using the Capture command.

1. Create a task bot and assign a name to it.
2. Use Recorder: Capture command,
   • Refresh the Browser window.
   • From the dropdown menu, select the Resiliency Challenge Page.
   • Click Capture Object to capture the Company Name.
   • Select the action to take on the Object as Set Text.
   • Provide the Company Name value as "Pathfinder Roofing and Siding".
3. Before running the bot, let's refine the object properties by unchecking all properties except DOMXPath.

   

    

4. The initial run will be successful, but when clicking the Add Button, the position and properties of the Company Name UI element changes, leading to bot failure.

Bot Error:

5. Imagine a scenario with your SAAS application where everything was coded, tested, and pushed to production smoothly. In such situations, it's incredibly costly and time-consuming for the development team to analyze the changes causing the issue, make necessary fixes, test them, and push them to production.
   • Solution with Generative Recorder Fallback Feature:
     • Select appropriate fallback option based on business need.
   • Benefits:
     • Avoids failures and reduces downtime.
     • Saves time and cost associated with analysis and fixes.
   • Utilization of Audit Logs:
     • Developers can use audit logs within the Control Room for troubleshooting.
   • Auto-Update Feature During Debugging:
     • Seamless updating of DOMXPATH for new changes during debugging.

By utilising these features and strategies, the development team can mitigate failures efficiently and maintain smooth operations of the SAAS application.

6. Let’s enable Native Fallback:

1. Follow the steps outlined above to enable this fallback, and then run the bot in attended mode to observe the issue. In this run, the bot will not fail instead, it will utilize the fallback to obtain the updated DOMXPath and complete the bot run successfully. A popup will display the updated DOMXPath for review, allowing you to make the necessary decisions.

2. In Step a, we understood there is change in Ui element properties and we have updated        DOMXPath for our review. As a developer, I can debug the bot and utilize the Auto-update feature to directly update the DOMXPath in the code. If we have more steps where we see this issue then we no need to update each and every line manually. It's taken care of by this feature which saves again a lot of time for the developer when resolving such issues.

100. Each time the automation utilizes a fallback, we can track it using the Notifications icon within the bot editor. By checking the Audit logs, we can gain insights into the Line Number, Object Details and the Fallback Status used for a successful bot run.

7. Let’s enable Generative AI Fallback for same problem statement mentioned above,

1. Follow the steps outlined above to enable this fallback, and then run the bot in attended mode to observe the issue. In this run, the bot will not fail instead, it will utilize the fallback to obtain the updated DOMXPath and complete the bot run successfully. A popup will display the updated DOMXPath for review, allowing you to make the necessary decisions.

2. Shown below in the Gen AI fallback, it utilizes live application details to determine the updated version of UI target element properties for a successful run.

100. In Step a, we understood there is change in UI element properties and we have updated DOMXPath for our review. As a developer, I can debug the bot and utilize the Auto-update feature to directly update the DOMXPath in the code. If we have more steps where we see this issue then we no need to update each and every line manually. It's taken care of by this feature which saves again a lot of time for the developer when resolving such issues.

500. Each time the automation utilizes a fallback, we can track it using the Notifications icon within the bot editor. By checking the Audit logs, we can gain insights into the Line Number, Object Details and the Fallback Status used for a successful bot run.

Conclusion