Hi @meghagowda ,

Please find the below,

Bot detection is a challenge for web developers who want to automate tasks or scrape data from websites. There are various techniques that websites use to detect and block bots, such as analyzing IP addresses, browser attributes, request patterns, and user behaviors. To avoid bot detection, you need to make your bot look more like a human user and avoid triggering any red flags. Here are some possible ways to do that:

• Use IP rotation or proxies to mask your original IP address and avoid being blacklisted by web servers. You can use services like ZenRows or ScrapingBee to get reliable proxies for web scraping.
• Disable the automation indicator WebDriver flags that Selenium sets by default, such as navigator.webdriver, useAutomationExtension, $cdc_, and $wdc_. These flags can be easily detected by anti-bot scripts and reveal that you are using Selenium. You can disable them by modifying the ChromeOptions or FirefoxOptions in your code12.
• Rotate HTTP header information and user agent to mimic different browsers and devices. You can use libraries like fake-useragent or user-agents to generate random user agents for your requests. You can also change other header fields, such as Accept, Accept-Language, Referer, and Cookie1.

These are some of the methods that you can use to avoid bot detection while developing or running bots in Automation Anywhere. However, keep in mind that bot detection is an ongoing arms race between web developers and web scrapers, and there is no guarantee that these methods will work forever. You should always monitor your bot performance and update your code accordingly if you encounter any issues.