December marked the inaugural meeting of our newly minted Pathfinder Community Product Club. In case you haven't heard about Product Club yet — the Pathfinder Community Product Club is a monthly virtual meetup led by Automation Anywhere product leaders that focuses on our latest proprietary product innovations. It offers a place for community members to stay informed, connect with product leaders, and gain insights into real-world applications of the latest innovations in intelligent automation.
If you can’t attend the meeting, no worries — we'll be dropping a recap of each month's session right here in our Product Club hub. So, without further ado, if you missed December's meeting, let's catch you up!
Hosts
- Shane Patel, Principal of Product Management
- Alexander Timoshenko, Director of Product Management
- Allison Able, Pathfinder Community Director
Topic: Automation Anywhere’s Document Automation + Generative AI
Document Automation offers out-of-the-box intelligent document extraction capabilities and, now supercharged with generative AI, the possibilities are limitless to analyze and extract data from structured, semi-structured, and even unstructured document types that you could never have before with automation. Best of all, these capabilities are native in the Automation Success Platform and can be embedded right in your flow of work.
Here’s a rundown of the session:
- Shane demoed Automation Co-Pilot infused with Document Automation from the business user perspective
- Alex demoed Document Automation from the developer perspective
- Allison facilitated Q&A from the group members with Shane and Alex
Want some additional context on Document Automation with Generative AI? Read our Getting Started Blog here.
DEMO 1: Automation Co-Pilot infused with Document Automation + Generative AI from the business user perspective
Zoe, the customer service agent, works in Salesforce with Automation Co-Pilot embedded right in the application. She is assisting a customer, Jimmy McGill, with a home insurance quotation and has several automations on the page that are contextually relevant available to her. Let’s walk through how she is able to complete this end-to-end process in mere minutes using automation, process automation, Document Automation, and generative AI.
- In Salesforce, Zoe first selects the “insurance quotation process” automation. Data will be automatically mapped from the host record directly into the process, so all Zoe has to do is verify everything looks accurate.
- The information looks correct, so Zoe submits that data and then needs to provide property details, which our automation will pull from Realtor.com. For this, we employ generative AI to gather the details from the webpage—this is a much quicker solution than building a complex automation that would need to locate individual property information on the dynamic webpage. Generative AI is able to make sense of all the information on the website and map it into variables so that it's presented back to Zoe in a structured manner.
- The information populates into the quotation. Next we want to include a fire risk assessment for the customer. Zoe selects “include fire risk assessment” and uploads the fire report, which is a PDF document in an unstructured format.
- The process is now paused awaiting validation from Zoe. From the unstructured document, Document Automation + Generative AI is able to extract the property address, report date, overall risk score, and any medium or high risks as those will impact the insurance quotation. This saves Zoe a good deal of time from having to review it manually, especially if it were a 20- or 50-page report.
- Enough information has now been gathered and Zoe can submit quote into the pricing and packaging automation which sends it into Guidewire, a very common insurance management system. It comes back and presents as of prices and packages to Zoe that are relevant to her customer, Jimmy McGill.
- The next step is to generate the final quote. All the pre-work was completed in a matter of minutes, all in Salesforce, using automation. Zoe generates the quote and is required to submit all quotations to a manager. She submits the quotation to her manager, who works in SAP.
- The manager receives a new approval task in Automation Co-Pilot that’s embedded into his SAP. He reviews and provides his approval and an approval notification is sent back to Zoe in Salesforce.
- The automation automatically completes some final steps for Zoe, such as attaching the quote to the Salesforce case and generating an email response to the customer using generative AI to gather all the interaction details around the quotation. Zoe just needs to review the email in the Salesforce email composer, make any edits, attach the quotation, and push the response to the customer.
Take note that throughout this entire process, we are involving the agent the whole time. We’re fetching information, displaying it for Zoe, and she enters additional information which impacts the process flow (in this instance she uploaded the fire risk document), etc. She remains the human-in-the-loop to validate information throughout the process.
DEMO 2: Automation Anywhere’s Document Automation + Generative AI tool from the developer perspective
Let’s envision a business department that requests us to extract data from employment verification letters. Each employer has provided a letter in a different layout with different information. If we were to take the traditional approach of extracting the data from these letters, it would be very complex because there are no value pairs per se. You must read the text to understand it, which is the challenge with traditional IDP solutions. However, using Automation Anywhere’s Document Automation + Generative AI, we can find a simpler, quicker solution for unstructured documents like these.
- First, we create a learning instance that defines what we want from the system and select our pre-trained generative AI model, which in this case will be for a generic unstructured document.
- Then, we need to create the data points we want to extract, like the employee name, employer, and employment dates in a specific format.
- We can also create validation rules to ensure certain conditions are met if needed. If a condition is not met, it will raise a warning and the document will go to a validation queue for a human to review.
- Finally, we initiate the process. It will extract the data and create a CSV file in a specific folder that we have defined.
Note that we did not need to train the system at all. We just entered the prompt for what we wanted it to return, e.g. “what is the employee name?” Even though the employment verification letters are written in different formats, the data is normalized.
Do we need to use generative AI for all different document types? No, not necessarily. If you have fixed forms, there are traditional technologies that work very well and very fast. But when it comes to unstructured documents, it's nearly impossible to use a traditional approach like regular expression keywords position on those. That's where this Document Automation + Generative AI solution is a game changer for our customers.
SESSION Q&A
Thank you to our audience for submitting their questions! Unfortunately, we couldn't answer them all during the live session. Based on your feedback, we’re excited to announce that we will extend the time of our next Product Club session. We also want to express our gratitude to our product leader co-hosts, Shane Patel and Alexander Timoshenko, for providing their responses.
**Please note that all answers were shared during the week of December 18th, 2023, and are subject to change. We strongly encourage you to contact your account management team for any licensing and pricing inquiries.
Q: How involved was the programming of the Gen AI in grabbing the information from the site? (Regarding shared demo)
A: It wasn't difficult at all. The key was to construct the right prompt to get consistent results in terms of the extraction of that data. We also found some LLMs were better than others. In this demo, we used OpenAI GPT3.5 to do the extraction.
Q: Is Zoe using Co-Pilot with an unattended bot? Or is this programmed as attended? (Regarding shared demo)
A: Zoe, in the demo, was interacting with a Process. A process in Automation 360 allows you to create automations that include automation tasks, human tasks, approval tasks, and other task types to represent a complex business process that should be automated. Within this example of a complex insurance quotation, we made use of unattended bots to perform the work for Zoe, while form tasks and document validation asks were used to involve Zoe throughout.
Q: What is the architecture or capabilities used in all this automation?
A: The entire automation demonstrated on the webinar is based on different components of the platform. we have a workflow defined in Process Composer. Workflow consists of different steps, some of them are automatic, some of them are human tasks. Automatic steps defined by users in the task editor and executed on bot runners. Example of such steps can be fetching data from a website, extracting data from documents using Document Automation, sending API requests to backend systems. Human steps are used to review data or fill in custom forms, approve actions.
Q: What are the activities involved in meshing these copilot screen with any enterprise app? Is it running like a browser plugin? What about thick client/legacy apps?
A: We showed a couple of examples where we embedded using iframes in Salesforce. No plugins are required as SF allows for easy direct embedding of 3rd party apps. However, we know some aren't as easy, such as SAP, so in that case, we also offer a browser plugin that allows you to use a side panel to display Co-Pilot without modifying the app.
Q: So, AA has its own GenAI data extraction interface for programming this into the automation?
A: We have a product for data extraction within a platform - Document Automation. This product can use GenAI for data extraction as an option.
Q: Can you set confidence thresholds on extraction? For example, if the confidence of the model extraction is under 98%, send to validation.
A: In general yes, you can set a confidence threshold but it's not applicable to GenAI. This model doesn't provide confidence.
Q: What if I want data from multiple pages in a table format? How can AI be applied for this task? Do I have to upload one page at a time? Also, how can Python logic be applied in data analytics for field-level formatting and filtering of the data?
A: GenAI for tables is under development. The goal is to have it available in Q1 next year. We have some actions that can help doing filtering or formatting without python logic. But if it doesn't do what you need, you can get data from the document in the task bot, apply your custom logic, and upload modified values back to the server. These new actions (get/upload) were introduced in .30 release.
Q: Will this work with scanned documents?
A: Yes, we use the OCR engine to retrieve the text layer.
Q: Can this read handwritten scanned images of documents from the 1980s or earlier?
A: Potentially, yes, depends on the quality of images and readability of the text
Q: What about long-running multiple-page data in a tabular way, i.e., repetitive data in the documents?
A: We are working on that at this moment and look forward to sharing updates.
Q: Can you customize the validation form and apply business rules to data that is keyed to ensure accuracy before passing the data on to the next step of the process?
A: You have the ability to establish validation rules at the field level and document level, which will be enforced during the validation process.
Q: How much DPI is acceptable in AI Document Automation?
A: We don't have strict limitations on DPI. Basically OCR engine will try to extract data from a document with any DPI, but if DPI is low, the extraction quality will be also low.
Q: Is this only available in the cloud, or is it also available on-prem?
A: At this moment it's available in the Cloud only. But we are working on on-premise support. It should be available end of Q1 next year.
Q: In the case of using Gen AI, which data flows out of the AA system to third parties? Also, what security controls are in place since some documents may be for internal consumption only?
A: We recommend referencing this documentation: https://docs.automationanywhere.com/bundle/enterprise-v2019/page/da-gen-ai-architecture-security.html
Q: If we use gen AI for date format, who is paying for the API call? AA or client?
A: The cost is covered as part of your Gen AI + Document Automation license. Please connect with your account management team for full licensing details.
Q: What licenses need to be purchased to see learning instances?
A: You need a Document Automation license and a corresponding role. Please connect with your account management team for full licensing details.
Q: Do we need to pay separately for each prebuilt model in Document Automation or are they all included in the licensing?
A: Different SKUs are required for different technologies (AA parsers, Google DocAI, Standard Forms), please connect with your account management team for full details.
Q: Is there a cost associated with using gen AI open source code?
A: We do not use open source GenAI in our product, but you can create your own package using open source libraries and use it in the product.
Q: Is there a token limitation for when we're sending the HTML page for extraction?
A: The token limitation will be the one associated with the model you select in the command package. For the demo example, we used GPT with a 4096 token limit and we were able to stay under that by only sending over the portion of the page we wanted answers on. Perhaps with newer 32k limits on models we would have been able to send the entire page.
PUT DOCUMENT AUTOMATION TO WORK FOR YOU
Document Automation + Generative AI is available right now, and there’s no time like the present to fortify your productivity and revolutionize your document extraction capabilities! If you're interested in this product, connect with your Sales or CS representative to learn more.