In the August Developer Meetup, Arjun Meda was joined by Rinku Sarkar (Director of Product Management), Jason Trent (Director of Product Management), and Smita Biswas (Lead Technical Writer) to deep dive into the newly released AI Agent Studio that allows you to easily build, manage, and govern custom AI Agents to responsibly execute cognitive tasks embedded in any automation workflow.
In this meeting recap, we’ll cover:
- What AI Agents are and the capabilities of AI Agent Studio
- A support case handling demo that leverages AI Agents and AI skills
- Security features built into our AI Governance
- Additional resources for AI Agent Studio on our docs portal
- Live audience Q&A
You can watch the full recording of the meeting below.
/https://www.youtube.com/watch?v=_yiIk687I30]
What is are AI Agents?
We define AI Agents as cognitive tasks that combine AI skills with contextual actions. AI Agents can make decisions based on knowledge inputs and perceptions, communicate in natural language, adapt to dynamic situations, and take action to reach the desired objective. With AI agents, organizations can implement agentic automation of complex workflows—work that was previously impossible to automate—to redefine the horizons of business process automation.
AI Agent Studio
Automation Anywhere continues prioritize deep innovation and investment in our AI System which encompasses products like Automator AI, Generative Recorder, Process Discovery, and Autopilot. The newest product that we introduced into this suite is AI Agent Studio, which is designed to enable you to build AI Agents across your deployment, infusing the power of generative AI in a responsible, safe manner.
Embedded directly within the Control Room, AI Agent Studio is a set of tools and capabilities that allow you to build AI Agent solutions and utilize those across your processes and across your organization.
Some key features and benefits for AI Agent Studio are:
- Power to integrate a variety of LLMs including OpenAI, Azure OpenAI, Google VertexAI, Amazon Bedrock, and more
- Ability to evaluate performance of different generative AI models to create the perfect AI skill for your use case
- Ability to leverage prompt templates to share and re-use skills across your organization
DEMO: Support Case Handling
Jason walked us through a live demo featuring an Automation Admin, Jake, who needs to establish a new connection to a foundational model and make that available to a Pro Developer, Marcus, who will fine tune prompts against that model connection for a support handling use case. Then, Marcus shares his AI skill with a citizen developer, Sue, who utilizes it in an email support triage process she’s been building.
- Jake received feedback from his team that the model connection wasn’t performing as expected. So, he is in Control Room and is going to create a new model connection from scratch. First, he clicks on "Create Model Connection." He names the new model connection and selects a vendor from a dropdown list. He selects Amazon Bedrock Anthropic Claude 2.1 model.
*Automation Anywhere has an open platform that allows you to connect to all the different learning models from all the different vendors. And in the future, we're going to be shipping support for custom models that you can either host in your own infrastructure or from other different hyperscalers.
- Next, Jake provides authentication details from his credential vault, then tests the connection. Once the connection is validated, he can invite roles, which will allow him to share this model connection with particular users, teams, roles, etc., ensuring that the right people are connecting to certain models at the right time. He selects a role and clicks “Create Model Connection.”
- Now, Marcus receives a notification from Jake that a new model connection is available to him, so he wants to see if that model improves the performance of an AI skill he already built. From Control Room, Marcus opens his AI Skill folder and selects a skill called “Tone Detection.”
- His current model is not returning enough information regarding the sentiment of the test prompt input text, so he clicks “Choose...” to change the model. Marcus selects the new model connection Jake created and clicks “Choose” again. He goes back to his Tone Detection skill and clicks “Get Response.” This time the model returns much more robust information on the test prompt input. Using model configuration parameters, Marcus could fine tune this model even further.
*Stay tuned below - Smita covers how you can access the documentation pages that cover model configuration properties in much greater detail!
- Moving to Process Composer, Sue has a process built for support case handling. In this process, an email comes into a monitored support inbox and if the tone is below a threshold, the case is routed to an L2 or L3 support agent. But, if it’s more of a general question, an AI Agent is utilized to respond and provide information to the customer. Sue has coordinated these several steps together and has 2 AI Agents already configured to detect sentiment from the support case and later suggest some KB articles to the customer. However, Sue’s AI Agents need to connect to a foundational model. She doesn’t know much about models, how to fine tune them, or what a prompt does, but that doesn’t matter because she knows Marcus has taken care of all that detail in the AI skill he created for her. Sue opens her “Evaluate support email and create case” AI Agent, and then searches for “generative AI” in the Action search bar. She locates a Generative AI Prompt Template, selects “Execute” and drags it to her canvas.
- Sue is presented with the option to select a Prompt Template. She clicks “Choose.” A window opens with the AI skills that Sue has access to. She locates the Tone Detection skill that Marcus created for her and clicks “Choose.”
Building Guardrails with AI Governance
Automation Anywhere provides full control and visibility to monitor all exchanges within the LLM models. We use industry standard algorithms for encrypting the prompts and model responses, as well as TLS for all the communications between the Control Rooms and LLM models.
AI governance can be accessed from the Administration section in Control Room where you’ll be able to:
- Securely store prompts data either within A360 or forward it to a SIEM platform with which you're already integrated
- Enforce AI use policies through model connections to ensure developers use only approved and compliant models
- Securely store credentials with our credential vault integrations
- Manage role-based access control so that user roles have relevant permissions and will be able to use and leverage specific features
- Monitor & audit all activities around AI with a 360° view of prompts and events, plus the ability to export data to a CSV so you can analyze, slice, and dice as desired.
In a future release, we will introduce access to a rich set of analytics where you’ll be able to see what models are being used across your automation state, what the token consumption looks like for all automations, and more.
Additional Resources
Automation Anywhere’s documentation portal is your go-to resource for comprehensive information about and surrounding AI Agent Studio, including super helpful illustrations and demo videos. Any questions you might have about AI Agent Studio’s AI tools (like permissions and prompt templates) or AI governance (like prompt logs and event logs) is thoroughly detailed here. You can also find a section on general FAQs regarding the data security for all Automation Anywhere products using Generative AI, as well as licensing pages that review the features and coverage for difference licenses by deployment.
Did You Know? If you are logged in, there are extra features you can access with our Product Documentation Portal including adding content to a Watch List to receive a notification when updates are published, saving topics to your Favorites, and leaving feedback that comes directly to us. We highly recommend taking advantage of these helpful features!
To learn more, check out:
Q&A
Q: How did you navigate to the prompt templates?
A: First, remember that this is all driven by governance and there are a set of permissions for both model connections and AI governance. So, if you don’t have access to those permissions, you won’t see these links within your environment. In the current .33 release, the first one will be under Manage > Model Connections, that will give you access to manage model connections. If you want to build an AI skill, that’s going to be an asset within the repository, so to do that you’ll hit “Create” and you’ll see “Prompt Template” as an option in the dropdown. Heads up for the next .34 release, we’ll be moving some of the navigation around, but we’ll share move about that closer to the October release.
Q: The way we can send the prompt, can we also send files?
A: In the current version, we don’t support files, only text-based right now. However, we do have a roadmap item for sending files as well and we’re going to introduce new models that are multimodal—meaning they can work with text, files, and video, and not only ingest that but also create that as part of the generation process.
Q: What is the size of the prompt we can send in?
A: The size of the prompt will be limited or managed by the model itself. Every model has different “context windows,” or information it can receive. I don't believe we have a limit size on the actual skill though.
Q: How can we send emails as input?
A: You would simply go and configure an automation to watch that e-mail inbox, extract the text from that, and then send that as a prompt input into the AI skill, which would, at runtime, generate a very large prompt and send that over to the foundational model.
Q: How do we validate the prompt result? Do we have a scoring mechanism?
A: We don’t have a scoring mechanism, per se. A lot of the validation happens in this experience. In a future release, we’re introducing new capabilities around guardrails where we will provide scoring capabilities based upon toxicity and others.
Q: Can we provide training to get the correct output?
A: In this particular example, there isn't necessarily training per se because every prompt is considered independent. However, the point of this whole screen is to effectively tune the way that you've written the prompt or maybe even the configurations for this given prompt so that you're able to get the best outcome out of that particular model. If you want to do something like expanded tuning, I've seen that happen two different ways:
- Use a fine-tuned version of a model. We're actually going to provide support for fine-tuned models in the next release. So you could take one of the models we have out-of-the-box today and we’ll provide you the ability to access and then fine tune that. So you're basically adding in your own information that will help the accuracy of those models.
- The second way is that concept around Enterprise Knowledge, which is that RAG capability. That's another way to kind of think about improving the accuracy through prompting other than sentiment analysis.
These foundational models are trained in multiple different capabilities. So in this analysis or sentiment and tone detection is one simple example just to prove the point of how the tools work. But you can actually ask models to do a various number of things. They're trained on the public Internet up until a certain date, usually within the last 12 months or so. And obviously the newer model you select would have a larger training set as well as more capabilities. These models are evolving really, really rapidly over time. And as I mentioned, these are models that we support today from various different hyperscalers. There's 600,000 models that are available for use that are more custom or bespoke and a lot of times we see customers instead of going with a very large model that can do a lot of different things, focusing on a bespoke or specific model or maybe even a model they built that's hyper-focused and has incredibly great results on a single task. Again, we will be providing support for custom models in a future release.
Q: Can you show us how the RBAC control looks for this?
A: It's going to be the standard RBAC that you would have. So for instance, whenever I'm in something like a folder, it would be the ability to create and share that folder with other people. A lot of this will depend on the roles that the current user is in, if they have the opportunity to even create an automation or see the repo in the first place. But then again, we can always manage who has access to these particular folders as well. So we manage access to this in multiple different layers, first starting with the model connection, then with that AI skill, and then ultimately with the AI Agent itself. And we can track and audit all of that under the covers as well.
Q: Can the RAG model provided by AA be fine-tuned? Can I use my internal documents to get responses?
A: There are a couple of different ways to manage RAG. We have separate generative AI package command packages that you can use that connect to the hyperscalers. We're going to be bringing native response augmented generation, or retrieval augmented generation, capabilities into AI Agent Studio from those hyperscalers in our next release. And then we do have this capability that's called Enterprise Knowledge. Enterprise Knowledge is a separate offering, but it's kind of related to AI Agent Studio. It allows you to upload your own information—that could be Word docs, PDFs, Excel files, etc—that might be relevant and allow you to query against that like you would writing a prompt and then getting that response back and then using that in automation.
Q: How does this system help with the existing generative AI command packages?
A: These features that we introduced through AI Agent Studio augment those capabilities.
Q: What’s the retention period of the log saved?
A: We’ve embedded the policy for the retention within the UI and the default is 180 days.
Q: Does it work as one session ID for one prompt?
A: No, a session ID here is to be treated as an ID that is associated with an automation execution, which can have multiple exchanges within one execution. So, treat the session ID as an automation session ID.
Q: Can we send both user and system prompts?
A: The way that prompt tooling works today is that you embed everything within that single prompt window. We actually send that over under the covers as a user prompt. However, we are looking to introduce new capabilities around specific system prompt tuning in a future release.
Q: Can we use it in a loop?
A: Yes! You could build individual AI skills and use those within an Agent or go and do looping within Process Composer and other capabilities like that as well.
Q: Do we need AI Governance option under enterprise Control Room? What's the licensing model essentially like would this have a separate license?
A: We introduced a new licensing capability called Enterprise SKU. That Enterprise SKU contains AI Agent Studio, everything you saw today, API tasks, as well as Connector Builder, and several other features. And AI Agent Studio, as I mentioned is, is contained within that offering.
Q: Will it use an open or private environment?
A: The way that AI Agent Studio model connections work is that you connect to, essentially, a publicly available model. But those models require your own credentials, so you’re connecting to your instances of those models. If you’re a customer that utilizes private cloud or you have deployments of models hosted somewhere else, then we are looking to bring what we would consider custom model connections that would allow you to go and connect to those types of models in the near future.
Q: For every request and responses, who will bear the license? Will it be Automation Anywhere or the respective LLMs?
A: It's actually gonna be the customer. So whenever the customer connects, whenever you're connecting using a model connection, you're providing, you know, very simply an API key or access, access keys as you would have seen when I created the connection to that Anthropic model that's hosted on Amazon Bedrock. So all of the licensing for those models would be and all the, the token consumption would be incurred within your, your hyperscaler or your particular LLM vendor.
Q: Is it inflexible for cloud, cloud enabled, or on-prem?
A: Currently we are talking about cloud only. We are looking to bring an on-prem version of the Enterprise SKU (that AI agent is included in) to customers either later this calendar year or the very beginning of next calendar year.
Q: Any rough ETA on when API tasks will be coming to on-premise?
A: API Tasks is part of the Enterprise SKU. It will not be coming in this initial version of the on-prem release of Enterprise SKU, mainly because of the infrastructure requirements and the way that we built API tasks is really cloud-first, cloud-native. We know that this is an ask for our on-prem customers and we're looking to figure out ways to provide that for them. We just don't have a timeline on that, or at least I don't personally.
Q: How do we store all these data in the cloud? Does that mean the clients need to procure additional cloud space or is it provided as part of the subscription by Automation Anywhere?
A: It’s part of the subscription, and of course subject to the retention period we mentioned. All of this will be stored within your tenancy within our production environment, secured with encryption algorithms we're using. Additionally, you can use your own SIM platform if you need longer retention periods and if you want to forward these logs to your in-house blunk or Q radar systems.
Q: Do we have a leverage to create our own models as well?
A: Yes, customers are free to create their own models. And then as I mentioned, we're going to be making it possible for customers to connect to custom models or maybe models that we don't provide support for natively out-of-the-box with AI Agent Studio in a future release.