Generative AI: Shaping the Future of Intelligent Automation
Generative AI has dominated headlines throughout 2024 and is set to remain in the spotlight well into 2025. While much of the focus has been on using Generative AI (GenAI) to answer questions, edit images, or summarize content, Ashling Partners has been exploring how Generative AI for Intelligent Automation can streamline business processes. We aim to automate and add intelligence to some or all processes to reduce the manual effort in completing a business process – providing immense value back to the business.
Although the applicability of Gen AI is extensive, the main questions that concern potential implementors revolve around the risk of an uncaptured error in a deterministic process and the ability to remediate those errors. Additionally, the extent of an error, if interpreted as a small mistake or something egregiously wrong and outside of the context of what was asked – so-called hallucinations – adds to hesitation in approaching Gen AI.
Despite GenAI’s vast potential, many organizations hesitate due to concerns about:
- Errors in Deterministic Processes – How do you catch an uncaptured error and manage remediation?
- Hallucinations – When output is drastically wrong or off-topic, trust in the AI solution can erode.
Many organizations hesitate due to these concerns. At Ashling Partners, our experts consistently assess Generative AI’s viability while identifying both its benefits and its drawbacks when paired with Intelligent Automation. Here’s what we’ve discovered so far.
Determining GenAI Value: Efficacy vs. Risk
Before you implement any use case, conduct a detailed statistical analysis of potential risks to understand possible outcomes and identify the necessary controls. Decide whether the use case needs human review— where value is provided by completing most of the work—or if programmatic controls is possible. Every use case is unique, so tailor your method of measuring both the efficacy and risk of the AI solution.
Examples of testing methodologies include:
- Error Matrix Analysis – Evaluate where and why mistakes occur.
- ROUGE Metric – Measure text summarization and generation quality.
- Levenshtein Distances – Compare predicted text accuracy to the correct output.
- Character Matching – Count precise character alignment between expected and predicted text.
Comparing an AI algorithm’s performance to a human’s performance can also yield valuable insights. For instance, if Gen AI’s false negativity rate stands at 3%—the highest-risk quadrant—but a human’s rate for the same task is 5%, automation not only saves time but also reduces erroneous outputs.
The Good: Real-Life Use Cases of GenAI + Automation
At Ashling Partners, we view Gen AI as a powerful tool within an intelligent automation strategy. You can center entire processes on GenAI or use it to boost the automation rate in existing workflows by replacing rules-based steps prone to high error margins.
Common Examples:
- Entity Extraction via regex
- Address Parsing using Levenshtein distance
- Name Matching between databases
- Form Extraction using simple string matching
These tasks often require human review, which lowers the overall automation rate. Replacing them with higher-performing GenAI engines can improve each step’s performance by 50–60%, significantly increasing the value delivered.
Notable Success: ETL with GenAI
One promising use case identified by our team is the use of Gen AI to extract, transform, and load (ETL) tasks of highly unstructured data. We have had incredibly positive results transforming raw data extracted from PDFs, excel, and web formats and standardizing and condensing the information to the desired format of the business. This is a common ask in automation use cases, where employees spend large amounts of time in non-value-added tasks of pulling information from tables to produce a report or create a business artifact on a routine basis.
Previous automation of these processes where the incoming data is unstructured has yet to be possible with rules-based automation or ML tools on the market due to the highly variable nature of the transformed content. Still, barriers remain.
The Bad: Limitations of Current GenAI Models
Like humans, context is needed in prompt engineering to achieve a desired output. The more complex the ETL step, for example, if we are trying to standardize 100 columns rather than just 5-10 columns, the degradation in output quality is very clear. This can be mitigated slightly by adding the necessary context for the column matching, but the business value is considerably diminished overall. The complexity of the use case will determine how much use intervention will be required and the percentage of use cases that will require extensive review.
Another limitation affecting the complexity dimension is the size of the requests that a given Gen AI model permits. Models that can accept larger data sets and more extensive context cost considerably more. The infrastructure required to host these more capable models is also much greater.
Finally, where math is required, there is consensus in the community that Gen AI should not be used, as it does not perform math calculations and only considers the most statistically likely answer, which may or may not be the same.
The Potential – Choosing a Trusted Partner to Implement AI for Your Organization
Our team at Ashling Partners continues to assess the strengths and limitations of models in the market, such as Open AI’s GPT and Meta’s LLAMA2 and their different model tiers. Price points, performance, infrastructure requirements, business risks, and testing methodologies are some of the many considerations required when considering bringing Gen AI to your business effectively. Let us help you design and implement the GenAI and intelligent automation path that is best for your enterprise.
Stay in the loop
Subscribe to our newsletter for the latest news, insights and resources on Agentic AI and Intelligent Automation.
