How LLMs Can Unlock Enterprise Data's Full Potential

Generative AI and Large Language Models (LLMs)  are advancing so quickly that it has led to overreaching marketing claims about what they can do, including solving very hard ML problems like energy optimization, sales forecasting, predicting failure of an asset, etc.

‍

While technical practitioners may recognize such claims are often misleading, many business leaders, seeing aggressive marketing from large companies, may take them at face value. So for executives, I want to answer two key questions in this post:

1) Where do Large Language Models fit within everything else your business uses for analytics and decision making?

2) Given what LLMs can and can't do right now, how can you best take advantage of their strengths?

‍

I'll cover some basics first, but if you are already well-versed with these, feel free to skip ahead to the framework section further below

‍

Understanding Enterprise Data

‍

To understand where LLMs fit in, it helps to look at how enterprises currently manage data. The Digital Age has created both opportunities, like leveraging data for insights, as well as challenges like analyzing unprecedented volumes. Let's quickly recap some key concepts of Enterprise Data to establish the foundation for where LLMs can help: Types of Data and Analytics Maturity Model.
‍

Differentiating Structured and Unstructured Data

Enterprise data mainly falls into two types: structured and unstructured. The kind of data you have determines the tools and methods needed to get value from it. The following table outlines the differences between these two types, helping to clarify how each is used in data analysis.

The Four Levels of Analytics Maturity

The Analytics Maturity Model helps organizations maximize the value derived from data in a methodical manner. It has 4 levels: Descriptive, Diagnostic, Predictive, and Prescriptive. Sometimes a fifth level, Cognitive, is used.‍

Organizations evolve through these levels as their analytics capabilities mature over time - from reactive reporting to predictive planning to AI-powered strategic advice.‍

The key is to leverage the most advanced analytics suitable for any business situation. While descriptive analytics works for rear-view reporting, predictive analytics enables smarter future planning. Over time, organizations scale their analytics capabilities vertically across all levels.

Strengths and Challenges of Large Language Models

While the umbrella term 'LLMs' refers broadly to all large language models, there are different types suited to particular use cases. Choosing the right LLM depends on your specific use case. It can be technical, so referring to previous blog posts might help.

Foundational models: These form the core and are broadly trained on vast datasets covering all knowledge areas. They can perform various tasks like understanding emotions, summarizing text, and more. Ex.: Llama2‍

Chat Models: These are fine-tuned on top of foundational models for meaningful conversations. They're used in chatbots and virtual assistants. Ex.: Llama Chat

Code Models: Specialized in understanding and generating code, such as translating between programming languages or automating coding tasks. Ex.: GitHub CoPilot.

(Passing over others in the interest of brevity.)

‍

What unites all LLMs is profound text comprehension, having learned from vast volumes of diverse writings. This allows them to deeply learn the patterns of human-written language and develop a strong comprehension ability. This enables remarkable skills like:

              Analyzing sentiment in passages

              Summarizing documents

              Answering text-based questions

While text-based training equips language models with language understanding, it doesn't grant them inherent numeric prediction abilities. They grasp structured data superficially, missing crucial mathematical relationships. Without an innate understanding of numeric dynamics, they recognize patterns but can't reliably project them into the future. Their interpretations offer insightful narratives but lack statistical validation and confidence intervals for reliable forecasting. In essence, this rules out language models from making structured data predictions in enterprises.
‍

Additionally, from a technical perspective, LLMs grapple with:

Handling Structured Data: Most business-critical data is stored in databases. While LLMs can create chat interfaces, there are challenges in using them for accurate database queries, including potential errors in SQL formulations to extract correct data.

Inconsistent Responses: Enterprises require consistent outputs, but LLMs provide probabilistic responses, making it hard to guarantee consistency, especially in chat interfaces.

A framework to introduce LLMs to advance the Analytics Maturity Model.

Now that we understand the strengths and limitations of LLMs, enterprises can maximize their value by focusing on their descriptive analytics strengths while remaining realistic about their limitations. The following is a simplified Analytics Maturity Model, focusing on descriptive and predictive analytics.

‍

The biggest value in deploying LLMs is to enhance access to past events and qualitative insights. Thinking they will improve statistical forecasts or algorithmic recommendations is unrealistic currently. By correctly leveraging LLMs for descriptive analytics, enterprises can maximize productivity and democratization. But, predictive capabilities will still rely on traditional Machine Learning solutions.

‍

While text-based training equips language models with language understanding, it doesn't grant them inherent numeric prediction abilities

‍

Getting Instant Insights on Past Events
LLMs transform accessing what happened through conversational interfaces. A chatbot on any device can now provide answers instantly on historical business data - be it yesterday's sales or last quarter's revenue. Performance metrics that took hours of reporting can be retrieved in seconds through natural language queries.

LLMs also create narratives from data automatically. Descriptions of trends, insights and events using everyday language broaden data accessibility beyond just visualizations. This mitigates the need for specialized data skills. Democratization empowers all employees to know business performance efficiently.

‍

Analyzing Unstructured Data
Processing qualitative data like customer feedback, social media posts or competitor intel is exponentially faster using LLMs. Their language mastery is unmatched for sentiment analysis, topic extraction and making sense of unstructured text data. This provides powerful new descriptive capabilities.

‍

Predictions - Out of Scope
However, predicting future outcomes relies on different specialized AI. LLMs cannot forecast sales or model asset failure scenarios the way machine learning models can. They enhance access to predictions but cannot make them independently. Prescriptive guidance also requires a rules/logic layer on top.

To begin, focus on tasks related to understanding "What Happened" on Structured and Unstructured Data. If you aim to find answers much times faster than current methods, LLMs are a suitable solution. You can gradually add more complexity.

‍
How Enterprises Can Capitalize from LLMs' Strengths

While the first order benefit of implementing large language models is going to be supercharged productivity, gaining efficiencies and automation through these advanced systems would lead to additional second order benefits as well, such as freeing up employee time and resources for more complex and strategic tasks, accelerating innovation cycles due to faster output, and increased revenue and profitability from the compounding advantages these models can enable across the organization.

Democratizing Data Access: Non-technical staff get self-serve data access via conversational interfaces. No specialized skills needed to extract insights

Supercharge Productivity: Answers business questions much faster than current reporting. Accelerates decisions by providing instant data retrievals

Encapsulating Process Knowledge: Capture and retrieve best practices concealed in documents. Unearth hidden insights from years of accumulated unstructured data

Augmenting Human Analysis: Surface non-obvious trends and outliers from structured datasets. Provide complementary qualitative insights to quantitative reporting

‍

‍

Conclusion

As we have seen, LLMs have unmatched language intelligence, allowing them to interpret texts and uncover insights rapidly from qualitative data. However, they currently lack the inherent mathematical logic needed for forecasting quantitative data.

‍
Nonetheless, companies can still strategically leverage LLMs to transform their descriptive analytics. By focusing on use cases tied to understanding "what happened" in both structured and unstructured data, enterprises can maximize productivity and democratization. Conversational interfaces can provide instant access to past events, metrics, trends and narratives without specialized data skills.

‍

As a reminder, the key strategic takeaways are:

              Leverage LLMs' strengths for text analysis to enhance descriptive analytics
              Manage expectations around limitations for numerical forecasting
              Democratize data access enterprise-wide through natural language

‍

With these insights, business leaders now have a framework to help address the two original questions: where LLMs fit alongside other analytics tools and how to capitalize on their capabilities.

‍

Stay tuned, the next post is about technical architectural patterns (fine-tuning, RAG, etc.) to make these applications a reality.