How Digital Teams Can Begin Planning for Language Models – Part 2

Image Source: Photo 64004277 / Puzzle © Mikael Damkier | Dreamstime.com

In part 1 of this series, I discussed how language models require careful attention to deliver accurate information. Since out-of-the-box language models are trained on general knowledge, using them to leverage your company’s information requires intentional information architecture workflows. In this post, I’ll describe how to bolster the information layer of your digital environment to promote the effective use of language models and realize human-centered digital transformation for your organization.

Lead with Value and Alignment

Large language models have quickly demonstrated their practical use and ability to complement how we make use of information.

While language models are widely available to the public, incorporating your company’s business and product data, and operationalizing their use in the enterprise, is not trivial. It requires a level of due diligence that’s similar to onboarding any new digital capability, but with the added layer of conceptual alignment and governance workflows to ensure that accurate information is retrieved and/or generated. Table 1 highlights some basic steps you’ll need to consider.

Use Cases	Identify and assess opportunities that bring measurable value to the business
Scope & Prioritize	Create a roadmap and a clear plan for implementation
Experience & Technical Development	Architect, design, and build the necessary user interfaces, technical systems, and integrations to enable the target use cases
Data & Information Modeling	Design and structure the data and information in a way that supports the identified use cases
Model Training	Curate the training datasets and iteratively train/augment the models to improve their performance over time
Information Governance & Operational Support	Establish policies, procedures, and controls to ensure the effective management, security, and compliance of information assets

As you can see, language models are not plug-and-play; that is they require more than a technical workflow to get them operational. 

Additionally, make sure your digital team is coming from a place of value. In addition to obvious data-driven areas like revenue, efficiency, and risk, consider other areas that are human-centered and critical to business outcomes. Our survey of independent industry reports concludes that customer experience, sales enablement, and employee engagement, among other areas, track directly to business value (see Table 2).

Customer Experience	Create positive interactions, intuitive interfaces, and personalized services to enhance customer satisfaction, loyalty, and overall brand perception
Marketing & Sales Enablement	Provide teams with the necessary tools, resources, and training to effectively engage with customers, accelerate the sales process, and drive revenue growth
Employee Experience & Productivity	Create a positive work environment and leverage technology to streamline workflows, ultimately boosting employee satisfaction, engagement, and overall productivity

When validating use cases that leverage language models, you should see a correlation between 1) the targeted objectives, 2) foundational human concepts that support efficient communication, 3) the language model’s ability to understand your business concepts, and 4) the type of UI interaction affordances needed by users.

To achieve the right level of due diligence, you’ll need to foster a shared appreciation of information and data across the organization. Start by expanding the collaboration between all of your digital disciplines including:

• Data Science & Analytics
• Data Architecture 
• Experience Research
• Content Strategy & Management
• UI & Experience Design
• Knowledge Management

If you’re not familiar with the information-data dichotomy, here is my quick explainer: Everything is information: a person, a thought, feelings, an object, a sequence of events, etc. Information that is represented in a digital or analog (non-digital) form for the purpose of being retrieved is called data. 

For example, if I observe ten students in a classroom and represent the students by:

1. Writing “10 students” on a piece of paper – then “10 students” would be the analog equivalent of data that represents the number of students in the room.
2. Storing “10 students” to a record in a database – then the value in the database field is digital data that represents the number of students in the room.
3. Saving a picture of the 10 students to a file – then the digital encoding of 0s and 1s is data that represents the number of students in the room.
4. Creating a conceptual model – then the following properties are data that’s used to represent the number of students in the room. Note the level of context that’s afforded by modeling the properties of students, classes, and calendar for retrieval. 

• Student
  • Student Has
    1. First Name
    2. Last Name
    3. Attendance
• Class
  • Class Has
    1. Class Name
    2. Class Calendar
    3. Students
• Class Calendar
  • Calendar Has
    1. Dates
    2. Start Time
    3. End Time

Ultimately, the digital team should work with their information architect to assess the activities that expose valuable information across their digital ecosystem. This will help to ensure the right concepts are modeled and used appropriately by language models, other data sources, and content delivery systems (see Figure 1).

Getting the entire team to think about information requires organizations to direct as much attention to human-centered information as they do to technology-centered information (data). Why? As discussed in part 1, while language models are good at language processing (via prediction), they fail at language understanding (via symbolic and semantic reasoning) without human training and ongoing supervision. 

Additionally, large language models will not be aware of organization-specific facts, norms, and biases. All of these impact the analysis of text and data related to your company (see Figure 2). Hence, domain experts (or stewards) must remain in the loop as you adopt a language model environment.

Having the digital competency in your organization to promote conceptual understanding and alignment is foundational to transforming information into quality data. This is where your company’s information layer comes into play.

Put Your Information Layer to Work

Every organization has an information layer. The information layer is where the conceptual alignment between your company’s Experience Layers and Technical Layer (see Figure 3) are rationalized and managed. 

While data architecture teams are no stranger to modeling business objects as data assets, they are less equipped to design and manage information models that semantically link to human-centered knowledge and behaviors–which should link to the data objects that business leaders care about most often. 

When digital teams take on information modeling, most end up with ad-hoc and often misaligned conceptual environments versus a shared, enterprise conceptual model. While an enterprise-wide conceptual lens requires specialization, it stands to deliver more valuable data to all parties involved.

Wrangling conceptual and semantic complexity has been the remit of human-centered information architecture over the last two decades. Yet, many (if not most) digital transformation and experience teams still chose to build expansive digital environments without information architecture specialists–and to their own peril. The imminent adoption of language models in the enterprise makes human-centered information architecture an essential component for the following reason: Large language models are pre-trained on generalized domains of knowledge that can get their structure and accuracy by referencing over 100 billion statistical parameters. Most organizations don’t have enough information to train models at this scale. As a result, all out-of-the-box language models won’t work reliably against your company’s private knowledge. Thus, post-training methods that add a layer of context on top of your model of choice are required.

To address this limitation, your digital team will have to assess and model information across business functions and digital experiences to give your language model environment the conceptual structure it needs to assure accuracy and alignment with your organization (see Figure 4). 

The good news is that the foundation of your entire digital ecosystem will improve from this effort by supplying your data architecture team with context-rich conceptual data. This means, as you improve your information layer and language model capability, all other capabilities benefit.

…as you improve your information layer and language model capability, all other capabilities benefit.

The following are three essential information models that are core to any information layer:

• Taxonomy – a controlled list of human-centered terms used to categorize user interaction, product information, and key aspects of the enterprise business model to optimize the usability of technology.
• Metadata – selected taxonomy terms designed for targeted business, analytics, and UI use cases in support of information retrieval, interaction, and measurement. 
• Ontology – a foundational model for representing conceptual relationships and frameworks that inform enterprise data catalogs, UI objects and functionality, human-digital experience, and graph database environments.

Taxonomy, metadata, and ontologies are the “connective tissue” between human behavior and technology. Solid information architecture workflows lead to the design of these models, and their governance and distribution to business, experience, and technical teams. Feel free to use Factor’s information layer framework to discuss the readiness of your organization’s information layer.

Getting Started

Language models provide an unparalleled degree of access and engagement with company knowledge and data. Enabling such capabilities will require new technical infrastructure, information professionals, domain experts, and an attention to transforming human-centered information into quality data.

As large language models become more instrumental in your digital strategy, so too will your information layer and human-centered information architecture practices.

In anticipation of considerable technical due diligence and business arguments for language models, digital teams should also jump-start their information modeling practices to begin sharing learnings that better inform their data architecture, artificial intelligence and technology teams on what types of data to collect and IT infrastructure to build. 

Additionally, as you plan to integrate large language models and determine your parameters for success, be sure to monitor your team’s ability to transform the information that matters to your company into context-rich conceptual data.

Finally, as you transform your digital environments in the dawn of large language models, do it effectively and responsibly — by facilitating shared understanding and alignment with conceptual clarity. 

We’d like to hear your thoughts! Was this post helpful or what approach is your team taking? Comment below or on LinkedIn. If your organization needs guidance on assessing its information layer to accommodate a language model environment or to simply get digital right using human-centered information architecture, Factor can help.