Taxonomies and AI Chatbots

This blog post is adapted from a Presentation by Gary Carlson at Taxonomy Bootcamp, November 2018

Taxonomies, AI, and Chatbots: Connecting People With Information

There is a natural relationship between AI chatbots and taxonomies—but how do they work together? More importantly, how can taxonomies/ontologies support best AI practices?

Taxonomies and ontologies interact with, support, and drive Artificial Intelligence (AI) technologies. On a fundamental level, AI chatbots are meant to connect a person with the information they need. And, fundamentally, that is exactly what taxonomies/ontologies are for.

First Important note: This blog post doesn’t explore any benefits or pitfalls of chatbots; nor is there an intention to discredit any underlying AI technology. Instead, I hope to outline a few ways that taxonomies or ontologies can support a well-designed chatbot experience. Taxonomists have a lot of great information that could assist the chatbot initiative, but are often brought into chatbot projects late in the design process.

What is a “Chatbot”?

We can define a chatbot as a computer program that:

• Interacts with a person via voice or text
• Interprets and clarifies a question
• Creates a query based on the question (and additional contextual information) to a data set(s)
• Provides an answer

A chatbot interprets a question using various technologies, but it must be able to:

• Address synonyms
• Determine context
• Disambiguate concepts
• Provide query expansion or refinement.

One thing to remember: chatbots aim to solve findability problems.  

Taxonomies serve the same function: reduce the friction or entropy that exists between the entity asking the question and the entity providing an answer.

Taxonomy/Ontology

Second Important Note: This blog post uses the terms “Taxonomy” and “Ontology” interchangeably, mainly for clarity and ease of understanding. There are concrete differences between a taxonomy and an ontology. However, on a basic level, both have concepts and define the relationships between those concepts.

Taxonomies and Ontologies define a common language and bridge one vernacular to another.

• Define relationships between concepts (hierarchical, associative, entry or synonym)
• Can be used to describe content
• Can be used to refine questions

Capabilities of Taxonomy for a Chatbot

1. Synonyms, homonyms, antonyms, etc.
2. Query disambiguation (I.e., “Turkey” the animal vs. “Turkey” the country)
3. Query expansion / refinement (i.e., “Terrier” → Dog)
4. Identify relationships across domains (i.e., Dog → Therapeutic Aids)
5. AI training and rules, entity extraction.

Practical Application and Implementation.

The more things change… The more they stay the same

Factor recently worked with some large teams of developers and AI engineers to build and implement new chatbots. (Taxonomists love working on projects from the ground up!) Going into our first chatbot project, we were quite interested to see how the project would differ from our standard Information Architecture projects. The Factor Team has developed a long standing and steady approach for things like e-commerce, portal, analytics, and other similar projects. But Artificial Intelligence was daunting. We were fully prepared to reassess our entire approach for chatbots but were surprised at how durable our methodology was!

A chatbot is just another system; another set of requirements.

Obviously, we placed a lot more emphasis on the actual technology requirement than was necessary for our other projects. But assessing all other areas was essential for designing and building a successful chatbot ontology. The assessment phase helped us determine the necessary domains, the role of variants, relationships, the need for the ontology to interact with the content, etc.

As a result of our assessment we were able to gather and provide requirements to the team that would design and build the chatbot. Each of these areas (in bold) had a direct impact on the overall ontology model.

Taxonomy Construction Inputs

• Entity extraction results from online and documentation content
• Review of search logs
• User questions documented by the support team
• Publicly available support documentation
• Previous user research
• Ontology tool training and documentation

Sourcing the taxonomy

Obviously there are so many more capabilities for a chatbot, but these are the capabilities that we supported with taxonomies/ontologies in our recent chatbot project.

Going through this capabilities list, it should be clear that chatbot capabilities are common across almost all findability projects. Realizing this helped us stick to the basics.

One of the great things about AI, is that we get to work with a team of extremely smart and information-centric engineers. We were really fortunate to be able to take advantage of the structures being built. So, we didn’t have to work very hard to convince people of the value of different types of entry terms (synonyms, antonyms, nicknames, etc). When the developers wrestled with the best way to interact with verbs, we were able to build them a “verb” taxonomy.

Berry Picking

In 1989, Marcia Bates published her “berrypicking” model, which has since proved foundational to the work of taxonomy.

Bates described key elements of information-seeking behavior that provides a backdrop for parsing and converting an utterance to a query. This required a pretty large mindshift for developers. The ontology provides the capability to support this process, but the interface and interactions still need to be incorporated.

• Typical search queries are not static, but rather evolve.
• Searchers commonly gather information in bits and pieces instead of in one grand best retrieved set.
• Searchers use a wide variety of search techniques which extend beyond those commonly associated with bibliographic databases.
• Searchers use a wide variety of sources other than bibliographic databases.
  

Not only does a query get refined, but it may change significantly and move from one modality to another.

5 things that every chatbot should consider

1. Synonyms, homonyms, antonyms, etc
2. Query disambiguation
3. Query expansion / refinement
4. Identify relationships across domains
5. AI training and rules, entity extraction

Final Thoughts

Don’t forget the end user goals. The ontology we built was directly impacted by the content domain and content structure. Building the ontology was relatively easy. Designing it to meet the different chatbot capabilities across the technology, content, user goals, business goals, and resources going forward was the real challenge.

Trust the process. Factor’s standard process and assessment for the ontology/taxonomy design was remarkably durable. AI Chatbots may seem daunting, but treating the process like any other taxonomy project helped us define and solve the problem on a fundamental level and work with developers, engineers to address findability issues.