Bias is intrinsic to humans. When we make an error steeped in bias, it is not uncommon to be consoled with “you are only human”.  Moreover, we tend to perpetuate our biased view of the world by seeking ideas and opinions that match our own. Individuals and organisations alike are vulnerable to this tendency, whether it’s just reading the news or building an understanding of the markets and applying that to branding, public relations, crisis management, product development and other strategies.

Unfortunately, biased search preferences, exacerbated by biases unknowingly built into AI algorithms that filter search results (see figure 1), obscure the true picture. What does it take to extract meaningful information from the internet in a less biased and yet more holistic way?

From bits to true insights

Every day, we are inundated by news, opinions, marketing messages and conversations on the internet. A single topic can be described and discussed by diverse voices of society across a multitude of platforms. While information-related content is immediately observable, the richer thinking and intention behind it is often less so. With empathy, we can better understand the perspective of the person or organisation behind the content, and their underlying motivations and emotions. These human-centric insights can enrich our understanding by painting a more accurate picture of the world.

Obviously, it takes more effort to empathise with content providers, especially those with a different perspective, at the scale of the World Wide Web. This is where information technology built around an understanding of human psychology, such as artificial intelligence, plays a critical role.

First, technology can help to overcome bias by enabling a systematic and broadened search at scale to identify relevant information, competing opinions and different perspectives. Second, using the power of association, empathy can be built into the search process by integrating the social, motivational and emotional aspects that surround each topic. This structured approach to web search, known as “associative hyper-search”, makes the collection of cognitive data possible by identifying the motivations and emotions of information providers.

The approach is “associative” since it identifies patterns connecting topics, motivations and emotions. It is also a “hyper-search” technology, relying on scale to reveal common patterns of association.

Uncover humanness though artificial intelligence

My collaborators* and I developed a tool, which we named NEMO (Needs and EMOtions), that dissect the different ways a topic, such as innovation, is talked about. It includes a process to specify a broad search, the software for conducting associative hyper-search, and machine learning to analyse and visualise the results of this search.

In the associative hyper-search phase, for any topic of interest, NEMO searches the web for combinations of topic-driven phrases, and words from defined lexicons of emotions and motivations. These data are then converted into word vectors, or unique numbers assigned to describe each combination of themes, emotions and motivations. Machine learning is then used to identify common themes within the topic, and their patterns of association with the various emotions and motivations (see figure 2 above).

To help the user better visualise and interpret the results, the findings are summarised graphically to reveal the importance of topics and subtopics, emotions and motivations, as well as their closeness in affiliation to other topics and subtopics, and their respective emotions and motivations.

For instance, on the topic of good food, themes concerning health, sustainable agriculture and accessibility of quality food might surface, as seen in the US in 2021 (see figure 3 below). Among the themes, the right to (quality) food was the most dominant and was clearly associated with the emotion “apprehension” and the motivation of “self-actualisation”.  In contrast, the other themes of health and sustainable agriculture were not only less dominant, but also lower on the motivational hierarchy and less skewed towards “apprehension” or “hope”.

Connecting content with the primary drivers of human behaviour allows us to better empathise with the diversity of minds behind the content. By introducing empathy in the search process, we can put that content into a human context and gain deeper insight into why the information is expressed the way it is. Ultimately, NEMO is meant to answer: “What is important to the world?” and “How is it evolving?”

A more “whole” world through the lens of empathy

Associative hyper-search is essentially a general technology that brings new possibilities. It offers another view of the world in real time, like a constant window to what the world is thinking.

By revealing the voices of society, it can help companies uncover drivers of corporate reputation, give early warning on social trends or technological disruptions, or identify unmet needs for innovation. For instance, a company might be interested in how its brand is perceived from the viewpoint of sustainability and how this relates to its target customers’ needs and emotions.

Beyond commercial applications, associative hyper-search is akin to the discovery of a global set of stories. These stories provide insights to politicians, social scientists, non-profit organisations and individuals on topics ranging from climate change to political sentiments and human trafficking. The possibilities are endless.

With deeper empathy, we stand to gain new insights that may lead us to question our assumptions, identify gaps in our understanding of the world and help us better frame our decisions (see figure 4 below).

As Henry Ford once said, “If there is any one secret of success, it lies in the ability to get the other person’s point of view and see things from that person’s angle as well as from your own.”

* Arnab Mukherjee and Sushmita Sarkar, Delta Analytics B.V.