AI-Infused Decision Making

Have you ever contemplated in the number of times you make decisions in a given day? It’s time consuming as well as irritating to be reluctant to purchase an item or make a request. What if you have been given a “magical” tool that could help narrow down your choices? What if we create simple tools that can adopt to your style, situation and the current context to give you more precise and accurate recommendations. If this resonates with you then continue reading this article, as I will share some thoughts around how new type of digital systems can help you and other organisations make better decisions.

What is AI-Infused Decision Making

Perhaps I’m endeavouring to coin a new term in the world of the business decision making and artificial intelligence. Working for so many years in the IT field and observing numerous successful and indeed as well as failed projects, one prevalent mistake I see is when organisations think the use of AI can solve any problem and will wondrously help them sell more and market better! well, the conspicuous answer is in the negative! I shall explain.

But first let me give a definition for what do I mean by “AI-Infused Decision Making”. For so many years, people have talked about Expert Systems and how they help make better decisions. According to wikipedia, in artificial intelligence, an expert system is a computer system emulating the decision-making ability of a human expert. The expert system is fed a set of knowledge base with the aim to solve complex problems by applying rule based systems. AI Decision Support systems on the other hand are systems that aid in making decisions. Normally, they provide means for users to interact and use the system to reason and finally conclude.

So what is AI-Infused Decision Making? it’s simply put a combination of both Expert Systems and the use of advanced sophisticated methods in decision support systems to help guide users reach possible decision choices. Now that we have defined the term, let us dive into why I believe it provides distinct view and perspective of how new Decision Support Platforms are made and how it contributes into transforming how organisations make strategic and operational decisions. Not just that, even on the personal level, many of us make decisions with the help of mobile or wearable devices. Hence, how significant making decision is!

Successful Transformation = Taking the Right Decisions on the Right Time

I would presume many would to concur that making a successful transformation is profoundly correlated with taking the right decisions on the right time. Numerous groundbreaking transformative (ideas, projects, products, etc.) were not completely novel or original but rather composed and packaged entirely in a manner that is truly exhibit solving real and challenging business problems.

So take a moment and think for a second. How many of us when he/she sees a successful business; say I thought about this many years ago? is not that right? well, what went wrong? why didn’t he or she started that business? Was it a lack of will? Perhaps, but I would reckon it’s to do with “Uncertainty“. Our fears of uncertainty heavily influence our decisions. You, for example, made a decision to read so far! if you were uncertain and skeptical about the value of reading this article you would not have spent time thus far reading.

Dealing with Uncertainty?

We live in a world that is for the most part uncertain, nevertheless we thrive to seek certainty, because certainty gives us confidence in the validity and effectiveness of our decisions. Consequently, most of the organisations look for visionary and talent people who can see through future and anticipate opportunities and make “The Right Decision On the Right Time”.

For so many years, IT companies have sought to create tools to help professionals better make decisions. From automations to the most advanced technologies and use of Machine Learning and to help solve complex problems and provide better recommendations.

A Proposed Model to Solve Uncertainty

Let us take a logical example of how most people would go about making a decision. Let’s say you want to buy a car. First you will go to the web and search for the car spec, features, reviews, pros and cons, …etc. Then you may contact a subject matter expert for consultation or a friend who happens to own one. Then finally, you look at the available data like cars sales, parts costs ..etc. Isn’t that a very logical approach?

This is precisely what I’m proposing here is to build an AI-Infused system that takes into consideration the previously described approach. The proposed platform will utilise the leading-data found on the Internet and source all reported news and social media feeds, including customer reviews and ratings. This measure represents the uncertainly observed on the social media platform likely to represents people’s views. Incorporate that with Subject Matter Experts Opinion using methods such as Delphi method to source feedback. After that, combine it with lagging-data such as company’s sales, past deals ..etc. The common methods of forecasting depend on choosing internal factors that are often available to the companies or service providers, such as prices, daily sales, etc. However, this method relies on combining the previously mentioned method with inputs from open data such as people’s sentiments about a product or the popularity of a product or company. As well as integrating economic indicators in the forecasting process and enhancing it with the participation of experts such as SMEs. Integrating all of these inputs into a deep learning-based system in an effort to give a more accurate prediction and forecasting than the currently available techniques.

To read and know more about this approach click here.

* The images in this article are licensed under Envato Elements

Big Data with AI for Business is just compelling

What is Big Data

Big data is a term that describes the large volume of data – both structured and unstructured – that inundates a business on a day-to-day basis. But it’s not the amount of data that’s important. It’s what organizations do with the data that matters. Big data can be analysed for insights that lead to better decisions and strategic business moves. Big Data refers to our ability to make sense of the vast amount of data that we generate every single second. In recent times, our world has become increasingly digitized, we produce more data than ever before. The amount of data in the world are simply exploding at the moment.

With the internet, more powerful computing and cheaper data storage helped to use data much better than ever before. Big Data means companies like Google can personalize our search results, Netflix and Amazon can understand our choices as a customer and recommend the right things for us. And we can use Big Data to even analyse the entire social media traffic around the world to spot trends.

Benefits of Big Data with AI

By bringing together big data and AI technology, companies can improve business performance and efficiency by:

• Anticipating and capitalizing on emerging industry and market trends

• Analyzing consumer behavior and automating customer segmentation

• Personalizing and optimizing the performance of digital marketing campaigns

• Using intelligent decision support systems fuelled by big data, AI, and predictive analytics

Realtime Examples of AI and Big Data in Business

Here are the examples of companies that use AI with Big Data and seen enormous success in their fields.

Case Study (a): Netflix – Big Data and AI Netflix uses AI and Big Data extensively and achieved great success as an organization. It has over 200 million subscribers around the world.

• Generate Content: AI with big data helps Netflix in understanding consumers more and more granular level, thereby it helps Netflix to generate ‘content’ that matches the consumers taste to a large extent. Other competitors have a 40% success rate, whereas Netflix enjoys an 80% success rate.

• Recommend Programmes: Netflix uses AI to recommend new movies and television programmes to consumers. 80% of what the consumers watch is driven by their AI recommendations. Netflix fine-tunes their algorithms in understanding the consumers and provides recommendations to the consumers about their programmes and movies.

• Auto Generate Thumbnails: Netflix uses AI to auto-generate thumbnails. Consumers spend limited time choosing the films on seeing just the thumbnails for few seconds to minutes. Netflix understood the importance of thumbnails for consumers choosing their favourite programmes. Using Artificial Intelligence, thumbnails are generated dynamically based on the consumers’ interests.

• Vary Streaming Speed: Netflix uses AI for Predicting the internet based on the consumers’ internet speed. AI algorithms help to scale up or scale down the streaming of movies based on the consumers’ real-time internet bandwidth.

• Assist Pre-production: Netflix uses AI in pre-production activities. It helps to find location spots to shoot a movie (based on actors availability, actors location, etc)

• Assist Post-production: Netflix uses AI widely in post-production activities as well. Although editing is manual, quality checks are driven by AI to avoid mistakes in post-production. There were several mistakes that happened due to negligence or lack of time, resources during post-production activities. But with the usage of AI algorithms, Netflix could eradicate these problems to a great extent.

Case Study (b): Disney (Theme Park and Cinemas) – Big Data and AI Disney uses Big Data and AI to give customers a more magical experience. Disney has always been a tech innovator in both Theme Park and in Cinemas to give the customer a wonderful experience.

• Magic band: Disney offers magic band to its customers while they enter the theme park. Its kind of fitness watch which helps to open hotel room, allows the customers to pay. It has a GPS tracker in the band, which keeps tracking the customers where there are walking within Disney Theme Park. It is to ensure, where they are going within the park, which rides they are spending time, how much time they spend in restaurants.

• Better Operational Management: It helps to schedule the workers to manage over crowding at one ride or at a single restaurant with in the park.

• Better Customer Experience: Better management of crowd, giving proper assistance within the park gives the customer a better experience. They might direct the customers to other rides, other restaurants to avoid delay in one place.

• Realtime Sentiment Analysis: Disney research team started using AI to understand real-time reactions when people watch in the live show or in the cinema. How they do is they are using ‘Machine Vision’ – AI coupled with a Camera, a night vision Camera looking at the audience. They do Sentiment analysis with the people in the show. Cameras will interpret the facial expressions by looking at how the people are responding to the shows or movies to see if they are sad, scared, having fun, etc. This would in turn help Disney to generate quality content based on the customers for their shows and movies.

Case Study (c): Big Data and AI with Motor Insurance Motor Insurance providers have started using AI with Big Data to provide a dynamic flexible insurance plan that will suit different customers based on their driving skills, ability and composure at different times.

• Motor Insurance companies generally determine the premium based on the age of the vehicle. The insurance providers then started to understand the Customer based on how they drive by considering the age factor. This gave the perception a person aged 18 would drive rashly on comparing with a person aged 55 who will show maturity in driving.

• Tracking Card: Motor Insurance providers started providing a tracking card to insert in the vehicle, which helps them to track and understand about the driving ability of the customer. This helped the provider to understand the customer better.

• Mobile App: Now replacing the card with the mobile connected with GPS, it just needs the providers to install a mobile app within the customers mobile. This helps the providers to collect information about the customer driving. With the implementation of AI with Big Data, the providers can study the customer to a granular level. It helps the provider to understand how the customer is driving in a highway, during a rainy day, or on a hilly mountain road. Also, the question comes, they are people aged 18 who can drive better than the people with higher age. With the AI algorithms, over a period of time, the providers can understand each individual, how he is driving in the morning or in the late night, during a rainy day or during peak hours. Hence the data with the granular detail of the customer helps the Insurance providers to provide flexibility based on their driving skills not just merely on the age of the vehicle or the age of the customer.

Conclusion

It’s no hype that AI with big data are another set of high five technologies just to boast with for the IT giants. It has been used widely in several sectors and industries starting from big organizations to small business. The implementation of AI with Big Data in every industry has proved a great success and has helped the company business to a great extent. As said in the beginning, the world is exploding with data at the moment. Big Data with AI is really making sense of the huge data with the internet, more powerful computing and cheaper data storage.

Sentiment Analysis is Worthless

Disclaimer: The article does not assume that readers have a data science background and thus excludes and masks any complexities behind sentiment analysis or data science.

Opinion mining has reached its peak with the introduction of tools that facilitates sharing ideas and thoughts with the public. Although subjectivity of opinions affects how factual information is, sentiment analysis plays a huge role in studying a targeted group’s perception of a certain entity or event. To mention a few applications where sentiment analysis shines: Discovering a public event’s reaction, improving the customer satisfaction process, and studying a certain brand’s or an entity’s reputation. However, there’s a huge disconnection between the mentioned valuable applications and sentiment analysis, thus, I will try to connect the dots here and illustrate how sentiment analysis should fulfil business needs. Let’s start with a brief explanation of how sentiment analysis works and then move to satisfy the title’s claim.

Sentiment Analysis

Sentiment analysis as a part of natural language processing is the task of discovering a certain text’s emotional tone that is perceived by readers. It receives a text and outputs how positive, negative, or neutral it is. There are other categories as well that are used for sentiment analysis such as [“Angry”, “Sad”, “Happy”, “Excited”] or [1, 2, 3, 4, 5] similar to a rating that goes from 1 being very negative to 5 that is very positive, and so on. I have chosen to group the techniques in terms of their limitations and end results, which will fall into two groups.

Word-Level

Intuition

There are many words that we categorize conceptually as negative, positive, or neutral. And that’s the very first trials of sentiment classification in the literature that was born right after the outburst of subjectivity analysis (Detecting whether a text is opinionated or not) in the 1990s where the paper “Recognizing subjective sentences: a computational investigation of narrative text” has given a huge contribution to.

Short Overview

Word-level-based models at their core check whether the text has more positive words/phrases than negative words or vice-versa, and then classifies based on that. I won’t go deeper on how it does that as there are many well-known approaches such as looking at the language morphology of a word, using hand-crafted rules, automated “rules” through machine learning, looking at the semantics of words. But the important point to take is that it only operates at the word level and doesn’t go far with the whole text’s semantics. Now let’s see how that works straightforwardly by only focusing on one category: “Negative” Sentiment.

Figure 2 — Translation: I told the cashier Khalid that I got the wrong order, and he said that he can’t change it, what a bad service!

The example is pretty simple here (“Wrong” & “Bad”) but what if it was negating a positive word like saying “Not Good” or “Not Correct”? Here we move to negation handling (Still word-level) where we check words surrounding a positive/negative word and see if they were negating their positivity/negativity.

Figure 3 — Translation: I told the cashier Khalid that my order is not correct, and he said that he can’t change it. The service is not good at all!

This solves the problem of negation. However, what if we have different examples like this:

Figure 4 — Translation: I got the wrong order but the cashier Khalid has solved my problem immediately

Figure 5 — Translation: The pistachio latte’s taste is too bitter. Couldn’t finish it!!

Word-level-based approaches struggle with these kinds of examples where we have in Figure 4 a negative word that precedes “But” and then the negativity gets cancelled by “solved my problem” and turns into a positive text. Figure 5 on the other hand falls into a deeper issue where we have the word in Arabic “مر” that might refer to “Pass” or “Bitter” and it can only be resolved by using Arabic diacritics that not so many people use, or employing an extremely complicated parser. The two problems can be solved through the usage of context and semantics.

Context-Level

Intuition

Words are never independent in a text, each word can change the meaning or opinion of the whole text. Although some natural language processing tasks can run away from the burden of context inclusion (A deeper dive into the semantics of words and their “interactions”), sentiment analysis cannot.

Time-Line Summary

Many trials in the past used rule-based approaches along with word morphology in order to include some semantics, then a movement towards models that try to create groups of words that are similar and by that, documents/sentences will have multiple topics based on the words mentioned (Topic Modeling) where Latent Dirichlet Allocation in 2003 wins as the strongest contributor. After that, deep learning has taken a long course starting from word-level semantics where the star was Word2Vec by Tomas Mikolov through “Efficient Estimation of Word Representations in Vector Space” paper and then moving towards context-level semantics (Contextualized Embedding), until reaching to Transformers to solve many efficiency and quality issues. The basic idea is that there was a huge past where the byproduct is the introduction of models that cater for the context and semantics of words within documents (There’s a huge amazing work on interpreting gigantic deep learning architectures, so the idea that these models cannot be interpreted is not fully true especially when analyzing the core concept of transformers; Attention)

Onto a quick simple example whereby the model includes a contextual representation of text and can understand that the word “مر” is not “pass” but “bitter”.

Figure 6 — Translation: The pistachio latte’s taste is too bitter. Couldn’t finish it!!

Sentiment Analysis and Business Value Disconnect

Disconnection

When we have millions of documents that could be coming from app store or google play comments for an app, google reviews for a place, complaints about a company, twitter region or hashtags tweets…etc. Applying sentiment analysis and getting 10% positive, 20% neutral, and 70% negative for an app or a Twitter hashtag let’s say, is basically useless due to the loss of connecting it to a certain topic. Knowing that some hashtag is too negative only tells you the what, not the why.

You might say that I’ll just filter the text by a keyword but that keyword was chosen by you, not the data! How many words are you going to account for? Are these words being used by customers? Heavily? The data (reviews, comments, tweets) should drive the process of deciding which aspects, or more elaborately, which collection of hundreds of keywords that you should look for. The key takeaway is that you need to know what the aspects are to know what exactly is so positive or negative about your place, app, Twitter marketing campaign, or generally speaking, your business, and then improve.

Connection

We (Brightaira) have researched this subject in order to solve this problem in a different methodology than what is well-known in the literature due to the following reasons:

1. Scarce Arabic NLP literature

2. Arabic NLP datasets are of low quality

3. Arabic NLP base components-of-the-shelf have low quality

4. Inherent domain-specificity for well-known algorithmic approaches in terms of practicality and generality

We have released our first Generalized Hybrid Aspect-Sentiment Detection and Tracking model which Figure-7 illustrates only its core capability (The model is integrated within Bloom System that is part of Customer-Success platform)

Figure 7 — Translation: I told the cashier Khalid that I got the wrong order, and he said that he can’t change it, what a bad service!

One more thing to notice is that the sentiment has gone through multiple layers of indexing and statistical calculations in order to be served as a comparable metric to the CSAT Score used in Customer-Success Management. However, the aforementioned does not address the issue!

Deeper Dive !

We have discovered that aspects are also not enough. We want to know a very well fine-grained problem specification of the aspects given in Figure 7. What was bad about customer-service above is “Order Exchange” & “Wrong Order” that should be detected by looking at “cannot change it” (ما اقدر اغير) and “Wrong Order” (طلبي غلط). Hence, through a combination of contextualized modeling and graph theory (our first text representation layer to solve the issue), we are currently researching in fully connecting the dots until reaching the core of the problem where Figure 8 will elaborate:

Figure 8 — Translation: I told the cashier Khalid that I got the wrong order, and he said that he can’t change it, what a bad service!

By that, Brightaira can now discover:

1. What the total CSAT Score is for a business

2. Why the total CSAT Score is as such

3. How to change the CSAT Score

and automatically generate an actionable well-defined recommendation that fits our Decision-Making Platform.