How Artificial Intelligence is Transforming Modern Marketing

Are you struggling to choose the best marketing strategy or measure the effectiveness and adequacy of your marketing campaign? You are not alone I’m too.

I’m no expert in marketing strategies so to set this straight before you go ahead and read the entire article, but I’m an expert in digital transformation and building intelligent systems that can advance your marketing strategy.

Today, most organizations follow a conventional and traditional approach to develop their marketing strategies. It involves a great deal of effort and requires good study of the market and alignment with the cooperate strategy. However, I would argue that these strategies are predominantly based on past experience and little to do with “your data”. It is rare to see organizations employ advanced analytics to build their strategies. Mostly, due to technical complexities or inability to harvest the data.

Data-Driven Organization

You must have seen this title before. Numerous organizations like to put this title in their strategies to indicate the organization puts data first. Although, this is a great direction to take, however, few organizations do manage to perfectly implement. Only those who really understand how to put “Data-First” manage to succeed in building a data-driven organization.

Building a “Data-Driven Organization” is a rather extremely challenging task. It would take the entire organization to achieve it. Many processes need to be redesigned, rules need to be rewritten and business logic needs to be rethought. Equally, the IT infrastructure needs to be ready to help achieve that from building systems to storing and manipulating data.

Data and Marketing

No matter how good and robust your strategy is, it will be extremely fragile if not based on facts and data. Strategy after all is a process; a thoughtful process; you need to collect data about your organization, products, customers, partners in order to tailor the strategy to work best for you.

The data is available in two places. One within your organization’s systems and the other outside your perimeters. The latter is mostly found in open data. Nowadays, social media and global news on the internet represent a big portion of that data. That is why organizations these days use social media monitoring tools to monitor and observe what people are exchanging about them and their brands.

Social Media and Marketing

Companies today are in a race to attract more customers and promote their products to consumers online and most specifically over social media platforms. It has become a practice to analyze what people say over social media platforms to measure the performance of the marketing and communication department. It’s really such a powerful tool and we have seen the impact they present on the social, economic and political life we have today.

Artificial Intelligence and Marketing

Artificial Intelligence was introduced to solve the inability to process a massive amount of data and spot important things like when people are happy or angry about a service or a product we have. Many tools today offer basic to advanced Natural Language Processing to read the unstructured data make sense of it and present insight that could help organizations improve their services.

AI can be used in various marketing scenarios and I will give a shortlist of potential scenarios where AI can be of help

• Personalized Recommendations: AI can be used to help deliver personalized content and therefore improve the chance customer click or choose a product or service. With proper data planning, you can collect information about your customer preferences (with consent) and display the relevant products and services.

• Customer Care: Customer care is a big umbrella that covers interacting with customers, receives feedback and process customers’ requests. AI can be used in various touchpoints within the customer journey.

• Conversational Agents (Basic & Advanced Chatbots): Chatbots and conversational agents are becoming more and more widely accepted due to the high adoption by many organizations.

• Content & Website Design: Today, there exist many tools that help in content generation and website designs recommendations. Organizations can easily leverage these tools to easily create and publish compelling contents.

• Advertisement Bidding: AI is used in all advertisement platforms and organizations can use these available features. For example, you can let google ads decide what best work for you! And without the need to understand how bidding strategies work.

• Understand Buyer Persona: Understand the buyer persona is key. You can use AI to determine the “intent” of the prospect request and then deliver the request to the right team.

• Audience Targeting: You can use analytics and advanced analytics to determine the right target audience. You can also use AI tools to screen the public data and generate insights that can help you define your target audience.

• Topic & Title Generations: Perhaps this is one of the most challenging tasks in AI and today we see quite good advancements in this field. You can generate titles and topics that attract more customers.

• Customer Churns: Identifying the customers churn is important. You can direct certain marketing campaigns or offer discounts for customers likely to churn.

• Lead Scoring and Health: You can use AI to assign a scoring for each lead to help sellers quality the lead. This helps optimize the quality of leads and the sales team’s ability to utilize the marketing efforts.

Marketing Recommendation Platform

Realizing the importance of digital marketing and the current gab in finding the right tools to help markers achieve better decisions. We at Brightaira decided to build a platform that helps people working in marketing and companies make decisions with regard to the services and products they provide.

Brightaira, is an advanced artificial intelligence global media platform that assists organizations in making decisions in the area of Marketing and Customer Success. Brightaira collects Millions of NEWS and SOCIAL MEDIA feeds, analyzes them and provides organizations with the insights and decision choices to help optimize customer experience and improve business outcomes.

Unlike other tools, Brightaira provides “recommendations”, we call them “Parameterized Recommendations”. Where the AI engine determines the best recommendations and then decide the values within these recommendations that fit your organization. For example, available tools identify your negative sentiment, Brightaira’s AI engine on the other hand tells you what improves your sentiment and by which percentage you will probably improve when following the recommendations!

Try it for Free!

We believe AI should be available to all. Most of the available tools are very expensive and few organizations can afford to bear the cost.

We provide a ton of features with very affordable subscriptions fee that is suitable for many. You can also try the tool before you commit to any payment. TRY NOW.

Recommendation Systems. Simple concept - Powerful applications!

What is Recommendation System?

It’s one of the most popular data since applications. It’s a system that predicts the likelihood that a user would prefer an item, based on his past behaviors. That can be done by employing a machine learning algorithm, which can predict user preferences for a particular entity. There are a wide variety of applications for the recommendation systems, and it is used by many of the big technology companies, in order to recommend products to their customers. For instance, Amazon used the recommendation systems for product recommendations, YouTube for video recommendations, Netflix and IMDB for movie recommendations, and Facebook and Twitter for friend recommendations.

The diagram below demonstrates the recommender systems method.

Recommendation System Mechanism:

The engine of the recommendation system filters the data via different machine learning algorithms, and based on that filtering, it can predict the most relevant entities to be recommended. After studying the previous behaviors of the users, it recommends products/services that the user may be interested in.

The engine’s working of a recommendation is classified in these 3 steps:

1- Data Collection: The techniques that can be used to collect data are:

• Explicit, where data are provided intentionally as information (e.g. user’s input such as movies rating)

• Implicit, where data are provided intentionally but gathered from the available data stream (e.g. search history, clicks, order history, etc…)

2- Data Storage: It can be stored in a cloud storage such as SQL database, NoSQL database, or some other kind of object storage. However, it depends on the data type and amount as well. The more data that the storage can have for the model, the better the recommendation system can be.

3- Recommendation System Methods:

There are several methods in recommendation systems, but there are two major approaches to filter data on the system:

1. Collaborative Filtering It is making recommendations according to the combination of your experience and the experiences of other people.

2. Content-Based Filtering (The one that I used in implementing my movie recommendation system)It is based on product attributes, which is the item description and the preferences of users’ profile. It calculates the similarity between different products on the basis of their attributes. It treats recommendation as a user-specific classification problem and learns a classifier for the user’s likes and dislikes based on product features.

The diagram below demonstrates content-based filtering recommender systems.

Recommendation System Applications:

There is a wide and variety of applications for recommendation systems, especially in the data science field. For example, music and video companies like Netflix, YouTube, and Spotify use them to generate music and video recommendations. Amazon uses it for product recommendations. Social media platforms such as Facebook and Twitter use them for friends and content recommendations. Restaurants and hotels use it to generate food-related recommendations. As well as in the research articles, financial services, and life insurance.

Implementing Movie Recommendation System in Python

One simple and direct way to develop a movie recommender system is to use the correlation between the attributes of the movie. Thus, it will find the similarities between the movies to make a suitable recommendation for the user. I used here MovieLense data from Kaggle, and I employed a Machine Learning algorithm to filter data using the content-based filtering method, in the purpose of making those evaluations and predictions. I also used the K-nearest neighbor classifier model, which finds the k most similar items to a particular instance based on a given distance metric.

The diagram below demonstrates the K-nearest neighbor classifier model.

After doing some Exploratory Data Analysis (EDA), I found out that there are only 6 features in the 2 datasets (merged). Thus, I decided to extract new features from the given ones as much as possible. Also, here are some noticed things from exploring the dataset.

About the dataset:

• Number of Movies in the Dataset: 10325 movies

• Number of Users in the Dataset: 668 users

Plot 1:

• Most of the rated movies are having a rate of 4.0

• Only 1198 Movies have a rate of 0.5 (lowest rate)

Plot 2:

• It shows the count of the top 10 genres that the movies in this dataset are categorized.

• The genre that represents the higher number of movies is Drama.

Experiments Results:

After using the K-nearest neighbor classifier as a model to predict the model, its accuracy score was 48.5% and it had beat the baseline’s, by 48.2%.

I tried to implement the model by optimizing it with the GridSearchCV best parameters, but the accuracy did not increase.

Further Recommendations:

Although that I extracted more than 20 features from the 6 ones, there was a shortage of information about the movies and their details! So, I believe that the accuracy score could be better if I had more details related to the movies. (e.g. actors & director)

How YouTube Recommends Videos

Introduction

Back in 2008, YouTube had passed Yahoo! to become the second largest search engine in the world, behind only Google. Today, we can ask a related question: “Is YouTube about to pass Amazon as the largest scaled and most sophisticated industrial recommendation system in existence?” This question isn’t rhetorical – because we don’t know the answer as YouTube fiercely competes with the Amazon recommendation system.

YouTube suggested videos are a force multiplier for YouTube’s search algorithm that we would need to understand.

Earlier YouTube Recommendation Process

To maximize your presence in YouTube search and suggested videos, you need to make sure your metadata is well-optimized. This includes your video’s title, description, and tags. Most SEOs focus on the search results – because that’s what matters in Google.

How to create metadata tags in YouTube?

We need to look at the relevant top-ranking video and then use as many of the tags as we could that were also relevant for our video.

Recent YouTube Recommendation Behaviour

The scenario with the YouTube Recommendation approach is changed now. To get repeated viewers, the video must be recognized by the YouTube Recommendation Process. But, most YouTube marketers know that appearing in suggested videos can generate almost as many views as appearing in YouTube’s search results.

Why? Because viewers tend to watch multiple videos during sessions that last about 40 minutes, on average. So, a viewer might conduct one search, watch a video, and then go on to watch a suggested video. In other words, you might get two or more videos viewed for each search that’s conducted on YouTube. That’s what makes suggested videos a force multiplier for YouTube’s search algorithm.

How does YouTube Recommend Videos – Lighter Approach

There is a video in YouTube on the YouTube Creators channel entitled “How YouTube’s Suggested Videos Work”.

As the video’s 300-word description explains:

“Suggested Videos are a personalized collection of videos that an individual viewer may be interested in watching next, based on prior activity.”

There’s no way that creators can influence a viewer’s prior behavior, but this also means that a sports channel can tap into sports fans. They are shown to viewers on the ri“`ght side of the watch page under ‘Up next’, below the video on the mobile app, and as the next video in autoplay. More than 70% of YouTube watch time comes from mobile devices, so you need a mobile-first strategy for suggested videos.

“Studies of YouTube consumption have shown that viewers tend to watch a lot more when they get recommendations from a variety of channels and suggested videos do just that. Suggested Videos are ranked to maximize engagement for the viewer.”

So, optimizing your metadata still helps, but you also need to create a compelling opening to your videos, maintain and build interest throughout the video, as well as engage your audience by encouraging comments and interacting with your viewers as part of your content.

How YouTube Recommends Videos – Recommender Systems

Recommender Systems are among the most common forms of Machine Learning that users will encounter, whether they’re aware of it or not. It powers curated timelines on Facebook and Twitter, and “suggested videos” on YouTube.

Previously formulated as a matrix factorization problem that attempts to predict a movie’s ratings for a particular user, many are now approaching this problem using Deep Learning; the intuition is that non-linear combinations of features may yield a better prediction than a traditional matrix factorization approach can.

In 2016, Covington, Adams, and Sargin demonstrated the benefits of this approach with “Deep Neural Networks for YouTube Recommendations”, making Google one of the first companies to deploy production-level deep neural networks for recommender systems.

Given that YouTube is the second most visited website in the United States, with over 400 hours of content uploaded per minute, recommending fresh content poses no straightforward task. In their research paper, Covington et al. demonstrate a two-stage information retrieval approach, where one network generates recommendations, and a second network ranks these generated recommendations. This approach is quite thoughtful; since recommending videos can be posed as an extreme multiclass classification problem, having one network to reduce the cardinality of the task from a few million data points into a few hundred data points permits the ranking network to take advantage of more sophisticated features which may have been too minute for the candidate generation model to learn.

Background

There were two main factors behind YouTube’s Deep Learning approach towards Recommender Systems:

• Scale: Due to the immense sparsity of these matrices, it’s difficult for previous matrix factorization approaches to scale amongst the entire feature space. Additionally, previous matrix factorization approaches have a difficult time handling a combination of categorical and continuous variables.

• Consistency: Many other product-based teams at Google have switched to deep learning as a general framework for learning problems. Since Google Brain has released TensorFlow, it is sufficiently easy to train, test, and deploy deep neural networks in a distributed fashion.

Network Structure

There are two networks at play:

• The candidate generation network takes the user’s activity history ****(eg. IDs of videos being watched, search history, and user-level demographics) and outputs a few hundred videos that might broadly apply to the user. The general idea is that this network should optimize for precision; each instance should be highly relevant, even if it requires forgoing some items which may be widely popular but irrelevant.

• In contrast, the ranking network takes a richer set of features for each video, and score each item from the candidate generation network. For this network, it’s important to have a high recall; it’s okay for some recommendations to not be very relevant as long as you’re not missing the most relevant items.

On the whole, this network is trained end-to-end; the training and test set consists of hold-out data. In other words, the network is given a user’s time history until some time t, and the network is asked what they would like to watch at time t+1! The authors believe this was among the best ways to recommend videos provided the episodic nature of videos on YouTube.

Performance Hacks

In both the candidate generation and candidate ranking networks, the authors leverage various tricks to help reduce dimensionality or performance from the model. We discuss these here, as they’re relevant to both models.

First, they trained a subnetwork to transform sparse features (such as video IDs, search tokens, and user IDs) into dense features by learning an embedding for these features. This embedding is learned jointly with the rest of the model parameters via gradient descent.

Secondly, to aid against the exploitation/exploration problem, they feed the age of the training example as a feature. This helps overcome the implicit bias in models which tend to recommend stale content, as a result of the average watch likelihood during training time. At serving time, they simply set the age of the example to be zero to compensate for this factor.

Ranking the Predictions

The fundamental idea behind partitioning the recommender system into two networks is that this provides the ability for the ranking network to examine each video with a finer tooth comb than the candidate generation model was able to.

For example, the candidate generation model may only have access to features such as video embedding, and the number of watches. In contrast, the ranking network can take features such as the thumbnail image and the interest of their peers to provide a much more accurate scoring.

The objective of the ranking network is to maximize the expected watch time for any given recommendation. Covington et al. decided to attempt to maximize watch time over the probability of a click, due to the common “clickbait” titles in videos.

Similar to the candidate generation network, the authors use embedding spaces to map sparse categorical features into dense representations. Any features which relate to multiple items (i.e. searches over multiple video IDs, etc) are averaged before being fed into the network. However, categorical features which depend upon the same underlying feature (i.e. video IDs of the impression, last video ID watched, etc) are shared between these categories to preserve memory and runtime requirements.

As far as continuous features go, they’re normalized in two ways.

• First, it follows the standard normalization between [0, 1), using a cumulative uniform distribution.

• Secondly, in addition to the standard normalization x, the form sqrt(x) and x² are also fed. This permits the model to create super and sub-linear functions of each feature, which is crucial to improving offline accuracy.

To predict expected watch time, the authors used logistic regression. Clicked impressions were weighed with the observed watch time, whereas negative examples all received unit weight. In practice, this is a modeled probability ET, where E[T] models the expected watch time of the impression, and P models the probability of clicking the video.

Finally, the authors demonstrated the impact of a wider and deeper network on per-user loss. The per-user loss was the total amount of mispredicted watch time, against the total watch time on held-out data. This permits the model to predict something that is a proxy to a good recommendation; rather than predicting a good recommendation itself.

Conclusion

“Deep Neural Networks for YouTube Recommendations” was one of the first papers to highlight the advancements that Deep Learning may provide for Recommender Systems, and appeared in ACM’s 2016 Conference on Recommender Systems. It laid the foundation for many papers afterward. So, it has been a fantastic journey for the YouTube in the past decade to improve the recommendation process which in turn helps to keep the viewers intact. There are statistics that YouTube app in mobiles has replaced watching television to a great extent around the world. Not at all a simple task, we must sincerely appreciate the people behind it to happen.

“We will soon trade in our clunky flat screens for its handheld cousin, the smartphone and its YouTube app.”