How to realize an automatic semantic analysis ?

Published on January 31, 2022  - Updated on December 05, 2023

How to realize an automatic semantic analysis?

Semantic analysis is often seen as a tedious process, very costly in time and resources. However, thanks to artificial intelligence this is no longer really the case today. We will see in this article how to carry out a semantic analysis and why the use of an automatic tool can allow you to automate all these tasks and analyze all your verbatims in record time!

Challenges

Understanding a message, taking into account its tone, meaning, and emotions is the primary goal of semantic analysis. Today, this approach harmonizes human understanding with technology to provide effective solutions.

Customer service departments are constantly inundated with inquiries from various channels. Semantic analysis enables the processing, analysis, and categorization of this information to better comprehend customer expectations and respond efficiently. This method converts data into concrete actions, playing a crucial role in operational improvement and customer service support.

Beyond customer service, semantic analysis becomes a consolidating tool for the entire company. It involves more than just analyzing emails, customer comments, or interpreting social media messages. The value of semantic analysis lies in its ability to offer the entire company an opportunity to enhance its understanding of customers.

Advantages:

• Automates the text analysis process
• Improves the accuracy of text analysis
• Reduces time and cost
• Provides valuable insights from textual data

Limitations:

• Requires a large amount of training data
• May be challenging to interpret results
• Cannot capture all nuances of human language

The preparation

First of all, we will see that the preparation of a good automatic semantic analysis can be separated into 7 distinct steps.

First of all, it is important to specify that most algorithms will be dependent on the language of the verbatims. Indeed, depending on the language, the grammatical and syntactic rules will be different and that is why it is important to define them well beforehand before embarking on a semantic analysis or ensure that the tool can automatically detect the language of the comment.

Once this is done, we can move on to the first step: sentence segmentation.

1)   Sentence segmentation

The verbatims left by your customers will in the vast majority of cases be made up of several sentences. The first step is therefore to split the data. Indeed, in this segmentation phase, each verbatim will be broken down sentence by sentence. The objective of this first step is to be able to place the words of each sentence in their context in order to establish the meaning of the sentence itself.

2)   Tokenization

This step is the logical continuation of the previous one.

Indeed, if the objective of segmentation is to separate sentences from each other within verbatim; the tokenization on the other hand will find the words in each of its sentences and assign them a "token". It is this token that will allow the algorithm to semantically analyze the words and identify them correctly and therefore move on to the next step: grammatical interpretation. 

3)   Grammatical interpretation

The third step in carrying out our semantic analysis is grammatical interpretation (or Part of speech tagging). The idea at this stage is to find out what are the adjectives, subjects, verbs, etc. This step is crucial since it is what will allow the semantic analysis algorithm to understand the sentence and to create links between the different words.

4)   Lemmatization

Once the grammatical interpretation has been made, the different words must be grouped by family of words, this is called lemmas. To put it simply, if in a sentence we find the word "eating"; the algorithm will automatically recognize that this word refers to the "eat" family and therefore the associated lemma will be "eat". The idea is to keep only the meaning of the word.

This greatly simplifies and increases the reliability of the semantic analysis. Indeed, if the algorithm is able to understand the meaning of the words within the sentences, it will be able to clearly identify what the customer is talking about in his comment and therefore be able to classify it in the appropriate theme effectively.

5)   The stop words cleaning

During the lemmatization, it will also be necessary to erase the words useless to the analysis within the sentences: this is called stop words. These are words that bring no value to the general analysis of the verbatim, they are obviously different depending on each language, hence the interest of having this information beforehand.

For example, in English, it can represent the words "and", "at", "the", etc. They are very often present in the verbatims and slow down the work without adding value to the understanding of the text. This is why the cleaning of stop words is not to be neglected.

6)   Dependency analysis

We can then move on to dependency analysis. This consists of establishing links between the different words found in the previous steps. What are the topics? What are adjectives? What are action verbs? The objective is to put in relation the words to each other regardless of their position in the sentence in order to identify the different subjects mentioned in the verbatim.

7)  Identification of co-references

Finally, the final step is to seek the relationships between these different subjects by identifying coreferences. This simply means finding all the terms that refer to the same subject. For example in the sentence “The steward served me my meal. He didn't even look at me. the word "he" is a co-reference of "Steward". Thanks to this work, the tool will be able to correctly associate the negative emotion of the second sentence with the word "Steward".

As we have just seen, the preparation work for a semantic analysis is very substantial, but once carried out, you will be assured of very high analysis reliability. We will come back to this later, but if you decide to use an automatic semantic analysis tool, all of these tasks will be carried out by it, which will save you considerable time.

B – Which approach should be used to perform an automatic semantic analysis?

Now that we've seen the prep work, it's time to look at the analysis itself. For this, there are two different approaches to carry out this one:

- Classification

- Text-clustering

1) Classification

As part of this approach, it is first necessary to define a “model” upstream. This consists in establishing all the themes (or classes) that we wish to find during the analysis of the verbatim statements. This will allow the algorithm to automatically classify the verbatims in the different themes that have been defined beforehand.

By coupling this classification with emotional analysis, it is possible to identify at a glance which themes are the least well felt by your customers (irritants) and conversely which are the points of enchantment.

This approach can also be applied to entities such as places, stages of the customer journey, etc. For example, if one of your customers mentions an in-store experience in their comment, it is possible to search in the data if the customer was at the checkout, in the fitting room, etc. It is this set of methods that will allow you to better qualify customer data and enhance the interest and ROI of semantic analysis.

2) Text-clustering

As part of this approach, it is first necessary to define a “model” upstream. This consists in establishing all the themes (or classes) that we wish to find during the analysis of the verbatim statements. This will allow the algorithm to automatically classify the verbatims in the different themes that have been defined beforehand.

By coupling this classification with emotional analysis, it is possible to identify at a glance which themes are the least well felt by your customers (irritants) and conversely which are the points of enchantment.

This approach can also be applied to entities such as places, stages of the customer journey, etc. For example, if one of your customers mentions an in-store experience in their comment, it is possible to search in the data if the customer was at the checkout, in front of the store, in the fitting room, etc. It is this set of methods that will allow you to better qualify customer data and enhance the interest and ROI of semantic analysis.

C - Why combine classification and text-clustering?

Very often, it will be interesting to set up both methods. Let's take an example to understand.

Imagine that you are looking for your priority irritants, in this case the classification and analysis of emotions will allow you to directly target the negative points that bring out the most sadness, anger or disgust. By adding a clustering approach, you will be able to group together the subjects that have caused these

negative emotions and therefore bring out your irritants. It is therefore by coupling these two approaches that you will obtain the best results in your search!

Semantic analysis: use cases & concrete examples

The use of semantic analysis has proven to be a powerful lever for many companies seeking to optimize their customer experience. Here are some concrete examples:

Floa Bank and semantic analysis of customer reviews

Since 2020, Floa Bank has been working with Q°emotion, a company specializing in semantic analysis of customer reviews. The goal is to allow Floa to collect and analyze customer reviews from various channels such as social media, online review sites, contact forms, and chat conversations.

Semantic analysis of customer reviews is a valuable tool for banking groups as it helps identify priority areas for improvement. However, it's crucial not to limit the analysis to negative or positive reviews. In the insurance sector, for example, verbatims are often negative due to customers being in stressful or angry situations.

It's essential to focus on the emotions expressed by customers. Emotional analysis solutions like Q°emotion's help better understand customer needs and expectations. It also identifies friction points in the customer journey, contributing to an overall improvement in the customer experience.

If you wish, you can check out our use case with Floa Bank at this link: https://www.qemotion.com/en/clientCase

Uber and social listening

Another use of semantic analysis, employed notably by Uber, is social listening. For instance, when the company releases a new version of its application, social media and user sentiments are meticulously analyzed. This is referred to as "social listening," aiming to measure user satisfaction or dissatisfaction.

"At Uber, we use this approach daily to understand how users feel about the changes we implement. As soon as we introduce a modification, we know what is received with enthusiasm and what needs improvement." - Krzysiek Radoszewski, Marketing Lead Eastern and Central Europe at Uber.

Google and the hummingbird semantic algorithm

In another use case focused on SEO, Google, the tech giant, developed its semantic tool to improve understanding of user search intent. Introduced in 2013, the "Hummingbird" algorithm enhances the relevance of Google's search results by analyzing users' search intentions. This algorithm also boosts natural referencing (SEO), benefiting businesses that integrate quality content on their website pages for better visibility.

In conclusion, semantic analysis proves to be a versatile tool with applications ranging from customer service improvements to SEO enhancements, illustrating its significance in shaping a more responsive and customer-centric business environment.

To conclude, as we have seen, performing a semantic analysis is a very complex and time-consuming task. But the good news is that this whole process can now be fully automated! If you are interested in this topic and want to know more, request your free demo of our solution.