NLP Overview

Natural Language Processing (NLP) has evolved with the growth of Artificial Intelligence. The famous turning test relies heavily on NLP, since the way an analyst differentiate between a machine and a human is through questions in Natural Language, be it any language, understood by the both.  

Back then in the 1950’s information retrieval automation work was in progress. So was interest in many other NLP tasks such as summarization, keyword extraction and even machine translation.. The first Natural Language based chatbot was ELIZA, this was developed in the 1960’s by an MIT Professor. However, the answers produced by ELIZA were not ever apt for many replies. Still this was a beginning, the beginning of chatbots using NLP and the NLP itself.

As AI progressed this led to opening up of avenues for NLP as well to solve more of NLP tasks with help of advancements in AI. The typical earlier tasks in NLP were limited to retrieval of documents based on query, wherein the documents were stored on a file system, to even other NLP tasks but all were limited to limited number of textual data in form of files. Advancements in computer science all together brought in more computational power and the boom of the internet provided the NLP researchers an abundance of textual data, to learn more, to predict more and to conclude more.

Now, we are here in the year 2022, where AI heights and computational powers of hardwares and efficiencies of softwares are at its best. Yes, there are yet more successes to be made for sure in all areas. But let us analyse in short  what we have in store for NLP given so many supporting hands it has now.  The key areas of NLP can be classified in several ways. Some broad classifications are as follows:

1. Linguist Methods

These are branches of NLP problems which deal with deep linguistic analysis. This not only deals with the study of parse trees. But also of rules which are written by Language Specialists themselves. These rules can be rules such as an English expert knows, when to use has and when to use have. How to use certain other language constructs, such as the apostrophes’ usage to punctuation signs. These linguistic specialists often embed the rules in language in the form of rules in either context free grammars (Lex and Yacc for example) or in any other way that can be comprehended by the programming language. Most important application areas of Linguist Methods are in shallow parsing, defining the rules of languages in Machine Translation, even in POS tagging at times.

2. Knowledge Based Methods

Knowledge based methods are useful in many applications of NLP. These methods can be formulated at the NLP developer to model in solutions such as query focused summarization or even information retrieval or WSD. This differ from Linguist methods as in KB based methods, typically an inference engine accumulates all intermediate findings to conclude with a finals output. There are inputs and stored knowledge about these systems in the form of rules and parameters. These methods can also be used in Sentiment Analysis, Recommender System and Word Sense Disambiguation problems.

3. Supervised Machine Learning Techniques

These are NLP methods which use supervised learning techniques. These learning techniques can be deep learning, Naive Bayes or even Logistic Regression. The choice of ML algorithm to be used with a particular NLP task is a different challenge. The aim here is to learn from how an NLP task had been performed in the past and how this data of input-output mapping in NLP tasks can be used to predict and compute outputs of unknown or relevant textual data problems. However, a key issue in these techniques is availability of training data. The amount and quality of training data plays a key role in output predictions on newer data. Mostly all NLP tasks have been tested on Supervised Machine Learning Tasks, be it chatbots, summarization, retrieval, sentiment analysis, Word Sense Disambiguation, Text Categorization, POS tagging, Keyword Extraction, Machine Translation or even Automated Image Tagging.

4. Unsupervised and Semi-Supervised Machine Learning Techniques

Unsupervised ML algorithms primarily use clustering, Self Organising Maps, or kind of Recurrent Neural Networks. These are methods which do not require excessive training data. These methods are used in problems in NLP such as spam detection, text summarization, word sense disambiguation, and fake news detection. These methods often lag behind in results as compared to supervised NLP results. However, one must note, this should not stop research in NLP from unsupervised techniques. The reason is the real benefit of unsupervised learning is that it is independent of training data. The training data collection can have bias and may not be reusable at times, unless extra care is made to assimilate the training data. Further, there may be problems in future for which we may need a quick solution, but training data is not there yet for the problem. For example, no one predicted that one day we would have to make an algorithm to detect fake news recommended by recommender systems.

5. Graph Based Methods and Heuristic Techniques

Graph theory is an important subject in Computer Science and even in Mathematics. There are a plethora of applications of Graph Theory in NLP. Some important areas of application of this area in NLP are in information retrieval, we all know the basis of Google PageRank algorithm is Graph Theory, further it can be used even in fake news detection, origin of fake news, summarization, sentence similarity, the WordNet is organised in form of hierarchy as well and use many Graph based computations to compute even word to word similarity, this is however taken for as an inbuilt feature in NLP.

6. Optimization and Numerical Mathematics Methods

Optimization techniques are used enormously in NLP tasks, the most important thing is gradient optimization used in many applications. Other techniques such as Archetypal Analysis, Greedy Solutions, Travelling Salesman Problems solutions to even Dynamic Programming, Knapsack Problems are some of the optimization techniques which have been used in many NLP applications such as sentiment analysis, information retrieval, text summarizations, keywords extraction, to mention a few for now.

Numerical mathematical techniques such as Singular Value Decomposition, Principal Component Analysis, Non-negative Matrix Factorization are also used in NLP. These are used to find the key words, key sentences and key phrases in the textual data.

7. Logics based techniques

Pure logic had been used historically in NLP applications, however its recent research applications of the latest state of the art are not that common. These works include initial propositional logics to more complex predicate analysis. However, it is well studied theoretically.

7. Reinforcement and Transfer Learning

Reinforcement  Learning – These are kind of Machine learning algorithms which learn based on feedback- well suited for chatbots, recommender systems and even to humanoid robots.

Transfer Learning is often used when one set of tasks are learned well on a problem or a dataset and is mapped on to be evaluated on another quite different data, however it must be compatible data.

8. Approximate Reasoning

NLP tasks often involve approximate-ness. There is a kind of uncertainty involved in textual data. Consider the task of automatic keyword tagging of an image, where the image is a red ball with a cat in a football ground. How should the algorithm tag this as – which attribute to be given importance. This is one such application of approximate reasoning which is not limited to just Fuzzy Approximation. However the conventional applications of Approximate reasoning in NLP are almost all NLP tasks.

9. Abstract Mathematical and Science Applications

There are abstract and other pure sciences which have applications in conjunction with some areas to NLP. Some examples of this include the Whale Optimization Techniques, The Particle Swarm Optimization, Genetic Algorithms. All these are however included in optimization techniques but the key science behind these are either in biology or physical sciences. These techniques are for instance used to optimise the parameters of a deep learning model in say a recommender system or a fake news detection NLP task.