Nidhika Yadav^a

Abstract– A recent news showed how AI algorithms can adversely effects children, teenagers and adults as well.  This article lays foundation to filtering system to pre-process and post-process the Large Language Models for both input and output to the system. Henceforth creating AI models that can be used in applications with much better surety of not hurting and adversely impacting children and adults both. The article proposes the use of sentiment analysis in pre-processing of input as well as in post-processing to make the users feel good about using social media.

Keywords: LLMs, Sentiment Analysis, Deep Learning, AI, LM, Tagging, Image Processing

1.    Introduction

Language Models (LMs) [2,3] takes as input textual data in a Natural Language format. The Large Language Models takes huge dumps form internet to learn useful things. And often end up learning some wrong things and then come in focus for bad things learned.

The detailed article whose solutions I am providing in this article is given in reference [1]. Some people call it statistical parroting system. Now do we call children learning new concepts as mugging things? No– They are learning—-Well then see from where we came in computing – The basic mode of communication used to be – ON signal and OFF signal as in a bitwise communication technology. From there we came to now broad options that can be well used with natural language-based instructions doing all work which used to be typed in in form of 0’s and 1’s once on mainframes. Hence what some people call statistical mugging system can be seen as an intermediate phase in lifecycle of development of a more AGI intelligent natural language system. Work is needed to make the systems more intelligent.

There are two things in Language Models being developed. One must see the variety of audience for a Language Model.

1. Children getting wrong contents,
2. Adults getting wrong contents.

Adults getting wrong contents is primarily due to presence of lot fake news and fake accounts on internet which are unknowingly fed into the AI system. The only way out of this is to feed the Language Models verified textual information. The verification can be on basis of authenticated accounts and trusted sources.

Think of AI model as a child trying to learn our world. If we feed in wrong things to it, it may learn wrong. So where does wrong things come in ? They come in from fake accounts, fake articles and fake news present on internet. This is how adult content can be taken care of. These things also hold for AI based image tagging and image processing using NLP techniques.

This article focuses on how to make AI languages safe for children and teenagers. Section 2 discusses the proposed approach.

2.    Proposed Approach

How does a small child learn wrong words? From wrong things he/she gets to hear. In the same way the system and AI algorithms being blamed are not being taught well, in technical terms it means pre-processing of input fed to train the AI brain is not filtered by filtering systems, hence it learns wrong things. This makes emphasis on filtering systems for  AI based Language Model learnings.

One need to change is the content being taught with some incremental changes in the teaching mythology viz the algorithms. And this does not mean we don’t need new algorithms; we need to make the intermediate stage AI more intelligent.

There is a need for a filtering system. A filtering system filters the input in form of sentences for apt inputs to be fed to the child–here the child is the AI language model–learning with the inputs given to it. Just like spam mails are separated from normal mails to secure us from harmful emails. In the same way, these harmful contents can be separated from AI language model. Which can be seen as a child learning things.

Regarding mentions of harmful contents for children. I suggest the filtering system which filters the Natural Language Inputs to the AI based Deep Neural Models to make three different Language Models. They are given as follows:

1. Language Model for children
2. Language Model for teenagers
3. Language Model for adults

The models for children and teenagers are fed in filtered input   i.e., the algorithm to non-adults will filter out all contents not meant for children. While adults do have right to know what is happening in world, and their model may not need so much of filtering. Just fake news needs to be separated from such adult Language  Model systems.

What we feed to the network is what we it will learn. These are basic pre-processing NLP task in LLMs in this process. This will solve many problems. Sentence level filtering need to be done here as many times a document is very useful but just few sentences are not meant for children.

Now, AI can help in filtering the sentences which are not apt for children and teenagers by learning using semi-supervised techniques. Or fully unsupervised way as well using sentiment of sentence as a classifying target class.

However, the first basic model can be filtering with keywords having negative sentiments. Sentiment analysis is a well-developed technique of Natural Language Processing and this can be through sentiment lexicons or sentiment models learned for some processes. Once sentiment of a sentence is computed as negative — one may delete this sentence from the training data fed to the AI-child in learning mode to make a model for children. Similar model for teenagers can be made where somewhat more flexibility can be provided to sentiment scores. But for children < 12 — only positive sentiment scores need to be put on the LLM AI model generation. For adults what needs to be filtered is fake accounts through which fake news is spread–not these words–but fake news——needs to be blocked from training — this needs to be learned.

Regarding gender bias and similar issues these are post processing NLP task which needs to be done after training LLMs. Not much from algorithm need to be changed for it, though algorithms can grow more for serving better. Some logic needs to be added in this — post-processing Natural Language Logic Rules Checks– Some hard work is always needed in Language tasks as against image processing things.

3.    Conclusions

To the best of the knowledge these things are not added in the current systems. Hence sentiment analysis based filtering system needs to be included in pre-processing the inputs of Language Models as well as in post-processing the outputs of Language Models and even image processing tasks using Language Models.

References

1. The race to understand the exhilarating, dangerous world of language AI. Weblink: (https://www.technologyreview.com/2021/05/20/1025135/ai-large-language-models-bigscience-project/?utm_campaign=site_visitor.unpaid.engagement&utm_medium=tr_social&utm_source=Facebook&fbclid=IwAR0-WA4AKBC0idlZRfM0ncoautcCos6xgjPlKpWAwVeon97F8DFYSdk2e4c)
2. Evaluating Large Language Models Trained on Code. [2107.03374] Evaluating Large Language Models Trained on Code (arxiv.org)
3. Efficient Large-Scale Language Model Training on GPU Clusters Using Megatron-LM. [2104.04473] Efficient Large-Scale Language Model Training on GPU Clusters Using Megatron-LM (arxiv.org)