Robo-Writers, Translators, Chatbots: Developments in NLP and What it Means for Education

Did a student write that essay or a robot? Did a teacher provide that six trait feedback or was it an automated feedback system? Did that student understand that Mandarin dialog or did they translate it on the fly? Is that student talking to a mental health professional or a therapy-bot?

If you are worried about plagiarism, things just got a lot more complicated with developments in Natural Language Processing (NLP), a branch of artificial intelligence that enables computers to understand, process, and generate language.  

AI, and particularly NLP, is changing the way humans create, consume, and evaluate content. Financial reports, sports news, and legal briefs are increasingly computer drafted. NLP is changing the nature of employment and will increasingly show up in classrooms–welcomed and otherwise.  

Language Processing

Natural language processing is a branch of computer science, specifically artificial intelligence, that enables computers to understand human language, process it and create language to interact with humans to communicate. Computers have been trained to process large quantities of data and language and then generate communication in return. It’s highly likely that everyone is interacting with NLP every single day because of all the places it is found.

Some examples include the use of chatbots, voice assistance, translation tools, and search engines. In my eighth grade STEAM course on emerging technologies, a good part of our year is focused on artificial intelligence and its various components. We explore a variety of materials such as AI4K12, ISTE U AI Course Materials, and have recently come across AIClub which has activities for students to see NLP in action and also work with some of the projects. They offer many free activities to explore AI. We want to move students from consumers to creators, especially in these areas, and with AI, it is impressive to see what students create, especially with chatbots. A group of sixth-grade students created the Calmzilla app to help middle school students deal with stress through a chatbot, showing a great way to focus on SEL through the use of chatbots and having students create.

How does NLP work?

NLP is possible through machine learning. Through algorithms, it is able to understand the meaning and structure of human language, analyze text, and even respond. The system stores all data whether that’s words, sentences, books, or any information that is loaded into the system. The computer then uses its data to find patterns to analyze.

Chatbots or messaging systems read the language that we use and can learn over time.

NLP can be used for different processes such as in documents for correcting spelling, translation of words, it can also respond to human language and react based on the information that it receives. Computers interpret what humans say, process human speech and then respond or react. It can evaluate speech based on context and intonation and then respond. We often use it for machine translation, text to speech, image description, and speech recognition.

Developments in NLP for accessibility and language creation

There have been many developments in NLP that promote accessibility and language uses from a variety of providers including Facebook, Google and Microsoft. For promoting accessibility, with NLP, companies can leverage the technology to provide more language capabilities.

Launched in 2020 by San Francisco R&D shop OpenAI, Generative Pre-trained Transformer 3 (GPT-3) is a third-generation model that demonstrated dramatic progress in creating human-like text.

Amazon Comprehend is NLP that locates insights and makes connections in text. There are many advances in large, “pre-trained transformer-based language models” such as BERT (Biodirectional Encoder Representations from Transformers). BERT, developed by Google is leveraged in its search engine and was trained on language modeling and next sentence prediction, and its use has continued to grow.

The use of NLP is expanding into all areas of work and life. There are some key trends with NLP to look out for in 2022 and here are some of the new developments (with thanks to Jack Clark for regular updates):

1. Facebook teaches language models to speak about 30 languages: These developments have been referred to as being “better than an equivalently sized GPT3 model…” To do this, Facebook trained a family of language models that are better at translation than GPT3. The XGLM ( an autoregressive language model) was trained on a mixture of 30 languages, varying from languages with a lot to very little data. They found that by training with more diverse language data than GPT3, the Facebook models performed better, especially when using ‘few-shot’ prompting. Few-shot prompting is where the model is fed a few examples of the target language, and then is asked to translate.
   
   
2. Facebook new AI supercomputer: Facebook introduced a new AI supercomputer to be used for AI research in what is called the AI Research SuperCluster (RSC). Facebook (Meta) believes it is one of the fastest AI supercomputers available. The use of this supercomputer will be for moderating the content on its platforms, creating new services for the metaverse and developing augmented reality tools.
   
   
3. Google’s big text model – LaMDA is a GPT-3 competitor which is part of a family of language models ranging in size from 2B to 137B parameters (GPT3: 175B), and that have been trained on a massive dataset of 1.56 trillion words. LaMDA is different from other big language models because it is centered around dialogue, with 50% of its data coming from “dialogues from public forums.” With this new capability, LaMDA can be used in a variety of useful applications that will be more practical and safe open-ended dialog, which is why we need people with the skills for the program to ensure the safety of these technologies.  
   
   
4. Microsoft makes MoEs easier to train: Microsoft is trying to scale-up mixture-of-experts (MoE) networks, which are one of the more promising routes for creating trillion-parameter-plus AI models. MoEs are more efficient to train than GPT3, which is a more dense model. MoEs work well for auto-regressive natural language generation tasks. With improved efficiency and training models, the result is great improvement in language accuracy. Beyond the efficiency, there is also a potential reduction in cost. Microsoft has stated that it can train 350M and 1.3B parameter MoE text models that have better (or the same) performance as GPT3 for a range of different tasks. What this means is that it nets out to models with the “same quality with 5X less training cost”. DeepSpeed-MoE offers massive MoE models with up to 4.5x faster and 9x cheaper inference when compared with quality-equivalent dense models.

NLP Applications

NLP has applicability in a variety of sectors of work and daily life.

• Sentiment analysis: NLP tools can “read” giant data sets to assess consumer brand sentiment and identify areas for improvement. It can detect negative comments as they are posted and can respond proactively.
• Chatbots: NLP streamlines communication in many areas of work and has been used to process customer requests because it is able to analyze the language and create a response to common requests. When a request is not understood, the message is sent to a human agent. There are many companies that are using virtual assistant technology to facilitate calls. For example, when you make a call to the electric company, airlines,  banks, or hotels, many of the most common requests can be responded to via chatbots or customers can manage their accounts using some of the voice commands. Companies often use the Alexa Skills Kit that is provided through Amazon.
• Reducing online bullying: detection of Cyberbullying is another way NLP is being utilized for good. On social media, classifiers are being developed to detect the usage of abusive and derogatory language, as well as hate speech.
• Fake news detection: While it may contribute to the problem, NLP has become a critical tool for detecting and reducing the spread of fake news and analyzing speech on social media. NLP is being used for automatic fake news identification this year, and this is an area we will likely see more of in 2022, with so much incorrect information about Covid-19 and other world issues circulating.
• World of Work: NLP is being used in various sectors such as finance, healthcare and human resources. In healthcare, NLP can analyze voice records and convert them into text to be added into patients’ records. NLP is being used to train algorithms on mental health disease which can then be used to provide cognitive behavior therapy or virtual therapy assistants to patients with PTSD, autism, or depression. In finance, NLP can extract data for credit scoring or combined with machine learning and predictive analytics to detect fraud. In human resources, NLP screens resumes and recruiting chatbots use NLP to schedule interviews, facilitate onboarding of candidates and more.

What NLP Means for Edu

Students should have an understanding of what AI is, the impact that it has on our lives and how it could shape the future–both good and bad. They should know that they are living through the first decade of widespread human-computer co-created content, art, and music–an amazing opportunity that will challenge historical views on intellectual property.  

Upper elementary school is a good time to teach digital literacy and critical consumption. Middle school is a good time to introduce AI ethics and begin the conversation about what it means to co-author with a machine.

High school should create opportunities for students to use machine learning tools to attack problems and create solutions. This could happen in a computer science course, a social studies and tech block, or an after school program (see AI4All).

And, keep an eye on student writing. Plagiarism detectors will be challenged by new content creation tools. Live performance assessment will remain the best way to determine if it was Robbie or robo-writer.

Finally, a real person wrote this post. But the next one you read, well, you may not be able to tell.