GLTR: A Tool to Detect Machine-Generated Text

The quick advancement of language models also increased the possibility of high abuse of text generation models. They can be used to spread hate comments and other malicious requirements. This created the urge to develop a model which can detect automated text. Giant Language model Test Room (GLTR) is a tool that helps to do that.

In this article, we will discuss what is GLTR and how does GLTR detect automated text generation in detail?

How Do Language Models Generate Text?

In our previous article on ChatGPT, we discussed what language models and the GPT language model are. The huge development of the language processing community led to the creation of larger language models with advanced features.

A language model is trained in such a way it can predict the next word when an input is provided. It is a machine-learning model which creates text by producing one word at a time, and to a great extent, language models have the capability of generating text which cannot be distinguished from human-written text to an inexperienced reader.

Language models are trained with large datasets, which helps them to accurately calculate and choose the words that follow in a provided context. When a language model generates text like that, it will look much more likely to be written by humans in the same context despite not having much knowledge of the situation. This feature is highly abused when malicious users use it to generate fake comments, reviews, or even news to stir the public’s point of view.

So, to prevent this abuse from happening, it was necessary to develop a tool that can detect machine-generated text from human-written text. The technique behind this technology is assuming that machine-generated text will have a certain set of words at each position predefined; however, when a human writes, there will be more unpredictable words regarding the context. GLTR can help to a great extent to identify this. This tool is a very visual tool that helps to detect automated text.

What is GLTR?

GLTR was developed by a group of US researchers from Harvard University and the Massachusetts Institute of Technology (MIT). GLTR uses the GPT-2 117M language model developed by OpenAI, and it is one of the largest open-source language models. This tool was developed to filter out inauthentic journalism to weed-out fake information. GLTR can detect generation artifacts by applying baseline statistical methods across common sampling schemes. The arrival of GLTR improved the detection rate of artificially generated text from 54% to 72% without any initial training. The best part is GLTR is open-sourced and publicly available.

GLTR Also has the ability to differentiate fake profiles on social media platforms such as Facebook, Twitter, etc., as this can be used to spread false information.

How does GLTR detect automated text generation?

GLTR relies on the unpredictability of human writing since computer-generated text has a pattern or a sequence while creating content. Since GLTR uses GPT-2 as the language model, It can use any input and analyze what words will be predicted by GPT-2 in each position. GLTR results have color coding for ease of understanding. GLTR is aware of the output generated by the language model, which allows it to rank all the words that are known to the language model.

This ranking of words will be used as positional information and will be color-coded over each text that is matching to the position in the ranking. The most likely word will be colored green for the top 10 ranked words, yellow for the top 100, and red for the top 1000 ranked words rest of the words will be marked in purple. This gives a visual representation of how the text looks and how likely was the usage of words.

To test this, let us use a text from this article itself.

Example of the human written text

We can see the colors purple and red appearing, which means this text contains more ‘surprising’ words than the words generated by a machine. Hence this text is more likely to be considered written by a human because of the unpredictable texts.

Three histograms are provided by GLTR that shows the aggregate information over the whole text. The first one shows how many words in each color-coded category appeared in the overall text.

The second histogram represents the ratio between what is the probability of the top predicted word and the word following it. The last histogram shows how the entropies are distributed in the prediction. That low uncertainty implies the language model is surer of the words predicted, and high uncertainty shows uncertainty in words. This tells that uncertainty is generally higher in human-created text.

In the above example, all the histogram shows that the text was not likely to be created by a machine.

Let us also test a machine-generated text and see the results. Below is a text search from an essay generated by Chat GPT about India.

Example of computer-generated text

Here we can clearly see the lack of red and purple colors in the analysis result by GLTR, which indicates the text is highly predictable and hence have a high possibility of being machine-generated.

We can access and analyze a text for free using GLTR from here.

Conclusion

GLTR has its own limitations, and it’s not perfect when it comes to identifying fake text. Another main limitation is this tool cannot be used in large-scale abuse detection but only for individual cases. But overall, GLTR is an awesome technology that is very helpful. I hope this post helped in understanding what GLTR is and how GLTR detects automated text generation. And how you should detect machine-generated text. Visit our social media page on Facebook, LinkedIn, Twitter, Telegram, Tumblr, Medium & Instagram, and subscribe to receive updates like this.