Generative text AIs and language models

The LSTM (Long Short-Term Memory) method is a piece of software that has been around since 1995 and is used to recognise certain patterns in data. This enables the software to remember previous information and reuse it when necessary.  

For this technology to work, LSTM uses a special neural network called a recurrent neural network (RNN), which works with repetitive processes. This means that the software is able to process information step by step.

However, LSTM also has its limitations. Since it processes information sequentially, it cannot handle large amounts of data well and tends to overlook more distant information.

In 2017, Google presented the scientific paper ‘Attention is All You Need’(opens in new tab). In it, the authors suggest that new attention mechanisms can significantly increase the quality and efficiency of previous models for neural networks:

The research team proposes a ‘simple network architecture, the Transformer, based exclusively on attention mechanisms [...]’(opens in new tab). This Transformer uses novel mechanisms to better understand context and process it more efficiently. Experiments were conducted with machine language translation tasks.

The Transformer has significantly advanced the development of generative text AI. Because it can better handle long dependencies in texts and be trained more efficiently, it is ideal for tasks such as text generation, chatbots and other applications that need to generate or understand natural language. No wonder, then, that all well-known text AIs today are based on the Transformer architecture.

What exactly do you want to know? If you ask vague questions, the AI will give vague answers. But you want a specific and useful answer for your exact use case, right?

What writing style should the AI use to respond? It's best to assign it a role so that it strikes the right tone, for example: ‘Respond as a teacher’ or ‘Explain in a way that a 10-year-old child can understand’.

Tip for writing tasks: Have the AI perform a style analysis of an existing text in advance so that it can train itself to use the desired style. Only then should you give it the actual task you want it to answer – in the style it has now been trained to use.

In what form should the answer be presented? As continuous text with subheadings, or as a table with columns for “Places”, “Sights” and “Restaurants”? Or perhaps even as a graphic or code? Tell the AI.

An AI usually understands and responds better if you give it only one clear task per prompt. If you want to know more after that, keep asking questions. During the conversation, the AI gradually gains an understanding of the entire context and you can have all your questions answered one by one.

For more complex tasks, it can also help to ask the AI to explain things to you step by step or to outline its thought processes.

If you want to receive information in a specific structure, subtopics at a certain altitude, or in a predefined style, provide the AI with an example in your prompt so that it understands you. This will save you time-consuming explanations and make it easier to get better results.

Chatting with a language model always means receiving answers that are not very helpful. The AI may also misunderstand you sometimes. That can happen. Rephrase your question; perhaps you can divide the task into two steps? Talk to the AI and tell it when and what it has misunderstood and what it should do instead.

A language model works by always finding the most probable next words or answers. However, this does not necessarily mean that this information is correct. Most language models therefore issue a justified warning (usually near the input field): ‘I may make mistakes. Please check my answers.’ Please adhere to this.

Not only can AI integrate metaphors into its responses, it can also vary the tone and pitch of an explanation. Use this to have a topic explained to you from multiple perspectives.  

Example: Answer the question ‘Should smartphones be allowed in class?’ from the perspective of a teacher, a 10-year-old pupil and his mother.

Not only can AI integrate metaphors into its responses, it can also vary the tone and pitch of an explanation. Use this to have a topic explained to you from multiple perspectives.

AI can also simulate a discussion by taking both sides equally and finding both arguments and counterarguments for a thesis and bringing them into the conversation.

Example: Simulate a discussion between two media experts on the thesis: ‘Teaching critical media literacy in schools is more important than teaching technical skills.’ Expert A takes the position that critical thinking forms the basis on which technical skills are built. Expert B argues that without sound technical knowledge, critical thinking and media literacy are not possible. Structure the discussion as a dialogue with 3-4 contributions per person. Take into account current challenges such as AI-generated content, disinformation and algorithmic filtering. Both experts should contribute arguments and examples from educational practice.

Although AI has no feelings and sometimes no opinion of its own, it can still be a great partner for brainstorming or feedback loops. Ask the AI to engage in an iterative process and exchange ideas on your topics digitally, just as you would with a team member.

Example: Prepare the AI for the exchange by informing it: ‘I am going to present an idea to you. You will then give me feedback on how I can develop and optimise it. I will then improve my idea. We will repeat this process three times.’

AI can slip into roles and bring stories or fictional scenes to life for you. This type of prompting is ideal for bringing historical figures or situations to life.

Example: Simulate a conversation between Marie Curie and Albert Einstein in 1925 about the future of physics. Marie Curie should defend her research on radioactivity, while Einstein asserts the importance of his theory of relativity. Take into account the state of knowledge at the time and the social circumstances for scientists, especially for a woman in science.

You can also ask the AI to rate its own answer on a scale of 1–10. It should also explain what it could have done better. This helps you to identify and classify the strengths and weaknesses of the answer.

For example, if you want to plan a trip and want the generative AI to suggest sightseeing tips or routes, ask the AI in advance what it needs to know about you in order to plan optimally. Answer all questions individually, in a structured manner, and provide all necessary details (no personal data!). This will help you optimise the suggested trip so that it matches your travel dates, preferences and budget as closely as possible.

Prompt schemata (known as frameworks) such as TIDD-EC support structured prompting. TIDD-EC is composed of the initial letters of its components: 

• Task Type: Give the language model a clear idea of what type of task it should perform.Instructions: Describe specific steps or guidelines for the language model to follow.
• Do: What actions will help the AI perform the task?
• Don't: What elements should the AI avoid when responding to the prompt?
• Examples: Provide concrete examples of the desired type of results or responses.
• Content: Provide specific data (note: only general data, no personal data!) that the AI can work with, use in its response, or refer to.
  

Example: ‘Explain to me in simple terms the health benefits of green tea. Refer to scientific findings and current research results. The answer should be easy to understand and not contain any complicated technical terms. An example would be: “Green tea can help strengthen the immune system.”’