State of the Art in AI

Original Editor - Michael Rowe

Top Contributors - Jess Bell and Stacy Schiurring  

Introduction

"If we want to embed AI [artificial intelligence] in society, we need to understand what it is. [...But defining] AI is not easy; in fact, there is no generally accepted definition of the concept."

AI is rapidly evolving, making it difficult to define. However, in basic terms, it can be described as a technology that allows machines to perform or simulate a number of complex tasks that would typically require human intelligence.

As AI develops and increases in competence, it is being used in more and more areas, including healthcare, rehabilitation and education. Many of us aren't aware of its capabilities or how to harness its potential. This page provides a general overview of the capabilities of Generative AI and simple prompt-writing strategies to help you get more out of AI models.

Generative AI

Generative AI (GenAI) is a type of AI that can create new content. GenAI models, such as Claude, ChatGPT, Preplexity, Gemini, etc., are next-word-predictors - i.e. they use patterns learned from their training data to anticipate and generate the most probable next word in the sequence.

Capabilities and advances in GenAI:

• GenAI is multimodal: this means it can operate across different media, translating between text, audio, image and video
• GenAI is increasing in competence and can handle tasks of greater complexity
• GenAI is everywhere: it is embedded in many devices and platforms, including operating systems, phones, software, etc.
• GenAI context windows are getting larger (see below)

Context Windows

GenAI models have context windows. A context window is the amount of data (or tokens) an AI model can "remember" when generating a response. In essence, it is GenAI's short-term memory. As context windows get larger, the AI model can handle more complex tasks and provide more accurate responses to increasingly complex prompts.

Please watch the following video if you would like a very brief summary of GenAI:

Frontier Models

Frontier models are the most advanced AI systems available. They are at the cutting edge of current technological capabilities in AI. As of September 2024, current frontier models are:

• Claude 3.5 Sonnet
• ChatGPT-4o
• Gemini 1.5 Pro
• Llama 3 (405B)

The capabilities of each model will change with time and new versions will be released. It is recommended that you experiment with different AI models to determine which models best suit your needs.

Large Language Models

Large language models (LLMs) are AI systems that are “trained” on vast amounts of data (e.g. books, websites, research articles, etc.). They learn to predict the next word in a sequence to generate new text, answer questions, and assist with different tasks, including creating images, videos, computer code, etc.

LLMs are becoming more capable: “... as the models expand in size and linguistic proficiency they increasingly display human-like intuitive system 1 thinking and associated cognitive errors”

Please note that system 1 thinking is automatic, quick, and can have errors whereas system 2 thinking is slow, effortful thinking that tends to be reliable.

Some important considerations:

• we may be approaching a hard limit to the available data for training
• available synthetic data (i.e. data produced by language models) is increasingly used to train language models
• there are high costs associated with training LLMs

Small Large Language Models

Small LLMs are running on edge devices - i.e. devices that don’t require an internet connection and are optimised for low-power mobile devices like phones and laptops. Examples include:

• Phi (Microsoft)
• Apple Intelligence

Mixture of Experts

AI architecture includes many specialised sub-networks or "experts". For each input or task, only a subset of these experts is activated. This allows the system to allocate resources more efficiently and helps to avoid redundancy by breaking complex tasks into smaller subtasks that are handled by specialised experts rather than a single large model.

For example, if an AI model is being used to help diagnose a medical condition, it would only activate experts specialised in analysing medical data.

GenAI vs Search

“Distinct from traditional AI systems, which are typically rule-based or rely on predefined datasets, generative AI models possess the unique ability to create new content that is original and not explicitly programmed.”

GenAI is not the same as an internet search:

• there is no ‘ground truth’ for LLMs, and no a priori model of the world they refer to - they don't have a built-in understanding of the world or a database of facts to reference
• responses are not retrieved from a database; they are generated, one word at a time, using the prompt as a starting point
  • when responding to a prompt, LLMs essentially create content word by word, based on statistical probabilities of what words tend to follow each other in certain contexts - they are not accessing a pre-existing "truth" or set of facts, but rather constructing responses based on learned patterns (this is why language models can sometimes produce incorrect or inconsistent information)
• it is difficult to determine the exact source of the data used by GenAI
• context windows are essentially GenAI’s short-term memory: larger context windows = more complex conversations

Writing Prompts for GenAI

Writing prompts is also different to keyword searches. The contextual richness of the prompt is an important indicator of the quality of the generated output.

The “Role. Goal. Instruct. Discuss” approach is a simple strategy to enhance your prompt writing. It provides context that guides GenAI to deliver more meaningful and specific responses. For example, "You are an experienced physiotherapist. I am a new graduate. I need to understand the physiotherapy management of osteoporosis."

Naive vs structured prompts:

• a naive prompt is very basic: e.g. how do you manage a patient who is day one post-total knee joint replacement?
• a structured prompt contains more contextual detail: e.g. "I am a novice physiotherapist who is managing a 73-year-old male patient who is day one post-total knee joint replacement. You are an experienced orthopaedic physiotherapist. I’m struggling with all aspects of managing this patient. Please help me develop a comprehensive plan. Start with the interview, then move on to the physical exam. Be detailed in your responses."

Your first prompt and the response it generates is not the end of the interaction, but rather a starting point. "That discussion, that back and forth interaction with the language model tends to generate really interesting, useful options." -- Michael Rowe

If you would like to learn more about writing prompts for AI, please see: Physiopedia AI Assistant Prompt Writing Guide.

The following, optional video also provides a general overview of AI, its capabilities and how to use it effectively. It also explores the art of prompt design:

Unanticipated Consequences

AI systems have demonstrated emergent behaviour or abilities at scale. In AI, emergent behaviour is defined as: "complex behavior that arises from the interaction of simple rules or elements, without any explicit programming for the resulting behavior". They have been able to create meaning from structure and semantics from syntax.

Some other surprising observations about AI include:

• using meaningless fillers (e.g. “…”) can sometimes produce better answers
• different personas (e.g. fleet commander vs spy in a political thriller) might get better results for different types of maths questions

It's important to remember that these findings were created in very specific situations and they probably won't scale to all prompting scenarios. But they highlight some of the unexpected / unusual features of GenAI.

Summary

GenAI is rapidly evolving and new features will continue to be released. It is essential to understand the capabilities and limitations of GenAI models to effectively harness their potential. Writing structured prompts that provide contextual details, rather than relying on basic, naive prompts will help you get more out of GenAI.

Additional Resources

• Physiopedia AI Assistant Prompt Writing Guide
• Hagendorff T, Fabi S, Kosinski M. Human-like intuitive behavior and reasoning biases emerged in large language models but disappeared in ChatGPT . Nat Comput Sci. 2023 Oct;3(10):833-838.
• Mollick E. Doing stuff with AI: opinionated midyear edition. One useful thing blog.
• Mollick E. Change blindness. One useful thing blog.
• Mollick E. Co-intelligence: living and working with AI. First edition. WH Allen, 2024 (not freely available).