A
C
Context Window
See also: AI Context, Token.
The context window is the amount of data an AI can process at once. It consists of its input, as well as any output it generates in response to that input.
Context windows can range from 4 thousand all the way to 2,000,000+ tokens, depending on the large language model (LLM) you are using.
Whatever the context window is, it is important to remember that the AI can only process a limited amount of information at once. If you exceed the context window, the AI will not be able to process the information correctly.
Some providers may also truncate the context window (usually dropping information from the middle) if it exceeds the maximum limit.
F
Fine Tune
Also known as: Fine Tuning.
Fine Tuning is a process where you take a pre-trained model and train it further on a specific dataset to improve its performance on a specific task, like making it sound more like your writing style.
The process involves presenting a model with training data (inputs and desired outputs) that results in a custom set of model weights that are introduced on top of the base. This allows the model to learn the nuances of the specific dataset and improve its performance on that dataset.
Due to the nature of fine-tuning, it is recommended to have a large dataset to train on, as well as a good understanding of the model you are fine-tuning. This also means that fine-tuning can be computationally expensive and time-consuming.
L
Large Language Model
Also known as: GPT, LLM.
A large language model (LLM) is a type of artificial intelligence that can generate human-like text. It is trained on a large corpus of text data and can generate text in response to a prompt.
LLMs are used in a variety of applications, including chatbots, content generation, and language translation. Some popular LLMs include OpenAI’s GPT’s and Anthropic’s Claude.
Its architecture is analogous to the fine neural structures inside our brains, but it operates digitally on numbers rather than the electrochemical reactions of human brains.
M
Message Role
Also known as: AI Message, System Message, User Message.
The message role is the part of the conversation with an AI model, and indicates the role that the message plays. There are three types of message roles:
System Message
Usually the first message in a conversation.
These set up the context for the AI and provide the AI with the necessary information to generate a response. You often specify the job or task you want the AI to perform in a system message.
However, some AI models don’t support a system message.
User Message
These are the messages that the contain the user’s input and instructions for the AI.
AI Message
These are the responses generated by the AI based on the user’s input and the context provided in the system message.
The AI message is the output of the AI model, but they can also be used to “prefill” the response of the AI (like putting words in the AI’s mouth) to guide the AI in a certain direction.
Moderation
Also known as: Content Moderation, External Moderation, Internal Moderation. See also: NSFW.
Some makers of AI models don’t want you writing anything that goes against their terms and conditions. To ensure you can’t break these, the manufacturers add in layers of moderation.
External moderation is seen when using (for example) the GPT models by Open AI - If moderation fails, you will get an error message.
This is done before the any of your messages are sent to the model, so you won’t be charged for these requests.
Internal moderation is built into the models by the makers, and may result in the model telling you that it is uncomfortable, or no message being returned.
You will still be charged for these messages as they have been sent to the LLM and required processing.
T
Temperature
A parameter for AI models that determines how coherent or crazy/creative the model is.
When the model decides on the next piece of text, the temperature value controls a bit of randomness. The naming can be thought of as how hot (chaos and bubbles) or cold (frozen and static) the model is.
Usually, the nearer to zero the temperature is, the less creative it is and for some models the closer it follows your instructions.
Token
Also known as: Input Tokens, Output Tokens.
Tokens are sequences of characters that LLMs convert into numeric representations, enabling the numeric processing of language at the core of large language models.
Tokens can sometimes be entire words, but more often are pieces of words. For a general rule of thumb, it takes 100 tokens to encode 75 words for the English language.
There are multiple types of tokens:
Input Tokens
The tokens used in your prompt that are sent to the AI for processing.
Output Tokens
The tokens that the AI generates as a result of your prompt.
Thinking Tokens
Some AI models (like Open AI’s o1) have “thinking” tokens that are used for internal processing. These tokens cannot be seen and are not part of the output tokens.
Some AI providers/models might also differentiate between image inputs, cached inputs (for example, when you use the same prompt multiple times), and other types of tokens.