Skip to content

4. Large Language Models and Translation

Learning Objectives

After completing this section, you should be able to:

  • Explain how LLM-based translation differs from conventional NMT and distinguish between general-purpose and translation-specific LLMs.

  • Identify the main factors involved in selecting an LLM for translation, including language coverage, model purpose, computational requirements, and licence.

  • Design translation prompts incorporating relevant context, terminology, examples, constraints, and output requirements.

  • Understand how LLMs can be used for translation and related tasks and how open models can be deployed locally within a translation workflow.

Suggested Reading

To gain a deeper understanding of the use of Large Language Models for machine translation, we recommend the following resources:

  • Hendy, A. et al. (2023). How Good Are GPT Models at Machine Translation? A Comprehensive Evaluation.
    Read Paper on arXiv

  • Alves, D. M. et al. (2024). Tower: An Open Multilingual Large Language Model for Translation-Related Tasks.
    Read Paper on arXiv

  • Hugging Face. Model Cards. Documentation explaining the information provided in model cards, including model uses, limitations, training information, and evaluation.
    Hugging Face Model Cards

4.0. From Neural Machine Translation to Large Language Models

In Section 2, we worked with Neural Machine Translation (NMT) models such as OpusMT and NLLB. These models are specifically designed to translate text from one language into another. Given a source text and the appropriate source and target language settings, the model generates a translation.

Large Language Models (LLMs) represent a different approach. Rather than being designed exclusively for translation, general-purpose LLMs are trained to process and generate language and can perform a wide variety of tasks. Translation is therefore one of many possible tasks that can be carried out by the same model.

This difference can be represented in simplified form as follows:

NMT

source text → translation model → target text

LLM

instruction + source text + optional additional information → language model → generated response

The instruction is particularly important. With a conventional NMT model, the task is already defined: the system is expected to translate. With a general-purpose LLM, we normally need to tell the model what task it should perform. This is done through a prompt.

For example:

Translate the following sentence from English into Spanish:

The meeting will take place next Monday.

The same LLM could receive the same sentence with a completely different instruction:

Summarize the following sentence:

The meeting will take place next Monday.

or:

Identify the named entities in the following sentence:

The meeting will take place next Monday.

The model itself has not changed; what has changed is the task specified in the prompt.

LLMs and the Evolution of Machine Translation

The use of LLMs for translation has increased rapidly in recent years. General-purpose LLMs and models specifically adapted for translation are now being explored and used as complete translation systems or as components of more complex machine translation workflows.

The distinction between NMT and LLM-based translation should therefore not be understood as a simple replacement of one technology by another. Dedicated NMT systems continue to be widely used and can offer important advantages in terms of speed, computational efficiency, predictability, and deployment. LLMs, on the other hand, introduce additional possibilities, particularly because they can use instructions and contextual information and perform multiple language-related tasks within the same model.

In the previous sections, the translation workflow was represented as:

NMT model → MTUOC-server → CAT tool → translator

The same general architecture can be used with an LLM:

LLM → MTUOC-server → CAT tool → translator

However, as we will see throughout this section, deploying an LLM for translation involves some additional decisions, particularly regarding model selection, prompts, generation parameters, and output processing.

4.1. Large Language Models for Translation

Not all LLMs used for translation have been developed for the same purpose. A useful initial distinction can be made between general-purpose LLMs and translation-specific LLMs.

4.1.1. General-purpose LLMs

A general-purpose LLM is designed to perform many different language-related tasks rather than translation alone. Depending on the model and its training, these tasks may include:

  • text generation;
  • question answering;
  • summarization;
  • information extraction;
  • classification;
  • rewriting and correction;
  • programming;
  • translation.

Many models are also instruction-tuned, meaning that they have been trained to respond to instructions formulated by users. This makes it possible to specify a translation task through natural-language prompts.

For example:

Translate the following text from English into Catalan:

Machine translation systems are increasingly integrated into professional translation workflows.

The main advantage of this approach is flexibility. The prompt can contain considerably more information than the source sentence alone. For example, we can tell the model who the intended reader is, specify the domain of the document, provide terminology, indicate the desired register, or supply examples of previous translations.

A general-purpose model can also be used for several stages of the same workflow. The same model could first translate a text, then revise a translation, identify terminology, or perform other language-processing tasks.

This flexibility also introduces an important difference from dedicated NMT. Since a general-purpose LLM is not exclusively a translation engine, its behaviour depends strongly on the instructions it receives. The model may also generate explanations, comments, alternative translations, or other text that was not explicitly requested. Controlling the prompt and the format of the output therefore becomes particularly important when LLMs are incorporated into automated translation workflows.

4.1.2. Translation-specific LLMs

Other LLMs have been trained or fine-tuned specifically for machine translation. These models retain the general architecture of an LLM but have been adapted so that translation becomes one of their main functions.

Examples include models such as TowerInstruct and SalamandraTA.

Translation-specific LLMs may differ from general-purpose models in several respects:

  • the training or fine-tuning data may contain substantial amounts of parallel or translation-oriented data;
  • the model may support a defined set of translation directions;
  • it may expect instructions to follow a particular format;
  • it may have been optimised specifically for translation quality or translation-related tasks.

An important practical consequence is that we should not assume that the same prompt is appropriate for every LLM.

For example, a general-purpose instruction model may respond well to a natural-language instruction such as:

Translate the following text from English into Spanish:

A translation-specific model may recommend a different prompt structure or a specific set of language identifiers. The appropriate format is normally documented in the model's model card.

4.1.3. Model Cards

Open models distributed through repositories such as Hugging Face normally include a model card. Before downloading or using an LLM for translation, it is important to consult this documentation.

A model card may provide information about:

  • the purpose of the model;
  • the base model from which it was derived;
  • supported languages;
  • training or fine-tuning data;
  • recommended tasks;
  • prompt or chat templates;
  • model size and available variants;
  • hardware or software requirements;
  • licence conditions;
  • known limitations.

This information is particularly important for translation. The fact that an LLM is multilingual does not necessarily mean that all its languages or translation directions are equally well supported.

Similarly, the ability of an LLM to translate a language pair that was not explicitly included as a translation direction during fine-tuning does not guarantee that the resulting translation will be of sufficient quality. LLMs may exhibit zero-shot translation capabilities, that is, the ability to perform translation in language combinations or settings for which they were not explicitly trained, but performance needs to be evaluated for the intended use case.

4.1.4. Selecting an LLM for Translation

Before using an LLM as a translation system, several factors should be considered:

  • Model purpose: Is the model a general-purpose instruction model or has it been specifically optimised for translation?

  • Supported languages: Does the model explicitly support the source and target languages required for the project?

  • Model size: How large is the model, and can it be run with the available computational resources?

  • Prompt format: Does the model use ordinary natural-language instructions, a predefined translation template, or a specific chat format?

  • Licence: Under what conditions can the model be used, particularly in professional or commercial applications?

  • Computational requirements: Can the model be deployed efficiently on the available hardware?

  • Translation performance: Does the model provide adequate quality for the relevant language pair, domain, and type of text?

The choice of an LLM is therefore not simply a question of selecting the largest or most recent model. It involves finding a suitable balance between translation capabilities, language coverage, computational requirements, controllability, and the intended translation scenario.

In the following section, we will focus on one of the most distinctive aspects of LLM-based translation: prompting.

4.2. Prompting LLMs for Translation

When using a conventional NMT system, the input normally consists of the source text to be translated, together with information specifying the source and target languages when required by the model. With an LLM, the input can contain much more information.

A prompt is the instruction and information provided to the model to specify what it should do. For translation, a prompt may include not only the source text and the translation direction, but also information about the domain, intended audience, terminology, style, context, or desired output format.

Prompt design is therefore an important component of LLM-based translation. Different prompts applied to the same source text and the same model may produce different translations.

4.2.1. Basic Translation Prompts

For a general-purpose instruction-tuned LLM, a translation request can be expressed using a simple natural-language instruction:

Translate the following text from English into Spanish:

This software can be installed on Windows, Linux and macOS.

The instruction explicitly identifies:

  • the task: translation;
  • the source language: English;
  • the target language: Spanish;
  • the source text to be translated.

In many cases, this is sufficient to obtain a translation. However, general-purpose LLMs may return additional information, such as an introductory sentence, comments, explanations, or alternative translations.

For use in an automated translation workflow, we generally want the model to return only the translated text. The instruction can therefore be made more explicit:

Translate the following text from English into Spanish.
Return only the translation and do not add any explanations or comments.

This software can be installed on Windows, Linux and macOS.

Translation-specific LLMs may use different instructions. Some have been fine-tuned with a particular prompt template and are expected to receive requests in that format.

For this reason, before designing a prompt, always check the model card to determine whether the developers recommend a specific prompt or chat template.

4.2.2. Adding Context and Translation Instructions

One of the main advantages of LLM-based translation is the possibility of incorporating additional information directly into the prompt.

Consider a simple translation instruction:

Translate the following text from English into Spanish:

The mouse is not working properly.

Without further information, the model has to infer the meaning of mouse from the sentence itself. We can reduce this ambiguity by providing contextual information:

You are translating the user manual of a computer peripheral from English into Spanish.

Translate the following text:

The mouse is not working properly.

Return only the Spanish translation.

Context can be provided at different levels. A translation prompt may specify, for example:

  • the domain of the text;
  • the text type or genre;
  • the communicative situation;
  • the intended audience;
  • the purpose of the translation;
  • the desired style or register;
  • information from the surrounding text.

The prompt can also assign a role to the model:

You are a professional translator specialising in medical translation.

Translate the following text from English into Spanish.
Use appropriate medical terminology and maintain a formal register.

[source text]

Role instructions are not a substitute for relevant contextual or terminological information, but they can form part of a more detailed set of translation instructions.

The amount of information included in the prompt should be appropriate to the task. Adding more instructions does not automatically produce a better translation. Instructions should be relevant, clear, and consistent.

4.2.3. Terminology and Glossaries

Terminology can also be incorporated directly into a translation prompt. This is particularly useful when a project requires specific translations for specialised terms, product names, institutional terminology, or other expressions.

For example:

Translate the following text from English into Spanish.

Use the following terminology:
machine translation = traducción automática
translation memory = memoria de traducción
language model = modelo de lenguaje

Return only the translation.

[source text]

The glossary becomes part of the information available to the model when generating the translation.

This approach can be useful for:

  • maintaining preferred terminology;
  • adapting translations to a particular client or organisation;
  • improving consistency;
  • preventing unwanted translations of product names or other expressions.

Terminological instructions can also be formulated as explicit constraints:

Translate the following text from English into Spanish.

Always translate "machine translation" as "traducción automática".
Do not translate "MTUOC-server".

Return only the translation.

[source text]

However, terminology provided through a prompt is an instruction rather than a hard constraint. An LLM may occasionally fail to apply a requested term consistently. Terminological compliance should therefore be checked when this is critical to the translation task.

4.2.4. Providing Translation Examples

Another way of guiding an LLM is to provide examples of the behaviour we expect.

A prompt containing only an instruction and the new source text is commonly referred to as zero-shot prompting: the model is asked to perform the task without being given examples within the prompt.

If one or more examples are provided before the new input, we can use few-shot prompting.

For example:

Translate from English into Spanish following the style of these examples.

English: Click Save to continue.
Spanish: Haga clic en Guardar para continuar.

English: Select the file you want to open.
Spanish: Seleccione el archivo que desea abrir.

English: Restart the application after installation.
Spanish:

The previous translations provide the model with information about the expected terminology, register, and style.

Examples can be particularly useful when:

  • a specific translation style must be reproduced;
  • previous approved translations are available;
  • the desired behaviour is difficult to describe through rules alone;
  • consistency with existing translations is important.

Examples should be carefully selected. Incorrect, inconsistent, or irrelevant examples may guide the model in the wrong direction.

4.2.5. Positive and Negative Constraints

Prompts can specify both what the model should do and what it should not do.

A positive instruction might be:

Use formal Spanish.

A negative instruction might be:

Do not translate product names.

Several constraints can be combined:

Translate the following text from English into Spanish.

Use a formal register.
Use the terminology provided in the glossary.
Do not translate product names.
Do not add explanations, notes, or alternative translations.
Return only the translated text.

[source text]

Negative constraints can be useful when a model repeatedly produces unwanted behaviour. However, prompts should remain as simple as possible. If a constraint is unnecessary, it does not need to be included.

4.2.6. Controlling the Output

Output control becomes particularly important when an LLM is connected to another application.

In an interactive chat, a response such as:

Here is the Spanish translation:

Este software puede instalarse en Windows, Linux y macOS.

may be perfectly acceptable. In a CAT-tool workflow, however, the introductory sentence is unwanted: only the translation should be inserted into the translation environment.

A prompt can explicitly request:

Return only the translation. Do not add explanations, labels, quotation marks, or comments.

Another possibility is to ask the model to place the translation between predefined delimiters:

Translate the following text from English into Spanish.

Return the translation between <translation> and </translation> tags.

[source text]

The expected response would then be:

<translation>Este software puede instalarse en Windows, Linux y macOS.</translation>

The translation can subsequently be extracted automatically from the response.

This distinction is important when moving from interactive use of an LLM to integration into a translation workflow. A response that is useful to a human user is not necessarily suitable for automatic processing.

MTUOC-server can use prompts to define the task sent to an LLM and regular expressions to extract the required translation from the generated response. We will return to this issue when configuring LLMs for MTUOC-server in Section 5.

Translation is not the only task that LLMs can perform within a translation workflow. Because instruction-tuned LLMs can respond to different types of prompts, the same model can potentially be used for several tasks simply by changing the instruction and the information provided to it.

4.3.1. Automatic Post-editing

An LLM can be asked to revise the output of another machine translation system. In this case, the prompt can contain both the original source text and the machine translation:

Improve the following machine translation.

Source:
[source text]

Machine translation:
[machine-translated text]

Correct translation errors while preserving the meaning of the source text.
Return only the revised translation.

This use of an LLM is generally referred to as automatic post-editing (APE).

The model is not generating a translation entirely from scratch. Instead, it receives an existing translation and is instructed to identify and correct problems in it.

4.3.2. Error and Grammar Correction

LLMs can also be instructed to correct linguistic errors without performing translation:

Correct any grammatical, spelling, and punctuation errors in the following Spanish text.
Do not change its meaning.
Return only the corrected text.

[text]

Depending on the workflow, correction can be applied to source texts, machine-translated texts, or other content.

The distinction between correction and rewriting should be made explicit in the prompt. If the model is simply asked to "improve" a text, it may make stylistic changes that go beyond error correction.

4.3.3. Paragraph-level and Context-aware Translation

Conventional translation workflows often process text segment by segment. However, individual sentences may not contain all the information needed to resolve ambiguity, maintain lexical consistency, or choose appropriate references.

LLMs can accept larger amounts of contextual information. Instead of translating an isolated sentence, we can provide a complete paragraph or additional surrounding text.

For example:

Translate the following paragraph from English into Spanish.
Take the full paragraph into account when resolving references and lexical ambiguity.
Return only the translation.

[paragraph]

Alternatively, contextual sentences can be provided while clearly identifying which segment should actually be translated:

Use the following previous sentence as context:

[previous sentence]

Translate only this sentence into Spanish:

[current sentence]

The ability to use wider context is one of the potentially important advantages of LLM-based translation. However, providing more context also increases the amount of input processed by the model and does not guarantee that the model will use all contextual information correctly.

4.3.4. Named Entity Recognition and Other Supporting Tasks

LLMs can also perform tasks that support translation without directly generating the final translation.

For example, a model can be asked to identify named entities:

Identify all person names, organisations, and locations in the following text.

Return one entity per line using this format:

entity | type

[text]

Other possible supporting tasks include terminology extraction, text classification, summarisation, and information extraction.

The important point is that an LLM can potentially perform different operations within the same translation workflow. The model remains the same; the task changes according to the prompt.

4.4. Running Open LLMs Locally

LLMs can be accessed through remote services, but open models can also be downloaded and executed on local hardware. Local execution is particularly relevant to the MTUOC approach because it allows translation technologies to be integrated into professional workflows while maintaining greater control over models and data.

4.4.1. Remote and Local LLMs

A remote LLM is normally accessed through an API. The source text and instructions are sent to an external service, which runs the model and returns the generated response.

This approach has several advantages:

  • no local model installation is required;
  • the computational resources are provided by the service;
  • powerful models can be accessed from relatively modest computers.

However, it also means that data are transmitted to an external service and that access may depend on connectivity, service availability, pricing, and provider conditions.

With a locally deployed LLM, the model weights are downloaded and inference takes place on hardware controlled by the user or organisation.

Potential advantages include:

  • greater control over data;
  • the possibility of processing data without sending them to an external provider;
  • greater control over the model and its configuration;
  • independence from external API availability and pricing.

Local execution also has costs. LLMs can require substantial amounts of RAM, GPU memory, storage space, and computational power.

The choice between remote and local execution therefore depends on the translation scenario, available hardware, data requirements, and the characteristics of the model.

4.4.2. Finding Open LLMs

Many open LLMs are distributed through model repositories such as Hugging Face.

A model repository normally contains the model files together with documentation describing how the model should be used. As discussed in Section 4.1, the model card should be consulted before selecting and downloading a model.

Particular attention should be paid to:

  • supported languages;
  • intended tasks;
  • recommended prompt format;
  • model size;
  • available model variants;
  • software dependencies;
  • licence;
  • hardware requirements.

The same model family may be available in several versions. Selecting the appropriate version is therefore part of the deployment process.

4.4.3. Model Size and Computational Requirements

LLM names often include an indication of the approximate number of parameters in the model. For example, a model described as 7B contains approximately seven billion parameters.

In general, increasing the number of parameters increases the amount of memory required to load and run the model. However, model size alone does not determine translation quality: training data, model architecture, fine-tuning, supported languages, and the translation task itself are also important.

Models can also be distributed using different numerical representations. Quantisation reduces the precision used to represent model parameters, making it possible to reduce memory and storage requirements.

For example, a quantised version of a model may be substantially smaller than the original full-precision model and may therefore be usable on hardware on which the original model could not be loaded.

Quantisation involves a trade-off between:

  • memory requirements;
  • storage requirements;
  • inference speed;
  • model quality.

For the purposes of this course, the important practical point is that the same model may be available in variants with very different hardware requirements. When selecting a model for local translation, you should therefore check not only the model family and parameter count but also the specific version you intend to run.

4.4.4. Tools for Local LLM Execution

Several frameworks and applications can be used to run open LLMs locally. Some work directly with model files distributed through Hugging Face, while others simplify model downloading, quantisation formats, model loading, and inference.

Regardless of the specific framework, the basic process is similar:

  1. identify an appropriate model;
  2. check its requirements and documentation;
  3. download the required model files;
  4. load the model using a compatible framework;
  5. provide the prompt and source text;
  6. generate and process the response.

This is conceptually similar to the workflow followed with OpusMT and NLLB in Section 2, where we first identified and tested a model before deploying it through MTUOC-server.

With LLMs, however, the prompt and the processing of the generated response become additional components of the deployment configuration.

4.5. Preparing an LLM for a Translation Workflow

The general architecture introduced earlier in this section can be used to integrate LLMs into professional translation workflows. However, deploying an LLM involves several additional elements that need to be configured:

  • Model selection: choosing a model that supports the required languages and translation task.
  • Prompt definition: specifying how the translation request will be formulated.
  • Generation parameters: controlling aspects of text generation where appropriate.
  • Output format: determining how the model should return the translation.
  • Output extraction: removing any additional content and extracting the text that should be sent back to the client application.

The processing workflow can therefore be represented more explicitly as:

source segment → prompt construction → LLM → response processing → MTUOC-server → CAT tool

This is particularly important in professional translation environments. The CAT tool expects a translation for the source segment it sends to the server. Any explanations, labels, comments, or other material generated by the LLM need to be prevented through prompting or removed before the response reaches the CAT tool.

In Section 5, we will put these concepts into practice by integrating open LLMs into MTUOC-server. We will work with different types of models and examine how model-specific prompts and configuration options are incorporated into the translation workflow.

Key Takeaways

After completing this section, you should remember that:

  • LLMs provide a different approach to machine translation from conventional NMT: they can perform translation as one of many language-related tasks and may be general-purpose or specifically adapted for translation.

  • Selecting an LLM for translation requires considering its intended purpose, supported languages, translation performance, model size, computational requirements, licence, and documentation.

  • Prompting is a central component of LLM-based translation and can incorporate context, terminology, examples, constraints, and output instructions to guide translation behaviour.

  • LLMs can support translation and related tasks and can be accessed remotely or run locally; when integrated into professional workflows, prompts and generated responses must be carefully controlled and processed.