Glossary

LLM training data explained

LLM training data comes in three kinds, for three stages of building a language model, and each raises different rights questions for sellers.

By

Updated

Definition

LLM training data is the text, code and human feedback used to build a large language model: broad material for pretraining, worked examples for fine-tuning, and ranked or rated responses for teaching the model which answers people prefer.

Three stages, three kinds of material

People often talk about training data for language models as if it were one pile of text. In practice labs buy and build different material for different stages, and the stage decides what a seller can offer.

StageWhat the material looks likeWhat the model gainsWhat makes it valuable
PretrainingVery large amounts of general text and codeGrammar, facts, reasoning habits, programming syntaxBreadth, quality and material that is not already public
Fine-tuningCurated examples of a prompt and a good responseHow to follow instructions or handle a specialist taskExpert authorship and accuracy
Preference dataSeveral answers to one prompt, ranked or scored by peopleWhich answers people find helpful or safeCareful, consistent human judgement

What sellers can add

Much pretraining text comes from public web archives. Common Crawl, for instance, has been collecting web pages since 2008 and makes its archive available at no charge. Because labs can already reach that material, what sellers can add is usually text that never reached the open web: archives, manuals, transcripts, internal documentation and code.

Mistakes sellers make

  • Assuming they own everything on their servers. User posts, licensed photos and contractor work often belong, in part, to someone else.
  • Leaving personal data in. Names and contact details in support chats or emails need removing or a lawful basis to share.
  • Ignoring duplicates. If half the archive is already on the public web, the buyer may value only the other half.
  • Granting unlimited rights. Training, evaluation, redistribution and use in outputs can be licensed separately, and each has a different value.

Sources

3 sources · checked Nov 7, 2026

  1. Common Crawl: OverviewThat the archive has been gathered since 2008 and is free to use · Checked
  2. US Copyright Office: Copyright and Artificial IntelligenceThe May 2025 pre-publication report on generative AI training, its fact-specific fair use view and its stance on licensing · Checked
  3. EU AI Act: Article 53Copyright policy and public training-content summary duties for general-purpose model providers · Checked

LLM training data FAQ

Where do LLMs get their training data?

From a mix of public web archives, books, code repositories, licensed collections bought from publishers and companies, and material written or rated by paid human experts. The mix differs by lab and is often only partly disclosed.

Can I sell data to AI companies for LLM training?

If you own it and it is not already freely available, possibly. Labs look for text, code or expert examples they cannot easily get elsewhere, with clear rights and personal data removed.

What format should LLM training data be in?

Clean text is the usual baseline, with headings and structure kept and page furniture removed. Fine-tuning sets are usually supplied as prompt and response pairs in a simple structured file format the buyer specifies.

Is web-scraped text legal to use for LLM training?

That is being argued in courts and policy reports in several countries. The answer depends on the jurisdiction, the material and how it is used, so licensed material with clear rights is easier for buyers to rely on.

Related

  • AI dataset

    An AI dataset is the material a model learns from: what it contains, how it is organised, and what buyers check before licensing one.

  • Data licensing

    Data licensing lets a buyer use a dataset under written conditions while the seller keeps control, which is how most data deals are structured.

  • Data ownership

    Data ownership is about who can lawfully control, use and license a dataset, which matters far more in a deal than who holds the file.

  • Data monetization

    Data monetization is the umbrella term for every way an organisation turns its data into money, inside the business or outside it.

  • Synthetic data

    Synthetic data is generated rather than collected, and labs now use it heavily; this page covers where it replaces real data and where it can't.

Join the DataBroker academy

The DataBroker academy and tools: find companies with data to sell, price it, license it and close the deal.