What is data labeling?

A model learns by comparing its guesses with the right answers. Labels are those answers. AWS describes the labelled set as the ground truth that a model's accuracy depends on, and gives examples by data type: boxes or pixel outlines around objects in images, sentiment, intent and named entities in text, and transcripts and tags for audio.

Labels for language models

Language models brought new kinds of label. In OpenAI's InstructGPT work, people wrote demonstrations of the desired answer to a prompt, used to fine-tune the model, and then ranked several model outputs from best to worst, used to train it further with reinforcement learning. A written answer or a ranking is a label too, just a more expensive one.

The second row is often overlooked. Many business records carry labels already, because people recorded what happened: the fix that worked, the diagnosis that was confirmed, the invoice that was disputed.

Outcomes already in the records

That pairing of input and real outcome can be worth more than labels added afterwards, because nobody had to guess. It can also serve as an evaluation set, a test with known answers.

• People label each item, often with several labelers per item and their answers combined, which AWS calls labeler consensus.
• A model labels the items it is confident about and sends the uncertain ones to people, whose answers then improve the model. AWS describes this as human-in-the-loop or active learning.
• Labels come from the records themselves, when the system that produced the data also recorded an outcome.

What buyers will ask

Whichever method is used, buyers will ask how it was done. The EU AI Act's Article 10 lists annotation, labelling, cleaning, enrichment and aggregation among the data-preparation steps that providers of high-risk AI systems must account for, and asks that training, validation and testing sets be relevant, representative and, as far as possible, free of errors.

Usually not until a buyer says what it wants. Labels only help if they match the buyer's task and categories, and a lab with its own scheme may relabel everything anyway.

Spend first on what every buyer needs: clear rights, removal of personal data, consistent formats and documentation of what each field means, which is closer to cleaning and enrichment than to labeling. Then value the dataset as it stands and ask buyers whether labels would change the price.

DataBroker doesn't label data, run a labeling service or offer labeling work. We help brokers and data owners decide what a dataset is worth as it stands, whether existing outcome fields make it more valuable, and how to license it. If you need a labeling vendor, look elsewhere.