Guide

Data sourcing: how companies get the data they need

Data sourcing means finding, assessing and obtaining outside data, through one of five routes that differ in cost, speed and legal risk.

By

Published

The short answer

Data sourcing is the process of identifying the data a project needs and obtaining it, usually from outside the company. There are five main routes: buy a listed product, license directly from the holder, partner and share, scrape public sources, or collect it yourself. Each trades speed against cost, control and legal risk, so start from the use case.

What data sourcing covers

Neudata, which researches alternative data for investment firms, describes data sourcing as the process organisations use to identify external datasets and build them into their work.

In practice the word stretches a little wider: it covers deciding what you need, finding who holds it, checking that it is good enough and lawful to use, agreeing terms and getting it delivered.

Other names for the same work

Procurement teams often call the same work data procurement, and engineers sometimes say data acquisition, although that phrase also names sensor and measurement hardware.

Start with the question

The order matters. Teams that start with a list of vendors tend to buy data they never use. Teams that start with one question the data must answer can judge every option against it: does this dataset cover the right period, region and population, at the right freshness, under terms that allow the use we have in mind?

The data sourcing process map

Most data reaches a company through one of five routes. The costs below are described rather than priced, because they depend on the dataset, the volume and the negotiation. Where a source publishes a real figure we give it.

Compiled from the sources listed at the end of this page, checked 2026-11-09.
RouteHow it worksWhat it costsMain riskSource
Buy a listed productSubscribe to a dataset on a marketplace or exchange under standard termsThe listing price; on AWS Data Exchange the seller pays a 3% listing fee on public offers, which is built into what you payStandard terms may not cover your use; the platform doesn't audit where every row came fromAWS Marketplace listing fees
License directlyNegotiate a data licence with the company that holds the dataA negotiated fee plus legal and diligence time on both sidesSlow; the holder may not have the rights it claims, so warranties matterNeudata; ICO via Skadden
Partner and shareSwap or pool data with another organisation, sometimes through an intermediary or industrial data platformOften little cash, but governance, integration and legal set-upUnequal benefit, unclear ownership of combined data, exit termsCDEI report on data intermediaries
Scrape public sourcesCollect pages or records from websites with your own crawlersEngineering, infrastructure and ongoing maintenance as sites changePersonal data, site terms and copyright; UK regulator expects you to show licensing wasn't a workable alternativeICO via Skadden
Collect it yourselfGather first-party data from your own users, products or sensorsThe slowest route: product changes, consent design and time to build volumeUsing customer data for a new purpose without clear notice; the FTC warns that quietly changing terms may be unfair or deceptiveFTC, February 2024

Steps in a data sourcing project

A sourcing project starts with the question and ends with terms; the sample test comes before rights work.
  1. 1

    Write down the question

    Name the decision or model the data feeds, the fields you need, the coverage and the refresh rate. This is the brief every seller will be judged against.

  2. 2

    Check what you already hold

    Internal logs, CRM records and past purchases often cover part of the need. Buying what another team already licensed is a common waste.

  3. 3

    Map candidate holders

    List who is likely to collect this data as a by-product of their business, which marketplaces carry related products, and whether a partner would share. Many useful datasets are never listed anywhere.

  4. 4

    Test a sample

    Ask for a sample or a trial period and check it against your question: coverage, gaps, duplicates, how fields are defined and how often they change.

  5. 5

    Check rights and provenance

    Ask how the data was collected, under what notice or consent, and whether the seller may license it for your use. A written record of origin, in the spirit of a datasheet for the dataset, saves arguments later.

  6. 6

    Agree terms and plan delivery

    Settle permitted use, term, refresh, deletion at the end, and warranties. Then decide how the data lands in your systems and who owns it internally.

Choosing a route

Speed points one way and control the other. A listed product can be in use within days, but you get what the seller packaged. A direct licence takes longer and costs more in legal time, but you can shape the fields, the refresh and the restrictions.

Partnerships and scraping

Partnerships suit data that both sides benefit from combining. Scraping looks cheap until you count maintenance and legal review. Collecting your own data gives the most control and the clearest provenance, at the cost of time.

Build or buy

For machine learning teams this choice is often framed as build or buy, and the same trade-offs apply. Regulators are also watching the order of preference: the UK Information Commissioner's Office has said that developers relying on scraped personal data must show why other collection methods, such as licensing from third parties, would not work.

Common data sourcing mistakes

  • Buying on the description alone without testing a sample against the real question.
  • Assuming a marketplace listing proves the data is lawful to use. The platform checks its own rules, not every row's origin.
  • Ignoring refresh: a dataset that was accurate at purchase can drift within months.
  • Leaving deletion and end-of-term rules out of the contract.
  • Scraping first and asking legal later.

Where DataBroker fits, and its limits

DataBroker sits on the other side of the table. Our paid academy and tools are for people who help companies sell or license the data they hold, which is often exactly the unlisted data a sourcing team is looking for.

If you are a broker, understanding how buyers source data tells you how to package an offer: a clear sample, a provenance note and a draft data licence shorten the buyer's checklist.

Sources

6 sources · checked Nov 9, 2026

The DataBroker team checked the facts on this page against 6 public sources, last on .

  1. Neudata: What is data sourcing and how should companies approach it?Definition of data sourcing and the advice to start from a defined need · Checked
  2. AWS Marketplace Seller Guide: listing fees3% listing fee on AWS Data Exchange public offers · Checked
  3. CDEI: Unlocking the value of data, exploring the role of data intermediaries (2021)Partnership routes: data exchanges, industrial data platforms and other intermediaries · Checked
  4. Skadden: ICO publishes outcomes of generative AI consultation (January 2025)ICO view that scrapers must show alternatives such as licensing from third parties were not feasible · Checked
  5. FTC: AI (and other) companies, quietly changing your terms of service could be unfair or deceptive (February 2024)Risk of reusing customer data under quietly changed terms · Checked
  6. Gebru et al.: Datasheets for DatasetsDocumenting a dataset's motivation, composition and collection process · Checked

Data sourcing FAQ

What is the difference between data sourcing and data collection?

Data collection usually means gathering data yourself, through forms, products or sensors. Data sourcing is the wider job of deciding what you need and obtaining it by any route, including buying, licensing, partnering and collecting.

Is data procurement the same as data sourcing?

Close enough in everyday use. Procurement teams tend to say data procurement and focus on the contract and vendor management; data teams say sourcing and focus on finding and testing the data. Both cover the same steps.

Where do companies source external data?

From marketplaces and exchanges run by cloud platforms, from specialist data providers, directly from companies that collect the data as part of their business, from public and open data, and from partners willing to share.

How do you evaluate a data source?

Test a sample against the question you need answered: coverage, accuracy, gaps, field definitions and refresh rate. Then check provenance and the licence terms, including permitted use, resale limits and deletion at the end of the term.

Is scraping a legal way to source data?

It depends on what is scraped, where, and how it is used. Personal data, website terms and copyright all apply. Regulators such as the UK ICO expect scrapers of personal data to justify why licensing or other methods would not work. Ask a lawyer before relying on it.

Related

  • Where to sell data

    Where to sell data depends on the dataset: compare cloud marketplaces, listing sites, brokers, direct deals and asset sales on fees and fit.

  • Data licensing

    Data licensing lets a buyer use a dataset under written conditions while the seller keeps control, which is how most data deals are structured.

  • Data exchange

    A data exchange connects organisations that hold data with those that want it, and handles the secure transfer between them.

  • How AI companies get training data

    How do AI companies get training data? From five sources: web crawls, licensed archives, paid human work, synthetic data and their own users.

Free tool

Try the data value calculator

Data value is what a dataset is worth to someone, estimated by what it would cost to recreate, what similar data sells for, or how much extra income or savings it brings the person using it.

  • Free
  • No sign-up
  • Runs in your browser