Guide
Data sourcing: how companies get the data they need
Data sourcing means finding, assessing and obtaining outside data, through one of five routes that differ in cost, speed and legal risk.
Published
The short answer
Data sourcing is the process of identifying the data a project needs and obtaining it, usually from outside the company. There are five main routes: buy a listed product, license directly from the holder, partner and share, scrape public sources, or collect it yourself. Each trades speed against cost, control and legal risk, so start from the use case.
What data sourcing covers
Neudata, which researches alternative data for investment firms, describes data sourcing as the process organisations use to identify external datasets and build them into their work.
In practice the word stretches a little wider: it covers deciding what you need, finding who holds it, checking that it is good enough and lawful to use, agreeing terms and getting it delivered.
Other names for the same work
Procurement teams often call the same work data procurement, and engineers sometimes say data acquisition, although that phrase also names sensor and measurement hardware.
Start with the question
The order matters. Teams that start with a list of vendors tend to buy data they never use. Teams that start with one question the data must answer can judge every option against it: does this dataset cover the right period, region and population, at the right freshness, under terms that allow the use we have in mind?
The data sourcing process map
Most data reaches a company through one of five routes. The costs below are described rather than priced, because they depend on the dataset, the volume and the negotiation. Where a source publishes a real figure we give it.
| Route | How it works | What it costs | Main risk | Source |
|---|---|---|---|---|
| Buy a listed product | Subscribe to a dataset on a marketplace or exchange under standard terms | The listing price; on AWS Data Exchange the seller pays a 3% listing fee on public offers, which is built into what you pay | Standard terms may not cover your use; the platform doesn't audit where every row came from | AWS Marketplace listing fees |
| License directly | Negotiate a data licence with the company that holds the data | A negotiated fee plus legal and diligence time on both sides | Slow; the holder may not have the rights it claims, so warranties matter | Neudata; ICO via Skadden |
| Partner and share | Swap or pool data with another organisation, sometimes through an intermediary or industrial data platform | Often little cash, but governance, integration and legal set-up | Unequal benefit, unclear ownership of combined data, exit terms | CDEI report on data intermediaries |
| Scrape public sources | Collect pages or records from websites with your own crawlers | Engineering, infrastructure and ongoing maintenance as sites change | Personal data, site terms and copyright; UK regulator expects you to show licensing wasn't a workable alternative | ICO via Skadden |
| Collect it yourself | Gather first-party data from your own users, products or sensors | The slowest route: product changes, consent design and time to build volume | Using customer data for a new purpose without clear notice; the FTC warns that quietly changing terms may be unfair or deceptive | FTC, February 2024 |
Steps in a data sourcing project
- 1
Write down the question
Name the decision or model the data feeds, the fields you need, the coverage and the refresh rate. This is the brief every seller will be judged against.
- 2
Check what you already hold
Internal logs, CRM records and past purchases often cover part of the need. Buying what another team already licensed is a common waste.
- 3
Map candidate holders
List who is likely to collect this data as a by-product of their business, which marketplaces carry related products, and whether a partner would share. Many useful datasets are never listed anywhere.
- 4
Test a sample
Ask for a sample or a trial period and check it against your question: coverage, gaps, duplicates, how fields are defined and how often they change.
- 5
Check rights and provenance
Ask how the data was collected, under what notice or consent, and whether the seller may license it for your use. A written record of origin, in the spirit of a datasheet for the dataset, saves arguments later.
- 6
Agree terms and plan delivery
Settle permitted use, term, refresh, deletion at the end, and warranties. Then decide how the data lands in your systems and who owns it internally.
Choosing a route
Speed points one way and control the other. A listed product can be in use within days, but you get what the seller packaged. A direct licence takes longer and costs more in legal time, but you can shape the fields, the refresh and the restrictions.
Partnerships and scraping
Partnerships suit data that both sides benefit from combining. Scraping looks cheap until you count maintenance and legal review. Collecting your own data gives the most control and the clearest provenance, at the cost of time.
Build or buy
For machine learning teams this choice is often framed as build or buy, and the same trade-offs apply. Regulators are also watching the order of preference: the UK Information Commissioner's Office has said that developers relying on scraped personal data must show why other collection methods, such as licensing from third parties, would not work.
Common data sourcing mistakes
- Buying on the description alone without testing a sample against the real question.
- Assuming a marketplace listing proves the data is lawful to use. The platform checks its own rules, not every row's origin.
- Ignoring refresh: a dataset that was accurate at purchase can drift within months.
- Leaving deletion and end-of-term rules out of the contract.
- Scraping first and asking legal later.
Where DataBroker fits, and its limits
DataBroker sits on the other side of the table. Our paid academy and tools are for people who help companies sell or license the data they hold, which is often exactly the unlisted data a sourcing team is looking for.
If you are a broker, understanding how buyers source data tells you how to package an offer: a clear sample, a provenance note and a draft data licence shorten the buyer's checklist.
- We don't sell datasets, leads or data feeds, and we don't run a data exchange.
- We don't publish typical dataset prices; they vary too much by type and use to quote honestly.
- Our dataset pricing calculator and data value calculator give rough ranges for a seller's data, not quotes from buyers.
Sources
6 sources · checked Nov 9, 2026
The DataBroker team checked the facts on this page against 6 public sources, last on .
- Neudata: What is data sourcing and how should companies approach it?Definition of data sourcing and the advice to start from a defined need · Checked
- AWS Marketplace Seller Guide: listing fees3% listing fee on AWS Data Exchange public offers · Checked
- CDEI: Unlocking the value of data, exploring the role of data intermediaries (2021)Partnership routes: data exchanges, industrial data platforms and other intermediaries · Checked
- Skadden: ICO publishes outcomes of generative AI consultation (January 2025)ICO view that scrapers must show alternatives such as licensing from third parties were not feasible · Checked
- FTC: AI (and other) companies, quietly changing your terms of service could be unfair or deceptive (February 2024)Risk of reusing customer data under quietly changed terms · Checked
- Gebru et al.: Datasheets for DatasetsDocumenting a dataset's motivation, composition and collection process · Checked
Data sourcing FAQ
What is the difference between data sourcing and data collection?
Data collection usually means gathering data yourself, through forms, products or sensors. Data sourcing is the wider job of deciding what you need and obtaining it by any route, including buying, licensing, partnering and collecting.
Is data procurement the same as data sourcing?
Close enough in everyday use. Procurement teams tend to say data procurement and focus on the contract and vendor management; data teams say sourcing and focus on finding and testing the data. Both cover the same steps.
Where do companies source external data?
From marketplaces and exchanges run by cloud platforms, from specialist data providers, directly from companies that collect the data as part of their business, from public and open data, and from partners willing to share.
How do you evaluate a data source?
Test a sample against the question you need answered: coverage, accuracy, gaps, field definitions and refresh rate. Then check provenance and the licence terms, including permitted use, resale limits and deletion at the end of the term.
Is scraping a legal way to source data?
It depends on what is scraped, where, and how it is used. Personal data, website terms and copyright all apply. Regulators such as the UK ICO expect scrapers of personal data to justify why licensing or other methods would not work. Ask a lawyer before relying on it.
Related
- Where to sell data
Where to sell data depends on the dataset: compare cloud marketplaces, listing sites, brokers, direct deals and asset sales on fees and fit.
- Data licensing
Data licensing lets a buyer use a dataset under written conditions while the seller keeps control, which is how most data deals are structured.
- Data exchange
A data exchange connects organisations that hold data with those that want it, and handles the secure transfer between them.
- How AI companies get training data
How do AI companies get training data? From five sources: web crawls, licensed archives, paid human work, synthetic data and their own users.
Free tool
Try the data value calculator
Data value is what a dataset is worth to someone, estimated by what it would cost to recreate, what similar data sells for, or how much extra income or savings it brings the person using it.
- Free
- No sign-up
- Runs in your browser








