Overview
Macroeconomic, social and firm-level research often depends on public or licensed databases rather than new data collection. The choices matter: GDP in current US dollars, constant prices or purchasing power parity gives different results; an indicator may change definition partway through a series; country codes and fiscal years may not line up between sources. Good sourcing starts from your variables and hypotheses, then finds the measure that fits them best.
We help you identify candidate sources, compare their coverage and definitions, and extract data into a clean, merged dataset with a data dictionary. Public sources such as the World Bank's World Development Indicators, IMF data and Eurostat are generally reusable with attribution, although each has exceptions for third-party data. Licensed databases such as Compustat, accessed through WRDS, are different: only authorized users at subscribing institutions may access them.
For licensed data, we do not supply or share data. Instead, we help you design the query, write extraction code you run under your own institutional access, and clean and merge what you download. This keeps you within the terms your library has agreed and keeps your data trail defensible.
Who it’s for and the problems it solves
This is for you if you are…
- Economics, finance, business and development students building a country or firm panel for a thesis
- Researchers who need to combine indicators from several international sources into one dataset
- Authors responding to reviewers who questioned a variable's definition, source or coverage
- Policy and consultancy teams needing a documented evidence base from official statistics
- Researchers with WRDS access who are unsure how to query Compustat or merge it with other data
Common problems we help with
- You do not know which database has the variable you need, or several sources report different values
- Your panel has gaps for many countries or years and you are unsure how to handle them
- Country names, ISO codes, currencies or fiscal years do not match across sources
- A series changed its base year or methodology and now has a break
- You need Compustat or other licensed data but do not know how to query it or what you may share
- Your methods section cannot explain exactly how the dataset was constructed
What’s included — and what isn’t
Included
- Mapping your variables and hypotheses to candidate indicators and sources
- Comparison of sources on definition, unit, coverage, frequency, revisions and license terms
- Extraction from public sources such as World Bank WDI, IMF data portal datasets, Eurostat, OECD data and national statistics offices, via download tools or APIs
- Query design and extraction code for licensed sources (for example Compustat through WRDS) that you run with your own access
- Harmonizing country codes, firm identifiers, currencies, deflators and fiscal or calendar years
- Merging sources into a tidy panel or cross-section, with missing-data and coverage reports
- Flagging structural breaks, base-year changes and outliers for your judgment
- Data dictionary with variable definitions, units, source codes, access dates and license notes
- Reproducible scripts in R, Python or Stata so you can refresh or extend the dataset
Not included
- Supplying, reselling or sharing licensed or subscription data (for example Compustat, CRSP, Bloomberg, Orbis) with anyone who lacks their own authorized access
- Using another person's database credentials or scraping sources in breach of their terms
- Filling gaps with invented values; any imputation or interpolation is transparent, justified and agreed with you
- Restricted-access microdata that require an approved application, unless you have that approval and the terms permit our help
What you receive
- Source assessment
- A short memo comparing candidate sources for each variable, with the recommended choice and the reason, ready to adapt for your methods.
- Analysis-ready dataset
- A merged file in your preferred format (CSV, Excel, Stata .dta, SPSS .sav or R) built from public sources, or from licensed data you downloaded and supplied to us.
- Data dictionary
- Variable names, definitions, units, original source codes, transformations, access dates and license or citation requirements.
- Coverage and gaps report
- Tables showing observations by country or firm and year, missing-data patterns and any breaks in series.
- Reproducible scripts
- Commented extraction and cleaning code so you or your examiner can reproduce the dataset from the original sources.
How it works
Variable planning
You share your research questions, model and preferred sample. We list the variables needed and any theoretical constraints on how each should be measured.
Source search and assessment
We identify candidate sources, check definitions, coverage and license terms, and agree the final choices with you.
Extraction
We extract public data directly. For licensed databases, we provide query steps or code and you run them with your own access, then send us the files if your license allows help from an assistant.
Cleaning and merging
We harmonize identifiers and units, merge sources, and produce coverage and missing-data reports for you to decide on sample and treatment.
Documentation and handover
You receive the dataset, dictionary and scripts, with a walkthrough so you can describe and defend your data construction.
Sources we commonly work with
- World Bank World Development Indicators (WDI)
- Cross-country development indicators accessible through the DataBank and API. World Bank datasets are generally CC BY 4.0, but some indicators come from third parties with their own terms, so we check each series' metadata.
- IMF data
- Macro-financial series such as balance of payments, prices, government finance and monetary statistics. The IMF data portal has been restructured, and former International Financial Statistics series are now organized into topic datasets, so we confirm current dataset names at extraction.
- Eurostat
- EU statistics reusable under CC BY 4.0 with source acknowledgment, with stated exceptions such as some data about non-EU countries and certain detailed trade data.
- OECD and UN sources
- OECD Data Explorer, UN Comtrade, UNCTAD and UNESCO Institute for Statistics for trade, investment and education indicators, each with its own reuse terms.
- National statistics offices and central banks
- Official national series for subnational or more detailed data, including handling of language, rebasing and publication-specific notes.
- Compustat and other WRDS databases
- Firm-level accounting data available only to authorized users at subscribing institutions for academic, non-commercial research. We design queries and cleaning code; you perform the download under your own access.
Matching definitions before merging
Merging data from different sources without checking definitions is one of the most common causes of reviewer criticism in panel studies. Before combining series we check the points below and record every decision in the data dictionary.
- Units and scale: millions versus billions, percent versus ratio, per capita versus total
- Price basis: current, constant (and which base year) or PPP-adjusted
- Identifiers: ISO3 country codes, territories and changes such as country splits; GVKEY, CUSIP or ISIN for firms
- Time alignment: calendar versus fiscal years, and end-of-period versus period-average values
- Revisions and vintages: which release you used and when you accessed it
Missing data and coverage
Cross-country panels are rarely complete. We report coverage honestly and give you options: restricting the sample, using a different indicator with better coverage, interpolation within short gaps, or multiple imputation where it can be justified. Each has consequences for your analysis, and the choice is yours, informed by what the literature in your field accepts. Once the data are assembled, panel data analysis support can take you through estimation.
What the work can look like
Illustrative example — fictional data
A remittances and financial inclusion panel for 60 countries
A fictional master's student wants to test whether remittance inflows are associated with account ownership across 60 low- and middle-income countries from 2011 to 2021, controlling for income, inflation and mobile phone penetration.
Source choices
Remittances, GDP per capita (constant PPP) and mobile subscriptions came from WDI; inflation from an IMF dataset; account ownership from the World Bank's Global Findex, which is survey-based and available only for selected years, so the panel was restructured around survey waves.
Harmonization
Country names were matched to ISO3 codes, three territories without Findex data were dropped, and all series were recorded with access dates and license notes in the dictionary.
Coverage report
The final panel had 4 waves and 57 countries, with 9% of control values missing, concentrated in inflation for two countries. The student chose to exclude those countries and reported this as a limitation.
Illustrative, invented scenario and figures. Actual sources and coverage depend on your variables, sample and period, and should be checked against the current databases.
What you’ll need to provide
- Research questions, hypotheses and your list of variables with any definitions from the literature
- Target sample: countries, regions, industries or firms, and the time period
- Details of any licensed databases your institution subscribes to and whether you have personal access
- Your institution's or the database's rules on research assistants handling licensed data
- Preferred software and file format
You don’t need everything ready to ask for a quote — tell us what you have and we’ll say what’s missing.
What affects price and turnaround
Price depends on
- Number of variables and sources to be combined
- Size of the sample (countries, firms, years) and frequency of data
- Amount of harmonization needed, such as currency conversion, rebasing or identifier matching
- Whether reproducible scripts or an API-based refresh workflow are required
- Complexity of licensed-data queries and merges
Turnaround depends on
- Availability and responsiveness of source websites and APIs
- Time for you to run licensed-data queries under your own access
- How quickly decisions on sample restrictions and missing data are made
Every project is quoted individually. How we price our work.
Questions people ask before starting
Can you just send me Compustat data?
No. Compustat and similar databases are licensed to subscribing institutions for their authorized users. Sharing extracts with people who lack access breaches those terms. We help you write the query and code, you download with your own access, and we help with cleaning if your institution permits it.
My university does not subscribe to the database I need. What are my options?
We look for open alternatives with comparable variables, such as company filings, open financial datasets or official statistics, and explain the trade-offs. Sometimes a design change, such as a different sample, is the more realistic route.
Why do WDI and the IMF report different values for the same indicator?
Different compilation dates, revisions, definitions or base years are the usual reasons. We document which source and vintage you used and why, so the choice is defensible.
Do I need to cite the databases?
Yes. Most sources specify a citation, and licenses such as CC BY 4.0 require attribution. WRDS also requires acknowledgment in publications. Our data dictionary lists the citation for each source.
Can you update the dataset later when new data are released?
If we build it with scripts or API calls, updating is usually straightforward. You can rerun the scripts yourself, or ask us to refresh and re-document the dataset.
Will you run the regressions too?
This service focuses on building the dataset. Estimation and interpretation are available through our econometrics services, and you remain responsible for interpreting and writing up the results.