Analysis Code Review and Reproducibility Package Preparation

Two different questions sit behind this service. First, is the analysis correct: do merges keep the right rows, are variables coded as intended, do the standard errors match the design? Second, is it reproducible: can someone else, on another computer, run your files and get the same numbers? A project can fail either test while looking fine to its author.

Our code review reads your scripts the way a careful coauthor or replicator would, runs them in a clean environment and compares the output with the numbers in your manuscript. We report problems by severity, from errors that change results to style issues that only affect readability, and suggest fixes.

For replication packages we follow widely used guidance, particularly the AEA Data Editor's recommendations and template README used across economics, and adapt it to your field and journal. The result is a package with a clear structure, documented data sources and a master script that runs everything without manual steps.

Code review is not about style preferences. We look first for errors that change results: merges that duplicate or drop observations, filters applied in the wrong order, variables recoded in one script and used raw in another, standard errors that ignore clustering, or a sample in the code that differs from the sample described in the paper. Each is checked against your manuscript's description.

Next we look for fragility: hard-coded paths, copy-pasted blocks that have drifted apart, results typed into tables by hand, and code that depends on objects created interactively. These may not be wrong today but will cause errors after the next revision.

• Critical: changes a reported number or conclusion
• Important: likely to break or to be misunderstood by a replicator
• Minor: readability, naming and comments

A replication package should let someone reproduce every table and figure by running code, with no manual intervention beyond setting a root directory. The AEA Data Editor's guidance, widely used beyond economics, recommends keeping data and code in separate folders so that code can be downloaded without very large data, citing every dataset not created by the authors, and stating software versions, required packages and expected runtimes.

The README is the heart of the package. It explains where each dataset comes from and whether and how it can be obtained, lists the computational requirements, gives step-by-step run instructions and maps each table and figure to the program that produces it. We prepare this with you, because only you know the full provenance of the data.

Confidential health records, licensed financial databases and administrative data often cannot be redistributed. That does not rule out reproducibility. The package can include all code, precise instructions for requesting access, the exact extraction query or file versions used, and sometimes a synthetic or simulated dataset with the same structure so replicators can check that the code runs.

We help you write data availability statements that are clear about what is and is not included and why, in line with your ethics approval and data management plan.

Bootstrapped confidence intervals, multiple imputation, cross-validation and simulation all use random numbers. Without a fixed seed, results change slightly every run, and replicators cannot tell a small random difference from an error. We set seeds once at the top of each relevant script and note any procedures where parallel processing requires extra care.

Software versions matter as much. Package updates can change defaults, and sometimes results. Recording versions with renv, a lockfile or a Stata version statement, and stating the operating system and runtime in the README, lets a replicator reconstruct your setup rather than guess at it.