{"id":836,"date":"2026-10-31T21:16:54","date_gmt":"2026-10-31T21:16:54","guid":{"rendered":"https:\/\/noesynthesis.org\/blog\/?p=836"},"modified":"2026-10-31T21:23:22","modified_gmt":"2026-10-31T21:23:22","slug":"reproducibility-vs-replicability","status":"publish","type":"post","link":"https:\/\/noesynthesis.org\/blog\/reproducibility-vs-replicability\/","title":{"rendered":"Reproducibility vs Replicability: What Was Actually Checked?"},"content":{"rendered":"

A second run can check the calculation. A second study asks whether the finding survives another encounter with the world.<\/h2>\n

I<\/span>magine a team studying whether a quieter reading room helps people finish a task. They collect observations, analyze a spreadsheet and publish a promising result. A colleague downloads their files, runs the analysis and obtains the same table. That is useful news. But nobody has yet invited a new group of readers into the room.<\/p>\n

This hypothetical example separates two achievements that a headline can easily collapse into \u201cscientists confirmed it.\u201d One checks the path from recorded observations to a reported answer. The other takes the question back to the world.<\/p>\n

\"Complete
Full conceptual diagram. No experimental data are represented.<\/figcaption><\/figure>\n

Same files, different question<\/h2>\n

The National Academies\u2019 2019 report on reproducibility and replicability<\/a> uses a specific distinction: computational reproducibility concerns consistent results from the same data and analytical procedure; replicability concerns consistency between studies that collect their own data to address the same question. Terminology varies between disciplines, so a paper\u2019s actual procedure matters more than the label alone.<\/p>\n

In our reading-room example, reproducing the table could catch a missing file, an undocumented exclusion or a calculation that does not match the published description. It gives another person a route through the work instead of asking them to trust a polished chart.<\/p>\n

Yet the same files contain the same limitations. Perhaps the study recruited only volunteers who already preferred silence. Rerunning their records does not bring in people who declined to participate. A recoverable answer is not automatically a broadly applicable answer.<\/strong><\/p>\n

A result can repeat with its mistake intact<\/h2>\n

The report\u2019s chapter on computational reproducibility<\/a> makes a crucial qualification: reusing erroneous code can reproduce an incorrect result. Agreement between two runs is not an independent inspection of every assumption. Documentation must also identify data, methods and the computational environment; a file that worked on its author\u2019s machine may depend on an unrecorded software version.<\/p>\n

Suppose our imaginary spreadsheet accidentally records seconds as minutes for one group. A faithful rerun might preserve the mistake perfectly. An independent check of the units asks something the rerun alone did not: does the recorded quantity mean what the analysis says it means?<\/p>\n

That is why \u201cthe code ran\u201d and \u201cthe reasoning was checked\u201d deserve separate sentences. A reader should be able to tell whether someone merely executed the package, inspected the transformations or independently rebuilt the calculation. None of those activities is worthless. Their evidential reach is different.<\/p>\n

New data do not require identical numbers<\/h2>\n

Now imagine a second team collecting fresh reading-room observations under comparable conditions. Their average difference will probably not match the first team\u2019s final decimal. The question is whether the results are consistent in light of their uncertainty, not whether the tables look like photocopies.<\/p>\n

The report\u2019s chapter on replicability<\/a> emphasizes comparing both the closeness of estimates and their uncertainty. A second study may be too imprecise to settle the issue. Disagreement can also expose differences in conditions worth investigating rather than immediately proving misconduct or making the original conclusion untouchable.<\/p>\n

For our hypothetical readers, room lighting, task difficulty or recruitment could matter. Those are candidate explanations to test, not excuses to invent after every disappointing result. A useful follow-up makes its comparison clear: which conditions were held similar, which changed and what outcome would count against the proposed explanation?<\/p>\n

Read the verb before trusting the headline<\/h2>\n

When an article says a finding was \u201cconfirmed,\u201d look for the action behind the word. Were the original files rerun? Were measurements collected again? Was the result tested in a different population? A classroom demonstration, a computational check and a new empirical study should not silently trade places.<\/p>\n

Our discussion of facts and interpretations<\/a> offers a related habit: keep the observation separate from what you conclude from it. Here the observation might be that an analyst recovered a published table. The stronger claim\u2014that the effect reliably holds for new people\u2014still needs its own support.<\/p>\n

A trustworthy research story does not need the biggest possible verb. It needs a precise account of what was checked, what survived that check and what remains open.<\/p>\n

Original conceptual illustration by M.R. Orion. Its bars are schematic, not experimental data. The reading-room study is an invented explanatory example.<\/em><\/p>\n

M.R. Orion<\/strong><\/p>\n","protected":false},"excerpt":{"rendered":"

A repeated calculation and a new study provide different kinds of evidence. Here is how to read the claim behind \u201cconfirmed.\u201d<\/p>\n","protected":false},"author":1,"featured_media":832,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"_monsterinsights_skip_tracking":false,"footnotes":""},"categories":[117,41],"tags":[165,164,166,149],"class_list":["post-836","post","type-post","status-publish","format-standard","has-post-thumbnail","category-editorial-en","category-technical-outreach","tag-replication","tag-reproducibility","tag-research-methods","tag-scientific-evidence"],"acf":[],"yoast_head":"\nReproducibility vs Replicability: What Was Checked?<\/title>\n<meta name=\"description\" content=\"Reproducibility vs replicability: why re-running an analysis differs from collecting new data, and what each check can tell us about a scientific claim.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/noesynthesis.org\/blog\/reproducibility-vs-replicability\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Reproducibility vs Replicability: What Was Checked?\" \/>\n<meta property=\"og:description\" content=\"Reproducibility vs replicability: why re-running an analysis differs from collecting new data, and what each check can tell us about a scientific claim.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/noesynthesis.org\/blog\/reproducibility-vs-replicability\/\" \/>\n<meta property=\"og:site_name\" content=\"Noesynthesis Blog\" \/>\n<meta property=\"article:published_time\" content=\"2026-10-31T21:16:54+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2026-10-31T21:23:22+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/noesynthesis.org\/blog\/wp-content\/uploads\/2026\/10\/reproduce-replicate-featured.webp\" \/>\n\t<meta property=\"og:image:width\" content=\"1440\" \/>\n\t<meta property=\"og:image:height\" content=\"810\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/webp\" \/>\n<meta name=\"author\" content=\"M.R. Orion\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"M.R. Orion\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"4 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/noesynthesis.org\\\/blog\\\/reproducibility-vs-replicability\\\/#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/noesynthesis.org\\\/blog\\\/reproducibility-vs-replicability\\\/\"},\"author\":{\"name\":\"M.R. Orion\",\"@id\":\"https:\\\/\\\/noesynthesis.org\\\/blog\\\/#\\\/schema\\\/person\\\/6081f8597aa01b89699f72d9beddb322\"},\"headline\":\"Reproducibility vs Replicability: What Was Actually Checked?\",\"datePublished\":\"2026-10-31T21:16:54+00:00\",\"dateModified\":\"2026-10-31T21:23:22+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/noesynthesis.org\\\/blog\\\/reproducibility-vs-replicability\\\/\"},\"wordCount\":727,\"commentCount\":0,\"image\":{\"@id\":\"https:\\\/\\\/noesynthesis.org\\\/blog\\\/reproducibility-vs-replicability\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/noesynthesis.org\\\/blog\\\/wp-content\\\/uploads\\\/2026\\\/10\\\/reproduce-replicate-featured.webp\",\"keywords\":[\"replication\",\"reproducibility\",\"research methods\",\"Scientific evidence\"],\"articleSection\":[\"English\",\"Science & Technology\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\\\/\\\/noesynthesis.org\\\/blog\\\/reproducibility-vs-replicability\\\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/noesynthesis.org\\\/blog\\\/reproducibility-vs-replicability\\\/\",\"url\":\"https:\\\/\\\/noesynthesis.org\\\/blog\\\/reproducibility-vs-replicability\\\/\",\"name\":\"Reproducibility vs Replicability: What Was Checked?\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/noesynthesis.org\\\/blog\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/noesynthesis.org\\\/blog\\\/reproducibility-vs-replicability\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/noesynthesis.org\\\/blog\\\/reproducibility-vs-replicability\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/noesynthesis.org\\\/blog\\\/wp-content\\\/uploads\\\/2026\\\/10\\\/reproduce-replicate-featured.webp\",\"datePublished\":\"2026-10-31T21:16:54+00:00\",\"dateModified\":\"2026-10-31T21:23:22+00:00\",\"author\":{\"@id\":\"https:\\\/\\\/noesynthesis.org\\\/blog\\\/#\\\/schema\\\/person\\\/6081f8597aa01b89699f72d9beddb322\"},\"description\":\"Reproducibility vs replicability: why re-running an analysis differs from collecting new data, and what each check can tell us about a scientific claim.\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/noesynthesis.org\\\/blog\\\/reproducibility-vs-replicability\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/noesynthesis.org\\\/blog\\\/reproducibility-vs-replicability\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/noesynthesis.org\\\/blog\\\/reproducibility-vs-replicability\\\/#primaryimage\",\"url\":\"https:\\\/\\\/noesynthesis.org\\\/blog\\\/wp-content\\\/uploads\\\/2026\\\/10\\\/reproduce-replicate-featured.webp\",\"contentUrl\":\"https:\\\/\\\/noesynthesis.org\\\/blog\\\/wp-content\\\/uploads\\\/2026\\\/10\\\/reproduce-replicate-featured.webp\",\"width\":1440,\"height\":810,\"caption\":\"Two panels contrast re-running the same records with testing a finding using new observations; schematic bars are not measured data\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/noesynthesis.org\\\/blog\\\/reproducibility-vs-replicability\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/noesynthesis.org\\\/blog\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Reproducibility vs Replicability: What Was Actually Checked?\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/noesynthesis.org\\\/blog\\\/#website\",\"url\":\"https:\\\/\\\/noesynthesis.org\\\/blog\\\/\",\"name\":\"Noesynthesis Blog\",\"description\":\"\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/noesynthesis.org\\\/blog\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/noesynthesis.org\\\/blog\\\/#\\\/schema\\\/person\\\/6081f8597aa01b89699f72d9beddb322\",\"name\":\"M.R. Orion\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/ccb9eddd5fed639653ff22c58d73d0dca07208f5e9ceef828bb3698d2d094381?s=96&d=mm&r=g\",\"url\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/ccb9eddd5fed639653ff22c58d73d0dca07208f5e9ceef828bb3698d2d094381?s=96&d=mm&r=g\",\"contentUrl\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/ccb9eddd5fed639653ff22c58d73d0dca07208f5e9ceef828bb3698d2d094381?s=96&d=mm&r=g\",\"caption\":\"M.R. Orion\"},\"sameAs\":[\"https:\\\/\\\/noesynthesis.org\\\/blog\"],\"url\":\"https:\\\/\\\/noesynthesis.org\\\/blog\\\/author\\\/admin_7hy3hjk8\\\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Reproducibility vs Replicability: What Was Checked?","description":"Reproducibility vs replicability: why re-running an analysis differs from collecting new data, and what each check can tell us about a scientific claim.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/noesynthesis.org\/blog\/reproducibility-vs-replicability\/","og_locale":"en_US","og_type":"article","og_title":"Reproducibility vs Replicability: What Was Checked?","og_description":"Reproducibility vs replicability: why re-running an analysis differs from collecting new data, and what each check can tell us about a scientific claim.","og_url":"https:\/\/noesynthesis.org\/blog\/reproducibility-vs-replicability\/","og_site_name":"Noesynthesis Blog","article_published_time":"2026-10-31T21:16:54+00:00","article_modified_time":"2026-10-31T21:23:22+00:00","og_image":[{"width":1440,"height":810,"url":"https:\/\/noesynthesis.org\/blog\/wp-content\/uploads\/2026\/10\/reproduce-replicate-featured.webp","type":"image\/webp"}],"author":"M.R. Orion","twitter_card":"summary_large_image","twitter_misc":{"Written by":"M.R. Orion","Est. reading time":"4 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/noesynthesis.org\/blog\/reproducibility-vs-replicability\/#article","isPartOf":{"@id":"https:\/\/noesynthesis.org\/blog\/reproducibility-vs-replicability\/"},"author":{"name":"M.R. Orion","@id":"https:\/\/noesynthesis.org\/blog\/#\/schema\/person\/6081f8597aa01b89699f72d9beddb322"},"headline":"Reproducibility vs Replicability: What Was Actually Checked?","datePublished":"2026-10-31T21:16:54+00:00","dateModified":"2026-10-31T21:23:22+00:00","mainEntityOfPage":{"@id":"https:\/\/noesynthesis.org\/blog\/reproducibility-vs-replicability\/"},"wordCount":727,"commentCount":0,"image":{"@id":"https:\/\/noesynthesis.org\/blog\/reproducibility-vs-replicability\/#primaryimage"},"thumbnailUrl":"https:\/\/noesynthesis.org\/blog\/wp-content\/uploads\/2026\/10\/reproduce-replicate-featured.webp","keywords":["replication","reproducibility","research methods","Scientific evidence"],"articleSection":["English","Science & Technology"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/noesynthesis.org\/blog\/reproducibility-vs-replicability\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/noesynthesis.org\/blog\/reproducibility-vs-replicability\/","url":"https:\/\/noesynthesis.org\/blog\/reproducibility-vs-replicability\/","name":"Reproducibility vs Replicability: What Was Checked?","isPartOf":{"@id":"https:\/\/noesynthesis.org\/blog\/#website"},"primaryImageOfPage":{"@id":"https:\/\/noesynthesis.org\/blog\/reproducibility-vs-replicability\/#primaryimage"},"image":{"@id":"https:\/\/noesynthesis.org\/blog\/reproducibility-vs-replicability\/#primaryimage"},"thumbnailUrl":"https:\/\/noesynthesis.org\/blog\/wp-content\/uploads\/2026\/10\/reproduce-replicate-featured.webp","datePublished":"2026-10-31T21:16:54+00:00","dateModified":"2026-10-31T21:23:22+00:00","author":{"@id":"https:\/\/noesynthesis.org\/blog\/#\/schema\/person\/6081f8597aa01b89699f72d9beddb322"},"description":"Reproducibility vs replicability: why re-running an analysis differs from collecting new data, and what each check can tell us about a scientific claim.","breadcrumb":{"@id":"https:\/\/noesynthesis.org\/blog\/reproducibility-vs-replicability\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/noesynthesis.org\/blog\/reproducibility-vs-replicability\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/noesynthesis.org\/blog\/reproducibility-vs-replicability\/#primaryimage","url":"https:\/\/noesynthesis.org\/blog\/wp-content\/uploads\/2026\/10\/reproduce-replicate-featured.webp","contentUrl":"https:\/\/noesynthesis.org\/blog\/wp-content\/uploads\/2026\/10\/reproduce-replicate-featured.webp","width":1440,"height":810,"caption":"Two panels contrast re-running the same records with testing a finding using new observations; schematic bars are not measured data"},{"@type":"BreadcrumbList","@id":"https:\/\/noesynthesis.org\/blog\/reproducibility-vs-replicability\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/noesynthesis.org\/blog\/"},{"@type":"ListItem","position":2,"name":"Reproducibility vs Replicability: What Was Actually Checked?"}]},{"@type":"WebSite","@id":"https:\/\/noesynthesis.org\/blog\/#website","url":"https:\/\/noesynthesis.org\/blog\/","name":"Noesynthesis Blog","description":"","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/noesynthesis.org\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/noesynthesis.org\/blog\/#\/schema\/person\/6081f8597aa01b89699f72d9beddb322","name":"M.R. Orion","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/secure.gravatar.com\/avatar\/ccb9eddd5fed639653ff22c58d73d0dca07208f5e9ceef828bb3698d2d094381?s=96&d=mm&r=g","url":"https:\/\/secure.gravatar.com\/avatar\/ccb9eddd5fed639653ff22c58d73d0dca07208f5e9ceef828bb3698d2d094381?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/ccb9eddd5fed639653ff22c58d73d0dca07208f5e9ceef828bb3698d2d094381?s=96&d=mm&r=g","caption":"M.R. Orion"},"sameAs":["https:\/\/noesynthesis.org\/blog"],"url":"https:\/\/noesynthesis.org\/blog\/author\/admin_7hy3hjk8\/"}]}},"_links":{"self":[{"href":"https:\/\/noesynthesis.org\/blog\/wp-json\/wp\/v2\/posts\/836","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/noesynthesis.org\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/noesynthesis.org\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/noesynthesis.org\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/noesynthesis.org\/blog\/wp-json\/wp\/v2\/comments?post=836"}],"version-history":[{"count":3,"href":"https:\/\/noesynthesis.org\/blog\/wp-json\/wp\/v2\/posts\/836\/revisions"}],"predecessor-version":[{"id":842,"href":"https:\/\/noesynthesis.org\/blog\/wp-json\/wp\/v2\/posts\/836\/revisions\/842"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/noesynthesis.org\/blog\/wp-json\/wp\/v2\/media\/832"}],"wp:attachment":[{"href":"https:\/\/noesynthesis.org\/blog\/wp-json\/wp\/v2\/media?parent=836"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/noesynthesis.org\/blog\/wp-json\/wp\/v2\/categories?post=836"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/noesynthesis.org\/blog\/wp-json\/wp\/v2\/tags?post=836"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}