Scientists increasingly write software as contributors to large-scale collaborative workflows. For example, data scientists write software to standardize and analyze data [17, 33, 41, 44] collected by field and experimental scientists. These analyses flow into software written by computational scientists to model natural phenomena, e.g. fish population dynamics [20, 50]. These models may then flow into integrative models, like those developed to predict the impact of ecological interventions and climate policies [13, 15].
Scientists engage in these collaborative computational workflows using a patchwork of:
When using these tools, scientists must put in consid-erable extra effort to follow identified best practices for data and software in the context of large-scale collabora-tive workflows, including the FAIR criteria for data and software [6, 53] (findability, accessibility, interoperabil-ity, reusability) and reproducibility. By default, artifacts are not FAIR (causing scientists to waste time reinventing the wheel) and not reproducible [23, 32, 46].
Reproducibility is a keystone of scientific discovery but unfortunately, many scientific disciplines are undergoing a replication crisis, including psychology [51], economics [7], and extending to the physical sciences: 70% of researchers reported that they failed to reproduce others' experiments [4]. In systems biology, 49% of published models were not reproducible [48]. A 2023 workshop at the National Center for Atmospheric Research explored the challenges of reproducibility in science that relies on computation, with presenters arguing that new techniques are needed to ensure that software-dependent results can be reproduced [32].
This work is licensed under a Creative Commons Attribution 4.0 International License.
_PROPL ’25, Singapore, Singapore_ © 2025 Copyright held by the owner/author(s). ACM ISBN 979-8-4007-2161-8/25/10 https://doi.org/10.1145/3759536.3763802
and whenever upstream data or models change) by Fairground's distributed parallelizing scheduler [31], which we call the _planetary compute engine_. Computations will leverage distributed hardware resources semi-automatically, with only the heaviest computations needing manual configuration. Computational resources, including CPUs, GPUs, and storage, will be contributed by participants and sponsors for use according to lightweight access policies (e.g. reserving a scientist's own individual and institutional hardware for the computations they author). Design research emphasizes the value of reasonable defaults in shaping user behavior [22]. The central design goal that distinguishes Fairground from competing systems is that we aim for artifacts authored in or imported into Fairground to be FAIR, reproducible, and live by default. Fairground is targeted toward the needs of a graduate student without substantial formal training in computing whose primary goal is to finish their next paper, rather than on one willing to invest substantial extra labor into achieving these criteria, which is rarely incentivized or rewarded. Let us give an overview of how the proposed design will achieve this critical goal.
_Open science_, in which data and analytic methods are made publicly available, is a key technique for improving reproducibility. When scientific work is conducted openly, other scientists and members of the public can study the work, check the analysis for errors, and re-analyze the data using their own methods [53]. Scientists in many domains, including in biology [3], ecology [23], and at NASA [46] have argued that open science is a key technique for improving reproducibility of scientific work and addressing the reproducibility crisis.
2 Design Considerations for Fairground Our vision is inspired by the success of other collaborative commons [42], particularly Wikipedia [52], GitHub [28], and Google Earth Engine [19].
2.1 Findability and Accessibility
On most existing platforms, notebooks are private by default, and substantial extra effort is needed to make them publicly findable and accessible by others who may benefit from their findings, functions, or pre-computed data (e.g. cleaned up versions of datasets). In Fairground, repositories will be public by default, closely following the example of GitHub, with opt-in access control to allow scientists to work with sensitive data. Each repository will receive a unique and persistent digital object identifier (DOI) [11] and be associated with its authors' ORCID identifiers [40] by default. Current repositories, such as those for oceanography [49], often include only data, not code, leaving scientists to re-implement analyses. Even when code is available in a repository such as Zenodo [12], it can be difficult to find and integrate into new contexts (e.g. due to outdated dependencies [43]). In Fairground, the entire system forms a single live program, enabling Fairground to offer _semantic search_ affordances such as find-all-uses (e.g. of a dataset, so that a scientist can find others who have already cleaned up that dataset, or of a statistical function, so that a scientist can find examples of similar analyses) and find-all-implementations of a particular interface (e.g. a data schema or a statistical function signature).
1.2 Towards live collaborations
Many fields of science, such as ecology and astronomy, are also _live_ collaborations: scientists continuously collect new data and refine modeling and analysis approaches. However, scientific models and analyses as reported in publications and scientific artifact repositories are frozen in time. Considerable extra effort and computational expertise possessed by few working scientists is needed to maintain liveness of analyses and models as new data are generated by sensors and experiments, and as upstream models and libraries improve. This leaves scientists and other stakeholders with a fragmented and outdated view of the state of the art [38]. We argue that _liveness_, in which the impact of program changes is visible immediately, promotes reuse because it enables programmers to more easily understand existing code and adapt it to their needs. We envision an ambitious but practicable transition to a collaborative scientific workflow where thousands of scientists and other stakeholders, assisted in the future by safely sandboxed AI agents [39], work together within a _computational commons_—which we call Fairground—where thousands of scientific artifacts are authored, collected, and maintained together _in executable form_ in a manner that is FAIR, reproducible, and live _by default_. This vision goes beyond prior efforts focused on simple archival of data and code to emphasize live code execution and the development of interoperable software components, and echoes other recent calls for integrative scientific workflow systems. Fairground will take the form of a collaborative computational notebook system that is as easy to use as competing platforms like JupyterLab or Google Earth Engine. It will consist of a collection of _computational notebooks_ and _data sources_ (including uploaded datasets and references to external datasets and streams) grouped into _repositories_ managed by participating individuals and organizations, much like on GitHub or GitLab (and we may indeed use a combination of these as our underlying version control platform). Unlike competing platforms, Fairground notebooks will be able to reference one another as libraries, collectively forming _a single planetary-scale live program_ being collaboratively edited in real-time or with optional lightweight version control, and executing live (i.e. as it is being edited
In addition to being able to access Fairground resources through a web browser, Fairground will offer an API so external applications can also access computational results. This will be particularly useful because Fairground is a live system: many results on Fairground will update continuously as new data are collected. Live data and visualizations maintained on Fairground might feed into external, public-facing projects, e.g. news articles or websites for public interest organizations; these feeds have been found to promote innovation in news media [2]. Users of these systems will be able to follow these embedded results back to their sources on Fairground, potentially increasing public trust in the results by inviting public scrutiny [36].
Computational notebooks typically operate as scripts; they cannot be understood as libraries that can be accessed from other notebooks (without wastefully re-executing them in their entirety in the new context). It requires enormous extra effort for a scientist to create reusable libraries and contribute them to public library repositories like pip [ the computational model underlying libraries like pandas for Python [47] or CRAN [1]. Consequently, it is common for scientists to re-implement behavior when they need an analysis that is not already in public package repositories. Reimplementing complex techniques can be error-prone, especially for non-experts who may not appreciate subtleties related to numerical accuracy [18] or leave statistical assumptions unchecked [16]. It is common to then copy-and-paste this functionality into each subsequent notebook where it might be useful, leading to messy and difficult-to-maintain code: 70% of all code snippets in Jupyter notebooks on GitHub are copies of other snippets [25]. On Fairground, every notebook is by default a library that exports its top-level definitions, including data and functions. Because the entire system operates as a single live program, importing one library into another does not require re-execution. This eliminates the process of packaging functionality into explicit libraries and submitting them to repositories—everything is immediately available for reuse across Fairground. To enable this sort of open reuse, Fairground cannot permit the importing notebook to modify the state of the imported notebook's analyses. Consequently, a central feature of Fairground is the development of Fair Python, a pure (i.e. immutable, functional) dataflow subset of Python that will serve as the native language of Fairground. Pure dataflow is fundamentally the same model used by spreadsheets: programs consist of elements that compute results, potentially on the basis of results produced by other parts of the program. Spreadsheets require that these elements be in a grid and restrict the kinds of computations that can be done; Fair Python will relax these constraints, allowing general-purpose computation. Pure dataflow programming is already in wide use in scientific code beyond spreadsheets, e.g. it is34], and most pandas code will be expressible in Fair Python without modification. By avoiding ephemeral state, we ensure that Fair Python programs have reproducible outputs. Of course, we do not expect scientists to (re)write all code in a restricted subset of Python—in reality, scientific code is written in a wide variety of languages, including scripting languages like Python, R, and Julia and native languages like C, C++ and Fortran. While we will design Fair Python to be a highly usable language for new scientific developments, we do not expect scientists working in Fairground to port existing code _en masse_ to Fair Python. Instead, we will develop a _pure foreign function interface_ (FFI) for Fair Python that adds _foreign nodes_ to the dataflow, which support executing code written in any language. To maintain liveness and interoperability, foreign nodes are restricted only in that they execute within a sandboxed environment and their inputs and outputs must be pure data. For example, a node might pass data and parameters from elsewhere on Fairground into an existing optimized numerical simulation package written in Fortran, then specify that the dataset generated when the simulation ends (e.g. saved to a file in the sandboxed file system) is an output value. Each sandboxed environment is configured using the Nix package manager, which produces reproducible execution environments. This solves the common problem scientists face of interoperating with code that requires old or conflicting versions of packages. To simplify common use cases, Fairground will be able to import existing Jupyter notebooks directly without modification, allowing them to be edited using the Jupyter web interface and automatically provisioning the necessary foreign node internally. Top-level definitions will be exported as outputs, and to maintain reproducibility, Fairground will persistently store the ephemeral state of each notebook. For native packages (those compiled for specific configurations from languages such as C++ or Fortran), we will require Nix configurations, which are written in a simple syntax that is widely documented. These configurations will be part of Fairground, so they can be reused by other scientists.
In traditional Python code, lines of code must be executed in order. This makes it difficult to leverage multicore hardware, which is ubiquitous even on laptops. Fairground eliminates these unnecessary sequential dependencies and automatically distributes computations, and live recomputations, for parallel execution across users' individual cores as well as computational resources contributed by participants. We call this system the Fairground _planetary compute engine_ [ can be searched to find relevant code and data. It will be open-source, enabling long-term maintenance and transparency of policies and behavior. It will support a distributed storage model, enabling both centralized, persistent storage and local replicas for high performance and customized access when needed. It will interoperate with other sources of data, such as NASA's PO.DAAC [15]. The commons will centralize thousands of data and code artifacts, which will have permanent identifiers that can be referenced from papers and other permanent artifacts, and24] and support computation in a variety of different languages.
Because Fairground programs will be live, it will seamlessly support reusability; any computation can be run, modified, and re-used in new contexts, even by users who have very different local computational infrastructure.
In summary, we propose three interleaved tracks of research that, together, will result in an operational prototype of a next-generation collaborative scientific workflow system. The proposed research will, in the process, lead to significant advances in the field of human-centered programming language design and implementation. In the first track, we will focus on the design and implementation of Fair Python. In the second, we will focus on interoperability with external data sources and code written in existing languages. In the third, we will advance the underlying planetary compute engine so that Fairground can scale to planetary-size datasets and analyses. We have already developed an operational prototype of the web-based user interface in the Hazel research programming environment [37] and plan, as a plenary approach, to integrate the research advances from all three tracks into this prototype, evolving it directly into Fairground.
To ensure that Fairground is as usable as possible for scientists, all three tracks will follow a _participatory design process_, following best practices for language design in which we iteratively make and evaluate design choices [9] with the specific goal of improving real-world scientific workflows being performed by scientists.
We will leverage our existing connections to involve scientists from several disciplines, with an initial focus on oceanography. In particular, we plan to develop case studies from the work of collaborators in the fields of oceanography and biology, and co-author Anil Madhavapeddy at the University of Cambridge. We will start with laptop-scale case studies from domain-specific data science courses with minimal external dependencies and then continue with full-scale case studies drawn from scientific research papers, ongoing projects, and graduate courses.
Encouragingly, our research direction into FAIR and live planetary computing is complemented by several ongoing efforts across the world, which we will discuss next.
We are currently working on a working prototype of Fairground, validated through a series of increasingly complex case studies and user studies. We anticipate that Fairground will first be ready for use by early adopters in the scientific community. At this point, we hope to continue development of Fairground by pursuing funding to develop a non-profit governance organization, guided by the TRUST principles [26], that will support both further technical development and, most importantly, develop the community around Fairground. While our initial focus will be on oceanography [21], biodiversity [5, 14] and forest protection [45] related case studies, we will broaden the scope of the project to other scientific communities at this time (and potentially earlier if opportunities arise). Our goal is for Fairground to be the most usable platform for doing day-to-day open science.
We anticipate that AI systems will be capable of assisting in the large-scale ingestion of existing computational notebooks and datasets into the Fairground system toward the end of the funding period, which we believe will help us bootstrap it to the point that it is quickly useful to working scientists rather than just early adopters. We believe that the time is right for scientific workflows to move to a nextgeneration platform designed with both usability and the FAIR criteria in mind from the start.
Our approach is motivated by multidisciplinary calls to develop tools that make it easier to follow FAIR practices for large-scale collaborative science and handle planetary-scale data and computation. Co-author Anil Madhavapeddy and colleagues in conservation made a case for “_planetary computing_ — infrastructure to handle the ingestion, transformation, analysis, and publication of global data products for furthering environmental science and enabling better informed policy-making” [15]. This article explains that scientists need access to large petabyte-scale input datasets consisting of:
Programmers then define computation over these datasets that: (i) is either algorithmic or machine learning-based, using a mix of CPUs and GPUs; (ii) needs to autoscale to permit local development followed by global analysis; and (iii) can be expressed by a non-CS expert, ideally with a visual interface.
The Global Biodiversity Information Facility observed: “There has been an explosion of data and information, and a concomitant paradigm shift to data-driven research and specifically biodiversity research and its myriad applications. [...] Redesigned scientific data organizations [...] are developing approaches designed specifically to take advantage of the unprecedented data opportunities in a cooperative framework" [10].
Participants at NCAR's Workshop on Correctness and Reproducibility for Climate and Weather Software [32] sought solutions for reproducibility problems. Running analyses on multiple systems results in discrepancies; our approach enables scientists to reuse common computations and results. At the 1st Programming for the Planet workshop [27], Cimadevilla argued [8] that acquisition and processing of data from heterogeneous data sources, followed by reuse of model outputs, is a key component of many scientific tasks.
Large enterprises have also recognized the need for commonly available computational infrastructure for earth science. In particular, Google offers the Google Earth Engine product [19] and Microsoft offers the Microsoft Planetary Computer [29] product upon request to environmental scientists. These tools allow scientists to develop computational notebooks and execute them using cloud hardware that has access to a common repository of data sets and software packages, much like our proposed system. However, as explained above, notebooks in these tools are not FAIR, not reproducible, and not live. The research described here may help these organizations address these problems for their proprietary planetary compute engines. We will instead lay the intellectual groundwork for a fully open complement to these platforms. Encouragingly, we can build upon emerging open and federated standards being established by initiatives such as OpenEO [30], which defines APIs independent of any single cloud provider or organization for the processing of earth observation data.
The most closely related effort to build a computational commons for scientists is the Observable community [35], which hosts a computational commons with support for live evaluation within individual notebooks (though not across notebooks). Notebooks can refer to explicitly exported values in other notebooks, which must be pure. The analyses must be written in JavaScript and run in the user's browser. While there is some support for controlling side effects, these are opt-in, so the system lacks reproducibility and automatic scalability. Although Observable does not meet our criteria, it demonstrates that it is possible to attract communities of users to a well-designed and curated computational commons.