Framework for Integrated Earth Science and Technology Applications

FIESTA allows new data repositories to be defined and formed by researchers from across the Earth Sciences—without any knowledge in geoinfomatics—in particular from domains with unique data and data needs that typically fit into small- to intermediate-sized scientific data repositories. FIESTA does this by focusing researchers on defining and managing of the data model primarily and championing community-driven data entry—the rest is taken care of by the FIESTA software and team. By using the same structured FIESTA data models and the same FIESTA REST API, each new data repository will be transparent in how to access it and, because the data models are standardized and versioned, it provides a direct way of reproducing the research results.

FIESTA nodes are customized using versioned YAML configuration files, including detailed data models that include (1) field annotations: descriptions, notes, groups, ordering, examples; (2) field validation rules: data types, numeric limits, (conditionally) required values, download-only fields, existence of values in other fields, post-processing derived fields; (3) upgrade paths between versions: previous field name(s); (4) vocabularies: controlled/suggested, internal/EarthRef.org-wide, external available via APIs; and (5) hierarchies: required/optional tables, ordering, and parent/child relationships. The configuration files support customizing the repository websites, which are live-editable with the TinaCMS open-source visual content management system, and which provide integrations such as to EarthRef.org’s FIESTA JupyterHub persistent notebook environment that is coupled to contributor’s private workspaces or the use of the FIESTA API for IGSN minting via DataCite.

Data can be uploaded into a FIESTA node using (1) multipart tab-delimited files: text-only, editable in a text editor, easily copied into Excel; and (2) zipped archives: multiple file formats, compressed, larger files. The FIESTA upload tool allows for (1) parsing: from multipart tab-delimited, Excel sheets; (2) preview: after parsing and before upload; (3) disabling: table/column/row disabling, skip N header rows; (4) data model mapping: table and field matching with dropdowns; and (5) import templates: saved data mapped to the data model.

All FIESTA data repositories support private workspaces to which data first get uploaded (see above) and where the users will prepare their data before making it public, allowing for (1) named contributions: for organizing multiple versions; (2) contribution metadata editing: description, DOI, lab names; (3) validation: invalidated after updates/edits, required before publishing; (4) sharing: private URL link to display contribution in Search Interface before publishing; and (5) custom plugins: Python or Typescript code in public GitHub repositories with permissible open-source licenses that are imported as git submodules into the FIESTA software after passing continuous integration tests to augment the Search Interface, Private Workspace, Upload Tool, and/or API for the specific requirements of each domain repository.

All FIESTA data repositories have websites with customizable features such as (1) authentication: ORCiD Member API and EarthRef username/passwords; (2) front page: links, recent contributions, news items; (3) content pages: about, workshops, help; (4) contact: report an issue, contacts, having trouble; and (5) footer: sponsors, supporters, license. These websites also include fully featured search interfaces with (1) full text search: any non-numeric field across the index of all contributions; (2) filters: numeric ranges, text buckets, with/without data; (3) results: tabbed table drilldown with infinite virtual scrolling; (4) structured metadata: injected JSON-LD headers when displaying a single contribution; and (5) download: contribution versions or search result rows across contributions.

These FIESTA websites - similarly to the FIESTA-powered OSU-MGR website - are managed using Typescript for faster and less error-prone development, Next.js for faster building and deploying, better static site generation, hydration, and delivery optimizations, and live editing with TinaCMS management of news, content pages, and repository configuration editing.

The FIESTA API provides the ability for other online domains to interrogate and re-use the FIESTA data repositories. The API uses (1) OpenAPI 3.0 definition: YAML definition with automatic documentation; (2) authentication: ORCiD Member API and EarthRef username/passwords; (3) public endpoints: download/search/validate without authentication; (4) private endpoints: create/update/delete/download/ search/validate private contributions; and (5) indexing: parse/summarize/index private contributions.