#

Metadata Generator for the Alaska Energy Data Gateway (AEDG)#

Ruff uv Checked with mypy pre-commit.ci status

A fundamental feature of the newly revised Alaska Energy Data Gateway (AEDG) is the documentation of data sources and products. To maximally make these data Findable, Accessible, Interoperable, and Reusable (FAIR), each data file ingested and produced will be accompanied by a metadata file in a standard format.

To standardize production of this multitude of files, we have implemented a command line tool written in Python. This tool utilizes the OEMetadata standard, an extension of Frictionless Data Packages:

Hülk, Ludwig, Jonas Huber, Christian Hofmann, and Christoph Muschner. “Open Energy Family - Open Energy Metadata (OEMetadata).” Python, January 2025. [https://github.com/OpenEnergyPlatform/oemetadata].

Hülk, Ludwig, Jonas Huber, Christian Hofmann, and Christoph Muschner. “Open Energy Metadata (OEMetadata),” March 13, 2025. [https://doi.org/10.5281/zenodo.15019562].

Installation#

This packages uses pyproject.toml to define the dependencies instead of requirements.txt. It was set-up using uv which is the preferred package management system, but pip with virtualenv can also be used.

Extra dependencies needed for rendering sphinx documentation (docs) and testing with pytest (test) are defined using optional dependencies and not groups. This is because compatibility with pip was important. As of 3/23/2025, uv was able to make “groups”, but the most recent version of pip (v25.0.1) did not have groups enabled yet (v25.1dev seems to). Installation instructions for these extras are included.

Clone the repository locally:

% git clone git@github.com:acep-aedg/aedg-metadata.git
% cd aedg-metadata

uv#

uv sync will create the .venv/ directory and it can also include the optional dependencies too.

% uv sync --all-extras  # Include all optional dependencies.
% source .venv/bin/activate

pip#

% python -m venv .venv
% source .venv/bin/activate
% pip install -e .
% pip install '.[test]'
% pip install '.[docs]'

Testing installation#

To check the package installed correctly: aedg_metadata generate --help

To check the testing installed correctly: pytest

To run all the pre-commit checks: uvx nox

To render the documentation:

% cd docs
% make html

Project Information#

Funding#

This project was built with support from State of Alaska capital appropriations for the Alaska Energy Data Gateway.

Additional Information#

Learn more about the Alaska Center for Energy and Power.

The Alaska Energy Data Gateway (AEDG) team has written several Google Docs to support development of the system. Access to these documents is restricted to team members, though the hope is to make them public-facing eventually. The following links are to requirements documentation created during the development of AEDG:

  1. 20250220- AEDG Requirements - Metadata: This doc analyses how metadata would be used to satisfy user stories and technical needs. The results of this analysis led to the identification of Open Energy Metadata as the best metadata standard to follow.

  2. AEDG Metadata Style Guide: As metadata began to be accumulated, it became apparent that a style guide was needed to standardize language and tone. This doc is under development.

  3. 2025-04-15 Keywords Requirements for Search: In the long term, a standard vocabulary will be used to guide selection of keywords in the metadata. This doc describes how search works in AEDG in hopes of guiding that future discussion.

Documentation#

User Guide#

API#

Source code for the data pipeline found in src/aedg_metadata.