Right now, fetching our data and building our site is essentially a four-step process. Each step requires the previous one to have been completed in the same flow.
- Fetch data from the GitHub API.
- Load that data directly into a
data.json file.
- Build the static site, including the
data.json file.
- Deploy the site to GitHub pages.
Since the data.json file doesn't get persisted beyond the build, we need to re-fetch data on each pass, and are also limited in the type of data we can store.
An alternative approach would be:
- Fetch data from the GitHub API.
- Load that data into a database somewhere (CosmosDB, for example).
- Before build, load the data from the database into a
data.json file.
- Build the static site, including the
data.json file.
- Deploy the site to GitHub pages.
This would have two big advantages:
- Deploying the site would no longer be dependent on fetching data from the GitHub API. This should make the overall build/deploy time faster.
- Storing in a database gives us much more flexibility on the type of data we store. For example, this could potentially make time-series calculations more feasible.
As a first pass, I'd propose the following course of action:
Once that's setup, we can look at adding additional metric stores in a followup.
Right now, fetching our data and building our site is essentially a four-step process. Each step requires the previous one to have been completed in the same flow.
data.jsonfile.data.jsonfile.Since the
data.jsonfile doesn't get persisted beyond the build, we need to re-fetch data on each pass, and are also limited in the type of data we can store.An alternative approach would be:
data.jsonfile.data.jsonfile.This would have two big advantages:
As a first pass, I'd propose the following course of action:
workflow_dispatchand a cron (every day?) that updates the new database.data.jsonfile instead of running the fetch script directly.Once that's setup, we can look at adding additional metric stores in a followup.