FITARA Remastered: Data

Hugo’s data directory is probably one of its most powerful components. Different formats are supported and accessing the data inside can be an exercise in creativity.

In the original dashboard, I stored each scorecard’s grades in a TSV file and then had some shell scripts convert the TSV to SQL to pipe into a SQLite file that got committed with the code and pushed up to the production server. All of the different analytics inside the dashboard came from queries against that database. Some of the data went into the HTML for agency grades, while most of the query results went into JavaScript configurations for the charts.

1
2
3
4

Agency  Grade   Cyber   ...
USDA    B       A
DoC     B       A
...

Knowing It would be some extra work to extract data, I went for a TOML data store instead of CSV. With the latter, I knew there would be way more iterating and I wasn’t sure how awful that would be within the templates. With the TOML structure, I could access most things as a map with keys to different parts of the data.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

title = "Scorecard 16.0 - September 2023"

metrics = [
  "Cyber",
  ...
]

[data.usda]
  grade = "B"
  metrics = ["A", "D", "C", "A", "A", "C", "F"]
  cioReports = "X"

[data.doc]
  grade = "B"
  metrics = ["A", "B", "A", "A", "B", "D", "F"]
  cioReports = "X"

Each TOML file is named with the year and month, like 202309.toml, then a small generate script takes all the TOML files in the data directory and creates empty files for each agency with the same name, but a .md extension. The reason for the empty files is the use of [[cascade]] in the _index.md file to cascade attributes down to every sub page in the bundle. The templates for each agency have some code in them like this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20

  <ol class="year-list">
    {{ range .CurrentSection.Pages.GroupByDate "2006" "asc" }}
      <li class="year-grades">
        <div class="year">{{ .Key }}</div>
        <ol class="grades-list">
          {{ range .Pages }}
            {{ $report := path.BaseName .File }}
            {{ $agency := strings.ToLower .Params.acronym }}
            {{ $grade := (index (index .Site.Data.reports $report).data $agency).grade }}
            <li class="{{ substr $grade 0 1 | strings.ToLower }}">
              <a href="{{ .RelPermalink }}">
                <time datetime="{{ .Date.Format `2006-01-02` }}">{{ .Date.Format "Jan 2006" }}</time>
                <span class="letter">{{ $grade }}</span>
              </a>
            </li>
          {{ end }}
        </ol>
      </li>
    {{ end }}
  </ol>

The star of the show is indexing .Site.Data.reports with the report and then the agency’s acronym. This particular block of code generates the entire grade history as a set of links that the user can click on to go to historic scorecards.