# Gauge

> Page counts for Knowledge Futures projects. No cookies, no fingerprints: one script tag, and daily numbers as files a static site can read.

Base URL: https://gauge.knowledgefutures.org

A **site** is one project's environment (for example a journal's production
website): the hosts its pages are served from, the KF billing account its
events are billed to, and a secret for its server. Pages send events with
one script tag; the project's server may send events with the secret. Every
night the events become daily summary files ("cubes") per site, which a
browser can read directly with HTTP Range requests.

## 1. Get a token (a person approves it in their browser)

Creating a site needs a token from a KF staff member. Use the device flow;
do not ask the person for a password or a secret.

```sh
curl -s -X POST https://gauge.knowledgefutures.org/v1/auth/device
# → {"device_code":"…","user_code":"BCDF-GHJK","verification_uri_complete":"https://gauge.knowledgefutures.org/device?code=BCDF-GHJK","expires_in":600,"interval":5}
```

Show the person `verification_uri_complete` and the `user_code`. They open
the link, sign in with KF Auth, check the code matches, and press Approve.
Meanwhile poll, no faster than `interval` seconds:

```sh
curl -s -X POST https://gauge.knowledgefutures.org/v1/auth/token \
  -H 'content-type: application/x-www-form-urlencoded' \
  -d 'grant_type=urn:ietf:params:oauth:grant-type:device_code&device_code=<device_code>'
# → 400 {"error":"authorization_pending"} until approved, then
# → 200 {"access_token":"…","token_type":"Bearer","expires_in":900}
```

Other errors: `slow_down` (poll less often), `access_denied` (the person
declined), `expired_token` (start again). The token lasts 15 minutes and is
handed out once. Keep it in memory only. Only KF staff can approve tokens
for now.

## 2. Create a site

```sh
curl -s -X POST https://gauge.knowledgefutures.org/v1/sites \
  -H "authorization: Bearer $TOKEN" -H 'content-type: application/json' \
  -d '{"name":"Example Journal","account":"<billing account id>","origins":["journal.example.org"]}'
```

Fields:

- `account` (required): the KF billing account id (the console account's
  id, a UUID). Ask the person if you don't know it.
- `origins` (required): the hosts the site's pages are served from, e.g.
  `["journal.example.org", "localhost:4321"]`. A host belongs to one site;
  a host another site holds answers 409 and names it.
- `name`: a label for people.
- `accountName`: what the console calls the account (`"PubPub"`), for
  the dashboards. The newest one given goes on all the account's sites.
- `project`: the KF project key, `<account>/<project>`, if it has one.
- `projectName`: what the console calls the project.
- `environment`: default `production`.
- `anyOrigin`: `true` lets a keyed beacon count from any host. For
  platforms whose customers bring their own domains.
- `cubes`: who may read the summary files from a browser: `origins`
  (default: pages on the site's own hosts), `public`, or `private` (the
  secret or a signed URL only). See section 5.

The answer (201) holds `secret`, shown once, plus `KF_GAUGE_URL`
and the `beacon` tag. Store the secret as a server-side secret of the
project (for example `KF_GAUGE_URL` in its secrets); never put it in
a page or a repository.

Also: `GET /v1/sites`, `GET /v1/sites/<id>`, `PATCH /v1/sites/<id>` (name,
accountName, projectName, origins, anyOrigin, cubes), `POST /v1/sites/<id>/rotate-secret`,
`DELETE /v1/sites/<id>` (stops collecting, keeps the data), and
`GET /v1/sites/<id>/usage?from=YYYY-MM-DD` (events per day: `n` from
readers, which is what is billed; `bots`, kept but not counted; `dropped`,
refused as attacks).

## 3. Add the tag to every page

```html
<script src="https://gauge.knowledgefutures.org/v1/b.js" defer></script>
```

The page's host decides the site, so the tag is the same for every project.
With `anyOrigin`, add `data-site="<site id>"`. It records one view per page
load, sends no cookies and stores nothing in the browser. Optional, in the
page's `<head>`:

```html
<meta name="kf:subject" content="pub:za65x7nh">      <!-- what the page is about: <type>:<id> -->
<meta name="kf:subject-uri" content="https://doi.org/10.1234/x">
<meta name="kf:scope" content="<tenant id>">         <!-- a platform's own customer, e.g. a community -->
```

Clicks on `data-kf-event="download:pdf"` elements are recorded as that
event; `window.kf.track(kind, name, props)` records anything else. Kinds:
`view`, `download`, `search`, `custom` (needs a name).

## 4. Server events (optional)

For numbers that matter (downloads you report to a funder), send from the
server, signed with the secret:

```sh
curl -s -X POST https://gauge.knowledgefutures.org/v1/e -u '<site id>:<secret>' -H 'content-type: application/json' \
  -d '{"kind":"download","subject":"pub:za65x7nh","name":"pdf","country":"DE","userAgent":"<reader UA>"}'
# → 204
```

A body is at most 4 KB; unknown keys are refused (422) — put extra detail in
`props` (up to 16 string values). `?echo=1` returns the stored row instead of
storing it, for checking. Pass the reader's `userAgent`: an event from a bot
is kept but not counted. An event whose values look like an attack (SQL or
script injection, path traversal and the like) is answered 204 and dropped;
`?echo=1` answers `{"dropped": "<field>: <why>"}` for it.

## 5. Read the numbers

Every night the events up to yesterday (UTC) become one Parquet file per cube:

- `days.parquet` — dimensions day, scope; counts views, visits, downloads, searches, custom
- `subjects.parquet` — dimensions day, scope, subject_type, subject_id; counts views, visits, downloads
- `pages.parquet` — dimensions day, scope, path; counts views, visits
- `referrers.parquet` — dimensions day, scope, referrer_host; counts views, visits
- `campaigns.parquet` — dimensions day, scope, utm_source, utm_medium, utm_campaign; counts views, visits
- `devices.parquet` — dimensions day, scope, device; counts views, visits
- `countries.parquet` — dimensions day, scope, country; counts views, visits
- `downloads.parquet` — dimensions day, scope, subject_type, subject_id, name; counts downloads
- `searches.parquet` — dimensions day, scope, name; counts searches

plus `meta.json` (first and last day, totals, when it was built).

```
https://gauge.knowledgefutures.org/v1/cubes/<site id>/days.parquet
```

Every row is one day, every number a count, so any date range is a sum of
rows. Bots are already removed. "visits" are views that arrived from another
site or from nowhere, not unique people. The files support HEAD and Range,
so a browser reads only the parts it needs, e.g. with hyparquet:

```js
import { asyncBufferFromUrl, parquetReadObjects } from 'hyparquet'
const file = await asyncBufferFromUrl({ url: 'https://gauge.knowledgefutures.org/v1/cubes/<site id>/days.parquet' })
const rows = await parquetReadObjects({ file })
```

Who can read them depends on the site's `cubes` setting:

- `origins` (default): a page on one of the site's hosts (checked by the
  browser's Origin header). Good for a static site's own stats page. Other
  websites can't embed them, but they are not secret.
- `public`: anyone.
- `private`: only with the secret (`-u '<site id>:<secret>'`) or a signed URL.

A signed URL lets a page read a file for a while without the secret, e.g.
a dashboard behind the project's own login. On the server:

```js
const hex = (b) => [...new Uint8Array(b)].map((x) => x.toString(16).padStart(2, '0')).join('')
const enc = new TextEncoder()
const keyHex = hex(await crypto.subtle.digest('SHA-256', enc.encode(secret)))   // the key is sha256(secret), hex
const key = await crypto.subtle.importKey('raw', enc.encode(keyHex), { name: 'HMAC', hash: 'SHA-256' }, false, ['sign'])
const exp = Math.floor(Date.now() / 1000) + 3600
const mac = new Uint8Array(await crypto.subtle.sign('HMAC', key, enc.encode(`${site}/${file}:${exp}`)))
const sig = btoa(String.fromCharCode(...mac)).replace(/\+/g, '-').replace(/\//g, '_').replace(/=+$/, '')
const url = `https://gauge.knowledgefutures.org/v1/cubes/${site}/${file}?exp=${exp}&sig=${sig}`
```

The raw events are never served.

A new site's first cubes appear the night after its first events. To see them
sooner, rebuild that site's cubes now (your token; it takes a few minutes, and
events reach the lake about five minutes after they are sent):

```sh
curl -s -X POST https://gauge.knowledgefutures.org/v1/compact -H "authorization: Bearer $TOKEN" \
  -H 'content-type: application/json' -d '{"site":"<site id>"}'
curl -s https://gauge.knowledgefutures.org/v1/compact -H "authorization: Bearer $TOKEN"   # "last" says how it went
```

## Not yet

- The console doesn't register sites yet; this API is the interim way, and the console will call
  it with its own key.
- Only KF staff can approve tokens.