APS AI Tracker

Reading the data

How the tracker records changes, reads statements and counts what they say. The examples and caveats below show how to check a result against the text.

Anatomy of a change

The tracker checks most statements nightly; some sites have to be checked by hand. After a check that the copy captured the statement properly, any copy whose text differs from the last one is a change. A language model reads each change twice: for how much the text changed, and for which of the statement’s answers changed.

Both readings record what a statement says, which may differ from what an agency does. Silence about a use-case register does not prove the agency lacks one; a claim to have one does not prove it exists.

An example

Here is one change as the timeline shows it, from the Federal Court of Australia.

  1. The header gives the date of the copy, the agency and how many characters the text gained or lost. The label is the change’s tier: substantive, because reporting an appointment is a new fact.
  2. The sentence and the points under it are the first reading: what changed, in the model’s words. A point starting “Removed” is shown in red.
  3. The coloured chips are the second reading: the answers that changed. Red (−) is an answer dropped, green (+) one added, grey (~) one altered.
  4. The words that changed are the diff both readings came from, with deleted text struck through in red and inserted text underlined in green.

The grey chip marks the promise to appoint a Chief AI Officer as kept: the new version reports the appointment, so it no longer needs to make the promise. A promise that disappears without being reported done shows in red, as dropped.

How much the text changed

Each change is put in one of four tiers.

Noise
Nothing the agency wrote changed: a sidebar rotated, a “last reviewed 3 days ago” counter ticked over, or the page was rebuilt around the same words.
Cosmetic
Wording, punctuation, formatting or a link changed, but nothing of substance.
Reworded
The text was rewritten or rearranged, but makes the same claims and commitments.
Substantive
A claim, commitment, fact or disclosure was added, removed or altered.

Substantive and reworded changes appear on the timeline and statement pages, each with a one-sentence summary and a list of the points that changed. “Changes of substance” counts only the substantive tier. Noise and cosmetic changes are hidden unless you ask for them.

Which answers changed

The model also reads every version of every statement into a profile: a fixed set of questions, each with a fixed set of possible answers. Most of the questions come from the three documents behind the statements. The DTA’s Standard lists eight things every statement must contain, policy v2.0 adds five requirements, and the APS AI Plan asks each agency for a Chief AI Officer. The rest are ours, and are marked as such.

Comparing a version’s profile with the one before shows which answers were added, dropped or altered, such as “Chief AI Officer: planned → in place”. A bullet point expanded into a paragraph about the same thing changes no answer. Profiles record job titles, never people’s names.

Where the questions come from

Who’s who explains how the three documents fit together. A question is a requirement only if its source makes it one: the Chief AI Officer comes from the AI Plan, not the policy. Usage patterns and domains use the DTA’s own categories.

DTA Standard for AI transparency statements v2.0 (minimum content)
  • Intentions behind AI use
  • Usage patterns and domains in use (Attachment A classification)
  • Whether the public interacts with or is affected by AI without human review
  • Measures to monitor effectiveness and protect the public
  • Compliance with the policy
  • Compliance with legislation
  • When the statement was last updated
  • A public contact
Policy for the responsible use of AI in government v2.0 (mandatory requirements)
  • Review cadence (annually, or sooner on a significant change)
  • Accountable official designated
  • Strategic position on AI (due within 6 months of v2.0)
  • Internal AI use-case register (due within 12 months)
  • Mandatory staff training (due within 12 months)
AI Plan for the Australian Public Service 2025 (Department of Finance)
  • Chief AI Officer (due July 2026)
This tracker's own reading, not required by any instrument
  • An explicit commitment to a human intermediary for public-facing AI
  • Explicit commitments: what the agency will do, will not do, or keep under human oversight
  • Named safeguards (risk assessment, audit, testing and so on)
  • Named AI tools
  • Which policy version the statement refers to
  • A stated first-published date

Checking a reading

Both readings come from a model, and either can be wrong. When an agency rewrites a section, a commitment it only reworded can show as dropped, with the new wording shown as added; the diff beside each one tells you which. A vaguely described role can also land in the wrong category.

Every summary names the model that wrote it and sits beside the diff, and every profile sits beside its statement, so you can check one against the other. The downloads have every reading in bulk.

When the record starts

Tracking began on 11 Nov 2025. Statements have been required since February 2025, so their first nine months aren’t in the record. Every statement that already existed enters on day one as “first tracked”, because we don’t know when it was published or how it read before then. If you have an earlier version, we’d like to add it.

Adoption over time

What the statements say charts, for each month, the share of tracked statements whose profile reports each item (a Chief AI Officer, a register, mandatory training, a reference to policy v2.0). Agencies joined the tracker in waves, so a jump between months can come from new statements rather than new text, and early months have fewer statements (121 are tracked now).

Two lists on that page use the statements’ own dates. A statement counts as updated since policy v2.0 if its stated last-updated date, or a change we saw, falls on or after 15 December 2025. It is listed as not updated if we have watched it since before then and seen neither; otherwise we can’t tell. The Standard asks for a review at least once a year, so the second list shows statements whose own date is more than a year old. A statement with no date can’t appear on it.

Shared wording

On each statement page, an amber tint marks a passage that also appears in other agencies’ statements. The deeper the amber, the more agencies share it:

Open the small count on an amber passage to see which agencies share it.

A grey tint marks wording from the policy or the DTA’s template, which most agencies start from. It has no count, because it can’t be traced to particular agencies.

First observed

For each shared passage we record which statement had it first and the order in which others picked it up. How far that order can be trusted puts each passage in one of three tiers:

This is the order we saw, and it doesn’t show who wrote a passage. The source could be an agency we don’t track, a draft that never went online, or the DTA template.

Originality scores

A statement’s originality score is the share of its text that is neither shared with other agencies nor from the template. A statement at 80% is mostly in its own words; one at 20% is mostly shared. The leaderboard and the agency grid colour statements on this scale, from borrowed to bespoke.

The score doesn’t measure effort. Shared text includes wording the Standard asks for (the title, the section headings, the OECD definition of an AI system), so following the Standard closely lowers the score. A statement published jointly by two bodies, such as the AASB and AUASB, counts as shared with itself.

Limitations

A capture sometimes picks up parts of the page around the statement. The model usually tells the two apart, but not always, so check the source link on any statement. The tracker is open source, for anyone who wants the technical method.