Skip to content

Conversation

@MathiasVP
Copy link
Collaborator

@MathiasVP MathiasVP commented Apr 18, 2025

@chanel-y ran the first campaign across all Microsoft PowerShell code yesterday (🥳)

However, they all failed (😭) because of a broken upgrade-script chain.

A broken what?

We want to be able to analyze database produced by old extractors. This is hard because we may make changes to the way the database is structured (i.e., the schema outputted by the extractor) over time.

How we solve dbscheme mismatches

Every time we make changes to the database schema file (colloquially called "the dbscheme") we need to add a pair of upgrade and downgrade scripts which tells the QL engine:

  1. How to transform a database produced by an old extractor to look like a database produced by a new extractor, and
  2. How to transform a database produced by a new extractor to look like a database produced by an old extractor.

(1) ensures that we can analyze old databases with new QL code, and (2) ensures that we can analyze new databases with old QL code. Arguably, the first one is the most important since old databases tend to stay in use for a long time.

GitHub has a document describing the process of creating up-and downgrade scripts here if you're curious about what these look like.

The current problem

For the few times I've modified the database scheme I've provided these scripts (see, for example, #88 and #100). However, it looks like this was missed at some point before I joined. So the current situation is that:

  • There are Microsoft DBs with dbscheme X1
  • The current dbscheme is Xn
  • We have upgrade scripts for X2 -> X3 -> ... -> Xn
  • We are missing an upgrade script for X1 -> X2

This PR adds that upgrade missing upgrade script. This completes the chain so that we can analyze databases with the X1 dbscheme using the current QL code (which assumes dbscheme Xn).

@MathiasVP MathiasVP changed the title Add missing downgrade and upgrade scripts PS: Add missing downgrade and upgrade scripts Apr 18, 2025
@MathiasVP MathiasVP merged commit 21210c6 into main Apr 18, 2025
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants