JS: Add overlay support to extractor

[asgerf]

This contains some of the initial changes needed for overlay support for JavaScript and TypeScript:

• Extractor changes
• DBscheme and upgrade/downgrade scripts
• Discard predicates

It does not include

• Overlay annotations (except for discard predicates)
• compileForOverlayEval, to be enabled separately when it does not cause a slowdown

[Copilot]

Pull Request Overview

This PR implements initial overlay support for JavaScript and TypeScript extractors to enable incremental analysis of codebases. It adds the necessary infrastructure for tracking file changes, discarding outdated entities, and managing database metadata while preparing for future overlay evaluation capabilities.

• Database schema updates with overlay-specific relations and support version
• Java extractor changes to filter extraction based on changed files and write overlay metadata
• CodeQL library support for identifying and discarding obsolete entities in overlay scenarios

Reviewed Changes

Copilot reviewed 13 out of 13 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
javascript/resources/codeql-extractor.yml	Adds overlay support version configuration
javascript/ql/lib/upgrades/ccefb5e2d49318eea4aeafd4c6ae2af9f94ac72a/upgrade.properties	Defines upgrade metadata for overlay relations
javascript/ql/lib/upgrades/ccefb5e2d49318eea4aeafd4c6ae2af9f94ac72a/semmlecode.javascript.dbscheme	New database schema with overlay support
javascript/ql/lib/upgrades/ccefb5e2d49318eea4aeafd4c6ae2af9f94ac72a/old.dbscheme	Previous database schema for upgrade comparison
javascript/ql/lib/semmlecode.javascript.dbscheme.stats	Statistics for overlay relations
javascript/ql/lib/semmlecode.javascript.dbscheme	Main schema with overlay metadata and changed files tables
javascript/ql/lib/semmle/javascript/internal/Overlay.qll	Overlay logic for discarding outdated entities
javascript/ql/lib/javascript.qll	Imports overlay support module
javascript/extractor/src/com/semmle/js/extractor/OverlayChanges.java	Data class for overlay change information
javascript/extractor/src/com/semmle/js/extractor/AutoBuild.java	Extractor modifications for overlay file filtering and metadata writing
javascript/downgrades/76a926a00d5f3bc199c203a1437796fd7b2835ba/upgrade.properties	Downgrade configuration for overlay relations
javascript/downgrades/76a926a00d5f3bc199c203a1437796fd7b2835ba/semmlecode.javascript.dbscheme	Downgrade schema without overlay support

[nickrolfe]

This looks sensible. Have you tested that it still works when the base and overlay are extracted using different source locations, so the CLI provides a path transformer when extracting the overlay?

[asgerf]

Have you tested that it still works when the base and overlay are extracted using different source locations, so the CLI provides a path transformer when extracting the overlay?

No. Does DCA test this?

[nickrolfe]

Have you tested that it still works when the base and overlay are extracted using different source locations, so the CLI provides a path transformer when extracting the overlay?

No. Does DCA test this?

Unfortunately not, it currently uses the same source location for both. Once you have overlay compilation enabled, we can write an integration test to cover this.

For now, I tested it by manually creating an overlay database, and I can see it's working as expected. Specifically:

• the different source location doesn't interfere with the filtering of filesToExtract.
• paths in the TRAP are transformed appropriately.