Skip to content

Commit 8c9400f

Browse files
author
Robert Marsh
committed
Merge branch 'master' into rdmarsh/cpp/ir-gvn-ast-wrapper
This expands the refactor to cover the C# IR GVN libraries and fixes up ValueNumberingInternal.qll to match API changes from master
2 parents b5baa9a + cd1a3e9 commit 8c9400f

File tree

2,054 files changed

+112386
-58137
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

2,054 files changed

+112386
-58137
lines changed

.codeqlmanifest.json

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
{ "provide": [ "*/ql/src/qlpack.yml",
2+
"*/ql/test/qlpack.yml",
3+
"*/upgrades/qlpack.yml",
4+
"misc/legacy-support/*/qlpack.yml",
5+
"misc/suite-helpers/qlpack.yml",
6+
"codeql/.codeqlmanifest.json" ] }
Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
---
2+
name: General issue
3+
about: Tell us if you think something is wrong or if you have a question
4+
title: General issue
5+
labels: question
6+
assignees: ''
7+
8+
---
9+
10+
**Description of the issue**
11+
12+
<!-- Please explain briefly what is the problem.
13+
If it is about an LGTM project, please include its URL.-->
14+

.github/labeler.yml

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
"C++":
2+
- cpp/**/*
3+
- change-notes/**/*cpp*
4+
5+
"C#":
6+
- csharp/**/*
7+
- change-notes/**/*csharp*
8+
9+
Java:
10+
- java/**/*
11+
- change-notes/**/*java.*
12+
13+
JS:
14+
- javascript/**/*
15+
- change-notes/**/*javascript*
16+
17+
Python:
18+
- python/**/*
19+
- change-notes/**/*python*
20+
21+
documentation:
22+
- "**/*.qhelp"
23+
- "**/*.md"
24+
- docs/**/*

.gitignore

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,3 +12,6 @@
1212
# Visual studio temporaries, except a file used by QL4VS
1313
.vs/*
1414
!.vs/VSWorkspaceSettings.json
15+
16+
# It's useful (though not required) to be able to unpack codeql in the ql checkout itself
17+
/codeql/

CODEOWNERS

Lines changed: 7 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,11 @@
1+
/cpp/ @Semmle/cpp-analysis
12
/csharp/ @Semmle/cs
23
/java/ @Semmle/java
34
/javascript/ @Semmle/js
4-
/cpp/ @Semmle/cpp-analysis
5-
/cpp/**/*.qhelp @semmledocs-ac
5+
/python/ @Semmle/python
6+
/cpp/**/*.qhelp @hubwriter
67
/csharp/**/*.qhelp @jf205
7-
/java/**/*.qhelp @felicity-semmle
8-
/javascript/**/*.qhelp @mc-semmle
9-
/python/**/*.qhelp @felicity-semmle
10-
/docs/language/ @felicity-semmle @jf205
8+
/java/**/*.qhelp @felicitymay
9+
/javascript/**/*.qhelp @mchammer01
10+
/python/**/*.qhelp @felicitymay
11+
/docs/language/ @shati-patel @jf205

CONTRIBUTING.md

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
# Contributing to QL
1+
# Contributing to CodeQL
22

33
We welcome contributions to our standard library and standard checks. Got an idea for a new check, or how to improve an existing query? Then please go ahead and open a pull request!
44

@@ -9,13 +9,13 @@ Before we accept your pull request, we require that you have agreed to our Contr
99
If you have an idea for a query that you would like to share with other Semmle users, please open a pull request to add it to this repository.
1010
Follow the steps below to help other users understand what your query does, and to ensure that your query is consistent with the other Semmle queries.
1111

12-
1. **Consult the QL documentation for query writers**
12+
1. **Consult the documentation for query writers**
1313

14-
There is lots of useful documentation to help you write QL, ranging from information about query file structure to language-specific tutorials. For more information on the documentation available, see [Writing QL queries](https://help.semmle.com/QL/learn-ql/writing-queries/writing-queries.html) on [help.semmle.com](https://help.semmle.com).
14+
There is lots of useful documentation to help you write queries, ranging from information about query file structure to tutorials for specific target languages. For more information on the documentation available, see [Writing CodeQL queries](https://help.semmle.com/QL/learn-ql/writing-queries/writing-queries.html) on [help.semmle.com](https://help.semmle.com).
1515

16-
2. **Format your QL correctly**
16+
2. **Format your code correctly**
1717

18-
All of Semmle's standard QL queries and libraries are uniformly formatted for clarity and consistency, so we strongly recommend that all QL contributions follow the same formatting guidelines. If you use QL for Eclipse, you can auto-format your query in the [QL editor](https://help.semmle.com/ql-for-eclipse/Content/WebHelp/ql-editor.html). For more information, see the [QL style guide](https://github.com/Semmle/ql/blob/master/docs/ql-style-guide.md).
18+
All of Semmle's standard queries and libraries are uniformly formatted for clarity and consistency, so we strongly recommend that all contributions follow the same formatting guidelines. If you use QL for Eclipse, you can auto-format your query in the [QL editor](https://help.semmle.com/ql-for-eclipse/Content/WebHelp/ql-editor.html). For more information, see the [CodeQL style guide](https://github.com/Semmle/ql/blob/master/docs/ql-style-guide.md).
1919

2020
3. **Make sure your query has the correct metadata**
2121

@@ -29,7 +29,7 @@ Follow the steps below to help other users understand what your query does, and
2929
The `select` statement of your query must be compatible with the query type (determined by the `@kind` metadata property) for alert or path results to be displayed correctly in LGTM and QL for Eclipse.
3030
For more information on `select` statement format, see [Introduction to query files](https://help.semmle.com/QL/learn-ql/writing-queries/introduction-to-queries.html#select-clause) on help.semmle.com.
3131

32-
5. **Save your query in a `.ql` file in correct language directory in this repository**
32+
5. **Save your query in a `.ql` file in the correct language directory in this repository**
3333

3434
There are five language-specific directories in this repository:
3535

@@ -54,7 +54,7 @@ repositories, which might be made public. We might also use this information
5454
to contact you in relation to your contributions, as well as in the
5555
normal course of software development. We also store records of your
5656
CLA agreements. Under GDPR legislation, we do this
57-
on the basis of our legitimate interest in creating the QL product.
57+
on the basis of our legitimate interest in creating the CodeQL product.
5858

5959
Please do get in touch ([email protected]) if you have any questions about
6060
this or our data protection policies.

README.md

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,16 +1,16 @@
1-
# Semmle QL
1+
# CodeQL
22

3-
This open source repository contains the standard QL libraries and queries that power [LGTM](https://lgtm.com), and the other products that [Semmle](https://semmle.com) makes available to its customers worldwide.
3+
This open source repository contains the standard CodeQL libraries and queries that power [LGTM](https://lgtm.com), and the other products that [Semmle](https://semmle.com) makes available to its customers worldwide.
44

5-
## How do I learn QL and run queries?
5+
## How do I learn CodeQL and run queries?
66

7-
There is [extensive documentation](https://help.semmle.com/QL/learn-ql/) on getting started with writing QL.
8-
You can use the [interactive query console](https://lgtm.com/help/lgtm/using-query-console) on LGTM.com or the [QL for Eclipse](https://lgtm.com/help/lgtm/running-queries-ide) plugin to try out your queries on any open-source project that's currently being analyzed.
7+
There is [extensive documentation](https://help.semmle.com/QL/learn-ql/) on getting started with writing CodeQL.
8+
You can use the [interactive query console](https://lgtm.com/help/lgtm/using-query-console) on LGTM.com or the [CodeQL for Visual Studio Code](https://help.semmle.com/codeql/codeql-for-vscode.html) extension to try out your queries on any open source project that's currently being analyzed.
99

1010
## Contributing
1111

12-
We welcome contributions to our standard library and standard checks. Do you have an idea for a new check, or how to improve an existing query? Then please go ahead and open a pull request! Before you do, though, please take the time to read our [contributing guidelines](CONTRIBUTING.md). You can also consult our [style guides](https://github.com/Semmle/ql/tree/master/docs) to learn how to format your QL for consistency and clarity, how to write query metadata, and how to write query help documentation for your query.
12+
We welcome contributions to our standard library and standard checks. Do you have an idea for a new check, or how to improve an existing query? Then please go ahead and open a pull request! Before you do, though, please take the time to read our [contributing guidelines](CONTRIBUTING.md). You can also consult our [style guides](https://github.com/Semmle/ql/tree/master/docs) to learn how to format your code for consistency and clarity, how to write query metadata, and how to write query help documentation for your query.
1313

1414
## License
1515

16-
The QL queries in this repository are licensed under [Apache License 2.0](LICENSE) by [Semmle](https://semmle.com).
16+
The code in this repository is licensed under [Apache License 2.0](LICENSE) by [Semmle](https://semmle.com).

change-notes/1.22/analysis-javascript.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,7 @@
3636
| Shift out of range (`js/shift-out-of-range`| Fewer false positive results | This rule now correctly handles BigInt shift operands. |
3737
| Superfluous trailing arguments (`js/superfluous-trailing-arguments`) | Fewer false-positive results. | This rule no longer flags calls to placeholder functions that trivially throw an exception. |
3838
| Undocumented parameter (`js/jsdoc/missing-parameter`) | No changes to results | This rule is now run on LGTM, although its results are still not shown by default. |
39+
| Missing space in string concatenation (`js/missing-space-in-concatenation`) | Fewer false positive results | The rule now requires a word-like part exists in the string concatenation. |
3940

4041
## Changes to QL libraries
4142

change-notes/1.23/analysis-cpp.md

Lines changed: 47 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -2,43 +2,65 @@
22

33
The following changes in version 1.23 affect C/C++ analysis in all applications.
44

5-
## General improvements
6-
75
## New queries
86

97
| **Query** | **Tags** | **Purpose** |
108
|-----------------------------|-----------|--------------------------------------------------------------------|
11-
| Hard-coded Japanese era start date (`cpp/japanese-era/exact-era-date`) | reliability, japanese-era | This query is a combination of two old queries that were identical in purpose but separate as an implementation detail. This new query replaces Hard-coded Japanese era start date in call (`cpp/japanese-era/constructor-or-method-with-exact-era-date`) and Hard-coded Japanese era start date in struct (`cpp/japanese-era/struct-with-exact-era-date`). |
9+
| Hard-coded Japanese era start date (`cpp/japanese-era/exact-era-date`) | reliability, japanese-era | This query is a combination of two old queries that were identical in purpose but separate as an implementation detail. This new query replaces Hard-coded Japanese era start date in call (`cpp/japanese-era/constructor-or-method-with-exact-era-date`) and Hard-coded Japanese era start date in struct (`cpp/japanese-era/struct-with-exact-era-date`). Results are not shown on LGTM by default. |
10+
| Pointer overflow check (`cpp/pointer-overflow-check`) | correctness, security | Finds overflow checks that rely on pointer addition to overflow, which has undefined behavior. Example: `ptr + a < ptr`. Results are shown on LGTM by default. |
11+
| Signed overflow check (`cpp/signed-overflow-check`) | correctness, security | Finds overflow checks that rely on signed integer addition to overflow, which has undefined behavior. Example: `a + b < a`. Results are shown on LGTM by default. |
12+
1213

1314
## Changes to existing queries
1415

1516
| **Query** | **Expected impact** | **Change** |
1617
|----------------------------|------------------------|------------------------------------------------------------------|
17-
| Query name (`query id`) | Expected impact | Message. |
18+
| Comparison of narrow type with wide type in loop condition (`cpp/comparison-with-wider-type`) | Higher precision | The precision of this query has been increased to "high" as the alerts from this query have proved to be valuable on real-world projects. With this precision, results are now displayed by default in LGTM. |
1819
| Hard-coded Japanese era start date in call (`cpp/japanese-era/constructor-or-method-with-exact-era-date`) | Deprecated | This query has been deprecated. Use the new combined query Hard-coded Japanese era start date (`cpp/japanese-era/exact-era-date`) instead. |
1920
| Hard-coded Japanese era start date in struct (`cpp/japanese-era/struct-with-exact-era-date`) | Deprecated | This query has been deprecated. Use the new combined query Hard-coded Japanese era start date (`cpp/japanese-era/exact-era-date`) instead. |
2021
| Hard-coded Japanese era start date (`cpp/japanese-era/exact-era-date`) | More correct results | This query now checks for the beginning date of the Reiwa era (1st May 2019). |
21-
| Too few arguments to formatting function (`cpp/wrong-number-format-arguments`) | Fewer false positive results | Fixed false positives resulting from mistmatching declarations of a formatting function. |
22-
| Too many arguments to formatting function (`cpp/too-many-format-arguments`) | Fewer false positive results | Fixed false positives resulting from mistmatching declarations of a formatting function. |
23-
24-
## Changes to QL libraries
25-
26-
* The data-flow library has been extended with a new feature to aid debugging.
27-
Instead of specifying `isSink(Node n) { any() }` on a configuration to
28-
explore the possible flow from a source, it is recommended to use the new
29-
`Configuration::hasPartialFlow` predicate, as this gives a more complete
30-
picture of the partial flow paths from a given source. The feature is
31-
disabled by default and can be enabled for individual configurations by
32-
overriding `int explorationLimit()`.
33-
* The data-flow library now allows flow through the address-of operator (`&`).
34-
* The `DataFlow::DefinitionByReferenceNode` class now considers `f(x)` to be a
35-
definition of `x` when `x` is a variable of pointer type. It no longer
36-
considers deep paths such as `f(&x.myField)` to be definitions of `x`. These
37-
changes are in line with the user expectations we've observed.
38-
* There is now a `DataFlow::localExprFlow` predicate and a
39-
`TaintTracking::localExprTaint` predicate to make it easy to use the most
40-
common case of local data flow and taint: from one `Expr` to another.
22+
| Non-constant format string (`cpp/non-constant-format`) | Fewer false positive results | Fixed false positive results triggrered by mismatching declarations of a formatting function. |
23+
| Sign check of bitwise operation (`cpp/bitwise-sign-check`) | Fewer false positive results | Results involving `>=` or `<=` are no longer reported. |
24+
| Too few arguments to formatting function (`cpp/wrong-number-format-arguments`) | Fewer false positive results | Fixed false positive results triggered by mismatching declarations of a formatting function. |
25+
| Too many arguments to formatting function (`cpp/too-many-format-arguments`) | Fewer false positive results | Fixed false positive results triggered by mismatching declarations of a formatting function. |
26+
| Unclear comparison precedence (`cpp/comparison-precedence`) | Fewer false positive results | False positive results involving template classes and functions have been fixed. |
27+
| Wrong type of arguments to formatting function (`cpp/wrong-type-format-argument`) | More correct results and fewer false positive results | This query now understands explicitly-specified argument numbers in format strings, such as the `1$` in `%1$s`. |
28+
29+
## Changes to libraries
30+
31+
* The data-flow library in `semmle.code.cpp.dataflow.DataFlow` and
32+
`semmle.code.cpp.dataflow.TaintTracking` have had extensive changes:
33+
* Data flow through fields is now more complete and reliable.
34+
* The data-flow library has been extended with a new feature to aid debugging.
35+
Previously, to explore the possible flow from all sources you could specify `isSink(Node n) { any() }` on a configuration.
36+
Now you can use the new `Configuration::hasPartialFlow` predicate,
37+
which gives a more complete picture of the partial flow paths from a given source, including flow that doesn't reach any sink.
38+
The feature is disabled by default and can be enabled for individual configurations by overriding `int explorationLimit()`.
39+
* There is now flow out of C++ reference parameters.
40+
* There is now flow through the address-of operator (`&`).
41+
* The `DataFlow::DefinitionByReferenceNode` class now considers `f(x)` to be a
42+
definition of `x` when `x` is a variable of pointer type. It no longer
43+
considers deep paths such as `f(&x.myField)` to be definitions of `x`. These
44+
changes are in line with the user expectations we've observed.
45+
* It's now easier to specify barriers/sanitizers
46+
arising from guards by overriding the predicate
47+
`isBarrierGuard`/`isSanitizerGuard` on data-flow and taint-tracking
48+
configurations respectively.
49+
* There is now a `DataFlow::localExprFlow` predicate and a
50+
`TaintTracking::localExprTaint` predicate to make it easy to use the most
51+
common case of local data flow and taint: from one `Expr` to another.
4152
* The member predicates of the `FunctionInput` and `FunctionOutput` classes have been renamed for
42-
clarity (e.g. `isOutReturnPointer()` to `isReturnValueDeref()`). The existing member predicates
53+
clarity (for example, `isOutReturnPointer()` to `isReturnValueDeref()`). The existing member predicates
4354
have been deprecated, and will be removed in a future release. Code that uses the old member
4455
predicates should be updated to use the corresponding new member predicate.
56+
* The predicate `Declaration.hasGlobalOrStdName` has been added, making it
57+
easier to recognize C library functions called from C++.
58+
* The control-flow graph is now computed in QL, not in the extractor. This can
59+
lead to changes in how queries are optimized because
60+
optimization in QL relies on static size estimates, and the control-flow edge
61+
relations will now have different size estimates than before.
62+
* Support has been added for non-type template arguments. This means that the
63+
return type of `Declaration::getTemplateArgument()` and
64+
`Declaration::getATemplateArgument` have changed to `Locatable`. For details, see the
65+
CodeQL library documentation for `Declaration::getTemplateArgument()` and
66+
`Declaration::getTemplateArgumentKind()`.

0 commit comments

Comments
 (0)