profile
viewpoint
Max Schaefer max-schaefer @Semmle and @github Oxford, UK

github/codeql-go 175

The CodeQL extractor and libraries for Go.

max-schaefer/codeql-go 0

The CodeQL extractor and libraries for Go.

max-schaefer/eosio-toppings 0

A monorepo with tools working on top of nodeos

max-schaefer/govim 0

govim is a Go development plugin for Vim8, written in Go

max-schaefer/pannellum 0

Pannellum is a lightweight, free, and open source panorama viewer for the web.

max-schaefer/ql 0

The standard QL libraries and queries that power LGTM.com and other Semmle Products

max-schaefer/serialize-to-js 0

serialize objects to javascript

max-schaefer/test-repo 0

A repository for testing various features of LGTM.com

max-schaefer/vscode-codeql-starter 0

Starter workspace to use with the CodeQL extension for Visual Studio Code.

shuvendu-lahiri/ql 0

The libraries and queries that power CodeQL and LGTM.com

create barnchmax-schaefer/ql

branch : js/api-graph-reexport

created branch time in 6 hours

PR opened github/codeql

JavaScript: Teach API graphs to recognise arguments supplied in partial function applications.

Evaluation (internal link) shows a few performance wobbles on the first run, which went away on rerunning. Also note the metrics towards the bottom of the report, which show no significant changes to API graph size or complexity.

+45 -12

0 comment

5 changed files

pr created time in 6 hours

PullRequestReviewEvent

Pull request review commentgithub/codeql

JS: Fixing an API-graph gotcha in `SQL.qll`

 private module MySql {  * Provides classes modelling the `pg` package.  */ private module Postgres {-  /** Gets an expression of the form `new require('pg').Client()`. */-  API::Node newClient() { result = API::moduleImport("pg").getMember("Client").getInstance() }+  /** Gets a reference to the `Client` constructor in the `pg` package. E.g: `require('pg').Client`. */
  /** Gets a reference to the `Client` constructor in the `pg` package, for example `require('pg').Client`. */

(I think we are still encouraged to avoid using "e.g." and "i.e.")

erik-krogh

comment created time in 12 hours

PullRequestReviewEvent
PullRequestReviewEvent

pull request commentgithub/codeql

JS: Fixing an API-graph gotcha in `SQL.qll`

I think getAnImmediateUse sounds good, it might be easier to understand for an outsider. But I'll let Max have the final word.

getAnImmediateUse is fine.

erik-krogh

comment created time in 12 hours

Pull request review commentgithub/codeql

JS: Fixing an API-graph gotcha in `SQL.qll`

 module API {    */   class Node extends Impl::TApiNode {     /**-     * Gets a data-flow node corresponding to a use of the API component represented by this node.+     * Gets a `SourceNode` corresponding to a use of the API component represented by this node.      *      * For example, `require('fs').readFileSync` is a use of the function `readFileSync` from the      * `fs` module, and `require('fs').readFileSync(file)` is a use of the result of that function.      *      * As another example, in the assignment `exports.plusOne = (x) => x+1` the two references to      * `x` are uses of the first parameter of `plusOne`.+     *+     * Note: The result from this predicate is always a `DataFlow::SourceǸode`, use `getAUse()` if+     * you want to follow purely local data-flow and get all `DataFlow::Node`s that corrospond to a+     * use of this API node.      */-    DataFlow::Node getAUse() {-      exists(DataFlow::SourceNode src | Impl::use(this, src) |-        Impl::trackUseNode(src).flowsTo(result)-      )+    DataFlow::SourceNode getAReference() {+      exists(DataFlow::SourceNode src | Impl::use(this, src) | result = Impl::trackUseNode(src))

And you meant getReturn() instead of getResult(), right?

Yes, totally! (Where is my context-sensitive autocomplete in the browser?)

erik-krogh

comment created time in 18 hours

PullRequestReviewEvent

Pull request review commentgithub/codeql

JS: Fixing an API-graph gotcha in `SQL.qll`

 module API {    */   class Node extends Impl::TApiNode {     /**-     * Gets a data-flow node corresponding to a use of the API component represented by this node.+     * Gets a `SourceNode` corresponding to a use of the API component represented by this node.      *      * For example, `require('fs').readFileSync` is a use of the function `readFileSync` from the      * `fs` module, and `require('fs').readFileSync(file)` is a use of the result of that function.      *      * As another example, in the assignment `exports.plusOne = (x) => x+1` the two references to      * `x` are uses of the first parameter of `plusOne`.+     *+     * Note: The result from this predicate is always a `DataFlow::SourceǸode`, use `getAUse()` if+     * you want to follow purely local data-flow and get all `DataFlow::Node`s that corrospond to a+     * use of this API node.      */-    DataFlow::Node getAUse() {-      exists(DataFlow::SourceNode src | Impl::use(this, src) |-        Impl::trackUseNode(src).flowsTo(result)-      )+    DataFlow::SourceNode getAReference() {+      exists(DataFlow::SourceNode src | Impl::use(this, src) | result = Impl::trackUseNode(src))

Actually, looking at this again, can you remind me why we need the type tracking at all? Couldn't we remove it, and then implement getACall like this?

DataFlow::CallNode getACall() {
  result = getResult().getAReference()
}

In other words, instead of dropping out of API graphs early, then do type tracking and local flow to find a call, why not use API graphs to find the call directly, and then drop out without further flow?

erik-krogh

comment created time in 18 hours

PullRequestReviewEvent

pull request commentgithub/codeql-go

Add Action to build and upload qhelp

I was imagining the submitter would fish out the interesting ones and post them

Can you update our internal guide to mention this step?

smowton

comment created time in a day

push eventgithub/codeql-go

Chris Smowton

commit sha 6b28c0705af6fc3a5cc2c981ee2803f1c8da344e

Add Action to build and upload qhelp

view details

Max Schaefer

commit sha 3490d35926255692674566c8ceca9164ec88def7

Merge pull request #358 from smowton/smowton/admin/qhelp-action Add Action to build and upload qhelp

view details

push time in a day

PR merged github/codeql-go

Add Action to build and upload qhelp

This tests that qhelp builds to markdown, and uploads an artefact containing the built help. I punted on determining which qhelp files you consider changed in a local checkout / push build / PR build since the total build time is about 10 seconds and the artefact is 40KB.

+31 -1

2 comments

3 changed files

smowton

pr closed time in a day

PullRequestReviewEvent
PullRequestReviewEvent

Pull request review commentgithub/codeql

JS: Fixing an API-graph gotcha in `SQL.qll`

 module API {    */   class Node extends Impl::TApiNode {     /**-     * Gets a data-flow node corresponding to a use of the API component represented by this node.+     * Gets a `SourceNode` corresponding to a use of the API component represented by this node.      *      * For example, `require('fs').readFileSync` is a use of the function `readFileSync` from the      * `fs` module, and `require('fs').readFileSync(file)` is a use of the result of that function.      *      * As another example, in the assignment `exports.plusOne = (x) => x+1` the two references to      * `x` are uses of the first parameter of `plusOne`.+     *+     * Note: The result from this predicate is always a `DataFlow::SourceǸode`, use `getAUse()` if+     * you want to follow purely local data-flow and get all `DataFlow::Node`s that corrospond to a+     * use of this API node.      */-    DataFlow::Node getAUse() {-      exists(DataFlow::SourceNode src | Impl::use(this, src) |-        Impl::trackUseNode(src).flowsTo(result)-      )+    DataFlow::SourceNode getAReference() {+      exists(DataFlow::SourceNode src | Impl::use(this, src) | result = Impl::trackUseNode(src))

Hm, as you say this is now becoming very similar to getAUse. Perhaps a bit too similar. The nice thing about the previous implementation (without trackUseNode) was that it had a clear correspondence to getARhs(), but this is now lost.

How about reverting to the previous definition without trackUseNode, and instead moving that ito getACall? The only other case where we need it seems to be a single use of getAReference().getAnInstantiation(); perhaps we can introduce a helper predicate analogous to getACall to cover that.

erik-krogh

comment created time in a day

PullRequestReviewEvent

Pull request review commentgithub/codeql

JS: Fixing an API-graph gotcha in `SQL.qll`

 private module MySql {   private class QueryCall extends DatabaseAccess, DataFlow::MethodCallNode {     QueryCall() {       exists(API::Node recv | recv = createPool() or recv = connection() |-        this = recv.getMember("query").getReturn().getAUse()+        this = recv.getMember("query").getASourceUse().getACall()

I think getACall() is not completely horrible.

erik-krogh

comment created time in a day

PullRequestReviewEvent

pull request commentgithub/codeql-go

Add Action to build and upload qhelp

I punted on determining which qhelp files you consider changed in a local checkout / push build / PR build since the total build time is about 10 seconds and the artefact is 40KB.

It's of course fine to do it like this as a first step, but wouldn't we want something a little more targeted in the long run, if only to help reviewers?

smowton

comment created time in a day

PullRequestReviewEvent

Pull request review commentgithub/codeql

JS: Fixing an API-graph gotcha in `SQL.qll`

 class C {         this.req = req;     }     send() {-        this.req.query('select 1 as number', (err, result) => {})+        this.req.query('select 1 as number', (err, result) => { })     } } new C(new sql.Request());++var obj = {+    foo: function () {+        return request.batch('create procedure #temporary as select * from table', (err, result) => {+            // ... error checks+        })+    }+}+obj.foo("foo", "bar", "baz"); // A API-graphs gotcha.
obj.foo("foo", "bar", "baz"); // An API-graphs gotcha: "baz" should not be considered a `SqlString`
erik-krogh

comment created time in a day

Pull request review commentgithub/codeql

JS: Fixing an API-graph gotcha in `SQL.qll`

 private module Postgres {     result = newPool().getMember("connect").getParameter(0).getParameter(1)   } -  /** Gets an expression that constructs a new connection pool. */-  API::Node newPool() {+  /** Gets a constructor that when invoked constructs a new connection pool. */+  API::Node newPoolCallee() {     // new require('pg').Pool()-    result = API::moduleImport("pg").getMember("Pool").getInstance()+    result = API::moduleImport("pg").getMember("Pool")     or     // new require('pg-pool')-    result = API::moduleImport("pg-pool").getInstance()+    result = API::moduleImport("pg-pool")   } +  /** Gets an expression that constructs a new connection pool. */+  API::Node newPool() { result = newPoolCallee().getInstance() }

Suggest renaming to pool, with newPoolCallee becoming newPool or poolConstructor.

erik-krogh

comment created time in a day

Pull request review commentgithub/codeql

JS: Fixing an API-graph gotcha in `SQL.qll`

 private module MySql {  * Provides classes modelling the `pg` package.  */ private module Postgres {+  /** Gets an expression of the form `require('pg').Client`. */+  API::Node newClientCallee() { result = API::moduleImport("pg").getMember("Client") }+   /** Gets an expression of the form `new require('pg').Client()`. */-  API::Node newClient() { result = API::moduleImport("pg").getMember("Client").getInstance() }+  API::Node newClient() { result = newClientCallee().getInstance() }

Only used once; I suggest inlining and renaming newClientCallee to newClient.

erik-krogh

comment created time in a day

Pull request review commentgithub/codeql

JS: Fixing an API-graph gotcha in `SQL.qll`

 private module MySql {  * Provides classes modelling the `pg` package.  */ private module Postgres {+  /** Gets an expression of the form `require('pg').Client`. */

This doc comment isn't accurate anymore (I forgot to update it).

erik-krogh

comment created time in a day

Pull request review commentgithub/codeql

JS: Fixing an API-graph gotcha in `SQL.qll`

 private module MySql {   private class QueryCall extends DatabaseAccess, DataFlow::MethodCallNode {     QueryCall() {       exists(API::Node recv | recv = createPool() or recv = connection() |-        this = recv.getMember("query").getReturn().getAUse()+        this = recv.getMember("query").getASourceUse().getACall()

It looks like getASourceUse().getACall() is quite a common pattern. Let's perhaps introduce a helper predicate that encapsulates it.

erik-krogh

comment created time in a day

Pull request review commentgithub/codeql

JS: Fixing an API-graph gotcha in `SQL.qll`

 private module MySql {   /** Gets the package name `mysql` or `mysql2`. */   API::Node mysql() { result = API::moduleImport(["mysql", "mysql2"]) } +  /** Gets a reference to `mysql.createConnection`. */+  API::Node createConnectionCallee() { result = mysql().getMember("createConnection") }+   /** Gets a call to `mysql.createConnection`. */-  API::Node createConnection() { result = mysql().getMember("createConnection").getReturn() }+  API::Node createConnection() { result = createConnectionCallee().getReturn() }++  /** Gets a reference to `mysql.createPool`. */+  API::Node createPoolCallee() { result = mysql().getMember("createPool") }

Again, I'd suggest calling this predicate createPool and renaming the predicate below, perhaps to pool() (by analogy with connection()).

erik-krogh

comment created time in a day

Pull request review commentgithub/codeql

JS: Fixing an API-graph gotcha in `SQL.qll`

 private module MySql {   /** Gets the package name `mysql` or `mysql2`. */   API::Node mysql() { result = API::moduleImport(["mysql", "mysql2"]) } +  /** Gets a reference to `mysql.createConnection`. */+  API::Node createConnectionCallee() { result = mysql().getMember("createConnection") }+   /** Gets a call to `mysql.createConnection`. */-  API::Node createConnection() { result = mysql().getMember("createConnection").getReturn() }+  API::Node createConnection() { result = createConnectionCallee().getReturn() }++  /** Gets a reference to `mysql.createPool`. */+  API::Node createPoolCallee() { result = mysql().getMember("createPool") }    /** Gets a call to `mysql.createPool`. */

In addition to renaming the predicate, I'd suggest also updating the doc comment to more clearly emphasise that this doesn't just syntactically match calls to createPool but also does type tracking.

erik-krogh

comment created time in a day

Pull request review commentgithub/codeql

JS: Fixing an API-graph gotcha in `SQL.qll`

 private module MySql {   /** Gets the package name `mysql` or `mysql2`. */   API::Node mysql() { result = API::moduleImport(["mysql", "mysql2"]) } +  /** Gets a reference to `mysql.createConnection`. */+  API::Node createConnectionCallee() { result = mysql().getMember("createConnection") }+   /** Gets a call to `mysql.createConnection`. */-  API::Node createConnection() { result = mysql().getMember("createConnection").getReturn() }+  API::Node createConnection() { result = createConnectionCallee().getReturn() }

Again, not a fan of the naming. I think the name createConnection is a better fit for what is currently called createConnectionCallee. This predicate is now only used once, so we might as well inline it.

erik-krogh

comment created time in a day

Pull request review commentgithub/codeql

JS: Fixing an API-graph gotcha in `SQL.qll`

 module API {       )     } +    /**+     * Gets a source-node corresponding to a use of the API component represented by this node.+     *+     * For example, `require('fs').readFileSync` is a use of the function `readFileSync` from the+     * `fs` module, and `require('fs').readFileSync(file)` is a use of the result of that function.+     *+     * As another example, in the assignment `exports.plusOne = (x) => x+1` the two references to+     * `x` are uses of the first parameter of `plusOne`.+     */+    DataFlow::SourceNode getASourceUse() { Impl::use(this, result) }

I don't like the name. How about something like getAReference? Also, you just copy-pasted the doc comment from getAUse; can you adjust it to clarify the difference between the two predicates?

erik-krogh

comment created time in a day

PullRequestReviewEvent
PullRequestReviewEvent

push eventmax-schaefer/ql

Erik Krogh Kristensen

commit sha ec2b3f0b6ca6d07613b0b68cb13d89fc80e6f9f4

better join-order fix in HTTP

view details

Robert Marsh

commit sha e9b1d817c757ee96747cdbba3c935e05a7a9bf92

C++: QLDoc for VirtualVariable in IR construction

view details

Robert Marsh

commit sha e51b9215e4b3c391d7227ca163b298baa316fec6

C++: QLDoc for Overlap in IR construction

view details

Robert Marsh

commit sha 1445b3186464a33e04d67b1e1bb019f790f901d7

C++: QLDoc for Operand

view details

Robert Marsh

commit sha 713bdae77adc8f91fb7f43ac69823d93d4088b2c

C++: sync identical files

view details

Erik Krogh Kristensen

commit sha 664342dd0f2f273d702c14a2c05d715700df5b85

change `SimpleParameter` to `Parameter` in the express model to support destructuring parameters

view details

Tamas Vajk

commit sha a6b62a383835f9ec50dc44aac4709bf2ebded438

C#: Add enum init value test

view details

Tamas Vajk

commit sha 77bb1b2cd9dbc80e174fad7307e3c79940ea0506

C#: Extract constant value of enum member equal clauses

view details

Tamas Vajk

commit sha 3577b27f49ce6b38453d1a7413c70e87356aac57

Fix to not report on enum member initialization

view details

Tamas Vajk

commit sha a635503be0987f4b3148f12f97cc093cd9596b54

Add test cases to UselessCastToSelf

view details

Nick Rolfe

commit sha 7609ce2d4718fbfc171e3ddfb4fdcbfb8cbc01c9

C++: accept test changes from extractor frontend upgrade

view details

Jonas Jensen

commit sha 165779ea09ab54587651bbd6da7340baaa4dd6cd

Merge pull request #4343 from rdmarsh2/rdmarsh2/cpp/ir-construction-qldoc C++: Add some IR QLDoc

view details

CodeQL CI

commit sha 75262ddace96633b3e1092764e4ce40f2e4a689d

Merge pull request #4328 from erik-krogh/indirect-fix2 Approved by esbena

view details

Tom Hvitved

commit sha 93edaa75ebeeba7a1506a1631ac73f5bfbcedaba

Merge pull request #4309 from tamasvajk/feature/enum-value-init Extract constant value of enum member equal clauses

view details

Ian Lynagh

commit sha 8a76195f04db356ae8e305939a6104ecb0709e5e

Merge pull request #4356 from github/igfoo/front_end C++: accept test changes from extractor frontend upgrade

view details

CodeQL CI

commit sha 060c19a063ec1939051ae22c30f5e844dca9b6a3

Merge pull request #4352 from erik-krogh/destructing-redirect Approved by esbena

view details

push time in 2 days

push eventgithub/codeql-go

Max Schaefer

commit sha 8667b64a1cc97078a2b2f5b0b8590329b7208e95

Make result variables aware of their index.

view details

Max Schaefer

commit sha 56f295f7415fa9df0ece1eb5dd9a591567ec5e0a

Add a few useful shortcuts.

view details

Max Schaefer

commit sha 97fb967d5c4e425e61bc6786ea87943577bbeacc

Add a few tests.

view details

Max Schaefer

commit sha 3dde501b5410a505c351c66c54007adbea0a8dee

Merge pull request #356 from max-schaefer/api-cleanup Add and move around a few convenience predicates

view details

push time in 2 days

PR merged github/codeql-go

Add and move around a few convenience predicates

In some cases I've moved predicates onto their superclass to make them more discoverable and avoid casts, in other cases the information computed by the predicate was already there, but not conveniently exposed.

+96 -23

4 comments

8 changed files

max-schaefer

pr closed time in 2 days

push eventmax-schaefer/ql

Max Schaefer

commit sha dfc44360129cb268330cccbdd84273b5883dcf7e

JavaScript: Teach API graphs to recognise arguments supplied in partial function applications.

view details

push time in 2 days

issue commentgithub/codeql-go

Go contextual queries (find definitions and references) are not working

Thank you for the update. I have now found the location you reported, but am unable to consistently reproduce. I think I got it to fail once, but since then it's been resolving references correctly every time I've tried again, so I'm kind of doubting my recollection of the time it failed, I may have just clicked on the wrong button then.

aeisenberg

comment created time in 2 days

push eventmax-schaefer/ql

lcartey@github.com

commit sha bdcf4198e60b9957f0da04a204d13fa028e362e1

Add additional Hibernate SQL sinks

view details

Tom Hvitved

commit sha 26639a113e3991e755464ce4089e67a6e34076dd

C#: Rename `Layout.Condition` to `FilePattern` and move to separate file

view details

Tom Hvitved

commit sha 14567f531408c47f9e7b58ef9b0a34bb2c5e4c65

C#: Support wild-cards in file patterns Implements the specification at https://wiki.semmle.com/display/SDmaster/project-layout+format by compiling file path specifications to regular expressions.

view details

Tom Hvitved

commit sha 37f1ce312256e36553c3951fba3c5c0c8f40ad9c

C#: Implement support for path transformers

view details

Tom Hvitved

commit sha 4d0a1ee8578a061f3286b6e4b7a626e4d9211921

Address review comments

view details

Anders Schack-Mulligen

commit sha b1e6e3a6f23c9f6c03c4f8b752aa5f6a38112da1

Java: Add 1.25 change notes.

view details

Rasmus Wriedt Larsen

commit sha 2979f9813ee0ba08779623a72106f00ab35e30d8

Python: Add missing change notes I looked through PRs between rc/1.24 and rc/1.25 and added missing change notes for: - https://github.com/github/codeql/pull/3314 - https://github.com/github/codeql/pull/3302 - https://github.com/github/codeql/pull/3212 - https://github.com/github/codeql/pull/3453 - https://github.com/github/codeql/pull/3407 - https://github.com/github/codeql/pull/3563 ``` git log --grep="Merge pull request" --format=oneline rc/1.24..rc/1.25 -- python/ ```

view details

Mathias Vorreiter Pedersen

commit sha faae2e782a7bfa7c2ff4742d1615d4af4055bc4b

C++: Implement field flow for operator[] writes and pointer deref writes.

view details

Mathias Vorreiter Pedersen

commit sha 41147d245dbc5a27e022655e0e388bd06c035c00

C++: Accept test changes

view details

Mathias Vorreiter Pedersen

commit sha 5d14688848fc79419093fb0bdf2003595c07e937

Merge branch 'main' into mathiasvp/array-field-flow

view details

Anders Schack-Mulligen

commit sha 02da80aa25fc7690bef9a4c77c057571b2c27406

Java: Remove "New Queries" section.

view details

Anders Schack-Mulligen

commit sha 86755215ad19e98b7e9490a7205ca4bd436bf1df

Merge pull request #4225 from aschackmull/java/changenote-1.25 Java: Add 1.25 change notes.

view details

Arthur Baars

commit sha 1f4028f4a08a7faa99b614a6af6ecc36ef2eeb66

Java: Add new SQL sinks for Hibernate versions 4 and 6

view details

Mathias Vorreiter Pedersen

commit sha 9659afdf09646fb51dc1159aefd46740dc94643f

C++: Accept more test changes

view details

Rasmus Wriedt Larsen

commit sha 038688a55c5519a40c15950a323fdf0ea45d13ae

Python: Minor updates to 1.25 change notes backporting fixes from `@sj`

view details

Rasmus Wriedt Larsen

commit sha fb3060dc3dc314c5181c69adb30992aea6397a65

Java: Minor fixup for SSA AdjacentUsesImpl::varBlockReaches This should not change anything in regards to correctness overall -- what we really care about is `varBlockStep`, and that checks `varOccursInBlock(v, b2)`. However, the comment is a bit easier to read together with the code now (and probably also gives slightly smaller predicate result size).

view details

Mathias Vorreiter Pedersen

commit sha ff0910408931d4e95d8ef0451d3fcbcabbb5f96d

Merge branch 'main' into mathiasvp/array-field-flow

view details

Calum Grant

commit sha 3414063f2eedf3444c65cf3232193e4270476e2c

Update change-notes/1.25/analysis-python.md Co-authored-by: Rasmus Wriedt Larsen <rasmuswl@github.com>

view details

Taus Brock-Nannestad

commit sha e0f5b208da911f8a9dfe4220943645e8e0124cbe

Python: Fix broken test of global typetracker flow The missing `global g` annotation meant `g = x` was interpreted as a local assignment.

view details

Mathias Vorreiter Pedersen

commit sha 78b24b76a09d680ac6581597f0ecfe25596ed18f

C++: Remove the problematic taint tracking rule. It seems like we get the flows from dataflow already now.

view details

push time in 2 days

PullRequestReviewEvent

Pull request review commentgithub/codeql

Add docs on CodeQL Design Patterns

+# CodeQL Design Patterns++A list of design patterns you are recommended to follow.++## `::Range` for extensibility and refinement++To allow both extensibility and refinement of classes, we use what is commonly referred to as the `::Range` pattern (since https://github.com/github/codeql/pull/727), but the actual implementation can use different names.++<details>+<summary>Generic example of how to define classes with ::Range</summary>++Instead of+```ql+/** <QLDoc...> */+abstract class MySpecialExpr extends Expr {+  /** <QLDoc...> */+  abstract int memberPredicate();+}+```+with+```ql+class ConcreteSubclass extends MySpecialExpr { ... }+```++use++```ql+/**+ * <QLDoc...>+ *+ * Extend this class to refine existing API models. If you want to model new APIs,+ * extend `MySpecialExpr::Range` instead.+ */+class MySpecialExpr extends Expr {+  MySpecialExpr::Range self;++  MySpecialExpr() { this = self }++  /** <QLDoc...> */+  int memberPredicate() { result = self.memberPredicate() }+}++/** Provides a class for modeling new <...> APIs. */+module MySpecialExpr {+  /**+   * <QLDoc...>+   *+   * Extend this class to model new APIs. If you want to refine existing API models,+   * extend `MySpecialExpr` instead.+   */+  abstract class Range extends Expr {+    /** <QLDoc...> */+    abstract int memberPredicate();+  }+}+```+with+```ql+class ConcreteSubclass extends MySpecialExpr::Range { ... }+```++</details>++### Rationale++Let's use an example from the Go libraries: https://github.com/github/codeql-go/blob/2ba9bbfd8ba1818b5ee9f6009c86a605189c9ef3/ql/src/semmle/go/Concepts.qll#L119-L157++`EscapeFunction`, as the name suggests, models various APIs that escape meta-characters. It has a member-predicate `kind()` that tells you what sort of escaping the modelled function does. For example, if the result of that predicate is `"js"`, then this means that the escaping function is meant to make things safe to embed inside JavaScript.+`EscapeFunction::Range` is subclassed to model various APIs, and `kind()` is implemented accordingly.+But we can also subclass `EscapeFunction` to, as in the above example, talk about all JS-escaping functions.++You can, of course, do the same without the `::Range` pattern, but it's a little cumbersome:+If you only had an `abstract class EscapeFunction { ... }`, then `JsEscapeFunction` would need to be implemented in a slightly tricky way to prevent it from extending `EscapeFunction` (instead of refining it). You would have to give it a charpred `this instanceof EscapeFunction`, which looks useless but isn't. And additionally, you'd have to provide trivial `none()` overrides of all the abstract predicates defined in `EscapeFunction`. This is all pretty awkward, and we can avoid it by distinguishing between `EscapeFunction` and `EscapeFunction::Range`.+++## Importing all subclasses of abstract base class++When providing an abstract class, you should ensure that all subclasses are included when the abstract class is (unless you have good reason not to). Otherwise you risk having different meanings of the abstract class depending on what you happen to import.

It's perhaps worth calling out the canonical example where we do not want this: DataFlow::Configuration and its variants are abstract, but we generally do not want to import all configurations into the same scope at once.

RasmusWL

comment created time in 2 days

push eventmax-schaefer/ql

lcartey@github.com

commit sha bdcf4198e60b9957f0da04a204d13fa028e362e1

Add additional Hibernate SQL sinks

view details

Tom Hvitved

commit sha 26639a113e3991e755464ce4089e67a6e34076dd

C#: Rename `Layout.Condition` to `FilePattern` and move to separate file

view details

Tom Hvitved

commit sha 14567f531408c47f9e7b58ef9b0a34bb2c5e4c65

C#: Support wild-cards in file patterns Implements the specification at https://wiki.semmle.com/display/SDmaster/project-layout+format by compiling file path specifications to regular expressions.

view details

Tom Hvitved

commit sha 37f1ce312256e36553c3951fba3c5c0c8f40ad9c

C#: Implement support for path transformers

view details

Tom Hvitved

commit sha 4d0a1ee8578a061f3286b6e4b7a626e4d9211921

Address review comments

view details

Anders Schack-Mulligen

commit sha b1e6e3a6f23c9f6c03c4f8b752aa5f6a38112da1

Java: Add 1.25 change notes.

view details

Rasmus Wriedt Larsen

commit sha 2979f9813ee0ba08779623a72106f00ab35e30d8

Python: Add missing change notes I looked through PRs between rc/1.24 and rc/1.25 and added missing change notes for: - https://github.com/github/codeql/pull/3314 - https://github.com/github/codeql/pull/3302 - https://github.com/github/codeql/pull/3212 - https://github.com/github/codeql/pull/3453 - https://github.com/github/codeql/pull/3407 - https://github.com/github/codeql/pull/3563 ``` git log --grep="Merge pull request" --format=oneline rc/1.24..rc/1.25 -- python/ ```

view details

Mathias Vorreiter Pedersen

commit sha faae2e782a7bfa7c2ff4742d1615d4af4055bc4b

C++: Implement field flow for operator[] writes and pointer deref writes.

view details

Mathias Vorreiter Pedersen

commit sha 41147d245dbc5a27e022655e0e388bd06c035c00

C++: Accept test changes

view details

Mathias Vorreiter Pedersen

commit sha 5d14688848fc79419093fb0bdf2003595c07e937

Merge branch 'main' into mathiasvp/array-field-flow

view details

Anders Schack-Mulligen

commit sha 02da80aa25fc7690bef9a4c77c057571b2c27406

Java: Remove "New Queries" section.

view details

Anders Schack-Mulligen

commit sha 86755215ad19e98b7e9490a7205ca4bd436bf1df

Merge pull request #4225 from aschackmull/java/changenote-1.25 Java: Add 1.25 change notes.

view details

Arthur Baars

commit sha 1f4028f4a08a7faa99b614a6af6ecc36ef2eeb66

Java: Add new SQL sinks for Hibernate versions 4 and 6

view details

Mathias Vorreiter Pedersen

commit sha 9659afdf09646fb51dc1159aefd46740dc94643f

C++: Accept more test changes

view details

Rasmus Wriedt Larsen

commit sha 038688a55c5519a40c15950a323fdf0ea45d13ae

Python: Minor updates to 1.25 change notes backporting fixes from `@sj`

view details

Rasmus Wriedt Larsen

commit sha fb3060dc3dc314c5181c69adb30992aea6397a65

Java: Minor fixup for SSA AdjacentUsesImpl::varBlockReaches This should not change anything in regards to correctness overall -- what we really care about is `varBlockStep`, and that checks `varOccursInBlock(v, b2)`. However, the comment is a bit easier to read together with the code now (and probably also gives slightly smaller predicate result size).

view details

Mathias Vorreiter Pedersen

commit sha ff0910408931d4e95d8ef0451d3fcbcabbb5f96d

Merge branch 'main' into mathiasvp/array-field-flow

view details

Calum Grant

commit sha 3414063f2eedf3444c65cf3232193e4270476e2c

Update change-notes/1.25/analysis-python.md Co-authored-by: Rasmus Wriedt Larsen <rasmuswl@github.com>

view details

Taus Brock-Nannestad

commit sha e0f5b208da911f8a9dfe4220943645e8e0124cbe

Python: Fix broken test of global typetracker flow The missing `global g` annotation meant `g = x` was interpreted as a local assignment.

view details

Mathias Vorreiter Pedersen

commit sha 78b24b76a09d680ac6581597f0ecfe25596ed18f

C++: Remove the problematic taint tracking rule. It seems like we get the flows from dataflow already now.

view details

push time in 3 days

push eventmax-schaefer/ql

Rasmus Lerchedahl Petersen

commit sha a2d006fe479ed4ca35457fa9c98da17cb2572da8

Python: Tests for field flow

view details

Rasmus Lerchedahl Petersen

commit sha 27b25565ca78a99c0565e65b767c8425d4ff28ee

Python: Implement field-stores, -reads, and -content

view details

Rasmus Lerchedahl Petersen

commit sha aa281671774d13f669c3e9d025997b0640d78ec6

Python: Add malloc nodes

view details

Rasmus Lerchedahl Petersen

commit sha e50b66554dfeba72c91b15cd58247ce133902022

Python: Add explorative test

view details

Rasmus Lerchedahl Petersen

commit sha e1323617363b902f3647e9457a4d04209e12338e

Python: Add missing .expected file

view details

Rasmus Lerchedahl Petersen

commit sha b2f1c435a864bf9ae8a3975b89d7fb215a890d6d

Python: update test expectations

view details

Rasmus Lerchedahl Petersen

commit sha 9aa0cfb35cd6e5455a321c3b67b7e8cca6a36ddd

Python: class callable -> class call Only have one type of callable, but have an extra type of call. A constructor call directs to an init callable (should also handle `call` overrides at some point).

view details

Rasmus Lerchedahl Petersen

commit sha 73d2d9b1f86fafee048949059ef5d74b5400b9c7

Python: Make constructor calls post-update nodes

view details

Rasmus Lerchedahl Petersen

commit sha 08b51e67c42eb257c7e9ad76516021ae206dc07f

Python: Update test annotation

view details

Rasmus Lerchedahl Petersen

commit sha 3e2331c87f6b30d846494e6a1e5b53e35b75fe2c

Merge branch 'main' of github.com:github/codeql into SharedDataflow_FieldFlow

view details

Rasmus Lerchedahl Petersen

commit sha b065d8724e75630271556beadb9c46fb28a0ee8a

Python: Fixup comments after merge

view details

Rasmus Lerchedahl Petersen

commit sha 131cf8d2ecc8ab772d35fde72916cb1f4b31860a

Python: Fix compilation error

view details

yoff

commit sha aece0ff65271219717e7dda416813d6352a88eeb

Apply suggestions from code review Co-authored-by: Taus <tausbn@github.com>

view details

Rasmus Lerchedahl Petersen

commit sha ef4461ce544b3c28c20e6546b1368fc46e4afed9

Python: Address review comments

view details

Tamas Vajk

commit sha 2868d5bf34648dddcf29ffa3f0d4107e08013740

C#: Add pointer cast test cases

view details

Erik Krogh Kristensen

commit sha 83f0514475e0a8a7054e403e3fef699db3cc409b

add req.files as a RequestInputAccess in the Express model

view details

Tamas Vajk

commit sha 48bf6d55aa3cb2807ea7776101e9e175588c936a

C#: Add implicit cast from array to pointer

view details

Erik Krogh Kristensen

commit sha 6163e6cf5f5dcb58e8fd7ab7d7e13c341cf987d2

adjust test case for XML entity expansion

view details

Joe

commit sha 45651cf1236a6aa2c4aeb723c7b49a664d63baa4

Java: PrintAst: Add a synthetic node for the initialisers of for statements

view details

Joe

commit sha 1f9960762425b676b64a9a150ded4015a6ff9606

Java: PrintAst: Improve test

view details

push time in 3 days

pull request commentgithub/codeql-go

Add and move around a few convenience predicates

I've added tests for Function.getResult(i), which is the main new predicate being added.

max-schaefer

comment created time in 3 days

push eventmax-schaefer/codeql-go

Max Schaefer

commit sha 97fb967d5c4e425e61bc6786ea87943577bbeacc

Add a few tests.

view details

push time in 3 days

pull request commentgithub/codeql-go

Add and move around a few convenience predicates

Sure; let me look into that.

max-schaefer

comment created time in 3 days

pull request commentgithub/codeql

JavaScript: Handle empty `package.json` files gracefully.

CI failure looks spurious; let me retry...

max-schaefer

comment created time in 3 days

issue commentgithub/codeql-go

Go contextual queries (find definitions and references) are not working

Thank you for your report. I'm trying to reproduce, but...

On line 178

verifydependencies.go only seems to have 101 lines. Am I looking at the wrong file?

I've tried a few other jump-to-definitions, and they all seem to work.

aeisenberg

comment created time in 3 days

pull request commentgithub/codeql-go

Add and move around a few convenience predicates

Evaluation came back clean.

max-schaefer

comment created time in 3 days

pull request commentgithub/codeql-go

Add and move around a few convenience predicates

Changes like these are conceptually simple but surprisingly fiddly due to prescriptive typing, so I think I'll run an evaluation over the weekend just to get some extra confidence that I didn't accidentally make some predicate empty.

max-schaefer

comment created time in 5 days

PR opened github/codeql-go

Add and move around a few convenience predicates

In some cases I've moved predicates onto their superclass to make them more discoverable and avoid casts, in other cases the information computed by the predicate was already there, but not conveniently exposed.

+47 -23

0 comment

3 changed files

pr created time in 5 days

push eventmax-schaefer/codeql-go

Max Schaefer

commit sha 56f295f7415fa9df0ece1eb5dd9a591567ec5e0a

Add a few useful shortcuts.

view details

push time in 5 days

create barnchmax-schaefer/codeql-go

branch : api-cleanup

created branch time in 5 days

PullRequestReviewEvent

Pull request review commentgithub/codeql-go

Improve extractor logging and a minor readability fix

 func ExtractWithFlags(buildFlags []string, patterns []string) error { 	packages.Visit(pkgs, func(pkg *packages.Package) bool { 		return true 	}, func(pkg *packages.Package) {+		log.Printf("Processing package %s.", pkg.PkgPath)

:+1:

sauyon

comment created time in 6 days

PullRequestReviewEvent

PR opened github/codeql

JavaScript: Handle empty `package.json` files gracefully.

Correct based branch to be determined.

+48 -1

0 comment

5 changed files

pr created time in 6 days

create barnchmax-schaefer/ql

branch : js/handle-empty-pkgjson

created branch time in 6 days

push eventmax-schaefer/ql

lcartey@github.com

commit sha bdcf4198e60b9957f0da04a204d13fa028e362e1

Add additional Hibernate SQL sinks

view details

Tom Hvitved

commit sha 26639a113e3991e755464ce4089e67a6e34076dd

C#: Rename `Layout.Condition` to `FilePattern` and move to separate file

view details

Tom Hvitved

commit sha 14567f531408c47f9e7b58ef9b0a34bb2c5e4c65

C#: Support wild-cards in file patterns Implements the specification at https://wiki.semmle.com/display/SDmaster/project-layout+format by compiling file path specifications to regular expressions.

view details

Tom Hvitved

commit sha 37f1ce312256e36553c3951fba3c5c0c8f40ad9c

C#: Implement support for path transformers

view details

Tom Hvitved

commit sha 4d0a1ee8578a061f3286b6e4b7a626e4d9211921

Address review comments

view details

Anders Schack-Mulligen

commit sha b1e6e3a6f23c9f6c03c4f8b752aa5f6a38112da1

Java: Add 1.25 change notes.

view details

Rasmus Wriedt Larsen

commit sha 2979f9813ee0ba08779623a72106f00ab35e30d8

Python: Add missing change notes I looked through PRs between rc/1.24 and rc/1.25 and added missing change notes for: - https://github.com/github/codeql/pull/3314 - https://github.com/github/codeql/pull/3302 - https://github.com/github/codeql/pull/3212 - https://github.com/github/codeql/pull/3453 - https://github.com/github/codeql/pull/3407 - https://github.com/github/codeql/pull/3563 ``` git log --grep="Merge pull request" --format=oneline rc/1.24..rc/1.25 -- python/ ```

view details

Mathias Vorreiter Pedersen

commit sha faae2e782a7bfa7c2ff4742d1615d4af4055bc4b

C++: Implement field flow for operator[] writes and pointer deref writes.

view details

Mathias Vorreiter Pedersen

commit sha 41147d245dbc5a27e022655e0e388bd06c035c00

C++: Accept test changes

view details

Mathias Vorreiter Pedersen

commit sha 5d14688848fc79419093fb0bdf2003595c07e937

Merge branch 'main' into mathiasvp/array-field-flow

view details

Anders Schack-Mulligen

commit sha 02da80aa25fc7690bef9a4c77c057571b2c27406

Java: Remove "New Queries" section.

view details

Anders Schack-Mulligen

commit sha 86755215ad19e98b7e9490a7205ca4bd436bf1df

Merge pull request #4225 from aschackmull/java/changenote-1.25 Java: Add 1.25 change notes.

view details

Arthur Baars

commit sha 1f4028f4a08a7faa99b614a6af6ecc36ef2eeb66

Java: Add new SQL sinks for Hibernate versions 4 and 6

view details

Mathias Vorreiter Pedersen

commit sha 9659afdf09646fb51dc1159aefd46740dc94643f

C++: Accept more test changes

view details

Rasmus Wriedt Larsen

commit sha 038688a55c5519a40c15950a323fdf0ea45d13ae

Python: Minor updates to 1.25 change notes backporting fixes from `@sj`

view details

Rasmus Wriedt Larsen

commit sha fb3060dc3dc314c5181c69adb30992aea6397a65

Java: Minor fixup for SSA AdjacentUsesImpl::varBlockReaches This should not change anything in regards to correctness overall -- what we really care about is `varBlockStep`, and that checks `varOccursInBlock(v, b2)`. However, the comment is a bit easier to read together with the code now (and probably also gives slightly smaller predicate result size).

view details

Mathias Vorreiter Pedersen

commit sha ff0910408931d4e95d8ef0451d3fcbcabbb5f96d

Merge branch 'main' into mathiasvp/array-field-flow

view details

Calum Grant

commit sha 3414063f2eedf3444c65cf3232193e4270476e2c

Update change-notes/1.25/analysis-python.md Co-authored-by: Rasmus Wriedt Larsen <rasmuswl@github.com>

view details

Taus Brock-Nannestad

commit sha e0f5b208da911f8a9dfe4220943645e8e0124cbe

Python: Fix broken test of global typetracker flow The missing `global g` annotation meant `g = x` was interpreted as a local assignment.

view details

Mathias Vorreiter Pedersen

commit sha 78b24b76a09d680ac6581597f0ecfe25596ed18f

C++: Remove the problematic taint tracking rule. It seems like we get the flows from dataflow already now.

view details

push time in 6 days

Pull request review commentgithub/codeql-go

Improve extractor logging and a minor readability fix

 func ExtractWithFlags(buildFlags []string, patterns []string) error { 			extractPackage(pkg, &wg, goroutineSem, fdSem) 			return 		}++		log.Printf("Skipping dependency package %s", pkg.PkgPath)
		log.Printf("Skipping dependency package %s.", pkg.PkgPath)
sauyon

comment created time in 6 days

Pull request review commentgithub/codeql-go

Improve extractor logging and a minor readability fix

 func ExtractWithFlags(buildFlags []string, patterns []string) error { 	packages.Visit(pkgs, func(pkg *packages.Package) bool { 		return true 	}, func(pkg *packages.Package) {+		log.Printf("Processing package %s.", pkg.PkgPath)

Does this (and the other uses of Printf) want a final \n, or do we get that from somewhere else?

sauyon

comment created time in 6 days

PullRequestReviewEvent
PullRequestReviewEvent

push eventmax-schaefer/codeql-go

Slavomir

commit sha 45dfc2bcf205ead8a80993c51744ea9102ec38e0

Move to stdlib, extend and refactor the `Io` module

view details

Slavomir

commit sha 073fae9ff83afab06a93a0233cf008cc43c6589b

Move to stdlib and extend the module for `io/ioutil` package

view details

Slavomir

commit sha f1cdfff3317af9ca45ccf36ca2d04db8438d60cb

Import IoIoutil module

view details

Slavomir

commit sha dc159eeee10239506e9e5977a64a582efd32ac31

Remove io.Writer model from io/ioutil

view details

Slavomir

commit sha c4a493f003962dc1ae4eeeb124d881b5866467d6

Remove models for methods where there are already interface models

view details

Slavomir

commit sha 07fdd3d17a1f8f36ec15b11f0876cab1807b5ae2

Fix TaintStep.expected: fix io.Pipe logic, remove rune/byte read/write

view details

Slavomir

commit sha 4127cb5cc47287a066acda1b1b6524845168f7fe

Remove rune/byte read/write from TaintStep/io.go

view details

Slavomir

commit sha 29382744fec8ef3d73d8c11c213572382850687f

Taint-track package `errors`

view details

Slavomir

commit sha 55a8e24055a0447849d947f7ebf5dd7879daddfb

Taint-track package `expvar`

view details

Slavomir

commit sha 6f0bfbfa96e533df448023c4c1a30c242f2c3899

Add taint-tracking to `database/sql` package in the SQL module

view details

Slavomir

commit sha 5e4d75561c28527b6915595b1538af6da1c6bd8e

Add `database/sql/driver` taint-tracking

view details

Slavomir

commit sha 24e8a18d224089227eba72dc8b4eabc06efc41b6

Add `database/sql/driver` taint-tracking

view details

Slavomir

commit sha 3fd6f9c40048bc1024a2b3d8f079f0879d6f57f2

Extend QueryString::Range with `database/sql/driver` interfaces' methods

view details

Slavomir

commit sha 5e7b279569a7061f85f63b9985d8c49678571d45

Remove model for a method that satisfies an interface that already has its own model.

view details

Slavomir

commit sha 4ecf9b0f6b2b870a627c235445f328b34c9bd391

Add taint-tracking for `container/heap` package

view details

Slavomir

commit sha 75e3ee6e7716b5f57a0ca3b5e2c9f7e5d24da192

Add taint-tracking for `container/list` package

view details

Slavomir

commit sha dedeb7bbf1b14f430ab0f76fbedde9fbbeeb8754

Add taint-tracking for `container/ring` package

view details

Chris Smowton

commit sha b6b7bd27176976c195e4cf24f5a34ffb29e9e051

Generalise model of HTTP libraries * Allow for HTTP response methods that define a content-type without a corresponding header write * Factor out stdlib-http-specific classification of fields that aren't vulnerable to an open-redirect exploit

view details

Chris Smowton

commit sha bdb3e54299c92bf65c5d58e3a837e00ef1232890

Add tests for stdlib-http fields that aren't supposed to cause open-redirect alerts

view details

Chris Smowton

commit sha 397282f41a060796043b4e3ac24501ed2f2a4f35

Add models for the Echo framework

view details

push time in 6 days

push eventgithub/codeql-go

Chris Smowton

commit sha 59138048bb5678591ae5f139d076fe7d00053ccc

Add query spotting probably-bad escapes in regular expressions. Inspired by js/useless-regexp-character-escape, but much much simpler because the Go source code parser forbids unrecognised escapes and its regex engine refuses to compile \\x where x is not a character class or other special token (e.g. start-of-word).

view details

Max Schaefer

commit sha 907ae20a16459830f5195c692178b661cfc0483b

Merge pull request #350 from smowton/smowton/feature/bad-regex-escape-query Add query spotting probably-bad escapes in regular expressions.

view details

push time in 7 days

PR merged github/codeql-go

Add query spotting probably-bad escapes in regular expressions.

Inspired by js/useless-regexp-character-escape, but much much simpler because the Go source code parser forbids unrecognised escapes and its regex engine refuses to compile \x where x is not a character class or other special token (e.g. start-of-word). The only potentially troublesome sequences are \a (confused for \\A or Vim's \\a perhaps) and \b (probably confused for \\b, start-of-word anchor).

Evaluated on lgtm.com, found two results, one probable and one true positive, so clearly this is a much rarer mistake in Go than in JS, but I figure it doesn't hurt to have the query.

+184 -0

4 comments

8 changed files

smowton

pr closed time in 7 days

PullRequestReviewEvent

pull request commentgithub/codeql

JavaScript: Fix and improve indirect command-argument tracking

An evaluation of the CWE-078 queries on nightly.slugs showed no performance or result changes.

max-schaefer

comment created time in 7 days

PR opened github/codeql

JavaScript: Fix and improve indirect command-argument tracking

The first two commits are a simple correctness fix: commandArgument was using type backtrackers wrongly, and hence was unable to backtrack into function calls.

The third commit introduces an extra bit of type tracking: we previously already type-tracked the 'sh' string (or similar) in an indirect command, now we also track the corresponding -c (or similar). With this improvement, we can derive a reasonable summary for https://github.com/tsertkov/exec-sh.

The final commit is an unrelated bit of clean-up to reduce the number of alert locations for the command-injection query.

+105 -14

0 comment

5 changed files

pr created time in 7 days

Pull request review commentgithub/codeql-go

Add query spotting probably-bad escapes in regular expressions.

+/**+ * @name Suspicious characters in a regular expression+ * @description If a literal bell character or backspace appears in a regular expression, the start of text or word boundary may have been intended.+ * @kind path-problem+ * @problem.severity warning+ * @precision high+ * @id go/suspicious-character-in-regex+ * @tags correctness+ *       security+ *       external/cwe/cwe-20+ */++import go+import DataFlow::PathGraph++/**+ * Holds if `source` corresponds to a string literal that contains an escaped `character`.+ *+ * `character` must be `"a"` or `"b"`, the only interesting escapes for this query.+ */+predicate containsEscapedCharacter(DataFlow::Node source, string character) {+  character in ["a", "b"] and+  exists(+    // Search for `character` preceded by an odd number of backslashes:+    source+        .asExpr()+        .(BasicLit)+        .getText()+        .regexpFind("(?<=(^|[^\\\\])\\\\(\\\\{2}){0,10})" + character, _, _)+  )+}++/** A dataflow configuration that traces strings containing suspicious escape sequences to a use as a regular expression. */+class Config extends DataFlow::Configuration {+  Config() { this = "SuspiciousRegexpEscape" }++  predicate isSource(DataFlow::Node source, string report) {+    containsEscapedCharacter(source, "a") and+    report =+      "the bell character \\a; did you mean \\\\a, the Vim alphabetic character class (use [[:alpha:]] instead) or \\\\A, the beginning of text?"+    or+    containsEscapedCharacter(source, "b") and+    report = "a literal backspace \\b; did you mean \\\\b, a word boundary?"+  }++  override predicate isSource(DataFlow::Node source) { isSource(source, _) }++  override predicate isSink(DataFlow::Node sink) { sink instanceof RegexpPattern }+}++from Config c, DataFlow::PathNode source, DataFlow::PathNode sink, string report+where c.hasFlowPath(source, sink) and c.isSource(source.getNode(), report)+select source, source, sink, "$@ used $@ contains " + report, source, "a regular expression", sink,
select source, source, sink, "$@ used $@ contains " + report, source, "A regular expression", sink,

(Will need updates to expected outputs, I'm afraid.)

smowton

comment created time in 7 days

PullRequestReviewEvent
PullRequestReviewEvent

push eventmax-schaefer/ql

lcartey@github.com

commit sha bdcf4198e60b9957f0da04a204d13fa028e362e1

Add additional Hibernate SQL sinks

view details

Mathias Vorreiter Pedersen

commit sha faae2e782a7bfa7c2ff4742d1615d4af4055bc4b

C++: Implement field flow for operator[] writes and pointer deref writes.

view details

Mathias Vorreiter Pedersen

commit sha 41147d245dbc5a27e022655e0e388bd06c035c00

C++: Accept test changes

view details

Mathias Vorreiter Pedersen

commit sha 5d14688848fc79419093fb0bdf2003595c07e937

Merge branch 'main' into mathiasvp/array-field-flow

view details

Arthur Baars

commit sha 1f4028f4a08a7faa99b614a6af6ecc36ef2eeb66

Java: Add new SQL sinks for Hibernate versions 4 and 6

view details

Mathias Vorreiter Pedersen

commit sha 9659afdf09646fb51dc1159aefd46740dc94643f

C++: Accept more test changes

view details

Rasmus Wriedt Larsen

commit sha fb3060dc3dc314c5181c69adb30992aea6397a65

Java: Minor fixup for SSA AdjacentUsesImpl::varBlockReaches This should not change anything in regards to correctness overall -- what we really care about is `varBlockStep`, and that checks `varOccursInBlock(v, b2)`. However, the comment is a bit easier to read together with the code now (and probably also gives slightly smaller predicate result size).

view details

Mathias Vorreiter Pedersen

commit sha ff0910408931d4e95d8ef0451d3fcbcabbb5f96d

Merge branch 'main' into mathiasvp/array-field-flow

view details

Taus Brock-Nannestad

commit sha e0f5b208da911f8a9dfe4220943645e8e0124cbe

Python: Fix broken test of global typetracker flow The missing `global g` annotation meant `g = x` was interpreted as a local assignment.

view details

Mathias Vorreiter Pedersen

commit sha 78b24b76a09d680ac6581597f0ecfe25596ed18f

C++: Remove the problematic taint tracking rule. It seems like we get the flows from dataflow already now.

view details

Taus Brock-Nannestad

commit sha 5fb33c90bc8cfc022916132766d7f7a3e6fc5d0f

Python: Add `ModuleVariableNode` to dataflow

view details

Taus Brock-Nannestad

commit sha 0b641c5ce9de69dbc178620b21ac19acce6658df

Python: Update type tracking and strange-essaflow tests

view details

Taus Brock-Nannestad

commit sha e197f52b6df76da1c3208eb80484ebed31e127c0

Merge branch 'main' into python-add-global-flow-steps

view details

Taus Brock-Nannestad

commit sha 0bb726f21ce231fbeac86bc6264ac5772e044ca0

Python: Fix up merge weirdness

view details

Taus Brock-Nannestad

commit sha 3727c482276f78ea23435e8b9ad14557f8d383ee

Python: Record test changes Some of the places where flow has disappeared look a bit suspect, so I don't consider this to be the final word on these tests.

view details

Mathias Vorreiter Pedersen

commit sha 7cd6137b343da1317ead8689ca6fd46ba753f5f2

Merge branch 'main' into mathiasvp/array-field-flow

view details

Mathias Vorreiter Pedersen

commit sha 3e56db7f83e29fe96e87036df237ca52086e9125

C++: Make fieldReadStep private

view details

Mathias Vorreiter Pedersen

commit sha 0c14e2b69a23d92cf56d85baa38f2b7928a5a3dd

C++: Fix annotations in taint.cpp

view details

Taus Brock-Nannestad

commit sha d5e9f367473fb9464450ebcc571bc6f221bfe0fc

Python: Add "enclosing callable" for `ModuleVariableNode` I've named this `DataFlowModuleScope` since it's not really a callable (and all of the relevant methods are empty anyway).

view details

Taus Brock-Nannestad

commit sha 2e737eda1e8d61bf00abdd140ef4d069d7cbc697

Python: Add a few function-local import tests

view details

push time in 8 days

Pull request review commentgithub/codeql-go

Add query spotting probably-bad escapes in regular expressions.

+package test++import "regexp"++func test() {++	// BAD: probably a mistake:+	regexp.MustCompile("hello\aworld")+	regexp.MustCompile("hello\bworld")+	// GOOD: more likely deliberate:+	regexp.MustCompile("hello\x07world")+	regexp.MustCompile("hello\007world")+	regexp.MustCompile("hello\u0007world")+	regexp.MustCompile("hello\U00000007world")+	regexp.MustCompile("hello\x08world")+	regexp.MustCompile("hello\010world")+	regexp.MustCompile("hello\u0008world")+	regexp.MustCompile("hello\U00000008world")+

:+1:

smowton

comment created time in 8 days

PullRequestReviewEvent

Pull request review commentgithub/codeql-go

Add query spotting probably-bad escapes in regular expressions.

+/**+ * @name Suspicious characters in a regular expression+ * @description If a literal bell character or backspace appears in a regular expression, the start of text or word boundary may have been intended.+ * @kind path-problem+ * @problem.severity warning+ * @precision high+ * @id go/suspicious-character-in-regex+ * @tags correctness+ *       security+ *       external/cwe/cwe-20+ */++import go+import DataFlow::PathGraph++/**+ * Holds if `source` corresponds to a string literal that contains a literal+ * bell character specified as `\a` (not `\x07` or other escapes that resolve to+ * the same character)+ */+predicate containsBellCharacter(DataFlow::Node source) {+  exists(+    source.asExpr().(BasicLit).getText().regexpFind("(?<=(^|[^\\\\])\\\\(\\\\{2}){0,10})a", _, _)

Well, I suppose I asked for it.

Could you combine these two predicates to avoid us having this, uh, very impressive regex in our code base twice?

smowton

comment created time in 8 days

Pull request review commentgithub/codeql-go

Add query spotting probably-bad escapes in regular expressions.

+package test++import "regexp"++func test() {++	// BAD: probably a mistake:+	regexp.MustCompile("hello\aworld")+	regexp.MustCompile("hello\bworld")+	// GOOD: more likely deliberate:+	regexp.MustCompile("hello\x07world")+	regexp.MustCompile("hello\007world")+	regexp.MustCompile("hello\u0007world")+	regexp.MustCompile("hello\U00000007world")+	regexp.MustCompile("hello\x08world")+	regexp.MustCompile("hello\010world")+	regexp.MustCompile("hello\u0008world")+	regexp.MustCompile("hello\U00000008world")+

Let's add a test to ensure we don't flag \\a.

smowton

comment created time in 8 days

PullRequestReviewEvent
PullRequestReviewEvent

push eventmax-schaefer/ql

Max Schaefer

commit sha 915fad4a7cbfadd036c9e30ec46394276b08c351

JavaScript: Add a few metric queries for API graphs.

view details

Max Schaefer

commit sha 49d621a6b149d31afd455d7c3ca59cfd76d1f0b9

JavaScript: Add two new command-injection tests.

view details

Max Schaefer

commit sha 55aff654d9ebf603468f40ac820693b337f48865

JavaScript: Fix use of type backtracker in `IndirectCommandArgument.qll`.

view details

Max Schaefer

commit sha e0761d8628441da0ae1cf31e6172871882c22f19

JavaScript: Do even more type tracking in command injection.

view details

Max Schaefer

commit sha 679a984b88e53ae85714f4065c300a19e25caa5e

JavaScript: Make alert locations for command injection more precise.

view details

push time in 8 days

create barnchmax-schaefer/ql

branch : js/fix-indirect-command-injection

created branch time in 8 days

Pull request review commentgithub/codeql-go

Add query spotting probably-bad escapes in regular expressions.

+/**+ * @name Suspicious characters in a regular expression+ * @description If a literal bell character or backspace appears in a regular expression, the start of text or word boundary may have been intended.+ * @kind path-problem+ * @problem.severity warning+ * @precision high+ * @id go/suspicious-character-in-regex+ * @tags correctness+ *       security+ *       external/cwe/cwe-20+ */++import go+import DataFlow::PathGraph++/**+ * Holds if `source` corresponds to a string literal that contains a literal+ * bell character that was not entered using a `\x07` or `\u0007` escape sequence.+ */+predicate containsBellCharacter(DataFlow::Node source) {+  exists(source.getStringValue().regexpFind("\\u0007", _, _)) and+  // Requiring this to be a BasicLit excludes considering uses of constants+  // as themselves sources (but sources whose raw text can't be interrogated)+  source.(DataFlow::ExprNode).asExpr() instanceof BasicLit and+  not source.(DataFlow::ExprNode).asExpr().(BasicLit).getText().matches(["%\\u0007%", "%\\x07%"])

by simply searching for an actual bell character in the cooked output.

It's admittedly an edge case, but that version has the additional weakness that it won't flag strings containing both an \a and an \x07.

smowton

comment created time in 8 days

PullRequestReviewEvent

Pull request review commentgithub/codeql-go

Add query spotting probably-bad escapes in regular expressions.

+/**+ * @name Suspicious characters in a regular expression+ * @description If a literal bell character or backspace appears in a regular expression, the start of text or word boundary may have been intended.+ * @kind path-problem+ * @problem.severity warning+ * @precision high+ * @id go/suspicious-character-in-regex+ * @tags correctness+ *       security+ *       external/cwe/cwe-20+ */++import go+import DataFlow::PathGraph++/**+ * Holds if `source` corresponds to a string literal that contains a literal+ * bell character that was not entered using a `\x07` or `\u0007` escape sequence.+ */+predicate containsBellCharacter(DataFlow::Node source) {+  exists(source.getStringValue().regexpFind("\\u0007", _, _)) and+  // Requiring this to be a BasicLit excludes considering uses of constants+  // as themselves sources (but sources whose raw text can't be interrogated)+  source.(DataFlow::ExprNode).asExpr() instanceof BasicLit and+  not source.(DataFlow::ExprNode).asExpr().(BasicLit).getText().matches(["%\\u0007%", "%\\x07%"])

if you're happy to vouch for its correctness

I'll let the tests vouch for that :smirk:

smowton

comment created time in 8 days

PullRequestReviewEvent

push eventgithub/codeql-go

Max Schaefer

commit sha 9c640fff4fc6ba6c16d407709a101cd6ad471b94

Add a new test for `FunctionInputsAndOutputs`.

view details

Max Schaefer

commit sha 4b565811223bcda6aab0990210a7c364a4e7557c

Fix input nodes for results that are not assigned to an SSA variable.

view details

Max Schaefer

commit sha 2d4f17c91cf2a61b26939759d9f09bac1223afdf

Ensure result inputs always have an entry node.

view details

Max Schaefer

commit sha 6130720e0024cdfdadfc1d00e23a1adb0634b5bc

Merge pull request #348 from max-schaefer/functioninput_entrynode Ensure `FunctionInput`s corresponding to results have an entry node

view details

push time in 8 days

PR merged github/codeql-go

Ensure `FunctionInput`s corresponding to results have an entry node

Currently, the entry node is required to be a post-update SSA node, so we get an entry node for the result of NewEncoder in this snippet:

enc := json.NewEncoder(w)
enc.Encode(data)

But not in this one:

json.NewEncoder(w).Encode(data)

This is unfortunate, since in this case it prevents us from stitching together the models for Encode (first argument -> receiver) and NewEncoder (result -> first argument).

This PR proposes to fix this by taking the entry node to be the result itself if it isn't assigned to an SSA variable. (We still do want it to be the SSA variable if there is one, since otherwise we'd lose flow in the first example.)

It feels like there should be a more principled solution, but for now I think this fixes our immediate problem.

For ease of reviewing, I have structured this PR into three commits:

  • First commit just adds a test; result changes can be reviewed lightly, since the implementation didn't change.
  • Second commit contains surgical change to just fix the problem above; result changes are small and hopefully show that the intended effect is achieved.
  • Third commit slightly generalises the fix.
+58 -6

4 comments

9 changed files

max-schaefer

pr closed time in 8 days

pull request commentgithub/codeql-go

Ensure `FunctionInput`s corresponding to results have an entry node

Evaluation came back fine.

max-schaefer

comment created time in 8 days

Pull request review commentgithub/codeql-go

Add query spotting probably-bad escapes in regular expressions.

+/**+ * @name Suspicious characters in a regular expression+ * @description If a literal bell character or backspace appears in a regular expression, the start of text or word boundary may have been intended.+ * @kind path-problem+ * @problem.severity warning+ * @precision high+ * @id go/suspicious-character-in-regex+ * @tags correctness+ *       security+ *       external/cwe/cwe-20+ */++import go+import DataFlow::PathGraph++/**+ * Holds if `source` corresponds to a string literal that contains a literal+ * bell character that was not entered using a `\x07` or `\u0007` escape sequence.+ */+predicate containsBellCharacter(DataFlow::Node source) {+  exists(source.getStringValue().regexpFind("\\u0007", _, _)) and+  // Requiring this to be a BasicLit excludes considering uses of constants+  // as themselves sources (but sources whose raw text can't be interrogated)+  source.(DataFlow::ExprNode).asExpr() instanceof BasicLit and+  not source.(DataFlow::ExprNode).asExpr().(BasicLit).getText().matches(["%\\u0007%", "%\\x07%"])+}++/**+ * Holds if `source` corresponds to a string literal that contains a literal+ * backspace that was not entered using a `\x08` or `\u0008` escape sequence.+ */+predicate containsBackspaceCharacter(DataFlow::Node source) {

Same question as above.

smowton

comment created time in 8 days

Pull request review commentgithub/codeql-go

Add query spotting probably-bad escapes in regular expressions.

+/**+ * @name Suspicious characters in a regular expression+ * @description If a literal bell character or backspace appears in a regular expression, the start of text or word boundary may have been intended.+ * @kind path-problem+ * @problem.severity warning+ * @precision high+ * @id go/suspicious-character-in-regex+ * @tags correctness+ *       security+ *       external/cwe/cwe-20+ */++import go+import DataFlow::PathGraph++/**+ * Holds if `source` corresponds to a string literal that contains a literal+ * bell character that was not entered using a `\x07` or `\u0007` escape sequence.+ */+predicate containsBellCharacter(DataFlow::Node source) {+  exists(source.getStringValue().regexpFind("\\u0007", _, _)) and+  // Requiring this to be a BasicLit excludes considering uses of constants+  // as themselves sources (but sources whose raw text can't be interrogated)+  source.(DataFlow::ExprNode).asExpr() instanceof BasicLit and+  not source.(DataFlow::ExprNode).asExpr().(BasicLit).getText().matches(["%\\u0007%", "%\\x07%"])

I'm probably missing some subtlety, but given that you require source to be a BasicLit anyway, couldn't you just do

  source.asExpr().(BasicLit).getText().regexpMatch(".*(?<!\\\\)\\\\a.*")
smowton

comment created time in 8 days

PullRequestReviewEvent
PullRequestReviewEvent

issue closedgithub/codeql-cli-binaries

How to specify a Golang version

Hello.

I tried to create a CodeQL database for kubernetes-1.9.0 (which is an old version) through LGTM.com.

It was built successfully, but I found a warning in the Golang 'Extraction' log, as follows.

...
[build] Detected go version: go version go1.15 linux/amd64.
[build] Kubernetes requires go1.9.1 or greater.
[build] Please install go1.9.1 or later.
...

As far as I know, I can specify the version through lgtm.yml for languages such as Python, C#, Java.

However, in this document, I could not find any words to specify the version of Golang.

Is there any way to specify the Golang version when creating a CodeQL database?

My LGTM project is https://lgtm.com/projects/g/donghyunlee00/v2-kubernetes-1.9.0, exactly same code with kubernetes-1.9.0 release code

Best Regards.

closed time in 8 days

donghyunlee00

issue commentgithub/codeql-cli-binaries

How to specify a Golang version

Discussed at https://github.com/github/codeql-go/issues/351.

donghyunlee00

comment created time in 8 days

PullRequestReviewEvent

issue commentgithub/codeql-go

Feature request: specify Go version used to build project

It's perhaps worth pointing out that when building a Go database, build errors aren't always a problem. The extractor will go ahead and build a database anyway. For example, you can see that there are now a few Go alerts on your project, suggesting that it ultimately managed to build and analyse a database.

donghyunlee00

comment created time in 8 days

Pull request review commentgithub/codeql-go

Add queries for CORS checking, Cookie attribute checking and unsafe signature generation

+/*+ * @name Unsafe signature calculation+ * @description Unsafety Signature that means the developer calculate signatures by just adding it like "a+b" which would bypass easily+ * @kind problem+ * @problem.severity recommendation+ */++import go+import semmle.go.dataflow.DataFlow++class UnsafetySignatureGenerate extends DataFlow::Configuration {+  UnsafetySignatureGenerate() { this = "UnsafetySignatureGenerate" }++  override predicate isSource(DataFlow::Node source) {+    exists(DataFlow::CallNode c1, DataFlow::Method m, DataFlow::ExprNode d |+      (+        m.hasQualifiedName("github.com/astaxie/beego.Controller", "GetString") or+        m.hasQualifiedName("github.com/astaxie/beego.Controller", "GetStrings") or+        m.hasQualifiedName("github.com/astaxie/beego.Controller", "GetInt") or+        m.hasQualifiedName("github.com/astaxie/beego.Controller", "GetFloat") or+        m.hasQualifiedName("github.com/astaxie/beego.Controller", "GetBool")+      ) and+      c1.getCalleeName() = "Write" and

I'm not sure what you mean. This line is almost useless. The only thing it does is to ensure that there is a call to Write somewhere in the program. However, that call is not required to be related to source in any way at all. That doesn't sound like a very useful thing to check.

onkyoworm

comment created time in 8 days

PullRequestReviewEvent

pull request commentgithub/codeql-go

Ensure `FunctionInput`s corresponding to results have an entry node

Yes, I think that's a fair summary.

max-schaefer

comment created time in 8 days

pull request commentgithub/codeql-go

Ensure `FunctionInput`s corresponding to results have an entry node

Initial evaluation on https://github.com/cockroachdb/cockroach looks promising, but I'm running a full comparison to be on the safe side.

max-schaefer

comment created time in 8 days

more