Path-based package wildcards don't work

[jonmcclintock]

With the recent refactor, path-based package wildcards don't work. Using a wildcard in the current directory works:

(env) jonm@jonm:~/go/src/github.com/GoASTScanner/gas$ gas ./... 2>&1 | head
[gas] 2018/03/07 19:47:46 including rules: default
[gas] 2018/03/07 19:47:46 excluding rules: default
[gas] 2018/03/07 19:47:46 Searching directory: /home/jonm/go/src/github.com/GoASTScanner/gas
[gas] 2018/03/07 19:47:46 Searching directory: /home/jonm/go/src/github.com/GoASTScanner/gas/cmd/gas
[gas] 2018/03/07 19:47:46 Searching directory: /home/jonm/go/src/github.com/GoASTScanner/gas/cmd/gasutil
[gas] 2018/03/07 19:47:46 Searching directory: /home/jonm/go/src/github.com/GoASTScanner/gas/cmd/tlsconfig
[gas] 2018/03/07 19:47:46 Searching directory: /home/jonm/go/src/github.com/GoASTScanner/gas/output

But using a real path doesn't:

[gas] 2018/03/07 19:48:29 including rules: default
[gas] 2018/03/07 19:48:29 excluding rules: default
warning: "/home/jonm/go/src/github.com/GoASTScanner/gas/..." matched no packages
[gas] 2018/03/07 19:48:29 no initial packages were loaded

[gcmurphy]

I tried to outsource this logic to https://github.com/kisielk/gotool. (usage here). Maybe raise an issue in that project?

[jonmcclintock]

We'll see what they say:

kisielk/gotool#19

[jonmcclintock]

The gotool maintainer closed it as won't-fix:

This matches how the official go tool works (unsurprisingly, since the code in this repository is copied from it.)

$ go list $(pwd)/...
warning: "/home/dominikh/prj/src/honnef.co/..." matched no packages

Since our goal is explicitly to match go's behaviour, this is working as intended. If you want the behaviour changed, you'll have to file the issue upstream.

[dominikh]

He also pointed out that

the go tool operates on import paths, which may be "absolute" (e.g. github.com/example/foo) or relative (e.g. ./example/foo). It does not operate on file system paths. C.f. go list $PWD

[gcmurphy]

@jonmcclintock see #187 as a potential workaround. (Bit of a hack so probably will be a bit buggy).

[jonmcclintock]

I think using gotool's package resolution is the wrong thing to do here. It also breaks running gas on a single file:

(env) jonm@jonm:~/go/src/github.com/GoASTScanner/gas$ gas analyzer.go 
[gas] 2018/03/20 22:09:07 including rules: default
[gas] 2018/03/20 22:09:07 excluding rules: default
[gas] 2018/03/20 22:09:07 Searching directory: /home/jonm/go/src/github.com/GoASTScanner/gas/analyzer.go
[gas] 2018/03/20 22:09:07 cannot find package "." in:
        analyzer.go

#187 as a short-term workaround is ok, but I don't know if it'll allow for the intuitive thing that you want to do, which is run gas on a specific set of files.

[gcmurphy]

I'm happy to revisit the single file use case but it was removed intentionally. I touched on the reasons for this here #193 (comment).

To elaborate a little more running a scan against a single file you lose a lot of context and cannot really leverage the type checker (if the file is not completely self contained). So the rules become glorified 'regex' for bad things. I wanted to move away from this into a more complete framework that can provide a higher level of accuracy and eventually move towards things like dataflow analysis etc.

For a typical CI use case you would most likely want to scan a whole project, just as you would want to test the whole project and tools like gometalinter account for this type of behaviour so you can still error checking in your editor integration. So I assumed that removing support for a single file wouldn't have a huge impact on users. But perhaps this is not the case..I'll have to think about if/how to resolve this use case.

[jonmcclintock]

I agree 100% that you can provide better, more insightful, findings if you consider the code in context.

But I think it's important to consider the intent of the developer when they say they want to run gas on a single file. They're not saying "only parse this source code when looking for findings", they're saying "only report findings in this source code". I think it's perfectly fair to load other packages and package source files from GOPATH to accomplish that.

The intuitive thing for a user to do is to try and run it on a single source file and see what it outputs. Right now that breaks in a very non-intuitive way, with an especially unhelpful error message.

[vtiwari-ring]

This still doesn't solve the recursive scan issue.

[gcmurphy]

@vtiwari-ring can you provide more detail of what isn't working for you?

The following work for me:

• Recursive scan of current working directory: gas ./...
• Recursive scan of entire path: gas ~/go/src/github.com/...
• Recursive scan by package name: github.com/BurntSushi/toml/...

[vtiwari-ring]

@gcmurphy the 2nd command worked for me. Not the other two. Thanks anyway.