profile
viewpoint
Simarpreet Singh simar7 @aquasecurity Earth https://simar.dev I like to create new things. ––––––––––––––––––––––––––––––––––––– Previously @facebook, @opendns, @cisco, @intel and @blackberry.

opendns/lemming 3

OpenDNS Data Systems Automation

simar7/adventures_in_opencl 1

A tutorial series for learning OpenCL

simar7/android_build 1

Reference Standpoint | CNA Build

raunaqsawhney/Alfred 0

Initial Commit

raunaqsawhney/mips-processor 0

5 Cycle-accurate Implementation of a pipelined MIPS processor

simar7/0x5umm3r86 0

what's all this fuss about.

simar7/8888resolver 0

an attempt to remake 1111resolver but for 8888

Pull request review commentaquasecurity/tracee

Rewrite Python code in Go

+test:

Could we this to the same Makefile but under a different target? Something like python-test for instance.

itaysk

comment created time in an hour

pull request commentaquasecurity/fanal

test(integration/bench): wait for an image load and remove images after tests

Nice find! lgtm!

knqyf263

comment created time in an hour

startedcristaloleg/go-advice

started time in 5 hours

push eventaquasecurity/fanal

Simarpreet Singh

commit sha a6daedf1d4eb99863aa01da02ffe885e574fbf1b

integration: Remove un-needed random num generation Signed-off-by: Simarpreet Singh <simar@linux.com>

view details

push time in 4 days

push eventaquasecurity/fanal

Simarpreet Singh

commit sha b8b68649f2744f0fea0deab07645d655f78f50da

utils_test: Add build tags Signed-off-by: Simarpreet Singh <simar@linux.com>

view details

push time in 4 days

push eventaquasecurity/trivy

Simarpreet Singh

commit sha 5c0addcccb75ef30b7ff50dbc0713b6708a73164

wip Signed-off-by: Simarpreet Singh <simar@linux.com>

view details

push time in 5 days

create barnchaquasecurity/trivy

branch : adding-image-name-build-time

created branch time in 5 days

push eventaquasecurity/fanal

Simarpreet Singh

commit sha 7f8ba06887ba3bc61770b0b7a44321e712484c02

tests: Fix invocation call sites for new JSON cache Signed-off-by: Simarpreet Singh <simar@linux.com>

view details

push time in 5 days

push eventsimar7/git-version-control-tools-clone

dependabot[bot]

commit sha 375511c6349533e890323e1d9a0f6659db1e6454

build(deps): bump django in /testing/docker/builder-eggbuild Bumps [django](https://github.com/django/django) from 1.11.23 to 1.11.28. - [Release notes](https://github.com/django/django/releases) - [Commits](https://github.com/django/django/compare/1.11.23...1.11.28) Signed-off-by: dependabot[bot] <support@github.com>

view details

Simarpreet Singh

commit sha aefcdd0b40a0ab2affb58dd3a1d9ff449e05a4ab

Merge pull request #5 from simar7/dependabot/pip/testing/docker/builder-eggbuild/django-1.11.28 build(deps): bump django from 1.11.23 to 1.11.28 in /testing/docker/builder-eggbuild

view details

push time in 6 days

PR merged simar7/git-version-control-tools-clone

build(deps): bump django from 1.11.23 to 1.11.28 in /testing/docker/builder-eggbuild dependencies

Bumps django from 1.11.23 to 1.11.28. <details> <summary>Commits</summary>

  • e09f09b [1.11.x] Bumped version for 1.11.28 release.
  • 001b063 [1.11.x] Fixed CVE-2020-7471 -- Properly escaped StringAgg(delimiter) parameter.
  • 7fd1ca3 [1.11.x] Fixed timezones tests for PyYAML 5.3+.
  • 121115d [1.11.x] Added CVE-2019-19844 to the security archive.
  • 2c4fb9a [1.11.x] Post-release version bump.
  • 358973a [1.11.x] Bumped version for 1.11.27 release.
  • f4cff43 [1.11.x] Fixed CVE-2019-19844 -- Used verified user email for password reset ...
  • a235574 [1.11.x] Refs #31073 -- Added release notes for 02eff7ef60466da108b1a33f1e4dc...
  • e8fdf00 [1.11.x] Fixed #31073 -- Prevented CheckboxInput.get_context() from mutating ...
  • 4f15016 [1.11.x] Post-release version bump.
  • Additional commits viewable in compare view </details> <br />

Dependabot compatibility score

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


<details> <summary>Dependabot commands and options</summary> <br />

You can trigger Dependabot actions by commenting on this PR:

  • @dependabot rebase will rebase this PR
  • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
  • @dependabot merge will merge this PR after your CI passes on it
  • @dependabot squash and merge will squash and merge this PR after your CI passes on it
  • @dependabot cancel merge will cancel a previously requested merge and block automerging
  • @dependabot reopen will reopen this PR if it is closed
  • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
  • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
  • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
  • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
  • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
  • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
  • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
  • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

You can disable automated security fix PRs for this repo from the Security Alerts page.

</details>

+1 -1

0 comment

1 changed file

dependabot[bot]

pr closed time in 6 days

push eventaquasecurity/fanal

Simarpreet Singh

commit sha 863e24dcea8bf63f902783f5e04901b518b0c492

analyzer_test: Prefer real extractor over mock Signed-off-by: Simarpreet Singh <simar@linux.com>

view details

Simarpreet Singh

commit sha f0c1a3aa83a91dbff95c3e94fcbdc35357da7fcb

analyzer_test: Add sad paths for Analyze Signed-off-by: Simarpreet Singh <simar@linux.com>

view details

push time in 6 days

push eventaquasecurity/fanal

Simarpreet Singh

commit sha e4eab8700d4df7aee979ff5902d7ebe7e00baa33

cache: remove mock_cache.go Signed-off-by: Simarpreet Singh <simar@linux.com>

view details

push time in 6 days

push eventaquasecurity/fanal

Simarpreet Singh

commit sha 397248488e789ba03e9cd6cce569c88df2a7d569

analyze_test: Add tests for Analzye() happy path Signed-off-by: Simarpreet Singh <simar@linux.com>

view details

push time in 6 days

push eventaquasecurity/fanal

Simarpreet Singh

commit sha 892f5a53e84ebc62fb08522ff8a741a7cf05281c

docker_test: remove old crufty tests Signed-off-by: Simarpreet Singh <simar@linux.com>

view details

Simarpreet Singh

commit sha 055601adbde0bd1f6b62054dc1f44ad16d18629c

docker_test: Add tests for ApplyLayers Signed-off-by: Simarpreet Singh <simar@linux.com>

view details

push time in 7 days

push eventaquasecurity/fanal

Simarpreet Singh

commit sha c430ac543bc12afea587f36d74ae262a11182844

docker_test: Add opq and wh file paths. Signed-off-by: Simarpreet Singh <simar@linux.com>

view details

Simarpreet Singh

commit sha bffad7cf734afec9c22efe858a1ac45315b4686a

docker_test: Add sad path for GetLayer Signed-off-by: Simarpreet Singh <simar@linux.com>

view details

Simarpreet Singh

commit sha 7f9d344731686a8a58a7941d9259db5f5d0874ab

docker_test: Add invalid file for extractFiles Signed-off-by: Simarpreet Singh <simar@linux.com>

view details

push time in 7 days

push eventaquasecurity/fanal

Simarpreet Singh

commit sha 6db4ecb0098373f2d506581c0459a648ea1e1f62

docker_test: Add opq and wh file paths. Signed-off-by: Simarpreet Singh <simar@linux.com>

view details

push time in 7 days

startedkelda/dksnap

started time in 8 days

startedrecruit-tech/duplayer

started time in 8 days

startedadrianosela/rdtp

started time in 8 days

startedrsc/goversion

started time in 8 days

startedtakama/daemon

started time in 8 days

push eventaquasecurity/tracee

Simarpreet Singh

commit sha 6c88a361bbf4e670da98590de30f38cd0add4279

tracee: Inject BPF code as an arg for testability Signed-off-by: Simarpreet Singh <simar@linux.com>

view details

Simarpreet Singh

commit sha 59c4cb4dc1a88d4568fe3b0f29bf570536838753

tracee: Move bpfcode stat validation inside of Validate() Signed-off-by: Simarpreet Singh <simar@linux.com>

view details

Simarpreet Singh

commit sha e942d8a4981213738fa614aa28b3878bccf18f5c

Makefile: Add -cover flag

view details

Simarpreet Singh

commit sha 5a4d264aadc03c7d05b2ad473106ff6a0deafe45

tracee_test: Add sad paths Signed-off-by: Simarpreet Singh <simar@linux.com>

view details

push time in 8 days

pull request commentaquasecurity/fanal

integration: Add dockerless mode tests

A few things I've tried and my observations:

  1. Github Actions comes with a very ancient version of Docker Engine (v3.0.8) and the latest being post-upgrade (v19.03.5). Upgrading it doesn't help but should be something we should do anyway.
  2. Upgrading containers/image to v5.1.0 didn't help either: https://github.com/aquasecurity/fanal/pull/83/checks?check_run_id=420067972
  3. GitHub Actions also fails intermittently when upgrading docker. Here's an example: https://github.com/aquasecurity/fanal/pull/81/checks?check_run_id=420105638
  4. GitHub Actions also fails intermittently with it's own configuration: https://github.com/aquasecurity/fanal/pull/81/checks?check_run_id=415876005
simar7

comment created time in 17 days

push eventaquasecurity/fanal

Simarpreet Singh

commit sha dd2b2590dc8cdd4d67bb4357791e736974fc5d32

integration: Remove un-needed random num generation Signed-off-by: Simarpreet Singh <simar@linux.com>

view details

push time in 17 days

PR opened aquasecurity/fanal

WIP: Upgrade containers image

Signed-off-by: Simarpreet Singh simar@linux.com

+305 -60

0 comment

23 changed files

pr created time in 17 days

create barnchaquasecurity/fanal

branch : upgrade-containers-image

created branch time in 17 days

startedoligot/go-mod-upgrade

started time in 17 days

push eventaquasecurity/fanal

Simarpreet Singh

commit sha 49c5f66931a0382234a98f4d920684b16f3392ec

github: Update docker version before running Signed-off-by: Simarpreet Singh <simar@linux.com>

view details

push time in 17 days

push eventaquasecurity/fanal

Simarpreet Singh

commit sha e0dc11d8387f37b707f08c5381db7a43851b3e6b

github: Update docker version before running Signed-off-by: Simarpreet Singh <simar@linux.com>

view details

push time in 17 days

push eventaquasecurity/tracee

Simarpreet Singh

commit sha 2e0b3b17e42251e6a449f26d0c800008156ab707

tracee_test: Add tests for readArgFromBuff Signed-off-by: Simarpreet Singh <simar@linux.com>

view details

push time in 18 days

startedjaemk/self_update

started time in 19 days

push eventaquasecurity/tracee

Simarpreet Singh

commit sha 7a63bee800d37800adc022cab52b8a402e87160e

tracee_go: Fix lint issues Signed-off-by: Simarpreet Singh <simar@linux.com>

view details

push time in 19 days

pull request commentaquasecurity/fanal

integration: Add dockerless mode tests

I'm not sure why cob is picking a git hash that doesn't even exists on this branch...

Run cob --threshold 0.7 --base origin/master --bench-cmd make --bench-args test-performance
2020/01/29 20:33:08 Run Benchmark: f3f6eaeb2c076f9edd828118b474bc4664342c9c origin/master
2020/01/29 20:41:27 Run Benchmark: 01b0a3bb60c567bd29b6c779029f8b4681d41a76 HEAD
2020/01/29 20:47:17 make: *** [test-performance] Error 1

01b0a3bb60c567bd29b6c779029f8b4681d41a76 is a bad ref and doesn't exist on this branch at all.

When I run it locally on this branch on my laptop it works fine:

➜  fanal git:(integration-test-dockerless-mode) cob --threshold 0.7 --base origin/master --bench-cmd make --bench-args test-performance
2020/01/29 12:51:16 Run Benchmark: f3f6eaeb2c076f9edd828118b474bc4664342c9c origin/master
2020/01/29 12:57:08 Run Benchmark: 333eb9b1a97d62162212cbbce990314c69cb86e9 HEAD

Result
======

+----------------------------------------------------------------------------------------+----------+----------------------+-------------------+
|                                          Name                                          |  Commit  |       NsPerOp        | AllocedBytesPerOp |
+----------------------------------------------------------------------------------------+----------+----------------------+-------------------+
| BenchmarkFanal_Library_DockerMode_WithoutCache/happy_path,_vulnimage_with_lock_files-8 |   HEAD   |  4449522065.00 ns/op |   25869615 B/op   |
+                                                                                        +----------+----------------------+-------------------+
|                                                                                        | HEAD@{1} |  4506801447.00 ns/op |   25883655 B/op   |
+----------------------------------------------------------------------------------------+----------+----------------------+-------------------+
|        BenchmarkFanal_Library_DockerMode_WithCache/happy_path,_amazonlinux:2-8         |   HEAD   |   97475484.00 ns/op  |   36928506 B/op   |
+                                                                                        +----------+----------------------+-------------------+
|                                                                                        | HEAD@{1} |  103030436.00 ns/op  |   36936039 B/op   |
+----------------------------------------------------------------------------------------+----------+----------------------+-------------------+
|       BenchmarkFanal_Library_DockerMode_WithoutCache/happy_path,_amazonlinux:2-8       |   HEAD   |  4365885645.00 ns/op |   36965192 B/op   |
+                                                                                        +----------+----------------------+-------------------+
|                                                                                        | HEAD@{1} |  4334010357.00 ns/op |   36975424 B/op   |
+----------------------------------------------------------------------------------------+----------+----------------------+-------------------+
|        BenchmarkFanal_Library_DockerMode_WithoutCache/happy_path,_photon:1.0-8         |   HEAD   |  3264917431.00 ns/op |   16698096 B/op   |
+                                                                                        +----------+----------------------+-------------------+
|                                                                                        | HEAD@{1} |  3779274012.00 ns/op |   16707363 B/op   |
+----------------------------------------------------------------------------------------+----------+----------------------+-------------------+
|  BenchmarkFanal_Library_DockerMode_WithoutCache/happy_path,_registry.redhat.io/ubi7-8  |   HEAD   |  6171049112.00 ns/op |   65907480 B/op   |
+                                                                                        +----------+----------------------+-------------------+
|                                                                                        | HEAD@{1} |  5984381306.00 ns/op |   65903012 B/op   |
+----------------------------------------------------------------------------------------+----------+----------------------+-------------------+
|  BenchmarkFanal_Library_DockerMode_WithCache/happy_path,_vulnimage_with_lock_files-8   |   HEAD   |  122892876.00 ns/op  |   24674955 B/op   |
+                                                                                        +----------+----------------------+-------------------+
|                                                                                        | HEAD@{1} |  148710901.00 ns/op  |   24675572 B/op   |
+----------------------------------------------------------------------------------------+----------+----------------------+-------------------+
|       BenchmarkFanal_Library_DockerMode_WithoutCache/happy_path,_debian:buster-8       |   HEAD   |  3000318343.00 ns/op |    7530569 B/op   |
+                                                                                        +----------+----------------------+-------------------+
|                                                                                        | HEAD@{1} |  3363852615.00 ns/op |    7540257 B/op   |
+----------------------------------------------------------------------------------------+----------+----------------------+-------------------+
|    BenchmarkFanal_Library_DockerMode_WithoutCache/happy_path,_opensuse_leap_15.1-8     |   HEAD   |  2460052588.00 ns/op |   23358625 B/op   |
+                                                                                        +----------+----------------------+-------------------+
|                                                                                        | HEAD@{1} |  2627778914.00 ns/op |   23378525 B/op   |
+----------------------------------------------------------------------------------------+----------+----------------------+-------------------+
|   BenchmarkFanal_Library_DockerMode_WithCache/happy_path,_registry.redhat.io/ubi7-8    |   HEAD   |  120135945.00 ns/op  |   65779538 B/op   |
+                                                                                        +----------+----------------------+-------------------+
|                                                                                        | HEAD@{1} |  127765444.00 ns/op  |   65775812 B/op   |
+----------------------------------------------------------------------------------------+----------+----------------------+-------------------+
|          BenchmarkFanal_Library_DockerMode_WithCache/happy_path,_photon:1.0-8          |   HEAD   |   78208201.00 ns/op  |   16583129 B/op   |
+                                                                                        +----------+----------------------+-------------------+
|                                                                                        | HEAD@{1} |   83761620.00 ns/op  |   16588219 B/op   |
+----------------------------------------------------------------------------------------+----------+----------------------+-------------------+
|        BenchmarkFanal_Library_DockerMode_WithoutCache/happy_path,_alpine:3.10-8        |   HEAD   |  186805673.00 ns/op  |   12121391 B/op   |
+                                                                                        +----------+----------------------+-------------------+
|                                                                                        | HEAD@{1} |  209313529.00 ns/op  |   12124596 B/op   |
+----------------------------------------------------------------------------------------+----------+----------------------+-------------------+
|         BenchmarkFanal_Library_DockerMode_WithCache/happy_path,_alpine:3.10-8          |   HEAD   |   61767327.00 ns/op  |   12002840 B/op   |
+                                                                                        +----------+----------------------+-------------------+
|                                                                                        | HEAD@{1} |   56735611.00 ns/op  |   12001876 B/op   |
+----------------------------------------------------------------------------------------+----------+----------------------+-------------------+
|        BenchmarkFanal_Library_DockerMode_WithCache/happy_path,_debian:buster-8         |   HEAD   |   58285838.00 ns/op  |    7396864 B/op   |
+                                                                                        +----------+----------------------+-------------------+
|                                                                                        | HEAD@{1} |   68083359.00 ns/op  |    7420259 B/op   |
+----------------------------------------------------------------------------------------+----------+----------------------+-------------------+
|      BenchmarkFanal_Library_DockerMode_WithCache/happy_path,_opensuse_leap_15.1-8      |   HEAD   |   51384260.00 ns/op  |   23274945 B/op   |
+                                                                                        +----------+----------------------+-------------------+
|                                                                                        | HEAD@{1} |   62158098.00 ns/op  |   23269427 B/op   |
+----------------------------------------------------------------------------------------+----------+----------------------+-------------------+
cpu
memory

Comparison
==========

+----------------------------------------------------------------------------------------+---------+-------------------+
|                                          Name                                          | NsPerOp | AllocedBytesPerOp |
+----------------------------------------------------------------------------------------+---------+-------------------+
| BenchmarkFanal_Library_DockerMode_WithoutCache/happy_path,_vulnimage_with_lock_files-8 |  1.27%  |       0.05%       |
+----------------------------------------------------------------------------------------+---------+-------------------+
|        BenchmarkFanal_Library_DockerMode_WithCache/happy_path,_amazonlinux:2-8         |  5.39%  |       0.02%       |
+----------------------------------------------------------------------------------------+---------+-------------------+
|       BenchmarkFanal_Library_DockerMode_WithoutCache/happy_path,_amazonlinux:2-8       |  0.74%  |       0.03%       |
+----------------------------------------------------------------------------------------+---------+-------------------+
|        BenchmarkFanal_Library_DockerMode_WithoutCache/happy_path,_photon:1.0-8         | 13.61%  |       0.06%       |
+----------------------------------------------------------------------------------------+---------+-------------------+
|  BenchmarkFanal_Library_DockerMode_WithoutCache/happy_path,_registry.redhat.io/ubi7-8  |  3.12%  |       0.00%       |
+----------------------------------------------------------------------------------------+---------+-------------------+
|  BenchmarkFanal_Library_DockerMode_WithCache/happy_path,_vulnimage_with_lock_files-8   | 17.36%  |       0.00%       |
+----------------------------------------------------------------------------------------+---------+-------------------+
|       BenchmarkFanal_Library_DockerMode_WithoutCache/happy_path,_debian:buster-8       | 10.81%  |       0.13%       |
+----------------------------------------------------------------------------------------+---------+-------------------+
|    BenchmarkFanal_Library_DockerMode_WithoutCache/happy_path,_opensuse_leap_15.1-8     |  6.38%  |       0.09%       |
+----------------------------------------------------------------------------------------+---------+-------------------+
|   BenchmarkFanal_Library_DockerMode_WithCache/happy_path,_registry.redhat.io/ubi7-8    |  5.97%  |       0.00%       |
+----------------------------------------------------------------------------------------+---------+-------------------+
|          BenchmarkFanal_Library_DockerMode_WithCache/happy_path,_photon:1.0-8          |  6.63%  |       0.03%       |
+----------------------------------------------------------------------------------------+---------+-------------------+
|        BenchmarkFanal_Library_DockerMode_WithoutCache/happy_path,_alpine:3.10-8        | 10.75%  |       0.03%       |
+----------------------------------------------------------------------------------------+---------+-------------------+
|         BenchmarkFanal_Library_DockerMode_WithCache/happy_path,_alpine:3.10-8          |  8.87%  |       0.00%       |
+----------------------------------------------------------------------------------------+---------+-------------------+
|        BenchmarkFanal_Library_DockerMode_WithCache/happy_path,_debian:buster-8         | 14.39%  |       0.32%       |
+----------------------------------------------------------------------------------------+---------+-------------------+
|      BenchmarkFanal_Library_DockerMode_WithCache/happy_path,_opensuse_leap_15.1-8      | 17.33%  |       0.02%       |
+----------------------------------------------------------------------------------------+---------+-------------------+

➜  fanal git:(integration-test-dockerless-mode) echo $?
0

Might be a bug in the latest release of cob, I'm not sure. The cob I have locally is not the latest and seems to work fine with it.

simar7

comment created time in 19 days

push eventaquasecurity/fanal

Simarpreet Singh

commit sha 333eb9b1a97d62162212cbbce990314c69cb86e9

library_test: Improve cache assertions with containers/image Signed-off-by: Simarpreet Singh <simar@linux.com>

view details

push time in 19 days

pull request commentaquasecurity/vuln-list-update

fix(amazon): handle removed ALAS

Curious if this is possible with other distros as well @knqyf263?

knqyf263

comment created time in 19 days

push eventaquasecurity/fanal

Teppei Fukuda

commit sha f3f6eaeb2c076f9edd828118b474bc4664342c9c

refactor: replace genuinetools/reg with containers/image (#70) * chore(ci): remove unused lines * feat(cache): add SetBytes * refactor(cache): replace Initialize with New * fix(cache): use ReadCloser instead of Reader * fix(option): update options according to containers/image * feat(image): add struct to manipulate an image * refactor(token): move the directory * chore(Makefile): fix test * chore(Makefile): add containers_image_storage_stub tag * refactor(docker): use Image * refactor(docker): remove unused functions * refactor(docker): update imports * test(docker): fix tests * refactor(analyer): use containers/image * chore(mod): update dependencies * fix(extractor): update interface * fix(main): use updated functions * test(integration): fix * refactor(image): remove unused definition * refactor(error): wrap errors * test(image): add TestNewImage * test(mock): prepare interfaces * test(mock): generate mocks * test(image): add TestImage_LayerInfos * test(image): add TestImage_ConfigBlob * test(image): add TestImage_GetBlob * chore(mod): update dependencies * refactor(error): wrap errors * fix(auth): pass nil when auth is empty * chore(Makefile): add a tag * test(bench): fix * chore(bench): introduce cob * chore(ci): restrict a push trigger * chore(bench): run benchmarks 10 times * test(bench): use a random tag * test(integration): remove ImageRemove * chore(cob): set threshold to 0.7 * image_test: Add unhappy paths for GetBlob Signed-off-by: Simarpreet Singh <simar@linux.com> * refactor(image): remove unused fuction * fix(image): close io.ReadCloser via cleanup function * test(image): do not skip populateSource Co-authored-by: Simarpreet Singh <simar@linux.com>

view details

Simarpreet Singh

commit sha 3552b1d5e4843f73563bacaba96421a28894e3dc

Merge branch 'master' into integration-test-dockerless-mode

view details

push time in 19 days

push eventaquasecurity/fanal

Simarpreet Singh

commit sha 93da8709000d0ffa9a3fbf03f15d5aa840607863

integration: Add dockerless mode tests Signed-off-by: Simarpreet Singh <simar@linux.com>

view details

Simarpreet Singh

commit sha 5475b01d79666bc36b1de169d3a1a121aac6081c

.github: Fix typo Signed-off-by: Simarpreet Singh <simar@linux.com>

view details

push time in 20 days

startedaquasecurity/fanal

started time in 20 days

Pull request review commentaquasecurity/fanal

integration: Add dockerless mode tests

 var testCases = []testCase{ 	}, } +func TestFanal_Library_DockerLessMode(t *testing.T) {+	for _, tc := range testCases {+		tc := tc // save a copy of tc for use in t.Run https://gist.github.com/posener/92a55c4cd441fc5e5e85f27bca008721+		t.Run(tc.name, func(t *testing.T) {+			t.Parallel()+			ctx := context.Background()+			d, _ := ioutil.TempDir("", "TestFanal_Library_DockerLessMode_*")+			defer os.RemoveAll(d)+			c := cache.Initialize(d)++			opt := types.DockerOption{+				Timeout:  600 * time.Second,+				SkipPing: true,+			}++			cli, err := client.NewClientWithOpts(client.FromEnv)+			require.NoError(t, err, tc.name)++			// clear existing Cache if any+			require.NoError(t, c.Clear(), tc.name)++			// remove existing Image if any+			_, _ = cli.ImageRemove(ctx, tc.dockerlessImageName, dtypes.ImageRemoveOptions{+				Force:         true,+				PruneChildren: true,+			})++			ext, err := docker.NewDockerExtractor(opt, c)+			require.NoError(t, err, tc.name)+			ac := analyzer.Config{Extractor: ext}++			// run tests twice, one without cache and with cache+			for i := 1; i <= 2; i++ {+				runChecks(t, "dockerless", ac, ctx, tc, d, c, nil)+			}++			// clear Cache+			require.NoError(t, c.Clear(), tc.name)+		})+	}+}+ func TestFanal_Library_DockerMode(t *testing.T) {-	t.Parallel()

I have removed these in favour of stability. I saw a race condition (didn't dive into why it was happening) amongst Tar and DockerMode tests when run in parallel (similar to what we saw previously). Also adding parallelism at this level doesn't save us much, in my tests it was only 1-2 seconds faster with t.Parallel

simar7

comment created time in 20 days

Pull request review commentaquasecurity/fanal

integration: Add dockerless mode tests

 import ( type testCase struct { 	name                 string 	imageName            string+	dockerlessImageName  string

This probably feels clunky but I thought about reusing imageName as the image name for docker less mode tests. It was even worse since we had to add extra test logic to massage it. I like this better, even though its slightly repetitive.

simar7

comment created time in 20 days

PR opened aquasecurity/fanal

Reviewers
integration: Add dockerless mode tests

Uses images from https://hub.docker.com/u/knqyf263

Signed-off-by: Simarpreet Singh simar@linux.com

+100 -38

0 comment

4 changed files

pr created time in 20 days

create barnchaquasecurity/fanal

branch : integration-test-dockerless-mode

created branch time in 20 days

issue commentaquasecurity/fanal

Integration tests in Dockerless mode

I've been looking into this and found some oddities:

The current test images we're using are not the ones found on docker hub. For instance:

  1. Alpine image 3.10.2 that we use doesn't seem to exist on docker hub, instead 3.10.4 exists.
  2. Amazon Linux version 2 seems to have a different OS version.
We have :analyzer.OS{Name:"2 (Karoo)", Family:"amazon"}
Docker Hub :analyzer.OS{Name:"AMI release 2018.03", Family:"amazon"}

How were the current images created? Should we revise them? Or should we instead have a separate list of test cases for docker less mode since the expectations might change tomorrow depending on if images get removed from docker hub.

This also raises another question, what should we do if we can't find images on docker hub anymore? Should the test fail?

knqyf263

comment created time in 21 days

pull request commentaquasecurity/fanal

refactor: replace genuinetools/reg with containers/image

lgtm!

knqyf263

comment created time in 21 days

Pull request review commentaquasecurity/fanal

refactor: replace genuinetools/reg with containers/image

+package image++import (+	"context"+	"encoding/json"+	"io"+	"io/ioutil"++	"github.com/containers/image/image"+	"github.com/containers/image/pkg/blobinfocache"+	"github.com/containers/image/pkg/compression"+	"github.com/containers/image/transports/alltransports"+	imageTypes "github.com/containers/image/types"+	"github.com/docker/distribution/reference"+	"github.com/opencontainers/go-digest"+	"golang.org/x/xerrors"++	"github.com/aquasecurity/fanal/cache"+	"github.com/aquasecurity/fanal/types"+)++type ImageSource interface {+	GetBlob(ctx context.Context, info imageTypes.BlobInfo, cache imageTypes.BlobInfoCache) (reader io.ReadCloser, n int64, err error)+}++type ImageCloser interface {+	LayerInfos() (layerInfos []imageTypes.BlobInfo)+	ConfigBlob(ctx context.Context) (blob []byte, err error)+}++type Reference struct {+	Name   string+	IsFile bool+}++type Image struct {+	name       string   // e.g. alpine:3.10+	isFile     bool     // from a tar file+	transports []string // e.g. "docker://"++	systemContext *imageTypes.SystemContext+	blobInfoCache imageTypes.BlobInfoCache+	rawSource     ImageSource+	src           ImageCloser++	cache cache.Cache+}++func NewImage(ctx context.Context, image Reference, transports []string, option types.DockerOption,+	c cache.Cache) (Image, error) {+	var domain string+	var auth *imageTypes.DockerAuthConfig++	if !image.IsFile {+		named, err := reference.ParseNormalizedNamed(image.Name)+		if err != nil {+			return Image{}, xerrors.Errorf("invalid image name: %w", err)+		}++		// add 'latest' tag+		named = reference.TagNameOnly(named)+		image.Name = named.String()++		// get a credential for Docker registry+		domain = reference.Domain(named)+		auth = GetToken(ctx, domain, option)+	}++	sys := &imageTypes.SystemContext{+		// TODO: make OSChoice configurable+		OSChoice:                          "linux",+		DockerAuthConfig:                  auth,+		DockerDisableV1Ping:               option.SkipPing,+		DockerInsecureSkipTLSVerify:       imageTypes.NewOptionalBool(option.InsecureSkipTLSVerify),+		OCIInsecureSkipTLSVerify:          option.InsecureSkipTLSVerify,+		DockerDaemonInsecureSkipTLSVerify: option.InsecureSkipTLSVerify,+	}++	return Image{+		name:       image.Name,+		isFile:     image.IsFile,+		transports: transports,++		systemContext: sys,+		blobInfoCache: blobinfocache.DefaultCache(sys),++		cache: c,+	}, nil+}++func (img *Image) populateSource() error {+	if img.rawSource != nil && img.src != nil {+		return nil+	}+	ctx := context.Background()+	var err error+	for _, transport := range img.transports {+		imgName := transport + img.name+		var ref imageTypes.ImageReference+		ref, err = alltransports.ParseImageName(imgName)+		if err != nil {+			return xerrors.Errorf("failed to parse an image name: %w", err)+		}++		var rawSource imageTypes.ImageSource+		rawSource, err = ref.NewImageSource(ctx, img.systemContext)+		if err != nil {+			// try next transport+			continue+		}++		var src imageTypes.ImageCloser+		src, err = image.FromSource(ctx, img.systemContext, rawSource)+		if err != nil {+			return xerrors.Errorf("failed to initialize: %w", err)+		}++		img.rawSource = rawSource+		img.src = src+		return nil+	}+	// return only the last error+	return err+}++func (img *Image) LayerInfos() ([]imageTypes.BlobInfo, error) {+	// ignore the cache error+	layerInfos, _ := img.getLayerInfosInCache(img.name, img.isFile)+	if layerInfos != nil {+		return layerInfos, nil+	}++	// When it doesn't hit a cache, it fetches the image from Docker Engine or Docker Registry+	if err := img.populateSource(); err != nil {+		return nil, xerrors.Errorf("failed population: %w", err)+	}++	layers := img.src.LayerInfos()++	// ignore the cache error+	_ = img.storeLayerInfosInCache(img.name, img.isFile, layers)

Yeah, I'm OK with it. Just thought we could remove some dead code if it will never see the light of the day. Let's keep it for now.

knqyf263

comment created time in 21 days

Pull request review commentaquasecurity/fanal

refactor: replace genuinetools/reg with containers/image

+package image++import (+	"context"+	"encoding/json"+	"io"+	"io/ioutil"++	"github.com/containers/image/image"+	"github.com/containers/image/pkg/blobinfocache"+	"github.com/containers/image/pkg/compression"+	"github.com/containers/image/transports/alltransports"+	imageTypes "github.com/containers/image/types"+	"github.com/docker/distribution/reference"+	"github.com/opencontainers/go-digest"+	"golang.org/x/xerrors"++	"github.com/aquasecurity/fanal/cache"+	"github.com/aquasecurity/fanal/types"+)++type ImageSource interface {+	GetBlob(ctx context.Context, info imageTypes.BlobInfo, cache imageTypes.BlobInfoCache) (reader io.ReadCloser, n int64, err error)+}++type ImageCloser interface {+	LayerInfos() (layerInfos []imageTypes.BlobInfo)+	ConfigBlob(ctx context.Context) (blob []byte, err error)+}++type Reference struct {+	Name   string+	IsFile bool+}++type Image struct {+	name       string   // e.g. alpine:3.10+	isFile     bool     // from a tar file+	transports []string // e.g. "docker://"++	systemContext *imageTypes.SystemContext+	blobInfoCache imageTypes.BlobInfoCache+	rawSource     ImageSource+	src           ImageCloser++	cache cache.Cache+}++func NewImage(ctx context.Context, image Reference, transports []string, option types.DockerOption,+	c cache.Cache) (Image, error) {+	var domain string+	var auth *imageTypes.DockerAuthConfig++	if !image.IsFile {+		named, err := reference.ParseNormalizedNamed(image.Name)+		if err != nil {+			return Image{}, xerrors.Errorf("invalid image name: %w", err)+		}++		// add 'latest' tag+		named = reference.TagNameOnly(named)+		image.Name = named.String()++		// get a credential for Docker registry+		domain = reference.Domain(named)+		auth = GetToken(ctx, domain, option)+	}++	sys := &imageTypes.SystemContext{+		// TODO: make OSChoice configurable+		OSChoice:                          "linux",+		DockerAuthConfig:                  auth,+		DockerDisableV1Ping:               option.SkipPing,+		DockerInsecureSkipTLSVerify:       imageTypes.NewOptionalBool(option.InsecureSkipTLSVerify),+		OCIInsecureSkipTLSVerify:          option.InsecureSkipTLSVerify,+		DockerDaemonInsecureSkipTLSVerify: option.InsecureSkipTLSVerify,+	}++	return Image{+		name:       image.Name,+		isFile:     image.IsFile,+		transports: transports,++		systemContext: sys,+		blobInfoCache: blobinfocache.DefaultCache(sys),++		cache: c,+	}, nil+}++func (img *Image) populateSource() error {+	if img.rawSource != nil && img.src != nil {+		return nil+	}+	ctx := context.Background()+	var err error+	for _, transport := range img.transports {+		imgName := transport + img.name+		var ref imageTypes.ImageReference+		ref, err = alltransports.ParseImageName(imgName)+		if err != nil {+			return xerrors.Errorf("failed to parse an image name: %w", err)+		}++		var rawSource imageTypes.ImageSource+		rawSource, err = ref.NewImageSource(ctx, img.systemContext)+		if err != nil {+			// try next transport+			continue+		}++		var src imageTypes.ImageCloser+		src, err = image.FromSource(ctx, img.systemContext, rawSource)+		if err != nil {+			return xerrors.Errorf("failed to initialize: %w", err)+		}++		img.rawSource = rawSource+		img.src = src+		return nil+	}+	// return only the last error+	return err+}++func (img *Image) LayerInfos() ([]imageTypes.BlobInfo, error) {+	// ignore the cache error+	layerInfos, _ := img.getLayerInfosInCache(img.name, img.isFile)+	if layerInfos != nil {+		return layerInfos, nil+	}++	// When it doesn't hit a cache, it fetches the image from Docker Engine or Docker Registry+	if err := img.populateSource(); err != nil {+		return nil, xerrors.Errorf("failed population: %w", err)+	}++	layers := img.src.LayerInfos()++	// ignore the cache error+	_ = img.storeLayerInfosInCache(img.name, img.isFile, layers)++	return layers, nil+}++func (img Image) getLayerInfosInCache(imageName string, isFile bool) ([]imageTypes.BlobInfo, error) {+	// tar file doesn't have info in cache+	if isFile {+		return nil, nil+	}+	rc := img.cache.Get("layerinfos::" + imageName)+	if rc == nil {+		return nil, nil+	}+	var layerInfos []imageTypes.BlobInfo+	if err := json.NewDecoder(rc).Decode(&layerInfos); err != nil {+		return nil, xerrors.Errorf("invalid cache: %w", err)+	}+	return layerInfos, nil+}++func (img Image) storeLayerInfosInCache(imageName string, isFile bool, layers []imageTypes.BlobInfo) error {+	// it doesn't store the information of a tar file+	if isFile {+		return nil+	}++	b, err := json.Marshal(layers)+	if err != nil {+		return xerrors.Errorf("invalid json: %w", err)+	}+	return img.cache.SetBytes("layerinfos::"+imageName, b)+}++func (img *Image) ConfigBlob(ctx context.Context) ([]byte, error) {+	b, _ := img.getConfigBlobInCache(img.name, img.isFile)+	if b != nil {+		return b, nil+	}++	if err := img.populateSource(); err != nil {+		return nil, xerrors.Errorf("failed population: %w", err)+	}++	blob, err := img.src.ConfigBlob(ctx)+	if err != nil {+		return nil, xerrors.Errorf("failed to get a config: %w", err)+	}++	if !img.isFile {+		_ = img.cache.SetBytes("configblob::"+img.name, blob)+	}++	return blob, nil+}++func (img Image) getConfigBlobInCache(imageName string, isFile bool) ([]byte, error) {+	// tar file doesn't have info in cache+	if isFile {+		return nil, nil+	}+	rc := img.cache.Get("configblob::" + imageName)+	if rc == nil {+		return nil, nil+	}+	b, err := ioutil.ReadAll(rc)+	if err != nil {+		return nil, xerrors.Errorf("unable to read a cache file: %w", err)+	}+	return b, nil+}++func (img *Image) GetBlob(ctx context.Context, dig digest.Digest) (io.ReadCloser, error) {+	rc := img.cache.Get(dig.String())+	if rc != nil {+		return rc, nil+	}++	if err := img.populateSource(); err != nil {

Looks much better now – thanks!

knqyf263

comment created time in 21 days

CommitCommentEvent

Pull request review commentaquasecurity/fanal

refactor: replace genuinetools/reg with containers/image

+package image++import (+	"context"+	"encoding/json"+	"io"+	"io/ioutil"++	"github.com/containers/image/image"+	"github.com/containers/image/pkg/blobinfocache"+	"github.com/containers/image/pkg/compression"+	"github.com/containers/image/transports/alltransports"+	imageTypes "github.com/containers/image/types"+	"github.com/docker/distribution/reference"+	"github.com/opencontainers/go-digest"+	"golang.org/x/xerrors"++	"github.com/aquasecurity/fanal/cache"+	"github.com/aquasecurity/fanal/types"+)++type ImageSource interface {+	GetBlob(ctx context.Context, info imageTypes.BlobInfo, cache imageTypes.BlobInfoCache) (reader io.ReadCloser, n int64, err error)+}++type ImageCloser interface {+	LayerInfos() (layerInfos []imageTypes.BlobInfo)+	ConfigBlob(ctx context.Context) (blob []byte, err error)+}++type Reference struct {+	Name   string+	IsFile bool+}++type Image struct {+	name       string   // e.g. alpine:3.10+	isFile     bool     // from a tar file+	transports []string // e.g. "docker://"++	systemContext *imageTypes.SystemContext+	blobInfoCache imageTypes.BlobInfoCache

Let's add this as part of a separate PR. Would be easier to keep track that way.

knqyf263

comment created time in 21 days

issue commentaquasecurity/kube-query

Add Images tables

I've been thinking about this one. I have a few thoughts:

  1. Currently k8s api has minimal information available on an image. It's just the image:tag information that's available.
  2. Should we take the above information of (image:tag) and use fanal/trivy to get a list of vulnerabilities to create a table instead? This could potentially be more useful that a simple list of images in use.

Thoughts? cc @lizrice

danielsagi

comment created time in 24 days

push eventaquasecurity/fanal

Simarpreet Singh

commit sha 91468b8e046069662c6d9b88a2dbafce3601b427

integration: Fix filenames to not include the `:` char (#79) Signed-off-by: Simarpreet Singh <simar@linux.com>

view details

push time in 24 days

PR merged aquasecurity/fanal

integration: Fix filenames to not include the `:` char

Only valid chars allowed: https://github.com/golang/go/blob/1fb7d5472e8f46faaa034fe6e16ca66a1e7c766f/src/cmd/go/internal/get/path.go#L151

Signed-off-by: Simarpreet Singh simar@linux.com

+7 -3

0 comment

9 changed files

simar7

pr closed time in 24 days

push eventaquasecurity/fanal

Simarpreet Singh

commit sha 40dece55138309d15c2eb2f4221c971eb5a28898

integration: Fix filenames to not include the `:` char Signed-off-by: Simarpreet Singh <simar@linux.com>

view details

push time in 24 days

Pull request review commentaquasecurity/fanal

integration: Fix filenames to not include the `:` char

 func checkPackageFromCommands(t *testing.T, actualFiles extractor.FileMap, osFou func checkPackages(actualFiles extractor.FileMap, t *testing.T, tc testCase) { 	actualPkgs, err := analyzer.GetPackages(actualFiles) 	require.NoError(t, err)-	data, _ := ioutil.ReadFile(fmt.Sprintf("testdata/goldens/%s.expectedpackages.golden", strings.ReplaceAll(tc.imageName, "/", "")))+	goldenFile := fmt.Sprintf("testdata/goldens/%s.expectedpackages.golden", strings.ReplaceAll(strings.ReplaceAll(tc.imageName, "/", ""), ":", ""))

thanks! I actually wanted to do something like that, didn't know about it. I will use it.

simar7

comment created time in 24 days

PR opened aquasecurity/fanal

Reviewers
integration: Fix filenames to not include the `:` char

Only valid chars allowed: https://github.com/golang/go/blob/1fb7d5472e8f46faaa034fe6e16ca66a1e7c766f/src/cmd/go/internal/get/path.go#L151

Signed-off-by: Simarpreet Singh simar@linux.com

+5 -3

0 comment

9 changed files

pr created time in 24 days

push eventaquasecurity/fanal

Simarpreet Singh

commit sha a22b1aedaabd8c33f3d845bb37e915d4243da529

integration: Fix filenames to not include the `:` char Signed-off-by: Simarpreet Singh <simar@linux.com>

view details

push time in 24 days

create barnchaquasecurity/fanal

branch : fix-filenames

created branch time in 24 days

push eventaquasecurity/fanal

Simarpreet Singh

commit sha e8e63e8283538ebad5aeffeddc4d96446605d4f6

image_test: Add unhappy paths for GetBlob Signed-off-by: Simarpreet Singh <simar@linux.com>

view details

Simarpreet Singh

commit sha fb76d82829cf40ab4dae50fec97671bc84ff56c4

Merge branch 'use_image' of github.com:aquasecurity/fanal into use_image

view details

push time in 24 days

Pull request review commentaquasecurity/fanal

refactor: replace genuinetools/reg with containers/image

+package image++import (+	"context"+	"encoding/json"+	"io"+	"io/ioutil"++	"github.com/containers/image/image"+	"github.com/containers/image/pkg/blobinfocache"+	"github.com/containers/image/pkg/compression"+	"github.com/containers/image/transports/alltransports"+	imageTypes "github.com/containers/image/types"+	"github.com/docker/distribution/reference"+	"github.com/opencontainers/go-digest"+	"golang.org/x/xerrors"++	"github.com/aquasecurity/fanal/cache"+	"github.com/aquasecurity/fanal/types"+)++type ImageSource interface {+	GetBlob(ctx context.Context, info imageTypes.BlobInfo, cache imageTypes.BlobInfoCache) (reader io.ReadCloser, n int64, err error)+}++type ImageCloser interface {+	LayerInfos() (layerInfos []imageTypes.BlobInfo)+	ConfigBlob(ctx context.Context) (blob []byte, err error)+}++type Reference struct {+	Name   string+	IsFile bool+}++type Image struct {+	name       string   // e.g. alpine:3.10+	isFile     bool     // from a tar file+	transports []string // e.g. "docker://"++	systemContext *imageTypes.SystemContext+	blobInfoCache imageTypes.BlobInfoCache+	rawSource     ImageSource+	src           ImageCloser++	cache cache.Cache+}++func NewImage(ctx context.Context, image Reference, transports []string, option types.DockerOption,+	c cache.Cache) (Image, error) {+	var domain string+	var auth *imageTypes.DockerAuthConfig++	if !image.IsFile {+		named, err := reference.ParseNormalizedNamed(image.Name)+		if err != nil {+			return Image{}, xerrors.Errorf("invalid image name: %w", err)+		}++		// add 'latest' tag+		named = reference.TagNameOnly(named)+		image.Name = named.String()++		// get a credential for Docker registry+		domain = reference.Domain(named)+		auth = GetToken(ctx, domain, option)+	}++	sys := &imageTypes.SystemContext{+		// TODO: make OSChoice configurable+		OSChoice:                          "linux",+		DockerAuthConfig:                  auth,+		DockerDisableV1Ping:               option.SkipPing,+		DockerInsecureSkipTLSVerify:       imageTypes.NewOptionalBool(option.InsecureSkipTLSVerify),+		OCIInsecureSkipTLSVerify:          option.InsecureSkipTLSVerify,+		DockerDaemonInsecureSkipTLSVerify: option.InsecureSkipTLSVerify,+	}++	return Image{+		name:       image.Name,+		isFile:     image.IsFile,+		transports: transports,++		systemContext: sys,+		blobInfoCache: blobinfocache.DefaultCache(sys),++		cache: c,+	}, nil+}++func (img *Image) populateSource() error {+	if img.rawSource != nil && img.src != nil {+		return nil+	}+	ctx := context.Background()+	var err error+	for _, transport := range img.transports {+		imgName := transport + img.name+		var ref imageTypes.ImageReference+		ref, err = alltransports.ParseImageName(imgName)+		if err != nil {+			return xerrors.Errorf("failed to parse an image name: %w", err)+		}++		var rawSource imageTypes.ImageSource+		rawSource, err = ref.NewImageSource(ctx, img.systemContext)+		if err != nil {+			// try next transport+			continue+		}++		var src imageTypes.ImageCloser+		src, err = image.FromSource(ctx, img.systemContext, rawSource)+		if err != nil {+			return xerrors.Errorf("failed to initialize: %w", err)+		}++		img.rawSource = rawSource+		img.src = src+		return nil+	}+	// return only the last error+	return err+}++func (img *Image) LayerInfos() ([]imageTypes.BlobInfo, error) {+	// ignore the cache error+	layerInfos, _ := img.getLayerInfosInCache(img.name, img.isFile)+	if layerInfos != nil {+		return layerInfos, nil+	}++	// When it doesn't hit a cache, it fetches the image from Docker Engine or Docker Registry+	if err := img.populateSource(); err != nil {+		return nil, xerrors.Errorf("failed population: %w", err)+	}++	layers := img.src.LayerInfos()++	// ignore the cache error+	_ = img.storeLayerInfosInCache(img.name, img.isFile, layers)++	return layers, nil+}++func (img Image) getLayerInfosInCache(imageName string, isFile bool) ([]imageTypes.BlobInfo, error) {+	// tar file doesn't have info in cache+	if isFile {+		return nil, nil+	}+	rc := img.cache.Get("layerinfos::" + imageName)+	if rc == nil {+		return nil, nil+	}+	var layerInfos []imageTypes.BlobInfo+	if err := json.NewDecoder(rc).Decode(&layerInfos); err != nil {+		return nil, xerrors.Errorf("invalid cache: %w", err)+	}+	return layerInfos, nil+}++func (img Image) storeLayerInfosInCache(imageName string, isFile bool, layers []imageTypes.BlobInfo) error {+	// it doesn't store the information of a tar file+	if isFile {+		return nil+	}++	b, err := json.Marshal(layers)+	if err != nil {+		return xerrors.Errorf("invalid json: %w", err)+	}+	return img.cache.SetBytes("layerinfos::"+imageName, b)+}++func (img *Image) ConfigBlob(ctx context.Context) ([]byte, error) {+	b, _ := img.getConfigBlobInCache(img.name, img.isFile)+	if b != nil {+		return b, nil+	}++	if err := img.populateSource(); err != nil {+		return nil, xerrors.Errorf("failed population: %w", err)+	}++	blob, err := img.src.ConfigBlob(ctx)+	if err != nil {+		return nil, xerrors.Errorf("failed to get a config: %w", err)+	}++	if !img.isFile {+		_ = img.cache.SetBytes("configblob::"+img.name, blob)+	}++	return blob, nil+}++func (img Image) getConfigBlobInCache(imageName string, isFile bool) ([]byte, error) {+	// tar file doesn't have info in cache+	if isFile {+		return nil, nil+	}+	rc := img.cache.Get("configblob::" + imageName)+	if rc == nil {+		return nil, nil+	}+	b, err := ioutil.ReadAll(rc)

How big these blobs can get? We're doing ioutil.ReadAll() here so should be mindful.

knqyf263

comment created time in 24 days

Pull request review commentaquasecurity/fanal

refactor: replace genuinetools/reg with containers/image

+package image++import (+	"context"+	"encoding/json"+	"io"+	"io/ioutil"++	"github.com/containers/image/image"+	"github.com/containers/image/pkg/blobinfocache"+	"github.com/containers/image/pkg/compression"+	"github.com/containers/image/transports/alltransports"+	imageTypes "github.com/containers/image/types"+	"github.com/docker/distribution/reference"+	"github.com/opencontainers/go-digest"+	"golang.org/x/xerrors"++	"github.com/aquasecurity/fanal/cache"+	"github.com/aquasecurity/fanal/types"+)++type ImageSource interface {+	GetBlob(ctx context.Context, info imageTypes.BlobInfo, cache imageTypes.BlobInfoCache) (reader io.ReadCloser, n int64, err error)+}++type ImageCloser interface {+	LayerInfos() (layerInfos []imageTypes.BlobInfo)+	ConfigBlob(ctx context.Context) (blob []byte, err error)+}++type Reference struct {+	Name   string+	IsFile bool+}++type Image struct {+	name       string   // e.g. alpine:3.10+	isFile     bool     // from a tar file+	transports []string // e.g. "docker://"++	systemContext *imageTypes.SystemContext+	blobInfoCache imageTypes.BlobInfoCache+	rawSource     ImageSource+	src           ImageCloser++	cache cache.Cache+}++func NewImage(ctx context.Context, image Reference, transports []string, option types.DockerOption,+	c cache.Cache) (Image, error) {+	var domain string+	var auth *imageTypes.DockerAuthConfig++	if !image.IsFile {+		named, err := reference.ParseNormalizedNamed(image.Name)+		if err != nil {+			return Image{}, xerrors.Errorf("invalid image name: %w", err)+		}++		// add 'latest' tag+		named = reference.TagNameOnly(named)+		image.Name = named.String()++		// get a credential for Docker registry+		domain = reference.Domain(named)+		auth = GetToken(ctx, domain, option)+	}++	sys := &imageTypes.SystemContext{+		// TODO: make OSChoice configurable+		OSChoice:                          "linux",+		DockerAuthConfig:                  auth,+		DockerDisableV1Ping:               option.SkipPing,+		DockerInsecureSkipTLSVerify:       imageTypes.NewOptionalBool(option.InsecureSkipTLSVerify),+		OCIInsecureSkipTLSVerify:          option.InsecureSkipTLSVerify,+		DockerDaemonInsecureSkipTLSVerify: option.InsecureSkipTLSVerify,+	}++	return Image{+		name:       image.Name,+		isFile:     image.IsFile,+		transports: transports,++		systemContext: sys,+		blobInfoCache: blobinfocache.DefaultCache(sys),++		cache: c,+	}, nil+}++func (img *Image) populateSource() error {+	if img.rawSource != nil && img.src != nil {+		return nil+	}+	ctx := context.Background()+	var err error+	for _, transport := range img.transports {+		imgName := transport + img.name+		var ref imageTypes.ImageReference+		ref, err = alltransports.ParseImageName(imgName)+		if err != nil {+			return xerrors.Errorf("failed to parse an image name: %w", err)+		}++		var rawSource imageTypes.ImageSource+		rawSource, err = ref.NewImageSource(ctx, img.systemContext)+		if err != nil {+			// try next transport+			continue+		}++		var src imageTypes.ImageCloser+		src, err = image.FromSource(ctx, img.systemContext, rawSource)+		if err != nil {+			return xerrors.Errorf("failed to initialize: %w", err)+		}++		img.rawSource = rawSource+		img.src = src+		return nil+	}+	// return only the last error+	return err+}++func (img *Image) LayerInfos() ([]imageTypes.BlobInfo, error) {+	// ignore the cache error+	layerInfos, _ := img.getLayerInfosInCache(img.name, img.isFile)+	if layerInfos != nil {+		return layerInfos, nil+	}++	// When it doesn't hit a cache, it fetches the image from Docker Engine or Docker Registry+	if err := img.populateSource(); err != nil {+		return nil, xerrors.Errorf("failed population: %w", err)+	}++	layers := img.src.LayerInfos()++	// ignore the cache error+	_ = img.storeLayerInfosInCache(img.name, img.isFile, layers)++	return layers, nil+}++func (img Image) getLayerInfosInCache(imageName string, isFile bool) ([]imageTypes.BlobInfo, error) {+	// tar file doesn't have info in cache+	if isFile {+		return nil, nil+	}+	rc := img.cache.Get("layerinfos::" + imageName)+	if rc == nil {+		return nil, nil+	}+	var layerInfos []imageTypes.BlobInfo+	if err := json.NewDecoder(rc).Decode(&layerInfos); err != nil {+		return nil, xerrors.Errorf("invalid cache: %w", err)+	}+	return layerInfos, nil+}++func (img Image) storeLayerInfosInCache(imageName string, isFile bool, layers []imageTypes.BlobInfo) error {+	// it doesn't store the information of a tar file+	if isFile {+		return nil+	}++	b, err := json.Marshal(layers)+	if err != nil {+		return xerrors.Errorf("invalid json: %w", err)+	}+	return img.cache.SetBytes("layerinfos::"+imageName, b)+}++func (img *Image) ConfigBlob(ctx context.Context) ([]byte, error) {+	b, _ := img.getConfigBlobInCache(img.name, img.isFile)+	if b != nil {+		return b, nil+	}++	if err := img.populateSource(); err != nil {+		return nil, xerrors.Errorf("failed population: %w", err)+	}++	blob, err := img.src.ConfigBlob(ctx)+	if err != nil {+		return nil, xerrors.Errorf("failed to get a config: %w", err)+	}++	if !img.isFile {+		_ = img.cache.SetBytes("configblob::"+img.name, blob)+	}++	return blob, nil+}++func (img Image) getConfigBlobInCache(imageName string, isFile bool) ([]byte, error) {+	// tar file doesn't have info in cache+	if isFile {+		return nil, nil+	}+	rc := img.cache.Get("configblob::" + imageName)+	if rc == nil {+		return nil, nil+	}+	b, err := ioutil.ReadAll(rc)+	if err != nil {+		return nil, xerrors.Errorf("unable to read a cache file: %w", err)+	}+	return b, nil+}++func (img *Image) GetBlob(ctx context.Context, dig digest.Digest) (io.ReadCloser, error) {+	rc := img.cache.Get(dig.String())+	if rc != nil {+		return rc, nil+	}++	if err := img.populateSource(); err != nil {

populateSource() doesn't seem to be unit tested on it's own nor does it seem to have a mock in a situation like this call site. We should add both as it seems to be an important function that's commonly used by various other functions.

knqyf263

comment created time in 24 days

Pull request review commentaquasecurity/fanal

refactor: replace genuinetools/reg with containers/image

+package image++import (+	"context"+	"encoding/json"+	"io"+	"io/ioutil"++	"github.com/containers/image/image"+	"github.com/containers/image/pkg/blobinfocache"+	"github.com/containers/image/pkg/compression"+	"github.com/containers/image/transports/alltransports"+	imageTypes "github.com/containers/image/types"+	"github.com/docker/distribution/reference"+	"github.com/opencontainers/go-digest"+	"golang.org/x/xerrors"++	"github.com/aquasecurity/fanal/cache"+	"github.com/aquasecurity/fanal/types"+)++type ImageSource interface {+	GetBlob(ctx context.Context, info imageTypes.BlobInfo, cache imageTypes.BlobInfoCache) (reader io.ReadCloser, n int64, err error)+}++type ImageCloser interface {+	LayerInfos() (layerInfos []imageTypes.BlobInfo)+	ConfigBlob(ctx context.Context) (blob []byte, err error)+}++type Reference struct {+	Name   string+	IsFile bool+}++type Image struct {+	name       string   // e.g. alpine:3.10+	isFile     bool     // from a tar file+	transports []string // e.g. "docker://"++	systemContext *imageTypes.SystemContext+	blobInfoCache imageTypes.BlobInfoCache+	rawSource     ImageSource+	src           ImageCloser++	cache cache.Cache+}++func NewImage(ctx context.Context, image Reference, transports []string, option types.DockerOption,+	c cache.Cache) (Image, error) {+	var domain string+	var auth *imageTypes.DockerAuthConfig++	if !image.IsFile {+		named, err := reference.ParseNormalizedNamed(image.Name)+		if err != nil {+			return Image{}, xerrors.Errorf("invalid image name: %w", err)+		}++		// add 'latest' tag+		named = reference.TagNameOnly(named)+		image.Name = named.String()++		// get a credential for Docker registry+		domain = reference.Domain(named)+		auth = GetToken(ctx, domain, option)+	}++	sys := &imageTypes.SystemContext{+		// TODO: make OSChoice configurable+		OSChoice:                          "linux",+		DockerAuthConfig:                  auth,+		DockerDisableV1Ping:               option.SkipPing,+		DockerInsecureSkipTLSVerify:       imageTypes.NewOptionalBool(option.InsecureSkipTLSVerify),+		OCIInsecureSkipTLSVerify:          option.InsecureSkipTLSVerify,+		DockerDaemonInsecureSkipTLSVerify: option.InsecureSkipTLSVerify,+	}++	return Image{+		name:       image.Name,+		isFile:     image.IsFile,+		transports: transports,++		systemContext: sys,+		blobInfoCache: blobinfocache.DefaultCache(sys),++		cache: c,+	}, nil+}++func (img *Image) populateSource() error {+	if img.rawSource != nil && img.src != nil {+		return nil+	}+	ctx := context.Background()+	var err error+	for _, transport := range img.transports {+		imgName := transport + img.name+		var ref imageTypes.ImageReference+		ref, err = alltransports.ParseImageName(imgName)+		if err != nil {+			return xerrors.Errorf("failed to parse an image name: %w", err)+		}++		var rawSource imageTypes.ImageSource+		rawSource, err = ref.NewImageSource(ctx, img.systemContext)+		if err != nil {+			// try next transport+			continue+		}++		var src imageTypes.ImageCloser+		src, err = image.FromSource(ctx, img.systemContext, rawSource)+		if err != nil {+			return xerrors.Errorf("failed to initialize: %w", err)+		}++		img.rawSource = rawSource+		img.src = src+		return nil+	}+	// return only the last error+	return err+}++func (img *Image) LayerInfos() ([]imageTypes.BlobInfo, error) {+	// ignore the cache error+	layerInfos, _ := img.getLayerInfosInCache(img.name, img.isFile)+	if layerInfos != nil {+		return layerInfos, nil+	}++	// When it doesn't hit a cache, it fetches the image from Docker Engine or Docker Registry+	if err := img.populateSource(); err != nil {+		return nil, xerrors.Errorf("failed population: %w", err)+	}++	layers := img.src.LayerInfos()++	// ignore the cache error+	_ = img.storeLayerInfosInCache(img.name, img.isFile, layers)++	return layers, nil+}++func (img Image) getLayerInfosInCache(imageName string, isFile bool) ([]imageTypes.BlobInfo, error) {+	// tar file doesn't have info in cache+	if isFile {+		return nil, nil+	}+	rc := img.cache.Get("layerinfos::" + imageName)+	if rc == nil {+		return nil, nil+	}+	var layerInfos []imageTypes.BlobInfo+	if err := json.NewDecoder(rc).Decode(&layerInfos); err != nil {+		return nil, xerrors.Errorf("invalid cache: %w", err)+	}+	return layerInfos, nil+}++func (img Image) storeLayerInfosInCache(imageName string, isFile bool, layers []imageTypes.BlobInfo) error {+	// it doesn't store the information of a tar file+	if isFile {+		return nil+	}++	b, err := json.Marshal(layers)+	if err != nil {+		return xerrors.Errorf("invalid json: %w", err)+	}+	return img.cache.SetBytes("layerinfos::"+imageName, b)+}++func (img *Image) ConfigBlob(ctx context.Context) ([]byte, error) {+	b, _ := img.getConfigBlobInCache(img.name, img.isFile)+	if b != nil {+		return b, nil+	}++	if err := img.populateSource(); err != nil {+		return nil, xerrors.Errorf("failed population: %w", err)+	}++	blob, err := img.src.ConfigBlob(ctx)+	if err != nil {+		return nil, xerrors.Errorf("failed to get a config: %w", err)+	}++	if !img.isFile {+		_ = img.cache.SetBytes("configblob::"+img.name, blob)+	}++	return blob, nil+}++func (img Image) getConfigBlobInCache(imageName string, isFile bool) ([]byte, error) {+	// tar file doesn't have info in cache+	if isFile {+		return nil, nil+	}+	rc := img.cache.Get("configblob::" + imageName)+	if rc == nil {+		return nil, nil+	}+	b, err := ioutil.ReadAll(rc)+	if err != nil {+		return nil, xerrors.Errorf("unable to read a cache file: %w", err)+	}+	return b, nil+}++func (img *Image) GetBlob(ctx context.Context, dig digest.Digest) (io.ReadCloser, error) {+	rc := img.cache.Get(dig.String())+	if rc != nil {+		return rc, nil+	}++	if err := img.populateSource(); err != nil {+		return nil, xerrors.Errorf("failed population: %w", err)+	}++	rc, _, err := img.rawSource.GetBlob(ctx, imageTypes.BlobInfo{Digest: dig, Size: -1}, img.blobInfoCache)+	if err != nil {+		return nil, xerrors.Errorf("failed to download the layer(%s): %w", dig, err)+	}++	stream, _, err := compression.AutoDecompress(rc)+	if err != nil {+		return nil, xerrors.Errorf("failed to download the layer(%s): %w", dig, err)+	}++	r, _ := img.cache.Set(dig.String(), stream)++	return ioutil.NopCloser(r), nil+}++func (img Image) RecordDigestUncompressedPair(dig digest.Digest, uncompressed digest.Digest) {+	img.blobInfoCache.RecordDigestUncompressedPair(dig, uncompressed)+}

This doesn't seem to be used anywhere. Remove?

knqyf263

comment created time in 24 days

Pull request review commentaquasecurity/fanal

refactor: replace genuinetools/reg with containers/image

+package image++import (+	"context"+	"encoding/json"+	"io"+	"io/ioutil"++	"github.com/containers/image/image"+	"github.com/containers/image/pkg/blobinfocache"+	"github.com/containers/image/pkg/compression"+	"github.com/containers/image/transports/alltransports"+	imageTypes "github.com/containers/image/types"+	"github.com/docker/distribution/reference"+	"github.com/opencontainers/go-digest"+	"golang.org/x/xerrors"++	"github.com/aquasecurity/fanal/cache"+	"github.com/aquasecurity/fanal/types"+)++type ImageSource interface {+	GetBlob(ctx context.Context, info imageTypes.BlobInfo, cache imageTypes.BlobInfoCache) (reader io.ReadCloser, n int64, err error)+}++type ImageCloser interface {+	LayerInfos() (layerInfos []imageTypes.BlobInfo)+	ConfigBlob(ctx context.Context) (blob []byte, err error)+}++type Reference struct {+	Name   string+	IsFile bool+}++type Image struct {+	name       string   // e.g. alpine:3.10+	isFile     bool     // from a tar file+	transports []string // e.g. "docker://"++	systemContext *imageTypes.SystemContext+	blobInfoCache imageTypes.BlobInfoCache+	rawSource     ImageSource+	src           ImageCloser++	cache cache.Cache+}++func NewImage(ctx context.Context, image Reference, transports []string, option types.DockerOption,+	c cache.Cache) (Image, error) {+	var domain string+	var auth *imageTypes.DockerAuthConfig++	if !image.IsFile {+		named, err := reference.ParseNormalizedNamed(image.Name)+		if err != nil {+			return Image{}, xerrors.Errorf("invalid image name: %w", err)+		}++		// add 'latest' tag+		named = reference.TagNameOnly(named)+		image.Name = named.String()++		// get a credential for Docker registry+		domain = reference.Domain(named)+		auth = GetToken(ctx, domain, option)+	}++	sys := &imageTypes.SystemContext{+		// TODO: make OSChoice configurable+		OSChoice:                          "linux",+		DockerAuthConfig:                  auth,+		DockerDisableV1Ping:               option.SkipPing,+		DockerInsecureSkipTLSVerify:       imageTypes.NewOptionalBool(option.InsecureSkipTLSVerify),+		OCIInsecureSkipTLSVerify:          option.InsecureSkipTLSVerify,+		DockerDaemonInsecureSkipTLSVerify: option.InsecureSkipTLSVerify,+	}++	return Image{+		name:       image.Name,+		isFile:     image.IsFile,+		transports: transports,++		systemContext: sys,+		blobInfoCache: blobinfocache.DefaultCache(sys),++		cache: c,+	}, nil+}++func (img *Image) populateSource() error {+	if img.rawSource != nil && img.src != nil {+		return nil+	}+	ctx := context.Background()+	var err error+	for _, transport := range img.transports {+		imgName := transport + img.name+		var ref imageTypes.ImageReference+		ref, err = alltransports.ParseImageName(imgName)+		if err != nil {+			return xerrors.Errorf("failed to parse an image name: %w", err)+		}++		var rawSource imageTypes.ImageSource+		rawSource, err = ref.NewImageSource(ctx, img.systemContext)+		if err != nil {+			// try next transport+			continue+		}++		var src imageTypes.ImageCloser+		src, err = image.FromSource(ctx, img.systemContext, rawSource)+		if err != nil {+			return xerrors.Errorf("failed to initialize: %w", err)+		}++		img.rawSource = rawSource+		img.src = src+		return nil+	}+	// return only the last error+	return err+}++func (img *Image) LayerInfos() ([]imageTypes.BlobInfo, error) {+	// ignore the cache error+	layerInfos, _ := img.getLayerInfosInCache(img.name, img.isFile)+	if layerInfos != nil {+		return layerInfos, nil+	}++	// When it doesn't hit a cache, it fetches the image from Docker Engine or Docker Registry+	if err := img.populateSource(); err != nil {+		return nil, xerrors.Errorf("failed population: %w", err)+	}++	layers := img.src.LayerInfos()++	// ignore the cache error+	_ = img.storeLayerInfosInCache(img.name, img.isFile, layers)++	return layers, nil+}++func (img Image) getLayerInfosInCache(imageName string, isFile bool) ([]imageTypes.BlobInfo, error) {+	// tar file doesn't have info in cache+	if isFile {+		return nil, nil+	}+	rc := img.cache.Get("layerinfos::" + imageName)+	if rc == nil {+		return nil, nil+	}+	var layerInfos []imageTypes.BlobInfo+	if err := json.NewDecoder(rc).Decode(&layerInfos); err != nil {+		return nil, xerrors.Errorf("invalid cache: %w", err)+	}+	return layerInfos, nil+}++func (img Image) storeLayerInfosInCache(imageName string, isFile bool, layers []imageTypes.BlobInfo) error {+	// it doesn't store the information of a tar file+	if isFile {+		return nil+	}++	b, err := json.Marshal(layers)+	if err != nil {+		return xerrors.Errorf("invalid json: %w", err)+	}+	return img.cache.SetBytes("layerinfos::"+imageName, b)+}++func (img *Image) ConfigBlob(ctx context.Context) ([]byte, error) {+	b, _ := img.getConfigBlobInCache(img.name, img.isFile)+	if b != nil {+		return b, nil+	}++	if err := img.populateSource(); err != nil {+		return nil, xerrors.Errorf("failed population: %w", err)+	}++	blob, err := img.src.ConfigBlob(ctx)+	if err != nil {+		return nil, xerrors.Errorf("failed to get a config: %w", err)+	}++	if !img.isFile {+		_ = img.cache.SetBytes("configblob::"+img.name, blob)+	}++	return blob, nil+}++func (img Image) getConfigBlobInCache(imageName string, isFile bool) ([]byte, error) {+	// tar file doesn't have info in cache+	if isFile {+		return nil, nil+	}+	rc := img.cache.Get("configblob::" + imageName)+	if rc == nil {+		return nil, nil+	}+	b, err := ioutil.ReadAll(rc)+	if err != nil {+		return nil, xerrors.Errorf("unable to read a cache file: %w", err)+	}+	return b, nil+}++func (img *Image) GetBlob(ctx context.Context, dig digest.Digest) (io.ReadCloser, error) {+	rc := img.cache.Get(dig.String())+	if rc != nil {+		return rc, nil+	}++	if err := img.populateSource(); err != nil {+		return nil, xerrors.Errorf("failed population: %w", err)+	}++	rc, _, err := img.rawSource.GetBlob(ctx, imageTypes.BlobInfo{Digest: dig, Size: -1}, img.blobInfoCache)+	if err != nil {+		return nil, xerrors.Errorf("failed to download the layer(%s): %w", dig, err)+	}++	stream, _, err := compression.AutoDecompress(rc)

Do we call Close() on stream? As per the documentation we need too it seems https://github.com/containers/image/blob/master/pkg/compression/compression.go#L130-L134

We should also add tests for this. Currently there aren't any. One of the things to assert in the test would be to ensure Close() is eventually called as it seems to be a requirement by the dependency.

knqyf263

comment created time in 24 days

Pull request review commentaquasecurity/fanal

refactor: replace genuinetools/reg with containers/image

+package image++import (+	"context"+	"encoding/json"+	"io"+	"io/ioutil"++	"github.com/containers/image/image"+	"github.com/containers/image/pkg/blobinfocache"+	"github.com/containers/image/pkg/compression"+	"github.com/containers/image/transports/alltransports"+	imageTypes "github.com/containers/image/types"+	"github.com/docker/distribution/reference"+	"github.com/opencontainers/go-digest"+	"golang.org/x/xerrors"++	"github.com/aquasecurity/fanal/cache"+	"github.com/aquasecurity/fanal/types"+)++type ImageSource interface {+	GetBlob(ctx context.Context, info imageTypes.BlobInfo, cache imageTypes.BlobInfoCache) (reader io.ReadCloser, n int64, err error)+}++type ImageCloser interface {+	LayerInfos() (layerInfos []imageTypes.BlobInfo)+	ConfigBlob(ctx context.Context) (blob []byte, err error)+}++type Reference struct {+	Name   string+	IsFile bool+}++type Image struct {+	name       string   // e.g. alpine:3.10+	isFile     bool     // from a tar file+	transports []string // e.g. "docker://"++	systemContext *imageTypes.SystemContext+	blobInfoCache imageTypes.BlobInfoCache+	rawSource     ImageSource+	src           ImageCloser++	cache cache.Cache+}++func NewImage(ctx context.Context, image Reference, transports []string, option types.DockerOption,+	c cache.Cache) (Image, error) {+	var domain string+	var auth *imageTypes.DockerAuthConfig++	if !image.IsFile {+		named, err := reference.ParseNormalizedNamed(image.Name)+		if err != nil {+			return Image{}, xerrors.Errorf("invalid image name: %w", err)+		}++		// add 'latest' tag+		named = reference.TagNameOnly(named)+		image.Name = named.String()++		// get a credential for Docker registry+		domain = reference.Domain(named)+		auth = GetToken(ctx, domain, option)+	}++	sys := &imageTypes.SystemContext{+		// TODO: make OSChoice configurable+		OSChoice:                          "linux",+		DockerAuthConfig:                  auth,+		DockerDisableV1Ping:               option.SkipPing,+		DockerInsecureSkipTLSVerify:       imageTypes.NewOptionalBool(option.InsecureSkipTLSVerify),+		OCIInsecureSkipTLSVerify:          option.InsecureSkipTLSVerify,+		DockerDaemonInsecureSkipTLSVerify: option.InsecureSkipTLSVerify,+	}++	return Image{+		name:       image.Name,+		isFile:     image.IsFile,+		transports: transports,++		systemContext: sys,+		blobInfoCache: blobinfocache.DefaultCache(sys),++		cache: c,+	}, nil+}++func (img *Image) populateSource() error {+	if img.rawSource != nil && img.src != nil {+		return nil+	}+	ctx := context.Background()+	var err error+	for _, transport := range img.transports {+		imgName := transport + img.name+		var ref imageTypes.ImageReference+		ref, err = alltransports.ParseImageName(imgName)+		if err != nil {+			return xerrors.Errorf("failed to parse an image name: %w", err)+		}++		var rawSource imageTypes.ImageSource+		rawSource, err = ref.NewImageSource(ctx, img.systemContext)+		if err != nil {+			// try next transport+			continue+		}++		var src imageTypes.ImageCloser+		src, err = image.FromSource(ctx, img.systemContext, rawSource)+		if err != nil {+			return xerrors.Errorf("failed to initialize: %w", err)+		}++		img.rawSource = rawSource+		img.src = src+		return nil+	}+	// return only the last error+	return err+}++func (img *Image) LayerInfos() ([]imageTypes.BlobInfo, error) {+	// ignore the cache error+	layerInfos, _ := img.getLayerInfosInCache(img.name, img.isFile)+	if layerInfos != nil {+		return layerInfos, nil+	}++	// When it doesn't hit a cache, it fetches the image from Docker Engine or Docker Registry+	if err := img.populateSource(); err != nil {+		return nil, xerrors.Errorf("failed population: %w", err)+	}++	layers := img.src.LayerInfos()++	// ignore the cache error+	_ = img.storeLayerInfosInCache(img.name, img.isFile, layers)

Looks like this is the only place where img.storeLayerInfosInCache is called. If the error is not important to handle we should remove it from the signature of the method as it is unexported anyways.

knqyf263

comment created time in 25 days

Pull request review commentaquasecurity/fanal

refactor: replace genuinetools/reg with containers/image

+package image++import (+	"context"+	"encoding/json"+	"io"+	"io/ioutil"++	"github.com/containers/image/image"+	"github.com/containers/image/pkg/blobinfocache"+	"github.com/containers/image/pkg/compression"+	"github.com/containers/image/transports/alltransports"+	imageTypes "github.com/containers/image/types"+	"github.com/docker/distribution/reference"+	"github.com/opencontainers/go-digest"+	"golang.org/x/xerrors"++	"github.com/aquasecurity/fanal/cache"+	"github.com/aquasecurity/fanal/types"+)++type ImageSource interface {+	GetBlob(ctx context.Context, info imageTypes.BlobInfo, cache imageTypes.BlobInfoCache) (reader io.ReadCloser, n int64, err error)+}++type ImageCloser interface {+	LayerInfos() (layerInfos []imageTypes.BlobInfo)+	ConfigBlob(ctx context.Context) (blob []byte, err error)+}++type Reference struct {+	Name   string+	IsFile bool+}++type Image struct {+	name       string   // e.g. alpine:3.10+	isFile     bool     // from a tar file+	transports []string // e.g. "docker://"++	systemContext *imageTypes.SystemContext+	blobInfoCache imageTypes.BlobInfoCache+	rawSource     ImageSource+	src           ImageCloser++	cache cache.Cache+}++func NewImage(ctx context.Context, image Reference, transports []string, option types.DockerOption,+	c cache.Cache) (Image, error) {+	var domain string+	var auth *imageTypes.DockerAuthConfig++	if !image.IsFile {+		named, err := reference.ParseNormalizedNamed(image.Name)+		if err != nil {+			return Image{}, xerrors.Errorf("invalid image name: %w", err)+		}++		// add 'latest' tag+		named = reference.TagNameOnly(named)+		image.Name = named.String()++		// get a credential for Docker registry+		domain = reference.Domain(named)+		auth = GetToken(ctx, domain, option)+	}++	sys := &imageTypes.SystemContext{+		// TODO: make OSChoice configurable+		OSChoice:                          "linux",+		DockerAuthConfig:                  auth,+		DockerDisableV1Ping:               option.SkipPing,+		DockerInsecureSkipTLSVerify:       imageTypes.NewOptionalBool(option.InsecureSkipTLSVerify),+		OCIInsecureSkipTLSVerify:          option.InsecureSkipTLSVerify,+		DockerDaemonInsecureSkipTLSVerify: option.InsecureSkipTLSVerify,+	}++	return Image{+		name:       image.Name,+		isFile:     image.IsFile,+		transports: transports,++		systemContext: sys,+		blobInfoCache: blobinfocache.DefaultCache(sys),++		cache: c,+	}, nil+}++func (img *Image) populateSource() error {+	if img.rawSource != nil && img.src != nil {+		return nil+	}+	ctx := context.Background()+	var err error+	for _, transport := range img.transports {+		imgName := transport + img.name+		var ref imageTypes.ImageReference+		ref, err = alltransports.ParseImageName(imgName)+		if err != nil {+			return xerrors.Errorf("failed to parse an image name: %w", err)+		}++		var rawSource imageTypes.ImageSource+		rawSource, err = ref.NewImageSource(ctx, img.systemContext)+		if err != nil {+			// try next transport+			continue+		}++		var src imageTypes.ImageCloser+		src, err = image.FromSource(ctx, img.systemContext, rawSource)+		if err != nil {+			return xerrors.Errorf("failed to initialize: %w", err)+		}++		img.rawSource = rawSource+		img.src = src+		return nil+	}+	// return only the last error+	return err+}++func (img *Image) LayerInfos() ([]imageTypes.BlobInfo, error) {+	// ignore the cache error+	layerInfos, _ := img.getLayerInfosInCache(img.name, img.isFile)+	if layerInfos != nil {+		return layerInfos, nil+	}++	// When it doesn't hit a cache, it fetches the image from Docker Engine or Docker Registry+	if err := img.populateSource(); err != nil {+		return nil, xerrors.Errorf("failed population: %w", err)+	}

What happens if this returns an error? I think this is untested right now.

knqyf263

comment created time in 25 days

Pull request review commentaquasecurity/fanal

refactor: replace genuinetools/reg with containers/image

+package image++import (+	"context"+	"encoding/json"+	"io"+	"io/ioutil"++	"github.com/containers/image/image"+	"github.com/containers/image/pkg/blobinfocache"+	"github.com/containers/image/pkg/compression"+	"github.com/containers/image/transports/alltransports"+	imageTypes "github.com/containers/image/types"+	"github.com/docker/distribution/reference"+	"github.com/opencontainers/go-digest"+	"golang.org/x/xerrors"++	"github.com/aquasecurity/fanal/cache"+	"github.com/aquasecurity/fanal/types"+)++type ImageSource interface {+	GetBlob(ctx context.Context, info imageTypes.BlobInfo, cache imageTypes.BlobInfoCache) (reader io.ReadCloser, n int64, err error)+}++type ImageCloser interface {+	LayerInfos() (layerInfos []imageTypes.BlobInfo)+	ConfigBlob(ctx context.Context) (blob []byte, err error)+}++type Reference struct {+	Name   string+	IsFile bool+}++type Image struct {+	name       string   // e.g. alpine:3.10+	isFile     bool     // from a tar file+	transports []string // e.g. "docker://"++	systemContext *imageTypes.SystemContext+	blobInfoCache imageTypes.BlobInfoCache

Question: What are they used for? It's not clear to me reading it right away.

knqyf263

comment created time in 25 days

push eventaquasecurity/fanal

Simarpreet Singh

commit sha fd370e6a54ee25ee63729cd8f416ffe2bb0638ec

Perf testing (#72) * integration: Add a test to use fanal as a library Signed-off-by: Simarpreet Singh <simar@linux.com> * integration: Table driven library_tests Signed-off-by: Simarpreet Singh <simar@linux.com> * integration: Add even more OSes to the docker mode test Signed-off-by: Simarpreet Singh <simar@linux.com> * library_test: run tests in parallel Signed-off-by: Simarpreet Singh <simar@linux.com> * .git: Update gitignore with trivy images dir Signed-off-by: Simarpreet Singh <simar@linux.com> * library_test: add golden files for packages Signed-off-by: Simarpreet Singh <simar@linux.com> * library_test: Run all tests in parallel Signed-off-by: Simarpreet Singh <simar@linux.com> * library_test: Refactor check logic to run twice. Once for no cache, once with cache. Signed-off-by: Simarpreet Singh <simar@linux.com> * library_test: Fix cache invocation Signed-off-by: Simarpreet Singh <simar@linux.com> * integration: Add a more comprehensive image for library_test Signed-off-by: Simarpreet Singh <simar@linux.com> * library_test: Introduce anon struct type Signed-off-by: Simarpreet Singh <simar@linux.com> * travis: add make test-integration Signed-off-by: Simarpreet Singh <simar@linux.com> * travis: Upgrade docker version Signed-off-by: Simarpreet Singh <simar@linux.com> * benchmark: Add benchmark tests Signed-off-by: Simarpreet Singh <simar@linux.com> * removeme: Add deliberate sleep Signed-off-by: Simarpreet Singh <simar@linux.com> * remove sleep Signed-off-by: Simarpreet Singh <simar@linux.com> * travis: Add cob to travis Signed-off-by: Simarpreet Singh <simar@linux.com> * chore(bench): use GitHub Actions * chore(bench): use GitHub Actions * chore(bench): install make * chore(bench): use GitHub Actions * chore(ci): move unit tests to GitHub Actions * benchmark_test: Remove assertions and goroutines Signed-off-by: Simarpreet Singh <simar@linux.com> * benchmark_test: Split with and without cache Signed-off-by: Simarpreet Singh <simar@linux.com> * benchamark_test: Add missing assertions, remove cruft Signed-off-by: Simarpreet Singh <simar@linux.com> * benchmark_test: Make tests indepedent of each other. Signed-off-by: Simarpreet Singh <simar@linux.com> * benchmark_test: Refactor teardown Signed-off-by: Simarpreet Singh <simar@linux.com> * benchmark_test: Clear cache per run for Without Cache Signed-off-by: Simarpreet Singh <simar@linux.com> Co-authored-by: Teppei Fukuda <knqyf263@gmail.com>

view details

push time in a month

PR merged aquasecurity/fanal

Perf testing

This PR adds performance testing with cob.

+259 -36

8 comments

4 changed files

simar7

pr closed time in a month

startedsaintmarina/undelete_jpg

started time in a month

startedpingcap/chaos-mesh

started time in a month

push eventaquasecurity/fanal

Simarpreet Singh

commit sha 2b08d887a3d8f1a3dd7656e1b3572cae548d9b8b

integration: Fanal as a library for tar mode (#76) * rename library_test > library_dockermode_test Signed-off-by: Simarpreet Singh <simar@linux.com> * integration: Introduce fanal as a library for tar mode tests. Signed-off-by: Simarpreet Singh <simar@linux.com> * integration: Refactor tar and dockermode together Signed-off-by: Simarpreet Singh <simar@linux.com> * integration: DRY check funcs Signed-off-by: Simarpreet Singh <simar@linux.com> * library_test: Refactor signatures Signed-off-by: Simarpreet Singh <simar@linux.com> * library_test: Remove removal of images for tar mode Signed-off-by: Simarpreet Singh <simar@linux.com>

view details

push time in a month

PR merged aquasecurity/fanal

integration: Fanal as a library for tar mode

This PR adds integration tests for Fanal being used as a library in Tar mode.

Also refactors code.

Addresses: https://github.com/aquasecurity/fanal/issues/73

Signed-off-by: Simarpreet Singh simar@linux.com

+165 -105

0 comment

1 changed file

simar7

pr closed time in a month

push eventaquasecurity/fanal

Simarpreet Singh

commit sha d4fb6be8e34b74d26225d7f509f279c25d1bc799

library_test: Remove removal of images for tar mode Signed-off-by: Simarpreet Singh <simar@linux.com>

view details

push time in a month

push eventaquasecurity/fanal

Simarpreet Singh

commit sha 4ea8185c4c704a479a0c004fd68d38a852e87ccd

benchmark_test: Clear cache per run for Without Cache Signed-off-by: Simarpreet Singh <simar@linux.com>

view details

push time in a month

push eventaquasecurity/fanal

Simarpreet Singh

commit sha 9bbe1c934c50689e71278ca677ef4f6ec3e39af3

benchmark_test: Refactor teardown Signed-off-by: Simarpreet Singh <simar@linux.com>

view details

push time in a month

push eventaquasecurity/fanal

Simarpreet Singh

commit sha 6698d17db188d8243f0d660086d682109e5be71a

benchmark_test: Refactor teardown Signed-off-by: Simarpreet Singh <simar@linux.com>

view details

push time in a month

Pull request review commentaquasecurity/fanal

Perf testing

+// +build performance++package integration++import (+	"context"+	"io/ioutil"+	"os"+	"testing"+	"time"++	_ "github.com/aquasecurity/fanal/analyzer/command/apk"+	_ "github.com/aquasecurity/fanal/analyzer/library/bundler"+	_ "github.com/aquasecurity/fanal/analyzer/library/cargo"+	_ "github.com/aquasecurity/fanal/analyzer/library/composer"+	_ "github.com/aquasecurity/fanal/analyzer/library/npm"+	_ "github.com/aquasecurity/fanal/analyzer/library/pipenv"+	_ "github.com/aquasecurity/fanal/analyzer/library/poetry"+	_ "github.com/aquasecurity/fanal/analyzer/library/yarn"+	_ "github.com/aquasecurity/fanal/analyzer/os/alpine"+	_ "github.com/aquasecurity/fanal/analyzer/os/amazonlinux"+	_ "github.com/aquasecurity/fanal/analyzer/os/debianbase"+	_ "github.com/aquasecurity/fanal/analyzer/os/photon"+	_ "github.com/aquasecurity/fanal/analyzer/os/redhatbase"+	_ "github.com/aquasecurity/fanal/analyzer/os/suse"+	_ "github.com/aquasecurity/fanal/analyzer/pkg/apk"+	_ "github.com/aquasecurity/fanal/analyzer/pkg/dpkg"+	_ "github.com/aquasecurity/fanal/analyzer/pkg/rpm"++	"github.com/aquasecurity/fanal/analyzer"+	"github.com/aquasecurity/fanal/cache"+	"github.com/aquasecurity/fanal/extractor/docker"+	"github.com/aquasecurity/fanal/types"+	dtypes "github.com/docker/docker/api/types"+	"github.com/docker/docker/client"+	"github.com/stretchr/testify/assert"+	"github.com/stretchr/testify/require"+)++type testCase struct {+	name      string+	imageName string+	imageFile string+}++var testCases = []testCase{+	{+		name:      "happy path, alpine:3.10",+		imageName: "alpine:3.10",+		imageFile: "testdata/fixtures/alpine-310.tar.gz",+	},+	{+		name:      "happy path, amazonlinux:2",+		imageName: "amazonlinux:2",+		imageFile: "testdata/fixtures/amazon-2.tar.gz",+	},+	{+		name:      "happy path, debian:buster",+		imageName: "debian:buster",+		imageFile: "testdata/fixtures/debian-buster.tar.gz",+	},+	{+		name:      "happy path, photon:1.0",+		imageName: "photon:1.0-20190823",+		imageFile: "testdata/fixtures/photon-10.tar.gz",+	},+	{+		name:      "happy path, registry.redhat.io/ubi7",+		imageName: "registry.redhat.io/ubi7",+		imageFile: "testdata/fixtures/ubi-7.tar.gz",+	},+	{+		name:      "happy path, opensuse leap 15.1",+		imageName: "opensuse/leap:latest",+		imageFile: "testdata/fixtures/opensuse-leap-151.tar.gz",+	},+	{+		name:      "happy path, vulnimage with lock files",+		imageName: "knqyf263/vuln-image:1.2.3",+		imageFile: "testdata/fixtures/vulnimage.tar.gz",+	},+}++func run(ac analyzer.Config, ctx context.Context, tc testCase, b *testing.B) {+	actualFiles, err := ac.Analyze(ctx, tc.imageFile)+	require.NoError(b, err)++	osFound, err := analyzer.GetOS(actualFiles)+	require.NoError(b, err)++	_, err = analyzer.GetPackages(actualFiles)+	require.NoError(b, err)++	_, err = analyzer.GetPackagesFromCommands(osFound, actualFiles)+	require.NoError(b, err)++	_, err = analyzer.GetLibraries(actualFiles)+	require.NoError(b, err)+}++func runChecksBench(b *testing.B, ac analyzer.Config, ctx context.Context, tc testCase) {+	for i := 0; i < b.N; i++ {+		run(ac, ctx, tc, b)+	}+}++func BenchmarkFanal_Library_DockerMode_WithoutCache(b *testing.B) {+	benchCache, _ := ioutil.TempDir("", "BenchmarkFanal_Library_DockerMode_WithoutCache_*")+	defer os.RemoveAll(benchCache)++	for _, tc := range testCases {+		ctx, _, _, ac := setup(b, tc, benchCache)+		b.Run(tc.name, func(b *testing.B) {+			b.ReportAllocs()+			b.ResetTimer()+			runChecksBench(b, ac, ctx, tc)+			b.StopTimer()+		})++		// teardown+		cli, err := client.NewClientWithOpts(client.FromEnv)+		require.NoError(b, err, tc.name)

good catch. ill fix it.

simar7

comment created time in a month

pull request commentaquasecurity/fanal

Perf testing

It could be my environment I'm not sure. But except alpine, all others are faster with cache.

simar7

comment created time in a month

pull request commentaquasecurity/fanal

refactor: replace genuinetools/reg with containers/image

I tried to review this PR but unfortunately it's a bit too big for me to wrap my head around so I'm still looking into it.

@knqyf263 would you mind rebasing/cherry-picking this PR with https://github.com/aquasecurity/fanal/pull/76 and https://github.com/aquasecurity/fanal/pull/72/ to verify its functional aspects?

knqyf263

comment created time in a month

create barnchaquasecurity/fanal

branch : pr-51-replica-2

created branch time in a month

pull request commentaquasecurity/fanal

Perf testing with cob

@knqyf263 sorry maybe I wasn't clear. That previous benchmark was just some feedback on cob, I ran it using an extra commit I added with a sleep to make it deliberately slower. Here's the output compared against PR #51

➜  fanal git:(pr-51-replica-2) cob -bench-args "test -v -tags=performance -timeout=0 -bench=. ./integration/..."

2020/01/21 15:58:25 Run Benchmark: 352ddcfcb5ce604d62b8b9fc444587c71730e9f3 HEAD~1
2020/01/21 15:59:10 Run Benchmark: 7323925f923c88e78c9754cf5a246e7fe4dd160f HEAD

Result
======

+----------------------------------------------------------------------------------------+----------+----------------------+-------------------+
|                                          Name                                          |  Commit  |       NsPerOp        | AllocedBytesPerOp |
+----------------------------------------------------------------------------------------+----------+----------------------+-------------------+
|         BenchmarkFanal_Library_DockerMode_WithCache/happy_path,_alpine:3.10-8          |   HEAD   |   26986071.00 ns/op  |   18075479 B/op   |
+                                                                                        +----------+----------------------+-------------------+
|                                                                                        | HEAD@{1} |   4255393.00 ns/op   |    525166 B/op    |
+----------------------------------------------------------------------------------------+----------+----------------------+-------------------+
|        BenchmarkFanal_Library_DockerMode_WithCache/happy_path,_amazonlinux:2-8         |   HEAD   |  688284072.00 ns/op  |   362858292 B/op  |
+                                                                                        +----------+----------------------+-------------------+
|                                                                                        | HEAD@{1} |  137790863.00 ns/op  |   36844241 B/op   |
+----------------------------------------------------------------------------------------+----------+----------------------+-------------------+
|        BenchmarkFanal_Library_DockerMode_WithCache/happy_path,_debian:buster-8         |   HEAD   |  4122623719.00 ns/op |   251644712 B/op  |
+                                                                                        +----------+----------------------+-------------------+
|                                                                                        | HEAD@{1} |   81617316.00 ns/op  |    7376742 B/op   |
+----------------------------------------------------------------------------------------+----------+----------------------+-------------------+
|   BenchmarkFanal_Library_DockerMode_WithCache/happy_path,_registry.redhat.io/ubi7-8    |   HEAD   |  833865316.00 ns/op  |   470322780 B/op  |
+                                                                                        +----------+----------------------+-------------------+
|                                                                                        | HEAD@{1} |  169343192.00 ns/op  |   65754536 B/op   |
+----------------------------------------------------------------------------------------+----------+----------------------+-------------------+
|      BenchmarkFanal_Library_DockerMode_WithCache/happy_path,_opensuse_leap_15.1-8      |   HEAD   |  419325751.00 ns/op  |   240911605 B/op  |
+                                                                                        +----------+----------------------+-------------------+
|                                                                                        | HEAD@{1} |   71547104.00 ns/op  |   23198659 B/op   |
+----------------------------------------------------------------------------------------+----------+----------------------+-------------------+
|  BenchmarkFanal_Library_DockerMode_WithCache/happy_path,_vulnimage_with_lock_files-8   |   HEAD   |  529047309.00 ns/op  |   336582308 B/op  |
+                                                                                        +----------+----------------------+-------------------+
|                                                                                        | HEAD@{1} |  106386674.00 ns/op  |   13313083 B/op   |
+----------------------------------------------------------------------------------------+----------+----------------------+-------------------+
|  BenchmarkFanal_Library_DockerMode_WithoutCache/happy_path,_registry.redhat.io/ubi7-8  |   HEAD   |  1272325860.00 ns/op |   470373480 B/op  |
+                                                                                        +----------+----------------------+-------------------+
|                                                                                        | HEAD@{1} |  161460923.00 ns/op  |   65758330 B/op   |
+----------------------------------------------------------------------------------------+----------+----------------------+-------------------+
|    BenchmarkFanal_Library_DockerMode_WithoutCache/happy_path,_opensuse_leap_15.1-8     |   HEAD   |  3501670642.00 ns/op |   796426736 B/op  |
+                                                                                        +----------+----------------------+-------------------+
|                                                                                        | HEAD@{1} |   71200488.00 ns/op  |   23208189 B/op   |
+----------------------------------------------------------------------------------------+----------+----------------------+-------------------+
|       BenchmarkFanal_Library_DockerMode_WithoutCache/happy_path,_debian:buster-8       |   HEAD   |  379387431.00 ns/op  |   251576954 B/op  |
+                                                                                        +----------+----------------------+-------------------+
|                                                                                        | HEAD@{1} |   78436121.00 ns/op  |    7378168 B/op   |
+----------------------------------------------------------------------------------------+----------+----------------------+-------------------+
| BenchmarkFanal_Library_DockerMode_WithoutCache/happy_path,_vulnimage_with_lock_files-8 |   HEAD   |  6321462510.00 ns/op |  1178750784 B/op  |
+                                                                                        +----------+----------------------+-------------------+
|                                                                                        | HEAD@{1} |  103507109.00 ns/op  |   13312119 B/op   |
+----------------------------------------------------------------------------------------+----------+----------------------+-------------------+
|        BenchmarkFanal_Library_DockerMode_WithoutCache/happy_path,_alpine:3.10-8        |   HEAD   |   30052824.00 ns/op  |   18075479 B/op   |
+                                                                                        +----------+----------------------+-------------------+
|                                                                                        | HEAD@{1} |   5104484.00 ns/op   |    525237 B/op    |
+----------------------------------------------------------------------------------------+----------+----------------------+-------------------+
|       BenchmarkFanal_Library_DockerMode_WithoutCache/happy_path,_amazonlinux:2-8       |   HEAD   |  1105224951.00 ns/op |   362903608 B/op  |
+                                                                                        +----------+----------------------+-------------------+
|                                                                                        | HEAD@{1} |  125111849.00 ns/op  |   36843315 B/op   |
+----------------------------------------------------------------------------------------+----------+----------------------+-------------------+
cpu
memory

Comparison
==========

+----------------------------------------------------------------------------------------+----------+-------------------+
|                                          Name                                          | NsPerOp  | AllocedBytesPerOp |
+----------------------------------------------------------------------------------------+----------+-------------------+
|         BenchmarkFanal_Library_DockerMode_WithCache/happy_path,_alpine:3.10-8          | 534.16%  |     3341.86%      |
+----------------------------------------------------------------------------------------+----------+-------------------+
|        BenchmarkFanal_Library_DockerMode_WithCache/happy_path,_amazonlinux:2-8         | 399.51%  |      884.84%      |
+----------------------------------------------------------------------------------------+----------+-------------------+
|        BenchmarkFanal_Library_DockerMode_WithCache/happy_path,_debian:buster-8         | 4951.16% |     3311.33%      |
+----------------------------------------------------------------------------------------+----------+-------------------+
|   BenchmarkFanal_Library_DockerMode_WithCache/happy_path,_registry.redhat.io/ubi7-8    | 392.41%  |      615.27%      |
+----------------------------------------------------------------------------------------+----------+-------------------+
|      BenchmarkFanal_Library_DockerMode_WithCache/happy_path,_opensuse_leap_15.1-8      | 486.08%  |      938.47%      |
+----------------------------------------------------------------------------------------+----------+-------------------+
|  BenchmarkFanal_Library_DockerMode_WithCache/happy_path,_vulnimage_with_lock_files-8   | 397.29%  |     2428.21%      |
+----------------------------------------------------------------------------------------+----------+-------------------+
|  BenchmarkFanal_Library_DockerMode_WithoutCache/happy_path,_registry.redhat.io/ubi7-8  | 688.01%  |      615.31%      |
+----------------------------------------------------------------------------------------+----------+-------------------+
|    BenchmarkFanal_Library_DockerMode_WithoutCache/happy_path,_opensuse_leap_15.1-8     | 4818.04% |     3331.66%      |
+----------------------------------------------------------------------------------------+----------+-------------------+
|       BenchmarkFanal_Library_DockerMode_WithoutCache/happy_path,_debian:buster-8       | 383.69%  |     3309.75%      |
+----------------------------------------------------------------------------------------+----------+-------------------+
| BenchmarkFanal_Library_DockerMode_WithoutCache/happy_path,_vulnimage_with_lock_files-8 | 6007.27% |     8754.72%      |
+----------------------------------------------------------------------------------------+----------+-------------------+
|        BenchmarkFanal_Library_DockerMode_WithoutCache/happy_path,_alpine:3.10-8        | 488.75%  |     3341.39%      |
+----------------------------------------------------------------------------------------+----------+-------------------+
|       BenchmarkFanal_Library_DockerMode_WithoutCache/happy_path,_amazonlinux:2-8       | 783.39%  |      884.99%      |
+----------------------------------------------------------------------------------------+----------+-------------------+

2020/01/21 16:26:10 This commit makes benchmarks worse
simar7

comment created time in a month

Pull request review commentaquasecurity/fanal

Perf testing with cob

+// +build performance++package integration++import (+	"context"+	"io/ioutil"+	"os"+	"testing"+	"time"++	_ "github.com/aquasecurity/fanal/analyzer/command/apk"+	_ "github.com/aquasecurity/fanal/analyzer/library/bundler"+	_ "github.com/aquasecurity/fanal/analyzer/library/cargo"+	_ "github.com/aquasecurity/fanal/analyzer/library/composer"+	_ "github.com/aquasecurity/fanal/analyzer/library/npm"+	_ "github.com/aquasecurity/fanal/analyzer/library/pipenv"+	_ "github.com/aquasecurity/fanal/analyzer/library/poetry"+	_ "github.com/aquasecurity/fanal/analyzer/library/yarn"+	_ "github.com/aquasecurity/fanal/analyzer/os/alpine"+	_ "github.com/aquasecurity/fanal/analyzer/os/amazonlinux"+	_ "github.com/aquasecurity/fanal/analyzer/os/debianbase"+	_ "github.com/aquasecurity/fanal/analyzer/os/photon"+	_ "github.com/aquasecurity/fanal/analyzer/os/redhatbase"+	_ "github.com/aquasecurity/fanal/analyzer/os/suse"+	_ "github.com/aquasecurity/fanal/analyzer/pkg/apk"+	_ "github.com/aquasecurity/fanal/analyzer/pkg/dpkg"+	_ "github.com/aquasecurity/fanal/analyzer/pkg/rpm"++	"github.com/aquasecurity/fanal/analyzer"+	"github.com/aquasecurity/fanal/cache"+	"github.com/aquasecurity/fanal/extractor/docker"+	"github.com/aquasecurity/fanal/types"+	dtypes "github.com/docker/docker/api/types"+	"github.com/docker/docker/client"+	"github.com/stretchr/testify/assert"+	"github.com/stretchr/testify/require"+)++type testCase struct {+	name                 string+	imageName            string+	imageFile            string+	expectedFiles        []string+	expectedOS           analyzer.OS+	expectedPkgsFromCmds string+	expectedLibraries    string+}++var testCases = []testCase{+	{+		name:          "happy path, alpine:3.10",+		imageName:     "alpine:3.10",+		imageFile:     "testdata/fixtures/alpine-310.tar.gz",+		expectedOS:    analyzer.OS{Name: "3.10.2", Family: "alpine"},+		expectedFiles: []string{"etc/alpine-release", "etc/os-release", "lib/apk/db/installed", "/config"},+	},+	{+		name:          "happy path, amazonlinux:2",+		imageName:     "amazonlinux:2",+		imageFile:     "testdata/fixtures/amazon-2.tar.gz",+		expectedFiles: []string{"etc/system-release", "var/lib/rpm/Packages", "etc/os-release", "/config"},+		expectedOS:    analyzer.OS{Name: "2 (Karoo)", Family: "amazon"},+	},+	{+		name:          "happy path, debian:buster",+		imageName:     "debian:buster",+		imageFile:     "testdata/fixtures/debian-buster.tar.gz",+		expectedFiles: []string{"var/lib/dpkg/status", "etc/debian_version", "etc/os-release", "usr/lib/os-release", "/config"},+		expectedOS:    analyzer.OS{Name: "10.1", Family: "debian"},+	},+	{+		name:          "happy path, photon:1.0",+		imageName:     "photon:1.0-20190823",+		imageFile:     "testdata/fixtures/photon-10.tar.gz",+		expectedFiles: []string{"var/lib/rpm/Packages", "etc/lsb-release", "etc/os-release", "/config", "usr/lib/os-release"},+		expectedOS:    analyzer.OS{Name: "1.0", Family: "photon"},+	},+	{+		name:          "happy path, registry.redhat.io/ubi7",+		imageName:     "registry.redhat.io/ubi7",+		imageFile:     "testdata/fixtures/ubi-7.tar.gz",+		expectedFiles: []string{"etc/redhat-release", "etc/system-release", "/config", "var/lib/rpm/Packages", "etc/os-release"},+		expectedOS:    analyzer.OS{Name: "7.7", Family: "redhat"},+	},+	{+		name:          "happy path, opensuse leap 15.1",+		imageName:     "opensuse/leap:latest",+		imageFile:     "testdata/fixtures/opensuse-leap-151.tar.gz",+		expectedFiles: []string{"usr/lib/os-release", "usr/lib/sysimage/rpm/Packages", "/config", "etc/os-release"},+		expectedOS:    analyzer.OS{Name: "15.1", Family: "opensuse.leap"},+	},+	{+		name:                 "happy path, vulnimage with lock files",+		imageName:            "knqyf263/vuln-image:1.2.3",+		imageFile:            "testdata/fixtures/vulnimage.tar.gz",+		expectedFiles:        []string{"etc/os-release", "node-app/package-lock.json", "python-app/Pipfile.lock", "ruby-app/Gemfile.lock", "rust-app/Cargo.lock", "/config", "etc/alpine-release", "lib/apk/db/installed", "php-app/composer.lock"},+		expectedOS:           analyzer.OS{Name: "3.7.1", Family: "alpine"},+		expectedLibraries:    "testdata/goldens/knqyf263vuln-image:1.2.3.expectedlibs.golden",+		expectedPkgsFromCmds: "testdata/goldens/knqyf263vuln-image:1.2.3.expectedpkgsfromcmds.golden",+	},+}++// benchCache is shared across benchmarks+var benchCache string++func runChecksBench(b *testing.B, ac analyzer.Config, ctx context.Context, tc testCase) {+	for i := 0; i < b.N; i++ {+		actualFiles, err := ac.Analyze(ctx, tc.imageFile)+		require.NoError(b, err)++		osFound, err := analyzer.GetOS(actualFiles)+		require.NoError(b, err)++		_, err = analyzer.GetPackages(actualFiles)+		require.NoError(b, err)++		_, err = analyzer.GetPackagesFromCommands(osFound, actualFiles)+		require.NoError(b, err)++		_, err = analyzer.GetLibraries(actualFiles)+	}+}++func BenchmarkFanal_Library_DockerMode_WithoutCache(b *testing.B) {+	benchCache, _ = ioutil.TempDir("", "TestFanal_Library_*")+	for _, tc := range testCases {+		tc := tc // save a copy of tc for use in t.Run https://gist.github.com/posener/92a55c4cd441fc5e5e85f27bca008721+		var ctx context.Context+		var ac analyzer.Config++		ctx, _, _, ac = setup(b, tc, benchCache)+		b.Run(tc.name, func(b *testing.B) {+			b.ReportAllocs()+			b.ResetTimer()+			runChecksBench(b, ac, ctx, tc)+			b.StopTimer()+		})+	}+}++func BenchmarkFanal_Library_DockerMode_WithCache(b *testing.B) {+	defer os.RemoveAll(benchCache)++	for _, tc := range testCases {+		tc := tc // save a copy of tc for use in t.Run https://gist.github.com/posener/92a55c4cd441fc5e5e85f27bca008721+		// setup+		opt := types.DockerOption{+			Timeout:  600 * time.Second,+			SkipPing: true,+		}++		// resuse cache created in previous test+		c := cache.Initialize(benchCache)

Excellent idea. I implemented this: https://github.com/aquasecurity/fanal/pull/72/commits/e26a4093de03bc069e0a6430681d38578b57c49c

simar7

comment created time in a month

push eventaquasecurity/fanal

Simarpreet Singh

commit sha f08b0a1624ef4c2184f2f48f939bc27f1c6fc29c

benchamark_test: Add missing assertions, remove cruft Signed-off-by: Simarpreet Singh <simar@linux.com>

view details

Simarpreet Singh

commit sha e26a4093de03bc069e0a6430681d38578b57c49c

benchmark_test: Make tests indepedent of each other. Signed-off-by: Simarpreet Singh <simar@linux.com>

view details

push time in a month

Pull request review commentaquasecurity/fanal

Perf testing with cob

+// +build performance++package integration++import (+	"context"+	"io/ioutil"+	"os"+	"testing"+	"time"++	_ "github.com/aquasecurity/fanal/analyzer/command/apk"+	_ "github.com/aquasecurity/fanal/analyzer/library/bundler"+	_ "github.com/aquasecurity/fanal/analyzer/library/cargo"+	_ "github.com/aquasecurity/fanal/analyzer/library/composer"+	_ "github.com/aquasecurity/fanal/analyzer/library/npm"+	_ "github.com/aquasecurity/fanal/analyzer/library/pipenv"+	_ "github.com/aquasecurity/fanal/analyzer/library/poetry"+	_ "github.com/aquasecurity/fanal/analyzer/library/yarn"+	_ "github.com/aquasecurity/fanal/analyzer/os/alpine"+	_ "github.com/aquasecurity/fanal/analyzer/os/amazonlinux"+	_ "github.com/aquasecurity/fanal/analyzer/os/debianbase"+	_ "github.com/aquasecurity/fanal/analyzer/os/photon"+	_ "github.com/aquasecurity/fanal/analyzer/os/redhatbase"+	_ "github.com/aquasecurity/fanal/analyzer/os/suse"+	_ "github.com/aquasecurity/fanal/analyzer/pkg/apk"+	_ "github.com/aquasecurity/fanal/analyzer/pkg/dpkg"+	_ "github.com/aquasecurity/fanal/analyzer/pkg/rpm"++	"github.com/aquasecurity/fanal/analyzer"+	"github.com/aquasecurity/fanal/cache"+	"github.com/aquasecurity/fanal/extractor/docker"+	"github.com/aquasecurity/fanal/types"+	dtypes "github.com/docker/docker/api/types"+	"github.com/docker/docker/client"+	"github.com/stretchr/testify/assert"+	"github.com/stretchr/testify/require"+)++type testCase struct {+	name                 string+	imageName            string+	imageFile            string+	expectedFiles        []string+	expectedOS           analyzer.OS+	expectedPkgsFromCmds string+	expectedLibraries    string

Sorry – I confused this PR with the other PR 😅 Yes they need to be removed. Thanks for catching!

simar7

comment created time in a month

Pull request review commentaquasecurity/fanal

Perf testing with cob

+// +build performance++package integration++import (+	"context"+	"io/ioutil"+	"os"+	"testing"+	"time"++	_ "github.com/aquasecurity/fanal/analyzer/command/apk"+	_ "github.com/aquasecurity/fanal/analyzer/library/bundler"+	_ "github.com/aquasecurity/fanal/analyzer/library/cargo"+	_ "github.com/aquasecurity/fanal/analyzer/library/composer"+	_ "github.com/aquasecurity/fanal/analyzer/library/npm"+	_ "github.com/aquasecurity/fanal/analyzer/library/pipenv"+	_ "github.com/aquasecurity/fanal/analyzer/library/poetry"+	_ "github.com/aquasecurity/fanal/analyzer/library/yarn"+	_ "github.com/aquasecurity/fanal/analyzer/os/alpine"+	_ "github.com/aquasecurity/fanal/analyzer/os/amazonlinux"+	_ "github.com/aquasecurity/fanal/analyzer/os/debianbase"+	_ "github.com/aquasecurity/fanal/analyzer/os/photon"+	_ "github.com/aquasecurity/fanal/analyzer/os/redhatbase"+	_ "github.com/aquasecurity/fanal/analyzer/os/suse"+	_ "github.com/aquasecurity/fanal/analyzer/pkg/apk"+	_ "github.com/aquasecurity/fanal/analyzer/pkg/dpkg"+	_ "github.com/aquasecurity/fanal/analyzer/pkg/rpm"++	"github.com/aquasecurity/fanal/analyzer"+	"github.com/aquasecurity/fanal/cache"+	"github.com/aquasecurity/fanal/extractor/docker"+	"github.com/aquasecurity/fanal/types"+	dtypes "github.com/docker/docker/api/types"+	"github.com/docker/docker/client"+	"github.com/stretchr/testify/assert"+	"github.com/stretchr/testify/require"+)++type testCase struct {+	name                 string+	imageName            string+	imageFile            string+	expectedFiles        []string+	expectedOS           analyzer.OS+	expectedPkgsFromCmds string+	expectedLibraries    string

Why not?

simar7

comment created time in a month

push eventaquasecurity/fanal

Simarpreet Singh

commit sha efd8c28420b2112a0338cbc97dc96bd0c906a3a0

library_test: Refactor signatures Signed-off-by: Simarpreet Singh <simar@linux.com>

view details

push time in a month

Pull request review commentaquasecurity/fanal

Perf testing with cob

+// +build performance++package integration++import (+	"context"+	"io/ioutil"+	"os"+	"testing"+	"time"++	_ "github.com/aquasecurity/fanal/analyzer/command/apk"+	_ "github.com/aquasecurity/fanal/analyzer/library/bundler"+	_ "github.com/aquasecurity/fanal/analyzer/library/cargo"+	_ "github.com/aquasecurity/fanal/analyzer/library/composer"+	_ "github.com/aquasecurity/fanal/analyzer/library/npm"+	_ "github.com/aquasecurity/fanal/analyzer/library/pipenv"+	_ "github.com/aquasecurity/fanal/analyzer/library/poetry"+	_ "github.com/aquasecurity/fanal/analyzer/library/yarn"+	_ "github.com/aquasecurity/fanal/analyzer/os/alpine"+	_ "github.com/aquasecurity/fanal/analyzer/os/amazonlinux"+	_ "github.com/aquasecurity/fanal/analyzer/os/debianbase"+	_ "github.com/aquasecurity/fanal/analyzer/os/photon"+	_ "github.com/aquasecurity/fanal/analyzer/os/redhatbase"+	_ "github.com/aquasecurity/fanal/analyzer/os/suse"+	_ "github.com/aquasecurity/fanal/analyzer/pkg/apk"+	_ "github.com/aquasecurity/fanal/analyzer/pkg/dpkg"+	_ "github.com/aquasecurity/fanal/analyzer/pkg/rpm"++	"github.com/aquasecurity/fanal/analyzer"+	"github.com/aquasecurity/fanal/cache"+	"github.com/aquasecurity/fanal/extractor/docker"+	"github.com/aquasecurity/fanal/types"+	dtypes "github.com/docker/docker/api/types"+	"github.com/docker/docker/client"+	"github.com/stretchr/testify/assert"+	"github.com/stretchr/testify/require"+)++type testCase struct {+	name                 string+	imageName            string+	imageFile            string+	expectedFiles        []string+	expectedOS           analyzer.OS+	expectedPkgsFromCmds string+	expectedLibraries    string+}++var testCases = []testCase{+	{+		name:          "happy path, alpine:3.10",+		imageName:     "alpine:3.10",+		imageFile:     "testdata/fixtures/alpine-310.tar.gz",+		expectedOS:    analyzer.OS{Name: "3.10.2", Family: "alpine"},+		expectedFiles: []string{"etc/alpine-release", "etc/os-release", "lib/apk/db/installed", "/config"},+	},+	{+		name:          "happy path, amazonlinux:2",+		imageName:     "amazonlinux:2",+		imageFile:     "testdata/fixtures/amazon-2.tar.gz",+		expectedFiles: []string{"etc/system-release", "var/lib/rpm/Packages", "etc/os-release", "/config"},+		expectedOS:    analyzer.OS{Name: "2 (Karoo)", Family: "amazon"},+	},+	{+		name:          "happy path, debian:buster",+		imageName:     "debian:buster",+		imageFile:     "testdata/fixtures/debian-buster.tar.gz",+		expectedFiles: []string{"var/lib/dpkg/status", "etc/debian_version", "etc/os-release", "usr/lib/os-release", "/config"},+		expectedOS:    analyzer.OS{Name: "10.1", Family: "debian"},+	},+	{+		name:          "happy path, photon:1.0",+		imageName:     "photon:1.0-20190823",+		imageFile:     "testdata/fixtures/photon-10.tar.gz",+		expectedFiles: []string{"var/lib/rpm/Packages", "etc/lsb-release", "etc/os-release", "/config", "usr/lib/os-release"},+		expectedOS:    analyzer.OS{Name: "1.0", Family: "photon"},+	},+	{+		name:          "happy path, registry.redhat.io/ubi7",+		imageName:     "registry.redhat.io/ubi7",+		imageFile:     "testdata/fixtures/ubi-7.tar.gz",+		expectedFiles: []string{"etc/redhat-release", "etc/system-release", "/config", "var/lib/rpm/Packages", "etc/os-release"},+		expectedOS:    analyzer.OS{Name: "7.7", Family: "redhat"},+	},+	{+		name:          "happy path, opensuse leap 15.1",+		imageName:     "opensuse/leap:latest",+		imageFile:     "testdata/fixtures/opensuse-leap-151.tar.gz",+		expectedFiles: []string{"usr/lib/os-release", "usr/lib/sysimage/rpm/Packages", "/config", "etc/os-release"},+		expectedOS:    analyzer.OS{Name: "15.1", Family: "opensuse.leap"},+	},+	{+		name:                 "happy path, vulnimage with lock files",+		imageName:            "knqyf263/vuln-image:1.2.3",+		imageFile:            "testdata/fixtures/vulnimage.tar.gz",+		expectedFiles:        []string{"etc/os-release", "node-app/package-lock.json", "python-app/Pipfile.lock", "ruby-app/Gemfile.lock", "rust-app/Cargo.lock", "/config", "etc/alpine-release", "lib/apk/db/installed", "php-app/composer.lock"},+		expectedOS:           analyzer.OS{Name: "3.7.1", Family: "alpine"},+		expectedLibraries:    "testdata/goldens/knqyf263vuln-image:1.2.3.expectedlibs.golden",+		expectedPkgsFromCmds: "testdata/goldens/knqyf263vuln-image:1.2.3.expectedpkgsfromcmds.golden",+	},+}++// benchCache is shared across benchmarks+var benchCache string++func runChecksBench(b *testing.B, ac analyzer.Config, ctx context.Context, tc testCase) {+	for i := 0; i < b.N; i++ {+		actualFiles, err := ac.Analyze(ctx, tc.imageFile)+		require.NoError(b, err)++		osFound, err := analyzer.GetOS(actualFiles)+		require.NoError(b, err)++		_, err = analyzer.GetPackages(actualFiles)+		require.NoError(b, err)++		_, err = analyzer.GetPackagesFromCommands(osFound, actualFiles)+		require.NoError(b, err)++		_, err = analyzer.GetLibraries(actualFiles)+	}+}++func BenchmarkFanal_Library_DockerMode_WithoutCache(b *testing.B) {+	benchCache, _ = ioutil.TempDir("", "TestFanal_Library_*")+	for _, tc := range testCases {+		tc := tc // save a copy of tc for use in t.Run https://gist.github.com/posener/92a55c4cd441fc5e5e85f27bca008721

good catch - no we don't.

simar7

comment created time in a month

Pull request review commentaquasecurity/fanal

Perf testing with cob

+// +build performance++package integration++import (+	"context"+	"encoding/json"+	"fmt"+	"io/ioutil"+	"os"+	"strings"+	"sync"+	"testing"+	"time"++	_ "github.com/aquasecurity/fanal/analyzer/command/apk"+	_ "github.com/aquasecurity/fanal/analyzer/library/bundler"+	_ "github.com/aquasecurity/fanal/analyzer/library/cargo"+	_ "github.com/aquasecurity/fanal/analyzer/library/composer"+	_ "github.com/aquasecurity/fanal/analyzer/library/npm"+	_ "github.com/aquasecurity/fanal/analyzer/library/pipenv"+	_ "github.com/aquasecurity/fanal/analyzer/library/poetry"+	_ "github.com/aquasecurity/fanal/analyzer/library/yarn"+	_ "github.com/aquasecurity/fanal/analyzer/os/alpine"+	_ "github.com/aquasecurity/fanal/analyzer/os/amazonlinux"+	_ "github.com/aquasecurity/fanal/analyzer/os/debianbase"+	_ "github.com/aquasecurity/fanal/analyzer/os/photon"+	_ "github.com/aquasecurity/fanal/analyzer/os/redhatbase"+	_ "github.com/aquasecurity/fanal/analyzer/os/suse"+	_ "github.com/aquasecurity/fanal/analyzer/pkg/apk"+	_ "github.com/aquasecurity/fanal/analyzer/pkg/dpkg"+	_ "github.com/aquasecurity/fanal/analyzer/pkg/rpm"++	"github.com/aquasecurity/fanal/analyzer"+	"github.com/aquasecurity/fanal/cache"+	"github.com/aquasecurity/fanal/extractor/docker"+	"github.com/aquasecurity/fanal/types"+	godeptypes "github.com/aquasecurity/go-dep-parser/pkg/types"+	dtypes "github.com/docker/docker/api/types"+	"github.com/docker/docker/client"+	"github.com/stretchr/testify/assert"+	"github.com/stretchr/testify/require"+)++type testCase struct {+	name                 string+	imageName            string+	imageFile            string+	expectedFiles        []string+	expectedOS           analyzer.OS+	expectedPkgsFromCmds string+	expectedLibraries    string+}++func runChecksBench(j int, b *testing.B, ac analyzer.Config, ctx context.Context, tc testCase, d string, c cache.Cache) {+	for i := 0; i < b.N; i++ {+		var wg sync.WaitGroup+		for i := 0; i <= j; i++ {+			wg.Add(1)+			go func(i int) {+				defer wg.Done()++				actualFiles, err := ac.Analyze(ctx, tc.imageFile)

It is possible with distributed tracing. OpenTracing is one option. It does require additional work to set it up (collection of traces, etc.) But it's something we can embed into a distributable like Trivy CLI and gather metrics. https://github.com/opentracing/opentracing-go

simar7

comment created time in a month

push eventaquasecurity/fanal

Simarpreet Singh

commit sha 598c234a8f480128beb20924d5b28148645ffee7

library_test: Refactor signatures Signed-off-by: Simarpreet Singh <simar@linux.com>

view details

push time in a month

Pull request review commentaquasecurity/fanal

integration: Fanal as a library for tar mode

 func TestFanal_Library_DockerMode(t *testing.T) { 	} } -func runChecks(t *testing.T, ac analyzer.Config, ctx context.Context, tc testCase, d string, c cache.Cache) {-	actualFiles, err := ac.Analyze(ctx, tc.imageFile)-	require.NoError(t, err)-	for file, _ := range actualFiles {-		assert.Contains(t, tc.expectedFiles, file, tc.name)+func TestFanal_Library_TarMode(t *testing.T) {+	t.Parallel()+	for _, tc := range testCases {+		tc := tc // save a copy of tc for use in t.Run https://gist.github.com/posener/92a55c4cd441fc5e5e85f27bca008721+		t.Run(tc.name, func(t *testing.T) {+			t.Parallel()+			ctx := context.Background()+			d, _ := ioutil.TempDir("", "TestFanal_Library_*")+			defer os.RemoveAll(d)+			c := cache.Initialize(d)++			opt := types.DockerOption{+				Timeout:  600 * time.Second,+				SkipPing: true,+			}++			cli, err := client.NewClientWithOpts(client.FromEnv)+			require.NoError(t, err, tc.name)++			// ensure image doesnt already exists+			_, _ = cli.ImageRemove(ctx, tc.imageFile, dtypes.ImageRemoveOptions{

Yes this makes the test idempotent.

simar7

comment created time in a month

pull request commentaquasecurity/fanal

Perf testing with cob

I tried adding time.Sleep(time.Second*2) to Analyze and these are the results:

2020/01/20 20:52:12 Run Benchmark: 9268652494b37ddd6683edea441c91d1bca808d3 HEAD~1
2020/01/20 20:53:15 Run Benchmark: 52df99cbc94ad4d08d00e3fc317c93c70fc08b51 HEAD

Result
======

+----------------------------------------------------------------------------------------+----------+----------------------+-------------------+
|                                          Name                                          |  Commit  |       NsPerOp        | AllocedBytesPerOp |
+----------------------------------------------------------------------------------------+----------+----------------------+-------------------+
|      BenchmarkFanal_Library_DockerMode_WithCache/happy_path,_opensuse_leap_15.1-8      |   HEAD   |   69825780.00 ns/op  |   23242297 B/op   |
+                                                                                        +----------+----------------------+-------------------+
|                                                                                        | HEAD@{1} |   74504263.00 ns/op  |   23243108 B/op   |
+----------------------------------------------------------------------------------------+----------+----------------------+-------------------+
|        BenchmarkFanal_Library_DockerMode_WithoutCache/happy_path,_alpine:3.10-8        |   HEAD   |   60429251.00 ns/op  |   11948800 B/op   |
+                                                                                        +----------+----------------------+-------------------+
|                                                                                        | HEAD@{1} |   60668475.00 ns/op  |   11951687 B/op   |
+----------------------------------------------------------------------------------------+----------+----------------------+-------------------+
| BenchmarkFanal_Library_DockerMode_WithoutCache/happy_path,_vulnimage_with_lock_files-8 |   HEAD   |  4815920556.00 ns/op |   24749160 B/op   |
+                                                                                        +----------+----------------------+-------------------+
|                                                                                        | HEAD@{1} |  6263285882.00 ns/op |   24738120 B/op   |
+----------------------------------------------------------------------------------------+----------+----------------------+-------------------+
|         BenchmarkFanal_Library_DockerMode_WithCache/happy_path,_alpine:3.10-8          |   HEAD   |   57912911.00 ns/op  |   11947491 B/op   |
+                                                                                        +----------+----------------------+-------------------+
|                                                                                        | HEAD@{1} |   62011062.00 ns/op  |   11953694 B/op   |
+----------------------------------------------------------------------------------------+----------+----------------------+-------------------+
|   BenchmarkFanal_Library_DockerMode_WithCache/happy_path,_registry.redhat.io/ubi7-8    |   HEAD   |  158639010.00 ns/op  |   65735156 B/op   |
+                                                                                        +----------+----------------------+-------------------+
|                                                                                        | HEAD@{1} |  189177784.00 ns/op  |   65738556 B/op   |
+----------------------------------------------------------------------------------------+----------+----------------------+-------------------+
|       BenchmarkFanal_Library_DockerMode_WithoutCache/happy_path,_debian:buster-8       |   HEAD   |  3148455073.00 ns/op |    7425912 B/op   |
+                                                                                        +----------+----------------------+-------------------+
|                                                                                        | HEAD@{1} |  3585126557.00 ns/op |    7425400 B/op   |
+----------------------------------------------------------------------------------------+----------+----------------------+-------------------+
|        BenchmarkFanal_Library_DockerMode_WithoutCache/happy_path,_photon:1.0-8         |   HEAD   |  3384472155.00 ns/op |   16588512 B/op   |
+                                                                                        +----------+----------------------+-------------------+
|                                                                                        | HEAD@{1} |  4999635858.00 ns/op |   16555600 B/op   |
+----------------------------------------------------------------------------------------+----------+----------------------+-------------------+
|          BenchmarkFanal_Library_DockerMode_WithCache/happy_path,_photon:1.0-8          |   HEAD   |  102068646.00 ns/op  |   16560638 B/op   |
+                                                                                        +----------+----------------------+-------------------+
|                                                                                        | HEAD@{1} |  108587133.00 ns/op  |   16580364 B/op   |
+----------------------------------------------------------------------------------------+----------+----------------------+-------------------+
|        BenchmarkFanal_Library_DockerMode_WithCache/happy_path,_amazonlinux:2-8         |   HEAD   |  126806459.00 ns/op  |   36855963 B/op   |
+                                                                                        +----------+----------------------+-------------------+
|                                                                                        | HEAD@{1} |  135358079.00 ns/op  |   36845914 B/op   |
+----------------------------------------------------------------------------------------+----------+----------------------+-------------------+
|        BenchmarkFanal_Library_DockerMode_WithCache/happy_path,_debian:buster-8         |   HEAD   |   79579785.00 ns/op  |    7415195 B/op   |
+                                                                                        +----------+----------------------+-------------------+
|                                                                                        | HEAD@{1} |   87481727.00 ns/op  |    7409803 B/op   |
+----------------------------------------------------------------------------------------+----------+----------------------+-------------------+
|  BenchmarkFanal_Library_DockerMode_WithCache/happy_path,_vulnimage_with_lock_files-8   |   HEAD   |  152463743.00 ns/op  |   24698618 B/op   |
+                                                                                        +----------+----------------------+-------------------+
|                                                                                        | HEAD@{1} |  157955019.00 ns/op  |   24709108 B/op   |
+----------------------------------------------------------------------------------------+----------+----------------------+-------------------+
|       BenchmarkFanal_Library_DockerMode_WithoutCache/happy_path,_amazonlinux:2-8       |   HEAD   |  3756398171.00 ns/op |   36869088 B/op   |
+                                                                                        +----------+----------------------+-------------------+
|                                                                                        | HEAD@{1} |  4981388690.00 ns/op |   36876400 B/op   |
+----------------------------------------------------------------------------------------+----------+----------------------+-------------------+
|  BenchmarkFanal_Library_DockerMode_WithoutCache/happy_path,_registry.redhat.io/ubi7-8  |   HEAD   |  6244091637.00 ns/op |   65818240 B/op   |
+                                                                                        +----------+----------------------+-------------------+
|                                                                                        | HEAD@{1} |  6487256481.00 ns/op |   65785584 B/op   |
+----------------------------------------------------------------------------------------+----------+----------------------+-------------------+
|    BenchmarkFanal_Library_DockerMode_WithoutCache/happy_path,_opensuse_leap_15.1-8     |   HEAD   |  2551770174.00 ns/op |   23293016 B/op   |
+                                                                                        +----------+----------------------+-------------------+
|                                                                                        | HEAD@{1} |  2581555879.00 ns/op |   23250560 B/op   |
+----------------------------------------------------------------------------------------+----------+----------------------+-------------------+
cpu
memory

Comparison
==========

+----------------------------------------------------------------------------------------+---------+-------------------+
|                                          Name                                          | NsPerOp | AllocedBytesPerOp |
+----------------------------------------------------------------------------------------+---------+-------------------+
|      BenchmarkFanal_Library_DockerMode_WithCache/happy_path,_opensuse_leap_15.1-8      |  6.28%  |       0.00%       |
+----------------------------------------------------------------------------------------+---------+-------------------+
|        BenchmarkFanal_Library_DockerMode_WithoutCache/happy_path,_alpine:3.10-8        |  0.39%  |       0.02%       |
+----------------------------------------------------------------------------------------+---------+-------------------+
| BenchmarkFanal_Library_DockerMode_WithoutCache/happy_path,_vulnimage_with_lock_files-8 | 23.11%  |       0.04%       |
+----------------------------------------------------------------------------------------+---------+-------------------+
|         BenchmarkFanal_Library_DockerMode_WithCache/happy_path,_alpine:3.10-8          |  6.61%  |       0.05%       |
+----------------------------------------------------------------------------------------+---------+-------------------+
|   BenchmarkFanal_Library_DockerMode_WithCache/happy_path,_registry.redhat.io/ubi7-8    | 16.14%  |       0.00%       |
+----------------------------------------------------------------------------------------+---------+-------------------+
|       BenchmarkFanal_Library_DockerMode_WithoutCache/happy_path,_debian:buster-8       | 12.18%  |       0.00%       |
+----------------------------------------------------------------------------------------+---------+-------------------+
|        BenchmarkFanal_Library_DockerMode_WithoutCache/happy_path,_photon:1.0-8         | 32.31%  |       0.20%       |
+----------------------------------------------------------------------------------------+---------+-------------------+
|          BenchmarkFanal_Library_DockerMode_WithCache/happy_path,_photon:1.0-8          |  6.00%  |       0.12%       |
+----------------------------------------------------------------------------------------+---------+-------------------+
|        BenchmarkFanal_Library_DockerMode_WithCache/happy_path,_amazonlinux:2-8         |  6.32%  |       0.03%       |
+----------------------------------------------------------------------------------------+---------+-------------------+
|        BenchmarkFanal_Library_DockerMode_WithCache/happy_path,_debian:buster-8         |  9.03%  |       0.07%       |
+----------------------------------------------------------------------------------------+---------+-------------------+
|  BenchmarkFanal_Library_DockerMode_WithCache/happy_path,_vulnimage_with_lock_files-8   |  3.48%  |       0.04%       |
+----------------------------------------------------------------------------------------+---------+-------------------+
|       BenchmarkFanal_Library_DockerMode_WithoutCache/happy_path,_amazonlinux:2-8       | 24.59%  |       0.02%       |
+----------------------------------------------------------------------------------------+---------+-------------------+
|  BenchmarkFanal_Library_DockerMode_WithoutCache/happy_path,_registry.redhat.io/ubi7-8  |  3.75%  |       0.05%       |
+----------------------------------------------------------------------------------------+---------+-------------------+
|    BenchmarkFanal_Library_DockerMode_WithoutCache/happy_path,_opensuse_leap_15.1-8     |  1.15%  |       0.18%       |
+----------------------------------------------------------------------------------------+---------+-------------------+

Few observations:

  1. Should cob issue a non-zero exit code if results are worse? Could be useful for CI.
  2. I think we should attach + and - signs in addition to color output. It's easier to read.
simar7

comment created time in a month

Pull request review commentaquasecurity/fanal

Perf testing with cob

+// +build performance++package integration++import (+	"context"+	"encoding/json"+	"fmt"+	"io/ioutil"+	"os"+	"strings"+	"sync"+	"testing"+	"time"++	_ "github.com/aquasecurity/fanal/analyzer/command/apk"+	_ "github.com/aquasecurity/fanal/analyzer/library/bundler"+	_ "github.com/aquasecurity/fanal/analyzer/library/cargo"+	_ "github.com/aquasecurity/fanal/analyzer/library/composer"+	_ "github.com/aquasecurity/fanal/analyzer/library/npm"+	_ "github.com/aquasecurity/fanal/analyzer/library/pipenv"+	_ "github.com/aquasecurity/fanal/analyzer/library/poetry"+	_ "github.com/aquasecurity/fanal/analyzer/library/yarn"+	_ "github.com/aquasecurity/fanal/analyzer/os/alpine"+	_ "github.com/aquasecurity/fanal/analyzer/os/amazonlinux"+	_ "github.com/aquasecurity/fanal/analyzer/os/debianbase"+	_ "github.com/aquasecurity/fanal/analyzer/os/photon"+	_ "github.com/aquasecurity/fanal/analyzer/os/redhatbase"+	_ "github.com/aquasecurity/fanal/analyzer/os/suse"+	_ "github.com/aquasecurity/fanal/analyzer/pkg/apk"+	_ "github.com/aquasecurity/fanal/analyzer/pkg/dpkg"+	_ "github.com/aquasecurity/fanal/analyzer/pkg/rpm"++	"github.com/aquasecurity/fanal/analyzer"+	"github.com/aquasecurity/fanal/cache"+	"github.com/aquasecurity/fanal/extractor/docker"+	"github.com/aquasecurity/fanal/types"+	godeptypes "github.com/aquasecurity/go-dep-parser/pkg/types"+	dtypes "github.com/docker/docker/api/types"+	"github.com/docker/docker/client"+	"github.com/stretchr/testify/assert"+	"github.com/stretchr/testify/require"+)++type testCase struct {+	name                 string+	imageName            string+	imageFile            string+	expectedFiles        []string+	expectedOS           analyzer.OS+	expectedPkgsFromCmds string+	expectedLibraries    string+}++func runChecksBench(j int, b *testing.B, ac analyzer.Config, ctx context.Context, tc testCase, d string, c cache.Cache) {+	for i := 0; i < b.N; i++ {+		var wg sync.WaitGroup+		for i := 0; i <= j; i++ {+			wg.Add(1)+			go func(i int) {+				defer wg.Done()++				actualFiles, err := ac.Analyze(ctx, tc.imageFile)

I personally feel this is nice to have but is a bit of an overkill for benchmarking. What you're really after here is tracing I feel. Knowing a breakdown by function/dependency is better off being done with APM/Tracing than measuring individual benchmarks as the former is more meaningful.

It would also make our tests excruciatingly long to read and brittle to maintain.

simar7

comment created time in a month

Pull request review commentaquasecurity/fanal

Perf testing with cob

+// +build performance++package integration++import (+	"context"+	"encoding/json"+	"fmt"+	"io/ioutil"+	"os"+	"strings"+	"sync"+	"testing"+	"time"++	_ "github.com/aquasecurity/fanal/analyzer/command/apk"+	_ "github.com/aquasecurity/fanal/analyzer/library/bundler"+	_ "github.com/aquasecurity/fanal/analyzer/library/cargo"+	_ "github.com/aquasecurity/fanal/analyzer/library/composer"+	_ "github.com/aquasecurity/fanal/analyzer/library/npm"+	_ "github.com/aquasecurity/fanal/analyzer/library/pipenv"+	_ "github.com/aquasecurity/fanal/analyzer/library/poetry"+	_ "github.com/aquasecurity/fanal/analyzer/library/yarn"+	_ "github.com/aquasecurity/fanal/analyzer/os/alpine"+	_ "github.com/aquasecurity/fanal/analyzer/os/amazonlinux"+	_ "github.com/aquasecurity/fanal/analyzer/os/debianbase"+	_ "github.com/aquasecurity/fanal/analyzer/os/photon"+	_ "github.com/aquasecurity/fanal/analyzer/os/redhatbase"+	_ "github.com/aquasecurity/fanal/analyzer/os/suse"+	_ "github.com/aquasecurity/fanal/analyzer/pkg/apk"+	_ "github.com/aquasecurity/fanal/analyzer/pkg/dpkg"+	_ "github.com/aquasecurity/fanal/analyzer/pkg/rpm"++	"github.com/aquasecurity/fanal/analyzer"+	"github.com/aquasecurity/fanal/cache"+	"github.com/aquasecurity/fanal/extractor/docker"+	"github.com/aquasecurity/fanal/types"+	godeptypes "github.com/aquasecurity/go-dep-parser/pkg/types"+	dtypes "github.com/docker/docker/api/types"+	"github.com/docker/docker/client"+	"github.com/stretchr/testify/assert"+	"github.com/stretchr/testify/require"+)++type testCase struct {+	name                 string+	imageName            string+	imageFile            string+	expectedFiles        []string+	expectedOS           analyzer.OS+	expectedPkgsFromCmds string+	expectedLibraries    string+}++func runChecksBench(j int, b *testing.B, ac analyzer.Config, ctx context.Context, tc testCase, d string, c cache.Cache) {+	for i := 0; i < b.N; i++ {+		var wg sync.WaitGroup+		for i := 0; i <= j; i++ {+			wg.Add(1)+			go func(i int) {+				defer wg.Done()++				actualFiles, err := ac.Analyze(ctx, tc.imageFile)+				require.NoError(b, err)+				for file, _ := range actualFiles {+					assert.Contains(b, tc.expectedFiles, file, tc.name)+				}+				assert.Equal(b, len(tc.expectedFiles), len(actualFiles), tc.name)++				// check OS+				osFound, err := analyzer.GetOS(actualFiles)+				require.NoError(b, err)+				assert.Equal(b, tc.expectedOS, osFound, tc.name)++				// check Packages+				actualPkgs, err := analyzer.GetPackages(actualFiles)+				require.NoError(b, err)+				data, _ := ioutil.ReadFile(fmt.Sprintf("testdata/goldens/%s.expectedpackages.golden", strings.ReplaceAll(tc.imageName, "/", "")))+				var expectedPkgs []analyzer.Package+				json.Unmarshal(data, &expectedPkgs)+				assert.ElementsMatch(b, expectedPkgs, actualPkgs, tc.name)++				// check Packges from Commands+				actualPkgsFromCmds, err := analyzer.GetPackagesFromCommands(osFound, actualFiles)+				require.NoError(b, err)+				if tc.expectedPkgsFromCmds != "" {+					data, _ := ioutil.ReadFile(tc.expectedPkgsFromCmds)+					var expectedPkgsFromCmds []analyzer.Package+					json.Unmarshal(data, &expectedPkgsFromCmds)+					assert.ElementsMatch(b, expectedPkgsFromCmds, actualPkgsFromCmds, tc.name)+				} else {+					assert.Equal(b, []analyzer.Package(nil), actualPkgsFromCmds, tc.name)+				}++				// check Libraries+				actualLibs, err := analyzer.GetLibraries(actualFiles)+				data, _ = json.MarshalIndent(actualLibs, "", "  ")+				require.NoError(b, err)+				if tc.expectedLibraries != "" {+					data, _ := ioutil.ReadFile(tc.expectedLibraries)+					var expectedLibraries map[analyzer.FilePath][]godeptypes.Library+					json.Unmarshal(data, &expectedLibraries)+					require.Equal(b, len(expectedLibraries), len(actualLibs), tc.name)+					for l := range expectedLibraries {+						assert.Contains(b, actualLibs, l, tc.name)+					}+				} else {+					assert.Equal(b, map[analyzer.FilePath][]godeptypes.Library{}, actualLibs, tc.name)+				}++				// check Cache+				actualCachedFiles, _ := ioutil.ReadDir(d + "/fanal/")+				require.Equal(b, 1, len(actualCachedFiles), tc.name)++				// check Cache contents+				r := c.Get(tc.imageFile)+				actualCacheValue, err := ioutil.ReadAll(r)+				require.NoError(b, err)+				assert.NotEmpty(b, actualCacheValue, tc.name)+			}(i)+		}+		wg.Wait()+	}+}++func BenchmarkFanal_Library_DockerMode_10(b *testing.B) {+	testCases := []testCase{+		{+			name:          "happy path, alpine:3.10",+			imageName:     "alpine:3.10",+			imageFile:     "testdata/fixtures/alpine-310.tar.gz",+			expectedOS:    analyzer.OS{Name: "3.10.2", Family: "alpine"},+			expectedFiles: []string{"etc/alpine-release", "etc/os-release", "lib/apk/db/installed", "/config"},+		},+		{+			name:          "happy path, amazonlinux:2",+			imageName:     "amazonlinux:2",+			imageFile:     "testdata/fixtures/amazon-2.tar.gz",+			expectedFiles: []string{"etc/system-release", "var/lib/rpm/Packages", "etc/os-release", "/config"},+			expectedOS:    analyzer.OS{Name: "2 (Karoo)", Family: "amazon"},+		},+		{+			name:          "happy path, debian:buster",+			imageName:     "debian:buster",+			imageFile:     "testdata/fixtures/debian-buster.tar.gz",+			expectedFiles: []string{"var/lib/dpkg/status", "etc/debian_version", "etc/os-release", "usr/lib/os-release", "/config"},+			expectedOS:    analyzer.OS{Name: "10.1", Family: "debian"},+		},+		{+			name:          "happy path, photon:1.0",+			imageName:     "photon:1.0-20190823",+			imageFile:     "testdata/fixtures/photon-10.tar.gz",+			expectedFiles: []string{"var/lib/rpm/Packages", "etc/lsb-release", "etc/os-release", "/config", "usr/lib/os-release"},+			expectedOS:    analyzer.OS{Name: "1.0", Family: "photon"},+		},+		{+			name:          "happy path, registry.redhat.io/ubi7",+			imageName:     "registry.redhat.io/ubi7",+			imageFile:     "testdata/fixtures/ubi-7.tar.gz",+			expectedFiles: []string{"etc/redhat-release", "etc/system-release", "/config", "var/lib/rpm/Packages", "etc/os-release"},+			expectedOS:    analyzer.OS{Name: "7.7", Family: "redhat"},+		},+		{+			name:          "happy path, opensuse leap 15.1",+			imageName:     "opensuse/leap:latest",+			imageFile:     "testdata/fixtures/opensuse-leap-151.tar.gz",+			expectedFiles: []string{"usr/lib/os-release", "usr/lib/sysimage/rpm/Packages", "/config", "etc/os-release"},+			expectedOS:    analyzer.OS{Name: "15.1", Family: "opensuse.leap"},+		},+		{+			name:                 "happy path, vulnimage with lock files",+			imageName:            "knqyf263/vuln-image:1.2.3",+			imageFile:            "testdata/fixtures/vulnimage.tar.gz",+			expectedFiles:        []string{"etc/os-release", "node-app/package-lock.json", "python-app/Pipfile.lock", "ruby-app/Gemfile.lock", "rust-app/Cargo.lock", "/config", "etc/alpine-release", "lib/apk/db/installed", "php-app/composer.lock"},+			expectedOS:           analyzer.OS{Name: "3.7.1", Family: "alpine"},+			expectedLibraries:    "testdata/goldens/knqyf263vuln-image:1.2.3.expectedlibs.golden",+			expectedPkgsFromCmds: "testdata/goldens/knqyf263vuln-image:1.2.3.expectedpkgsfromcmds.golden",+		},+	}++	for _, tc := range testCases {+		tc := tc // save a copy of tc for use in t.Run https://gist.github.com/posener/92a55c4cd441fc5e5e85f27bca008721+		b.Run(tc.name, func(b *testing.B) {+			//b.Parallel()+			ctx := context.Background()+			d, _ := ioutil.TempDir("", "TestFanal_Library_*")+			defer os.RemoveAll(d)+			c := cache.Initialize(d)++			opt := types.DockerOption{+				Timeout:  600 * time.Second,+				SkipPing: true,+			}++			cli, err := client.NewClientWithOpts(client.FromEnv)+			require.NoError(b, err, tc.name)++			// ensure image doesnt already exists+			_, _ = cli.ImageRemove(ctx, tc.imageFile, dtypes.ImageRemoveOptions{+				Force:         true,+				PruneChildren: true,+			})++			testfile, err := os.Open(tc.imageFile)+			require.NoError(b, err)++			// load image into docker engine+			_, err = cli.ImageLoad(ctx, testfile, true)+			require.NoError(b, err, tc.name)++			// tag our image to something unique+			err = cli.ImageTag(ctx, tc.imageName, tc.imageFile)+			require.NoError(b, err, tc.name)++			ext, err := docker.NewDockerExtractor(opt, c)+			require.NoError(b, err, tc.name)+			ac := analyzer.Config{Extractor: ext}++			b.ReportAllocs()+			b.ResetTimer()+			// run tests twice, one without cache and with cache+			for i := 1; i <= 2; i++ {+				runChecksBench(5, b, ac, ctx, tc, d, c)++			}

There was also a bug in this benchmark code that I found, I think these results are much more meaningful:

BenchmarkFanal_Library_DockerMode_WithoutCache/happy_path,_alpine:3.10-8                      19          58401294 ns/op        11952605 B/op     156570 allocs/op
BenchmarkFanal_Library_DockerMode_WithoutCache/happy_path,_amazonlinux:2-8                     1        3391656811 ns/op        36893008 B/op     566862 allocs/op
BenchmarkFanal_Library_DockerMode_WithoutCache/happy_path,_debian:buster-8                     1        3209189846 ns/op         7467096 B/op     228334 allocs/op
BenchmarkFanal_Library_DockerMode_WithoutCache/happy_path,_photon:1.0-8                        1        3694983184 ns/op        16589568 B/op     397234 allocs/op
BenchmarkFanal_Library_DockerMode_WithoutCache/happy_path,_registry.redhat.io/ubi7-8                   1        6131400243 ns/op        65793568 B/op     831039 allocs/op
BenchmarkFanal_Library_DockerMode_WithoutCache/happy_path,_opensuse_leap_15.1-8                        1        2519825637 ns/op        23268648 B/op     453900 allocs/op
BenchmarkFanal_Library_DockerMode_WithoutCache/happy_path,_vulnimage_with_lock_files-8                 1        4892505695 ns/op        24762120 B/op     414372 allocs/op
BenchmarkFanal_Library_DockerMode_WithCache/happy_path,_alpine:3.10-8                                 18          59800578 ns/op        11953707 B/op     156572 allocs/op
BenchmarkFanal_Library_DockerMode_WithCache/happy_path,_amazonlinux:2-8                                8         132578903 ns/op        36849505 B/op     566711 allocs/op
BenchmarkFanal_Library_DockerMode_WithCache/happy_path,_debian:buster-8                               13          83306147 ns/op         7420395 B/op     228188 allocs/op
BenchmarkFanal_Library_DockerMode_WithCache/happy_path,_photon:1.0-8                                  10         106737022 ns/op        16565464 B/op     397098 allocs/op
BenchmarkFanal_Library_DockerMode_WithCache/happy_path,_registry.redhat.io/ubi7-8                      6         170368824 ns/op        65745706 B/op     830901 allocs/op
BenchmarkFanal_Library_DockerMode_WithCache/happy_path,_opensuse_leap_15.1-8                          16          71487653 ns/op        23241808 B/op     453769 allocs/op
BenchmarkFanal_Library_DockerMode_WithCache/happy_path,_vulnimage_with_lock_files-8                    7         155990412 ns/op        24704204 B/op     413966 allocs/op

simar7

comment created time in a month

push eventaquasecurity/fanal

Simarpreet Singh

commit sha 9268652494b37ddd6683edea441c91d1bca808d3

benchmark_test: Split with and without cache Signed-off-by: Simarpreet Singh <simar@linux.com>

view details

push time in a month

Pull request review commentaquasecurity/fanal

Perf testing with cob

+// +build performance++package integration++import (+	"context"+	"encoding/json"+	"fmt"+	"io/ioutil"+	"os"+	"strings"+	"sync"+	"testing"+	"time"++	_ "github.com/aquasecurity/fanal/analyzer/command/apk"+	_ "github.com/aquasecurity/fanal/analyzer/library/bundler"+	_ "github.com/aquasecurity/fanal/analyzer/library/cargo"+	_ "github.com/aquasecurity/fanal/analyzer/library/composer"+	_ "github.com/aquasecurity/fanal/analyzer/library/npm"+	_ "github.com/aquasecurity/fanal/analyzer/library/pipenv"+	_ "github.com/aquasecurity/fanal/analyzer/library/poetry"+	_ "github.com/aquasecurity/fanal/analyzer/library/yarn"+	_ "github.com/aquasecurity/fanal/analyzer/os/alpine"+	_ "github.com/aquasecurity/fanal/analyzer/os/amazonlinux"+	_ "github.com/aquasecurity/fanal/analyzer/os/debianbase"+	_ "github.com/aquasecurity/fanal/analyzer/os/photon"+	_ "github.com/aquasecurity/fanal/analyzer/os/redhatbase"+	_ "github.com/aquasecurity/fanal/analyzer/os/suse"+	_ "github.com/aquasecurity/fanal/analyzer/pkg/apk"+	_ "github.com/aquasecurity/fanal/analyzer/pkg/dpkg"+	_ "github.com/aquasecurity/fanal/analyzer/pkg/rpm"++	"github.com/aquasecurity/fanal/analyzer"+	"github.com/aquasecurity/fanal/cache"+	"github.com/aquasecurity/fanal/extractor/docker"+	"github.com/aquasecurity/fanal/types"+	godeptypes "github.com/aquasecurity/go-dep-parser/pkg/types"+	dtypes "github.com/docker/docker/api/types"+	"github.com/docker/docker/client"+	"github.com/stretchr/testify/assert"+	"github.com/stretchr/testify/require"+)++type testCase struct {+	name                 string+	imageName            string+	imageFile            string+	expectedFiles        []string+	expectedOS           analyzer.OS+	expectedPkgsFromCmds string+	expectedLibraries    string+}++func runChecksBench(j int, b *testing.B, ac analyzer.Config, ctx context.Context, tc testCase, d string, c cache.Cache) {+	for i := 0; i < b.N; i++ {+		var wg sync.WaitGroup+		for i := 0; i <= j; i++ {+			wg.Add(1)+			go func(i int) {+				defer wg.Done()++				actualFiles, err := ac.Analyze(ctx, tc.imageFile)+				require.NoError(b, err)+				for file, _ := range actualFiles {+					assert.Contains(b, tc.expectedFiles, file, tc.name)+				}+				assert.Equal(b, len(tc.expectedFiles), len(actualFiles), tc.name)++				// check OS+				osFound, err := analyzer.GetOS(actualFiles)+				require.NoError(b, err)+				assert.Equal(b, tc.expectedOS, osFound, tc.name)++				// check Packages+				actualPkgs, err := analyzer.GetPackages(actualFiles)+				require.NoError(b, err)+				data, _ := ioutil.ReadFile(fmt.Sprintf("testdata/goldens/%s.expectedpackages.golden", strings.ReplaceAll(tc.imageName, "/", "")))+				var expectedPkgs []analyzer.Package+				json.Unmarshal(data, &expectedPkgs)+				assert.ElementsMatch(b, expectedPkgs, actualPkgs, tc.name)++				// check Packges from Commands+				actualPkgsFromCmds, err := analyzer.GetPackagesFromCommands(osFound, actualFiles)+				require.NoError(b, err)+				if tc.expectedPkgsFromCmds != "" {+					data, _ := ioutil.ReadFile(tc.expectedPkgsFromCmds)+					var expectedPkgsFromCmds []analyzer.Package+					json.Unmarshal(data, &expectedPkgsFromCmds)+					assert.ElementsMatch(b, expectedPkgsFromCmds, actualPkgsFromCmds, tc.name)+				} else {+					assert.Equal(b, []analyzer.Package(nil), actualPkgsFromCmds, tc.name)+				}++				// check Libraries+				actualLibs, err := analyzer.GetLibraries(actualFiles)+				data, _ = json.MarshalIndent(actualLibs, "", "  ")+				require.NoError(b, err)+				if tc.expectedLibraries != "" {+					data, _ := ioutil.ReadFile(tc.expectedLibraries)+					var expectedLibraries map[analyzer.FilePath][]godeptypes.Library+					json.Unmarshal(data, &expectedLibraries)+					require.Equal(b, len(expectedLibraries), len(actualLibs), tc.name)+					for l := range expectedLibraries {+						assert.Contains(b, actualLibs, l, tc.name)+					}+				} else {+					assert.Equal(b, map[analyzer.FilePath][]godeptypes.Library{}, actualLibs, tc.name)+				}++				// check Cache+				actualCachedFiles, _ := ioutil.ReadDir(d + "/fanal/")+				require.Equal(b, 1, len(actualCachedFiles), tc.name)++				// check Cache contents+				r := c.Get(tc.imageFile)+				actualCacheValue, err := ioutil.ReadAll(r)+				require.NoError(b, err)+				assert.NotEmpty(b, actualCacheValue, tc.name)+			}(i)+		}+		wg.Wait()+	}+}++func BenchmarkFanal_Library_DockerMode_10(b *testing.B) {+	testCases := []testCase{+		{+			name:          "happy path, alpine:3.10",+			imageName:     "alpine:3.10",+			imageFile:     "testdata/fixtures/alpine-310.tar.gz",+			expectedOS:    analyzer.OS{Name: "3.10.2", Family: "alpine"},+			expectedFiles: []string{"etc/alpine-release", "etc/os-release", "lib/apk/db/installed", "/config"},+		},+		{+			name:          "happy path, amazonlinux:2",+			imageName:     "amazonlinux:2",+			imageFile:     "testdata/fixtures/amazon-2.tar.gz",+			expectedFiles: []string{"etc/system-release", "var/lib/rpm/Packages", "etc/os-release", "/config"},+			expectedOS:    analyzer.OS{Name: "2 (Karoo)", Family: "amazon"},+		},+		{+			name:          "happy path, debian:buster",+			imageName:     "debian:buster",+			imageFile:     "testdata/fixtures/debian-buster.tar.gz",+			expectedFiles: []string{"var/lib/dpkg/status", "etc/debian_version", "etc/os-release", "usr/lib/os-release", "/config"},+			expectedOS:    analyzer.OS{Name: "10.1", Family: "debian"},+		},+		{+			name:          "happy path, photon:1.0",+			imageName:     "photon:1.0-20190823",+			imageFile:     "testdata/fixtures/photon-10.tar.gz",+			expectedFiles: []string{"var/lib/rpm/Packages", "etc/lsb-release", "etc/os-release", "/config", "usr/lib/os-release"},+			expectedOS:    analyzer.OS{Name: "1.0", Family: "photon"},+		},+		{+			name:          "happy path, registry.redhat.io/ubi7",+			imageName:     "registry.redhat.io/ubi7",+			imageFile:     "testdata/fixtures/ubi-7.tar.gz",+			expectedFiles: []string{"etc/redhat-release", "etc/system-release", "/config", "var/lib/rpm/Packages", "etc/os-release"},+			expectedOS:    analyzer.OS{Name: "7.7", Family: "redhat"},+		},+		{+			name:          "happy path, opensuse leap 15.1",+			imageName:     "opensuse/leap:latest",+			imageFile:     "testdata/fixtures/opensuse-leap-151.tar.gz",+			expectedFiles: []string{"usr/lib/os-release", "usr/lib/sysimage/rpm/Packages", "/config", "etc/os-release"},+			expectedOS:    analyzer.OS{Name: "15.1", Family: "opensuse.leap"},+		},+		{+			name:                 "happy path, vulnimage with lock files",+			imageName:            "knqyf263/vuln-image:1.2.3",+			imageFile:            "testdata/fixtures/vulnimage.tar.gz",+			expectedFiles:        []string{"etc/os-release", "node-app/package-lock.json", "python-app/Pipfile.lock", "ruby-app/Gemfile.lock", "rust-app/Cargo.lock", "/config", "etc/alpine-release", "lib/apk/db/installed", "php-app/composer.lock"},+			expectedOS:           analyzer.OS{Name: "3.7.1", Family: "alpine"},+			expectedLibraries:    "testdata/goldens/knqyf263vuln-image:1.2.3.expectedlibs.golden",+			expectedPkgsFromCmds: "testdata/goldens/knqyf263vuln-image:1.2.3.expectedpkgsfromcmds.golden",+		},+	}++	for _, tc := range testCases {+		tc := tc // save a copy of tc for use in t.Run https://gist.github.com/posener/92a55c4cd441fc5e5e85f27bca008721+		b.Run(tc.name, func(b *testing.B) {+			//b.Parallel()+			ctx := context.Background()+			d, _ := ioutil.TempDir("", "TestFanal_Library_*")+			defer os.RemoveAll(d)+			c := cache.Initialize(d)++			opt := types.DockerOption{+				Timeout:  600 * time.Second,+				SkipPing: true,+			}++			cli, err := client.NewClientWithOpts(client.FromEnv)+			require.NoError(b, err, tc.name)++			// ensure image doesnt already exists+			_, _ = cli.ImageRemove(ctx, tc.imageFile, dtypes.ImageRemoveOptions{+				Force:         true,+				PruneChildren: true,+			})++			testfile, err := os.Open(tc.imageFile)+			require.NoError(b, err)++			// load image into docker engine+			_, err = cli.ImageLoad(ctx, testfile, true)+			require.NoError(b, err, tc.name)++			// tag our image to something unique+			err = cli.ImageTag(ctx, tc.imageName, tc.imageFile)+			require.NoError(b, err, tc.name)++			ext, err := docker.NewDockerExtractor(opt, c)+			require.NoError(b, err, tc.name)+			ac := analyzer.Config{Extractor: ext}++			b.ReportAllocs()+			b.ResetTimer()+			// run tests twice, one without cache and with cache+			for i := 1; i <= 2; i++ {+				runChecksBench(5, b, ac, ctx, tc, d, c)++			}

Yes I had asked similar questions here https://github.com/aquasecurity/fanal/issues/69 I wasn't sure what direction we wanted to head in so I've left it open for comments.

simar7

comment created time in a month

push eventaquasecurity/fanal

Simarpreet Singh

commit sha d1991fb25af906e39531579c5970143df4611f17

benchmark_test: Remove assertions and goroutines Signed-off-by: Simarpreet Singh <simar@linux.com>

view details

push time in a month

Pull request review commentaquasecurity/fanal

Perf testing with cob

+// +build performance++package integration++import (+	"context"+	"encoding/json"+	"fmt"+	"io/ioutil"+	"os"+	"strings"+	"sync"+	"testing"+	"time"++	_ "github.com/aquasecurity/fanal/analyzer/command/apk"+	_ "github.com/aquasecurity/fanal/analyzer/library/bundler"+	_ "github.com/aquasecurity/fanal/analyzer/library/cargo"+	_ "github.com/aquasecurity/fanal/analyzer/library/composer"+	_ "github.com/aquasecurity/fanal/analyzer/library/npm"+	_ "github.com/aquasecurity/fanal/analyzer/library/pipenv"+	_ "github.com/aquasecurity/fanal/analyzer/library/poetry"+	_ "github.com/aquasecurity/fanal/analyzer/library/yarn"+	_ "github.com/aquasecurity/fanal/analyzer/os/alpine"+	_ "github.com/aquasecurity/fanal/analyzer/os/amazonlinux"+	_ "github.com/aquasecurity/fanal/analyzer/os/debianbase"+	_ "github.com/aquasecurity/fanal/analyzer/os/photon"+	_ "github.com/aquasecurity/fanal/analyzer/os/redhatbase"+	_ "github.com/aquasecurity/fanal/analyzer/os/suse"+	_ "github.com/aquasecurity/fanal/analyzer/pkg/apk"+	_ "github.com/aquasecurity/fanal/analyzer/pkg/dpkg"+	_ "github.com/aquasecurity/fanal/analyzer/pkg/rpm"++	"github.com/aquasecurity/fanal/analyzer"+	"github.com/aquasecurity/fanal/cache"+	"github.com/aquasecurity/fanal/extractor/docker"+	"github.com/aquasecurity/fanal/types"+	godeptypes "github.com/aquasecurity/go-dep-parser/pkg/types"+	dtypes "github.com/docker/docker/api/types"+	"github.com/docker/docker/client"+	"github.com/stretchr/testify/assert"+	"github.com/stretchr/testify/require"+)++type testCase struct {+	name                 string+	imageName            string+	imageFile            string+	expectedFiles        []string+	expectedOS           analyzer.OS+	expectedPkgsFromCmds string+	expectedLibraries    string+}++func runChecksBench(j int, b *testing.B, ac analyzer.Config, ctx context.Context, tc testCase, d string, c cache.Cache) {+	for i := 0; i < b.N; i++ {+		var wg sync.WaitGroup+		for i := 0; i <= j; i++ {+			wg.Add(1)+			go func(i int) {

I had an idea to increase the sample size by running several tests in each goroutine. I realized this might not work as I intend it to be after reading about b.N and -benchtime. I will remove it.

simar7

comment created time in a month

Pull request review commentaquasecurity/fanal

Perf testing with cob

+// +build performance++package integration++import (+	"context"+	"encoding/json"+	"fmt"+	"io/ioutil"+	"os"+	"strings"+	"sync"+	"testing"+	"time"++	_ "github.com/aquasecurity/fanal/analyzer/command/apk"+	_ "github.com/aquasecurity/fanal/analyzer/library/bundler"+	_ "github.com/aquasecurity/fanal/analyzer/library/cargo"+	_ "github.com/aquasecurity/fanal/analyzer/library/composer"+	_ "github.com/aquasecurity/fanal/analyzer/library/npm"+	_ "github.com/aquasecurity/fanal/analyzer/library/pipenv"+	_ "github.com/aquasecurity/fanal/analyzer/library/poetry"+	_ "github.com/aquasecurity/fanal/analyzer/library/yarn"+	_ "github.com/aquasecurity/fanal/analyzer/os/alpine"+	_ "github.com/aquasecurity/fanal/analyzer/os/amazonlinux"+	_ "github.com/aquasecurity/fanal/analyzer/os/debianbase"+	_ "github.com/aquasecurity/fanal/analyzer/os/photon"+	_ "github.com/aquasecurity/fanal/analyzer/os/redhatbase"+	_ "github.com/aquasecurity/fanal/analyzer/os/suse"+	_ "github.com/aquasecurity/fanal/analyzer/pkg/apk"+	_ "github.com/aquasecurity/fanal/analyzer/pkg/dpkg"+	_ "github.com/aquasecurity/fanal/analyzer/pkg/rpm"++	"github.com/aquasecurity/fanal/analyzer"+	"github.com/aquasecurity/fanal/cache"+	"github.com/aquasecurity/fanal/extractor/docker"+	"github.com/aquasecurity/fanal/types"+	godeptypes "github.com/aquasecurity/go-dep-parser/pkg/types"+	dtypes "github.com/docker/docker/api/types"+	"github.com/docker/docker/client"+	"github.com/stretchr/testify/assert"+	"github.com/stretchr/testify/require"+)++type testCase struct {+	name                 string+	imageName            string+	imageFile            string+	expectedFiles        []string+	expectedOS           analyzer.OS+	expectedPkgsFromCmds string+	expectedLibraries    string+}++func runChecksBench(j int, b *testing.B, ac analyzer.Config, ctx context.Context, tc testCase, d string, c cache.Cache) {+	for i := 0; i < b.N; i++ {+		var wg sync.WaitGroup+		for i := 0; i <= j; i++ {+			wg.Add(1)+			go func(i int) {+				defer wg.Done()++				actualFiles, err := ac.Analyze(ctx, tc.imageFile)+				require.NoError(b, err)+				for file, _ := range actualFiles {+					assert.Contains(b, tc.expectedFiles, file, tc.name)+				}+				assert.Equal(b, len(tc.expectedFiles), len(actualFiles), tc.name)

Yeah I'm not sure about the overhead but certainly there must be some. I can take them off.

simar7

comment created time in a month

Pull request review commentaquasecurity/fanal

integration: Fanal as a library for tar mode

+// +build integration++package integration++import (+	"context"+	"io/ioutil"+	"os"+	"testing"+	"time"++	"github.com/aquasecurity/fanal/analyzer"+	"github.com/aquasecurity/fanal/extractor/docker"+	dtypes "github.com/docker/docker/api/types"+	"github.com/docker/docker/client"+	"github.com/stretchr/testify/assert"+	"github.com/stretchr/testify/require"++	_ "github.com/aquasecurity/fanal/analyzer/command/apk"+	_ "github.com/aquasecurity/fanal/analyzer/library/bundler"+	_ "github.com/aquasecurity/fanal/analyzer/library/cargo"+	_ "github.com/aquasecurity/fanal/analyzer/library/composer"+	_ "github.com/aquasecurity/fanal/analyzer/library/npm"+	_ "github.com/aquasecurity/fanal/analyzer/library/pipenv"+	_ "github.com/aquasecurity/fanal/analyzer/library/poetry"+	_ "github.com/aquasecurity/fanal/analyzer/library/yarn"+	_ "github.com/aquasecurity/fanal/analyzer/os/alpine"+	_ "github.com/aquasecurity/fanal/analyzer/os/amazonlinux"+	_ "github.com/aquasecurity/fanal/analyzer/os/debianbase"+	_ "github.com/aquasecurity/fanal/analyzer/os/photon"+	_ "github.com/aquasecurity/fanal/analyzer/os/redhatbase"+	_ "github.com/aquasecurity/fanal/analyzer/os/suse"+	_ "github.com/aquasecurity/fanal/analyzer/pkg/apk"+	_ "github.com/aquasecurity/fanal/analyzer/pkg/dpkg"+	_ "github.com/aquasecurity/fanal/analyzer/pkg/rpm"++	"github.com/aquasecurity/fanal/cache"+	"github.com/aquasecurity/fanal/types"+)++type testCase struct {+	name                 string+	imageName            string+	imageFile            string+	expectedFiles        []string+	expectedOS           analyzer.OS+	expectedPkgsFromCmds string+	expectedLibraries    string+}++func TestFanal_Library_DockerMode(t *testing.T) {+	testCases := []testCase{+		{+			name:          "happy path, alpine:3.10",+			imageName:     "alpine:3.10",+			imageFile:     "testdata/fixtures/alpine-310.tar.gz",+			expectedOS:    analyzer.OS{Name: "3.10.2", Family: "alpine"},+			expectedFiles: []string{"etc/alpine-release", "etc/os-release", "lib/apk/db/installed", "/config"},+		},+		{+			name:          "happy path, amazonlinux:2",+			imageName:     "amazonlinux:2",+			imageFile:     "testdata/fixtures/amazon-2.tar.gz",+			expectedFiles: []string{"etc/system-release", "var/lib/rpm/Packages", "etc/os-release", "/config"},+			expectedOS:    analyzer.OS{Name: "2 (Karoo)", Family: "amazon"},+		},+		{+			name:          "happy path, debian:buster",+			imageName:     "debian:buster",+			imageFile:     "testdata/fixtures/debian-buster.tar.gz",+			expectedFiles: []string{"var/lib/dpkg/status", "etc/debian_version", "etc/os-release", "usr/lib/os-release", "/config"},+			expectedOS:    analyzer.OS{Name: "10.1", Family: "debian"},+		},+		{+			name:          "happy path, photon:1.0",+			imageName:     "photon:1.0-20190823",+			imageFile:     "testdata/fixtures/photon-10.tar.gz",+			expectedFiles: []string{"var/lib/rpm/Packages", "etc/lsb-release", "etc/os-release", "/config", "usr/lib/os-release"},+			expectedOS:    analyzer.OS{Name: "1.0", Family: "photon"},+		},+		{+			name:          "happy path, registry.redhat.io/ubi7",+			imageName:     "registry.redhat.io/ubi7",+			imageFile:     "testdata/fixtures/ubi-7.tar.gz",+			expectedFiles: []string{"etc/redhat-release", "etc/system-release", "/config", "var/lib/rpm/Packages", "etc/os-release"},+			expectedOS:    analyzer.OS{Name: "7.7", Family: "redhat"},+		},+		{+			name:          "happy path, opensuse leap 15.1",+			imageName:     "opensuse/leap:latest",+			imageFile:     "testdata/fixtures/opensuse-leap-151.tar.gz",+			expectedFiles: []string{"usr/lib/os-release", "usr/lib/sysimage/rpm/Packages", "/config", "etc/os-release"},+			expectedOS:    analyzer.OS{Name: "15.1", Family: "opensuse.leap"},+		},+		{+			name:                 "happy path, vulnimage with lock files",+			imageName:            "knqyf263/vuln-image:1.2.3",+			imageFile:            "testdata/fixtures/vulnimage.tar.gz",+			expectedFiles:        []string{"etc/os-release", "node-app/package-lock.json", "python-app/Pipfile.lock", "ruby-app/Gemfile.lock", "rust-app/Cargo.lock", "/config", "etc/alpine-release", "lib/apk/db/installed", "php-app/composer.lock"},+			expectedOS:           analyzer.OS{Name: "3.7.1", Family: "alpine"},+			expectedLibraries:    "testdata/goldens/knqyf263vuln-image:1.2.3.expectedlibs.golden",+			expectedPkgsFromCmds: "testdata/goldens/knqyf263vuln-image:1.2.3.expectedpkgsfromcmds.golden",+		},+	}++	for _, tc := range testCases {+		tc := tc // save a copy of tc for use in t.Run https://gist.github.com/posener/92a55c4cd441fc5e5e85f27bca008721+		t.Run(tc.name, func(t *testing.T) {+			t.Parallel()+			ctx := context.Background()+			d, _ := ioutil.TempDir("", "TestFanal_Library_*")+			defer os.RemoveAll(d)+			c := cache.Initialize(d)++			opt := types.DockerOption{+				Timeout:  600 * time.Second,+				SkipPing: true,+			}++			cli, err := client.NewClientWithOpts(client.FromEnv)+			require.NoError(t, err, tc.name)++			// ensure image doesnt already exists+			_, _ = cli.ImageRemove(ctx, tc.imageFile, dtypes.ImageRemoveOptions{+				Force:         true,+				PruneChildren: true,+			})++			testfile, err := os.Open(tc.imageFile)+			require.NoError(t, err)++			// load image into docker engine+			_, err = cli.ImageLoad(ctx, testfile, true)+			require.NoError(t, err, tc.name)++			// tag our image to something unique+			err = cli.ImageTag(ctx, tc.imageName, tc.imageFile)+			require.NoError(t, err, tc.name)++			ext, err := docker.NewDockerExtractor(opt, c)+			require.NoError(t, err, tc.name)+			ac := analyzer.Config{Extractor: ext}++			// run tests twice, one without cache and with cache+			for i := 1; i <= 2; i++ {+				runChecksDockerMode(t, ac, ctx, tc, d, c)+			}++			// clear Cache+			require.NoError(t, c.Clear(), tc.name)++			// remove Image+			_, err = cli.ImageRemove(ctx, tc.imageFile, dtypes.ImageRemoveOptions{+				Force:         true,+				PruneChildren: true,+			})+			assert.NoError(t, err, tc.name)+		})+	}+}++func runChecksDockerMode(t *testing.T, ac analyzer.Config, ctx context.Context, tc testCase, d string, c cache.Cache) {+	actualFiles, err := ac.Analyze(ctx, tc.imageFile)

Also this way we can run docker mode and tar mode in parallel.

simar7

comment created time in a month

Pull request review commentaquasecurity/fanal

integration: Fanal as a library for tar mode

+// +build integration++package integration++import (+	"context"+	"io/ioutil"+	"os"+	"testing"+	"time"++	"github.com/aquasecurity/fanal/analyzer"+	"github.com/aquasecurity/fanal/extractor/docker"+	dtypes "github.com/docker/docker/api/types"+	"github.com/docker/docker/client"+	"github.com/stretchr/testify/assert"+	"github.com/stretchr/testify/require"++	_ "github.com/aquasecurity/fanal/analyzer/command/apk"+	_ "github.com/aquasecurity/fanal/analyzer/library/bundler"+	_ "github.com/aquasecurity/fanal/analyzer/library/cargo"+	_ "github.com/aquasecurity/fanal/analyzer/library/composer"+	_ "github.com/aquasecurity/fanal/analyzer/library/npm"+	_ "github.com/aquasecurity/fanal/analyzer/library/pipenv"+	_ "github.com/aquasecurity/fanal/analyzer/library/poetry"+	_ "github.com/aquasecurity/fanal/analyzer/library/yarn"+	_ "github.com/aquasecurity/fanal/analyzer/os/alpine"+	_ "github.com/aquasecurity/fanal/analyzer/os/amazonlinux"+	_ "github.com/aquasecurity/fanal/analyzer/os/debianbase"+	_ "github.com/aquasecurity/fanal/analyzer/os/photon"+	_ "github.com/aquasecurity/fanal/analyzer/os/redhatbase"+	_ "github.com/aquasecurity/fanal/analyzer/os/suse"+	_ "github.com/aquasecurity/fanal/analyzer/pkg/apk"+	_ "github.com/aquasecurity/fanal/analyzer/pkg/dpkg"+	_ "github.com/aquasecurity/fanal/analyzer/pkg/rpm"++	"github.com/aquasecurity/fanal/cache"+	"github.com/aquasecurity/fanal/types"+)++type testCase struct {+	name                 string+	imageName            string+	imageFile            string+	expectedFiles        []string+	expectedOS           analyzer.OS+	expectedPkgsFromCmds string+	expectedLibraries    string+}++func TestFanal_Library_DockerMode(t *testing.T) {+	testCases := []testCase{+		{+			name:          "happy path, alpine:3.10",+			imageName:     "alpine:3.10",+			imageFile:     "testdata/fixtures/alpine-310.tar.gz",+			expectedOS:    analyzer.OS{Name: "3.10.2", Family: "alpine"},+			expectedFiles: []string{"etc/alpine-release", "etc/os-release", "lib/apk/db/installed", "/config"},+		},+		{+			name:          "happy path, amazonlinux:2",+			imageName:     "amazonlinux:2",+			imageFile:     "testdata/fixtures/amazon-2.tar.gz",+			expectedFiles: []string{"etc/system-release", "var/lib/rpm/Packages", "etc/os-release", "/config"},+			expectedOS:    analyzer.OS{Name: "2 (Karoo)", Family: "amazon"},+		},+		{+			name:          "happy path, debian:buster",+			imageName:     "debian:buster",+			imageFile:     "testdata/fixtures/debian-buster.tar.gz",+			expectedFiles: []string{"var/lib/dpkg/status", "etc/debian_version", "etc/os-release", "usr/lib/os-release", "/config"},+			expectedOS:    analyzer.OS{Name: "10.1", Family: "debian"},+		},+		{+			name:          "happy path, photon:1.0",+			imageName:     "photon:1.0-20190823",+			imageFile:     "testdata/fixtures/photon-10.tar.gz",+			expectedFiles: []string{"var/lib/rpm/Packages", "etc/lsb-release", "etc/os-release", "/config", "usr/lib/os-release"},+			expectedOS:    analyzer.OS{Name: "1.0", Family: "photon"},+		},+		{+			name:          "happy path, registry.redhat.io/ubi7",+			imageName:     "registry.redhat.io/ubi7",+			imageFile:     "testdata/fixtures/ubi-7.tar.gz",+			expectedFiles: []string{"etc/redhat-release", "etc/system-release", "/config", "var/lib/rpm/Packages", "etc/os-release"},+			expectedOS:    analyzer.OS{Name: "7.7", Family: "redhat"},+		},+		{+			name:          "happy path, opensuse leap 15.1",+			imageName:     "opensuse/leap:latest",+			imageFile:     "testdata/fixtures/opensuse-leap-151.tar.gz",+			expectedFiles: []string{"usr/lib/os-release", "usr/lib/sysimage/rpm/Packages", "/config", "etc/os-release"},+			expectedOS:    analyzer.OS{Name: "15.1", Family: "opensuse.leap"},+		},+		{+			name:                 "happy path, vulnimage with lock files",+			imageName:            "knqyf263/vuln-image:1.2.3",+			imageFile:            "testdata/fixtures/vulnimage.tar.gz",+			expectedFiles:        []string{"etc/os-release", "node-app/package-lock.json", "python-app/Pipfile.lock", "ruby-app/Gemfile.lock", "rust-app/Cargo.lock", "/config", "etc/alpine-release", "lib/apk/db/installed", "php-app/composer.lock"},+			expectedOS:           analyzer.OS{Name: "3.7.1", Family: "alpine"},+			expectedLibraries:    "testdata/goldens/knqyf263vuln-image:1.2.3.expectedlibs.golden",+			expectedPkgsFromCmds: "testdata/goldens/knqyf263vuln-image:1.2.3.expectedpkgsfromcmds.golden",+		},+	}++	for _, tc := range testCases {+		tc := tc // save a copy of tc for use in t.Run https://gist.github.com/posener/92a55c4cd441fc5e5e85f27bca008721+		t.Run(tc.name, func(t *testing.T) {+			t.Parallel()+			ctx := context.Background()+			d, _ := ioutil.TempDir("", "TestFanal_Library_*")+			defer os.RemoveAll(d)+			c := cache.Initialize(d)++			opt := types.DockerOption{+				Timeout:  600 * time.Second,+				SkipPing: true,+			}++			cli, err := client.NewClientWithOpts(client.FromEnv)+			require.NoError(t, err, tc.name)++			// ensure image doesnt already exists+			_, _ = cli.ImageRemove(ctx, tc.imageFile, dtypes.ImageRemoveOptions{+				Force:         true,+				PruneChildren: true,+			})++			testfile, err := os.Open(tc.imageFile)+			require.NoError(t, err)++			// load image into docker engine+			_, err = cli.ImageLoad(ctx, testfile, true)+			require.NoError(t, err, tc.name)++			// tag our image to something unique+			err = cli.ImageTag(ctx, tc.imageName, tc.imageFile)+			require.NoError(t, err, tc.name)++			ext, err := docker.NewDockerExtractor(opt, c)+			require.NoError(t, err, tc.name)+			ac := analyzer.Config{Extractor: ext}++			// run tests twice, one without cache and with cache+			for i := 1; i <= 2; i++ {+				runChecksDockerMode(t, ac, ctx, tc, d, c)+			}++			// clear Cache+			require.NoError(t, c.Clear(), tc.name)++			// remove Image+			_, err = cli.ImageRemove(ctx, tc.imageFile, dtypes.ImageRemoveOptions{+				Force:         true,+				PruneChildren: true,+			})+			assert.NoError(t, err, tc.name)+		})+	}+}++func runChecksDockerMode(t *testing.T, ac analyzer.Config, ctx context.Context, tc testCase, d string, c cache.Cache) {+	actualFiles, err := ac.Analyze(ctx, tc.imageFile)

Yeah I agree. I have addressed this concern in the next commits. I think I'm happy with having a single file and a common set of use cases as seen here: https://github.com/aquasecurity/fanal/pull/76/commits/41881b073d3f0bb9ba2e57069a7e3f7fbaebcf0f

simar7

comment created time in a month

push eventaquasecurity/fanal

Simarpreet Singh

commit sha 41881b073d3f0bb9ba2e57069a7e3f7fbaebcf0f

integration: Refactor tar and dockermode together Signed-off-by: Simarpreet Singh <simar@linux.com>

view details

Simarpreet Singh

commit sha d3e859c5831ac1ad33df48f407eeff8a5a43c8a1

integration: DRY check funcs Signed-off-by: Simarpreet Singh <simar@linux.com>

view details

push time in a month

PR opened aquasecurity/fanal

Reviewers
integration: Fanal as a library for tar mode

This PR adds integration tests for Fanal being used as a library in Tar mode.

Also refactors code.

Signed-off-by: Simarpreet Singh simar@linux.com

+251 -71

0 comment

2 changed files

pr created time in a month

more