profile
viewpoint
If you are wondering where the data of this site comes from, please visit https://api.github.com/users/misterpickypants/events. GitMemory does not store any data, but only uses NGINX to cache data for a period of time. The idea behind GitMemory is simply to give users a better reading experience.
Augusto Roman misterpickypants @tanium Longmont, CO I'm a software engineer at Tanium. I have high standards for code reviews.

tanium/octobot 21

github bot with slack and jira integration

augustoroman/chompy 1

Software for a snack-dispensing motivator!

misterpickypants/consul 0

Consul is a distributed, highly available, and data center aware solution to connect and configure applications across dynamic, distributed infrastructure.

misterpickypants/proposal 0

Go Project Design Documents

misterpickypants/recorder 0

HTTP traffic record & replay for go

misterpickypants/rules_go 0

Go rules for Bazel

tanium/badger 0

Fast key-value DB in Go.

tanium/protobuf.js 0

Protocol Buffers for JavaScript (& TypeScript).

tanium/service 0

Run go programs as a service on major platforms.

issue commentbazelbuild/bazel

Bazel analysis phase re-evaluates if the WORKSPACE file is touched but not changed.

Not knowing very much about the bazel internals, I would expect that the file system watcher that detects writes to cached files to invalidate them could store a checksum of such files and, after any writes, only invalidate if the checksum changes. 🤷‍♂️

misterpickypants

comment created time in 3 months

issue openedbazelbuild/bazel

Bazel analysis phase re-evaluates if the WORKSPACE file is touched but not changed.

Description of the problem / feature request:

Bazel analysis cache appears to be re-evaluated if the WORKSPACE file is touched (but not changed), significantly slowing the bazel execution when no changes have been made. This appears to happen even if the modification & access times of the file do not change.

The impact of this in our internal repo is significant during development. Running gazelle regularly to sync go.mod/go.sum to the WORKSPACE dependencies is fast (<1s), but it rewrites the WORKSPACE file even if there are no changes. Touching the WORKSPACE file (with no actual changes) incurs a 30-60 second penalty to the subsequent bazel run.

A similar but reduced effect can be seen if the root BUILD file is touched, but seems to incur only ~50% of the time impact.

Bugs: what's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

  • Clone https://github.com/connyay/rules_go_deterministic_builder_demo and build once to populate the cache.
    git clone https://github.com/connyay/rules_go_deterministic_builder_demo
    cd rules_go_deterministic_builder_demo
    bazel build //cmd/main:main
    
  • Time builds to verify it's cached and fast. Yay!
    » time bazel build //cmd/main:main
    Target //cmd/main:main up-to-date:
      bazel-bin/cmd/main/main_/main
    bazel build //cmd/main:main  0.03s user 0.04s system 13% cpu 0.497 total
    » time bazel build //cmd/main:main
    Target //cmd/main:main up-to-date:
      bazel-bin/cmd/main/main_/main
    bazel build //cmd/main:main  0.03s user 0.04s system 13% cpu 0.465 total
    
  • Touch WORKSPACE with a fixed timestamp.
    » touch -t 202106211618.11 -a -m WORKSPACE
    
  • See that builds are slower the first time, then fast again.
    » time bazel build //cmd/main:main
    INFO: Analyzed target //cmd/main:main (4 packages loaded, 34 targets configured).
    INFO: Found 1 target...
    Target //cmd/main:main up-to-date:
      bazel-bin/cmd/main/main_/main
    INFO: Elapsed time: 2.698s, Critical Path: 0.01s
    INFO: 1 process: 1 internal.
    INFO: Build completed successfully, 1 total action
    bazel build //cmd/main:main  0.02s user 0.04s system 2% cpu 2.732 total          <-- slow ~3sec
    » time bazel build //cmd/main:main
    INFO: Analyzed target //cmd/main:main (0 packages loaded, 0 targets configured).
    INFO: Found 1 target...
    Target //cmd/main:main up-to-date:
      bazel-bin/cmd/main/main_/main
    INFO: Elapsed time: 0.457s, Critical Path: 0.01s
    INFO: 1 process: 1 internal.
    INFO: Build completed successfully, 1 total action
    bazel build //cmd/main:main  0.03s user 0.04s system 13% cpu 0.497 total        <-- fast! ~.5 sec
    
    Target //cmd/main:main up-to-date:
      bazel-bin/cmd/main/main_/main
    bazel build //cmd/main:main  0.03s user 0.05s system 15% cpu 0.486 total   
    
  • Touch WORKSPACE with the SAME timestamp again, then see that builds are slower the first time, then fast again. Ooof!
    » touch -t 202106211618.11 -a -m WORKSPACE
    » time bazel build //cmd/main:main
    INFO: Analyzed target //cmd/main:main (4 packages loaded, 34 targets configured).
    INFO: Found 1 target...
    Target //cmd/main:main up-to-date:
      bazel-bin/cmd/main/main_/main
    INFO: Elapsed time: 2.516s, Critical Path: 0.01s
    INFO: 1 process: 1 internal.
    INFO: Build completed successfully, 1 total action
    bazel build //cmd/main:main  0.02s user 0.03s system 1% cpu 2.543 total          <-- slow again!
    » time bazel build //cmd/main:main
    INFO: Analyzed target //cmd/main:main (0 packages loaded, 0 targets configured).
    INFO: Found 1 target...
    Target //cmd/main:main up-to-date:
      bazel-bin/cmd/main/main_/main
    INFO: Elapsed time: 0.414s, Critical Path: 0.00s
    INFO: 1 process: 1 internal.
    INFO: Build completed successfully, 1 total action
    bazel build //cmd/main:main  0.02s user 0.03s system 11% cpu 0.441 total         <-- fast again!
    »
    

What operating system are you running Bazel on?

OSX Big Sur 11.4

Darwin 20.5.0 / Darwin Kernel Version 20.5.0 / root:xnu-7195.121.3~9/RELEASE_X86_64 x86_64

What's the output of bazel info release?

release 4.1.0

What's the output of git remote get-url origin ; git rev-parse master ; git rev-parse HEAD ?

» git remote get-url origin ; git rev-parse main ; git rev-parse HEAD
git@github.com:connyay/rules_go_deterministic_builder_demo.git
f0496b41c0c927d9936d82b1ab9f37e7f6faad4e
f0496b41c0c927d9936d82b1ab9f37e7f6faad4e
»

Have you found anything relevant by searching the web?

No :(

Any other information, logs, or outputs that you want to share?

I've tried looking at execution logs and profile data data. While the profile data does show that the workspace is being re-evaluated, I cannot figure out why.

created time in 3 months