profile
viewpoint

androidthings/sample-googleassistant 465

Google Assistant API sample for Android Things

drigz/aiyprojects-raspbian 1

Source code for the AIYProjects "Voice Kit"

drigz/aip 0

API Improvement Proposals. https://aip.dev/

drigz/api-linter 0

A linter for APIs defined in protocol buffers.

drigz/async_grpc 0

Framework for building asynchronous, multi-threaded gRPC services

drigz/bazel 0

a fast, scalable, multi-language and extensible build system

drigz/bazel-deps 0

Generate bazel dependencies for maven artifacts

drigz/bazel_gomock 0

Code to create Go mocks for bazel targets using mockgen

drigz/bazel_python_directory_conflict 0

Repro for directory conflict with py_library's imports parameter

issue commentgoogle/glog

LOG_EVERY_N stops printing when using threads

if (LOG_OCCURRENCES_MOD_N++ == n) LOG_OCCURRENCES_MOD_N -= n; feels risky to me as well. If a thread terminates between incrementing and subtracting (unlikely but possible?) we would end up in the same position. Or:

  • n == 2 and LOG_OCCURRENCES_MOD_N == 1
  • three threads simultaneously execute LOG_OCCURRENCES_MOD_N++, seeing 2, 3, and 4 respectively
  • the first thread subtracts, the others do nothing, so LOG_OCCURRENCES_MOD_N==2
  • the log message is not emitted until the counter wraps around

I'm wondering if we can somehow salvage the previous behavior, which was working for a long time. Maybe keeping unsigned and checking x=LOG_OCCURRENCES_MOD_N++; if (n < x && x < MAXINT)? This would continue to count up even if we subtract too far. Or maybe we could revert to the signed int, and add an annotation to tell the compiler that LOG_OCCURRENCES_MOD_N -= n is very unlikely to overflow so we don't mind what it does - and maybe adjust the docs to recommend avoiding the combination of N close to MAXINT and multi-threaded logging.

lucabergamini

comment created time in 3 days

issue commentgoogle/glog

LOG_EVERY_N stops printing when using threads

I haven't tried with a signed int but I guess you're on the right track. I caught the bad state in gdb:

(gdb) info locals
occurrences_9 = {<std::__atomic_base<unsigned int>> = {static _S_alignment = 4, _M_i = 65324}, <No data fields>}
occurrences_mod_n_9 = {<std::__atomic_base<unsigned int>> = {static _S_alignment = 4, _M_i = 4294731290}, <No data fields>}

_M_i = 4294731290 suggests that LOG_OCCURRENCES_MOD_N -= n happened twice in both threads and it wrapped around to ~MAXINT, and now every thread is executing LOG_OCCURRENCES_MOD_N -= n every time.

A signed int would avoid that problem. OTOH it could plausibly cause undefined behavior through overflows if N is close to MAXINT, but maybe that's just a theoretical concern.

lucabergamini

comment created time in 3 days

issue commentgoogle/glog

LOG_EVERY_N stops printing when using threads

I can repro this sometimes after 30s or so but not reliably, so I haven't been able to bisect. I built with ThreadSantizer but it didn't report any problem, and still crashed after a while. I've run a few times under gdb with and without -c dbg but haven't managed to repro there. I haven't tried with CMake.

lucabergamini

comment created time in 3 days

Pull request review commentkubernetes/kubernetes

fix subPath unmount issue due to smb server down

 func removePathIfNotMountPoint(mountPath string, mounter Interface, extensiveMou // PathExists returns true if the specified path exists. // TODO: clean this up to use pkg/util/file/FileExists func PathExists(path string) (bool, error) {-	_, err := os.Stat(path)+	_, err := os.Lstat(path)

@tsmetana You are entirely correct: syscall.Access works fine for this on Linux, and I was testing the wrong path. Sorry for the noise!

andyzhangx

comment created time in 17 days

PullRequestReviewEvent
PullRequestReviewEvent

Pull request review commentkubernetes/kubernetes

fix subPath unmount issue due to smb server down

 func removePathIfNotMountPoint(mountPath string, mounter Interface, extensiveMou // PathExists returns true if the specified path exists. // TODO: clean this up to use pkg/util/file/FileExists func PathExists(path string) (bool, error) {-	_, err := os.Stat(path)+	_, err := os.Lstat(path)

Forgive the drive-by comment, but since I had a repro I've tested the suggestions, and they all get "no such file or directory" on my system:

$ sudo ls /var/lib/kubelet/pods/823a42ab-1f77-4caa-b3ea-4e33fe43dd53/volume-subpaths/pv-smb/deployment-smb  
0
$ sudo ./stat /var/lib/kubelet/pods/5e50975f-6331-4e76-9968-09627864d30d/volume-subpaths/pv-smb/deployment-smb/0
checking /var/lib/kubelet/pods/5e50975f-6331-4e76-9968-09627864d30d/volume-subpaths/pv-smb/deployment-smb/0
os.Stat: stat /var/lib/kubelet/pods/5e50975f-6331-4e76-9968-09627864d30d/volume-subpaths/pv-smb/deployment-smb/0: no such file or directory
os.Lstat: lstat /var/lib/kubelet/pods/5e50975f-6331-4e76-9968-09627864d30d/volume-subpaths/pv-smb/deployment-smb/0: no such file or directory
syscall.Access: no such file or directory

from

        _, err := os.Stat(p)
        fmt.Printf("os.Stat: %v\n", err)
        _, err = os.Lstat(p)
        fmt.Printf("os.Lstat: %v\n", err)
        err = syscall.Access(p, syscall.F_OK)
        fmt.Printf("syscall.Access: %v\n", err)

Perhaps it would be better to call umount without trying to check for path existence? If the path doesn't exist after calling umount, it could suppress the error and print the expected "skipping" message, if that's the motivation for the check.

andyzhangx

comment created time in 17 days

issue openedkubernetes-csi/csi-driver-smb

"Unmount skipped because path does not exist" when subPath is deleted

What happened:

I noticed that sometimes our pods get stuck in Terminating, and when I look at the kubelet logs I see:

mount_helper_common.go:33] Warning: Unmount skipped because path does not exist: /var/lib/kubelet/pods/5e50975f-6331-4e76-9968-09627864d30d/volume-subpaths/pv-smb/deployment-smb/0
nestedpendingoperations.go:301] [...] remove /var/lib/kubelet/pods/5e50975f-6331-4e76-9968-09627864d30d/volume-subpaths/pv-smb/deployment-smb: directory not empty"

What you expected to happen:

Pods can be deleted, even if a directory has been deleted on the smbserver.

How to reproduce it:

I set up a volume mount based on the examples pv-smb.yaml, pvc-smb-static.yaml, deployment.yaml, with some exceptions:

  • I created foo/bar within the SMB share.
  • I changed the deployment to run sleep 9999 and set a subPath:
          volumeMounts:
            - name: smb
              mountPath: "/mnt/smb"
              readOnly: false
              subPath: foo

After the deployment was running, I confirmed it had mounted the SMB share, then deleted foo/ followed by the deployment:

$ kubectl exec -it deploy/deployment-smb -- ls /mnt/smb                                         
bar                               
$ sudo rm -r /path/to/smb/share/foo                                                                                                                                                                  
$ kubectl delete deploy deployment-smb                                                          
deployment.apps "deployment-smb" deleted                                                                 
$ kubectl get pods                                                                              
NAME                                          READY   STATUS        RESTARTS   AGE                       
deployment-smb-67cfbbd475-q7jjb               1/1     Terminating   0          50s                       

The pod is stuck in terminating. kubelet shows the following errors:

kubelet[1558913]: W1118 21:30:46.620913 1558913 mount_helper_common.go:33] Warning: Unmount skipped because path does not exist: /var/lib/kubelet/pods/5e50975f-6331-4e76-9968-09627864d30d/volume-subpaths/pv-smb/deployment-smb/0
kubelet[1558913]: E1118 21:30:46.621117 1558913 nestedpendingoperations.go:301] Operation for "{volumeName:kubernetes.io/csi/smb.csi.k8s.io^unique-volumeid podName:5e50975f-6331-4e76-9968-09627864d30d nodeName:}" failed. No retries permitted until 2021-11-18 21:31:50.621060134 +0100 CET m=+801376.094888036 (durationBeforeRetry 1m4s). Error: "error cleaning subPath mounts for volume \"smb\" (UniqueName: \"kubernetes.io/csi/smb.csi.k8s.io^unique-volumeid\") pod \"5e50975f-6331-4e76-9968-09627864d30d\" (UID: \"5e50975f-6331-4e76-9968-09627864d30d\") : error deleting /var/lib/kubelet/pods/5e50975f-6331-4e76-9968-09627864d30d/volume-subpaths/pv-smb/deployment-smb: remove /var/lib/kubelet/pods/5e50975f-6331-4e76-9968-09627864d30d/volume-subpaths/pv-smb/deployment-smb: directory not empty"

csi-smb-node doesn't report an error - perhaps because it doesn't know about the subPath? (although there a warning "... is not a mountpoint, deleting")

I1118 20:28:37.399545       1 utils.go:118] GRPC call: /csi.v1.Node/NodePublishVolume
I1118 20:28:37.399595       1 utils.go:119] GRPC request: {"staging_target_path":"/var/lib/kubelet/plugins/kubernetes.io/csi/pv/pv-smb/globalmount","target_path":"/var/lib/kubelet/pods/5e50975f-6331-4e76-9968-09627864d30d/volumes/kubernetes.io~csi/pv-smb/mount","volume_capability":{"AccessType":{"Mount":{"mount_flags":["dir_mode=0555","file_mode=0444","vers=3.0"]}},"access_mode":{"mode":5}},"volume_context":{"csi.storage.k8s.io/ephemeral":"false","csi.storage.k8s.io/pod.name":"deployment-smb-67cfbbd475-q7jjb","csi.storage.k8s.io/pod.namespace":"default","csi.storage.k8s.io/pod.uid":"5e50975f-6331-4e76-9968-09627864d30d","csi.storage.k8s.io/serviceAccount.name":"default","source":"//smb-server.default.svc.cluster.local/share"},"volume_id":"unique-volumeid"}
I1118 20:28:37.404346       1 nodeserver.go:85] NodePublishVolume: mounting /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pv-smb/globalmount at /var/lib/kubelet/pods/5e50975f-6331-4e76-9968-09627864d30d/volumes/kubernetes.io~csi/pv-smb/mount with mountOptions: [bind] volumeID(unique-volumeid)
I1118 20:28:37.404403       1 mount_linux.go:175] Mounting cmd (mount) with arguments ( -o bind /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pv-smb/globalmount /var/lib/kubelet/pods/5e50975f-6331-4e76-9968-09627864d30d/volumes/kubernetes.io~csi/pv-smb/mount)
I1118 20:28:37.411311       1 mount_linux.go:175] Mounting cmd (mount) with arguments ( -o bind,remount /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pv-smb/globalmount /var/lib/kubelet/pods/5e50975f-6331-4e76-9968-09627864d30d/volumes/kubernetes.io~csi/pv-smb/mount)
I1118 20:28:37.414453       1 nodeserver.go:92] NodePublishVolume: mount /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pv-smb/globalmount at /var/lib/kubelet/pods/5e50975f-6331-4e76-9968-09627864d30d/volumes/kubernetes.io~csi/pv-smb/mount volumeID(unique-volumeid) successfully
I1118 20:28:37.414487       1 utils.go:125] GRPC response: {}
I1118 20:28:37.512580       1 utils.go:118] GRPC call: /csi.v1.Node/NodeUnpublishVolume
I1118 20:28:37.512620       1 utils.go:119] GRPC request: {"target_path":"/var/lib/kubelet/pods/3d219b8d-4683-4ee7-8a75-525e03099cbe/volumes/kubernetes.io~csi/pv-smb/mount","volume_id":"unique-volumeid"}
I1118 20:28:37.513728       1 nodeserver.go:107] NodeUnpublishVolume: unmounting volume unique-volumeid on /var/lib/kubelet/pods/3d219b8d-4683-4ee7-8a75-525e03099cbe/volumes/kubernetes.io~csi/pv-smb/mount
I1118 20:28:37.513746       1 mount_linux.go:266] Unmounting /var/lib/kubelet/pods/3d219b8d-4683-4ee7-8a75-525e03099cbe/volumes/kubernetes.io~csi/pv-smb/mount
W1118 20:28:37.521540       1 mount_helper_common.go:133] Warning: "/var/lib/kubelet/pods/3d219b8d-4683-4ee7-8a75-525e03099cbe/volumes/kubernetes.io~csi/pv-smb/mount" is not a mountpoint, deleting
I1118 20:28:37.521655       1 nodeserver.go:112] NodeUnpublishVolume: unmount volume unique-volumeid on /var/lib/kubelet/pods/3d219b8d-4683-4ee7-8a75-525e03099cbe/volumes/kubernetes.io~csi/pv-smb/mount successfully
I1118 20:28:37.521675       1 utils.go:125] GRPC response: {}

If I inspect the mountpoint:

$ sudo ls /var/lib/kubelet/pods/5e50975f-6331-4e76-9968-09627864d30d/volume-subpaths/pv-smb/deployment-smb   
0
$ sudo ls -ld /var/lib/kubelet/pods/5e50975f-6331-4e76-9968-09627864d30d/volume-subpaths/pv-smb/deployment-smb/0
ls: cannot access '/var/lib/kubelet/pods/5e50975f-6331-4e76-9968-09627864d30d/volume-subpaths/pv-smb/deployment-smb/0': No such file or directory
$ sudo umount /var/lib/kubelet/pods/5e50975f-6331-4e76-9968-09627864d30d/volume-subpaths/pv-smb/deployment-smb/0
(successful umount)

After umount, the pod terminates normally. I've actually installed a hacky workaround on the crontab of our clusters:

* * * * * journalctl -u kubelet -n 3 | sed -n "s/.*Unmount skipped because path does not exist: \(.*\)/\1/p" | xargs umount

Anything else we need to know?:

This might be a kubelet bug - I don't know much about the interface between the plugin and the kubelet itself. I'm also not sure how to reproduce without using csi-driver-smb, so not sure how it ought to be reported upstream.

I found https://github.com/kubernetes/kubernetes/issues/83089 but it just mentions the warning, not the pod stuck in Terminating.

Environment:

  • CSI Driver version: 1.2.0
  • Kubernetes version (use kubectl version): 1.20.11
  • OS (e.g. from /etc/os-release): Similar to Debian testing
  • Kernel (e.g. uname -a): 5.10.46
  • Install tools: helm

created time in 18 days

pull request commentkubernetes/website

Remove "basic" from supported API auth methods

Thanks for taking a look!

we're gonna need an approval from someone from SIG Auth before we can merge this

Do I need to tag SIG Auth here, or should I just wait? (please excuse my inexperience, I've only sent a couple of PRs and was never sure when I should be pinging people)

Can you please also link to where it's documented that "HTTP basic auth" is no longer supported?

https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.19.md#:~:text=Support%20for%20basic%20authentication

drigz

comment created time in 20 days

issue commentgooglecloudrobotics/core

No valid versions with the prefix "1.18" found.

Sorry for the slow response - I haven't had time to dig into this this week. Maybe cluster.tf's 1.18 needs to be bumped to 1.20 if GKE no longer offers 1.18. You could try that out with the deploy from source instructions: https://googlecloudrobotics.github.io/core/how-to/deploy-from-sources - a PR would be welcome if it works!

michidk

comment created time in 24 days

issue commentgooglecloudrobotics/core

Find a way to install go tools directly

Yay!

➜  ~ go install github.com/googlecloudrobotics/core/src/go/cmd/synk@14a096b951f982d23f00a5d4ad4a38862ddc51d0
go: downloading github.com/googlecloudrobotics/core/src v0.0.0-20211105123335-14a096b951f9
go: downloading github.com/googlecloudrobotics/core v0.0.0-20211105123335-14a096b951f9
➜  ~ ~/go/bin/synk --help 
A tool to sync manifests with a cluster.
[...]
ensonic

comment created time in a month

issue commentgooglecloudrobotics/core

Find a way to install go tools directly

http-relay error is gone for now at least:

$ go get github.com/googlecloudrobotics/core/src/go/...@98a8851a
go: downloading github.com/googlecloudrobotics/core/src v0.0.0-20211105094632-98a8851a1ff1
go get: installing executables with 'go get' in module mode is deprecated.
        Use 'go install pkg@version' instead.
        For more information, see https://golang.org/doc/go-get-install-deprecation
        or run 'go help get' or 'go help install'.

$ go install github.com/googlecloudrobotics/core/src/go/cmd/synk@98a8851a
go install: github.com/googlecloudrobotics/core/src/go/cmd/synk@98a8851a (in github.com/googlecloudrobotics/core/src@v0.0.0-20211105094632-98a8851a1ff1):
        The go.mod file for the module providing named packages contains one or
        more replace directives. It must not contain directives that would cause
        it to be interpreted differently than if it were the main module.
ensonic

comment created time in a month

PR opened kubernetes/website

Remove "basic" from supported API auth methods

This was removed in v1.19.

<!-- 🛈

Hello!

Remember to ADD A DESCRIPTION and delete this note before submitting your pull request. The description should explain what will change, and why.

PLEASE title the FIRST commit appropriately, so that if you squash all your commits into one, the combined commit message makes sense. For overall help on editing and submitting pull requests, visit: https://kubernetes.io/docs/contribute/start/#improve-existing-content

Use the default base branch, “main”, if you're documenting existing features in the English localization.

If you're working on a different localization (not English), see https://kubernetes.io/docs/contribute/new-content/overview/#choose-which-git-branch-to-use for advice.

If you're documenting a feature that will be part of a future release, see https://kubernetes.io/docs/contribute/new-content/new-features/ for advice.

-->

+1 -1

0 comment

1 changed file

pr created time in a month

push eventdrigz/website

Rodrigo Queiro

commit sha f3921c90282b92733edc93fdc79a9573b58a253e

Remove "basic" from supported API auth methods This was removed in v1.19.

view details

push time in a month

issue closedwarmcat/libwebsockets

libwebsockets fails inside a container: OOM allocating 1073741816 fds

I observed this when running mosquitto inside a container, using containerd 1.5.5:

/ # /usr/sbin/mosquitto -v -c /mosquitto/config/mosquitto.conf 
1634294021: mosquitto version 1.6.12 starting                                                            
1634294021: Config loaded from /mosquitto/config/mosquitto.conf.
1634294021: Opening websockets listen socket on port 9090.
1634294021: libuv support not compiled in         
1634294021: OOM allocating 1073741816 fds           
1634294021: ZERO RANDOM FD                      
1634294021: Error: Unable to create websockets listener on port 9090.

The issue appears to be caused by https://github.com/containerd/containerd/pull/4475, which sets LimitNOFILE=infinity, apparently in favor of container-local accounting. However, https://github.com/kubernetes/kubernetes/issues/3595 prevents me from easily avoiding this crash. It can be worked around by running ulimit -n 1024 before starting mosquitto.

This was reported before in #1769, but closed. I've reopened because I expect more people will run into this as newer versions of containerd are more widely used.

closed time in 2 months

drigz

issue commentwarmcat/libwebsockets

libwebsockets fails inside a container: OOM allocating 1073741816 fds

Thanks for the pointer and apologies for the imprecise issue title. mosquitto 1.6.12 pins lws 2.4.2 (source). This was bumped to 4.2.0 in May, and I can't reproduce the issue with mosquitto 1.6.15.

I would argue that the high limit is not unreasonable; https://man7.org/linux/man-pages/man2/getrlimit.2.html doesn't guarantee that a process can allocate that many FDs or that much memory, just "This specifies a value one greater than the maximum file descriptor number that can be opened by this process. Attempts (open(2), pipe(2), dup(2), etc.) to exceed this limit yield the error EMFILE."

Either way, the case of LimitNOFILE=infinity is fixed, so I'll close this. Sorry for the noise!

drigz

comment created time in 2 months

issue openedwarmcat/libwebsockets

libwebsockets fails inside a container: OOM allocating 1073741816 fds

I observed this when running mosquitto inside a container, using containerd 1.5.5:

/ # /usr/sbin/mosquitto -v -c /mosquitto/config/mosquitto.conf 
1634294021: mosquitto version 1.6.12 starting                                                            
1634294021: Config loaded from /mosquitto/config/mosquitto.conf.
1634294021: Opening websockets listen socket on port 9090.
1634294021: libuv support not compiled in         
1634294021: OOM allocating 1073741816 fds           
1634294021: ZERO RANDOM FD                      
1634294021: Error: Unable to create websockets listener on port 9090.

The issue appears to be caused by https://github.com/containerd/containerd/pull/4475, which sets LimitNOFILE=infinity, apparently in favor of container-local accounting. It can be worked around by running ulimit -n 1024 before starting mosquitto.

This was reported before in #1769, but closed. I've reopened because I expect more people will run into this as newer versions of containerd are more widely used.

created time in 2 months

Pull request review commentgoogle/glog

ci: replace generated headers by templates

 jobs:           '*/src/mock-log.h' \           '/usr/*' \           --output-file coverage.info++        for file in $(ls src/glog/*.h.in); do

nit: I don't think ls is needed here, for f in *.h; do should work just fine.

sergiud

comment created time in 2 months

PullRequestReviewEvent
PullRequestReviewEvent
MemberEvent

pull request commentgoogle/glog

add linux github workflow

I "invited" the bot, let's see if the comments appear.

sergiud

comment created time in 2 months

pull request commentgooglecloudrobotics/core

Update controller-runtime to v0.10.2 / client-go to v0.22.2

Thanks! I modified this to delete the commented-out test, as the comment on https://github.com/kubernetes/kubernetes/pull/78630#issuecomment-500424163 suggests this won't be fixed, and the code would otherwise bitrot; I've sent it for internal review.

oliver-goetz

comment created time in 2 months

pull request commentgoogle/glog

add linux github workflow

I couldn't find an explanation on how to do it, but I signed into coveralls.io and "enabled" this repo. I also found this doc about the GitHub Action: https://github.com/marketplace/actions/coveralls-github-action - is that sufficient for you to enable it here?

sergiud

comment created time in 2 months

pull request commentgoogle/glog

add linux github workflow

Sorry, I'm afraid I've not used GitHub actions or any of those coverage tools before, my knowledge is all from the days of running make coverage and inspecting a text file...

sergiud

comment created time in 2 months

delete branch drigz/glog

delete branch : werror

delete time in 2 months

push eventgoogle/glog

Rodrigo Queiro

commit sha 831a6f823276216d89daa3b50e2c542c35480e96

Add -Werror to Bazel presubmits This avoids issues like #717.

view details

push time in 2 months

PR merged google/glog

Add -Werror to Bazel presubmits cla: yes

This avoids issues like #717.

+6 -0

2 comments

1 changed file

drigz

pr closed time in 2 months

push eventdrigz/glog

Artur

commit sha c34dbe9873f63d1941d5477e0dcfae74b4cff243

Fix `syscall` warning in Bazel Build This commit resolves [#717](https://github.com/google/glog/issues/717) issue.

view details

Rodrigo Queiro

commit sha ed14894bcc3daa76af207f93990e4eae4ecefcb5

Add -Werror to Bazel presubmits This avoids issues like #717.

view details

push time in 2 months

push eventgoogle/glog

Artur

commit sha c34dbe9873f63d1941d5477e0dcfae74b4cff243

Fix `syscall` warning in Bazel Build This commit resolves [#717](https://github.com/google/glog/issues/717) issue.

view details

push time in 2 months

more