profile
viewpoint

charleswhchan/dwave-ocean-qubo-examples 2

My attempt to implement problems described "A Tutorial on Formulating and Using QUBO Models" (https://arxiv.org/pdf/1811.11538.pdf) using D-Wave Ocean SDK in Python and solving on D-Wave Quantum Computer

charleswhchan/celery 1

Distributed Task Queue (development branch)

charleswhchan/amazon-kinesis-client-python 0

Amazon Kinesis Client Library for Python

charleswhchan/ansible 0

Ansible is a radically simple IT automation platform that makes your applications and systems easier to deploy. Avoid writing scripts or custom code to deploy and update your applications— automate in a language that approaches plain English, using SSH, with no agents to install on remote systems.

charleswhchan/ansible-lint 0

Best practices checker for Ansible

charleswhchan/apidocjs.com 0

apidocjs.com Website

charleswhchan/atom 0

The hackable text editor

charleswhchan/awesome-go 0

A curated list of awesome Go frameworks, libraries and software

charleswhchan/aws-cli 0

Universal Command Line Interface for Amazon Web Services

charleswhchan/backbone 0

Give your JS App some Backbone with Models, Views, Collections, and Events

starteddjango-guardian/django-guardian

started time in 2 days

starteddfunckt/django-rules

started time in 2 days

startedvintasoftware/django-role-permissions

started time in 2 days

startedvintasoftware/django-role-permissions

started time in 2 days

startedjazzband/django-authority

started time in 2 days

startedyandex/zero-downtime-migrations

started time in 4 days

startedReact-Proto/react-proto

started time in 4 days

startedocadotechnology/django-closuretree

started time in 7 days

starteddjango-mptt/django-mptt

started time in 7 days

starteddjango-treebeard/django-treebeard

started time in 7 days

issue commentlocalstack/localstack

Simulate S3 BadDigest and InvalidDigest errors

Thanks @whummer, I can confirm with the latest image, the not base64 encoded checksum case is now fixed (and returns InvalidDigest). 👏👍

I did mention another case in my last post (not in gist): 12345678901234567890 returns BadDigest while 1234567890123456789 returns InvalidDigest. It don't really matter to me but I am not sure if you want to take a look also?

charleswhchan

comment created time in 7 days

issue commentlocalstack/localstack

Simulate S3 BadDigest and InvalidDigest errors

Hi @whummer : I really appreciate your effort in troubleshooting this issue 👍

After reviewing your tests and comparing to my test, I found something interesting that triggers the error. I have created a gist

Specifically, if you refer to the checksum used in Test #3:

	fmt.Println("Test #3: InvalidDigest ---------------------------------------")
	contentMD5 = "not base64 encoded checksum"
	uploadPartInput = &s3.UploadPartInput{
		Body:       aws.ReadSeekCloser(bytes.NewReader(data)),
		Bucket:     aws.String(bucket),
		Key:        aws.String(key),
		ContentMD5: aws.String(contentMD5),
		PartNumber: aws.Int64(int64(partNumber)),
		UploadId:   aws.String(uploadID),
	}

	fmt.Println("UploadPart ...")
	_, err = svc.UploadPart(uploadPartInput)
	fmt.Printf("expect InvalidDigest; actual %+v\n", err)

Output

Test #3: InvalidDigest ---------------------------------------
UploadPart ...
expect InvalidDigest; actual BadDigest: The Content-MD5 you specified did not match what we received.

I did a bit more investigation and it appears it might have to something with the length of the contentMD5? If the length is <= 19, localstack would return InvalidDigest (correctly): contentMD5 = "1234567890123456789" Otherwise, localstack would return BadDigest? contentMD5 = "12345678901234567890"

Strange problem. I hope this information helps 😃

charleswhchan

comment created time in 8 days

issue commentRichardKnop/machinery

Redis SSL

@yap-studios : sorry but I don't have permission. Please check with @RichardKnop. 😃

yap-studios

comment created time in 12 days

issue commentRichardKnop/machinery

Redis SSL

Yeah. PR looks reasonable to me.

yap-studios

comment created time in 13 days

issue commentlocalstack/localstack

Incorrect response code when Downloading S3 object with invalid range

Thanks @whummer for the quick turnaround. You are right, it is working with the following image:

$ docker pull localstack/localstack:latest
latest: Pulling from localstack/localstack
8a4e9a21a873: Pull complete 
Digest: sha256:3db28dbf80b9587376d11306398e1d2e8ee32bfbb5bf7528d4dd84a0f314cdc5
Status: Downloaded newer image for localstack/localstack:latest
docker.io/localstack/localstack:latest

Please close the ticket. 👍

charleswhchan

comment created time in 13 days

issue commentlocalstack/localstack

Incorrect response code when Downloading S3 object with invalid range

Hi @whummer, I tried the latest docker image and I am still seeing the error: "error":"RequestedRangeNotSatisfiable: Requested Range Not Satisfiable\n\tstatus code: 416

Looking at the test case and comparing to my code. I am using Download().

Perhaps the fix needs to be applied some where else as well? Maybe a test using download_file() or download_fileobj() will help expose the problem?

charleswhchan

comment created time in 14 days

issue commentlocalstack/localstack

Simulate S3 BadDigest and InvalidDigest errors

Hi @whummer.

I just tried the latest docker image (link) and it seems to broken in a different way -- it is now returning BadDigest all the time.

I am using go-lang, and here is my 2 test cases:

Test 1. checksum := "not base64 encoded checksum"

  • Expect InvalidDigest
  • Actual 0.11.1: InvalidDigest
  • Actual latest: BadDigest

Test 2

		checksum := base64.StdEncoding.EncodeToString(wrongCheckSum[:])```
- Expect `BadDigest`
- Actual 0.11.1: `InvalidDigest` ❌  
- Actual `latest`: `BadDigest` ✅ 

Note: I also run the same test against S3 and MinIO to to confirm the expected behaviour.
charleswhchan

comment created time in 14 days

issue commentRichardKnop/machinery

Redis SSL

The factory currently looks for the prefix "redis://": https://github.com/RichardKnop/machinery/blob/master/v1/factories.go#L41

Looking at the underlying Redis libraries, both of them supports rediss://:

  • redigo https://github.com/gomodule/redigo/pull/208
  • go-redis https://github.com/go-redis/redis/pull/420

It should be pretty easy to modify the factory to support this. Want to give this a try and create a PR for it?

yap-studios

comment created time in 15 days

startedhshoff/vx

started time in 15 days

startedplouc/nivo

started time in 15 days

startedFormidableLabs/victory

started time in 15 days

starteduber/react-vis

started time in 15 days

startedgithub/super-linter

started time in 15 days

issue openedlocalstack/localstack

Incorrect response code when Downloading S3 object with invalid range

Type of request: This is a ...

[x] bug report [ ] feature request

Detailed description

When download invalid a range of bytes from S3 with an invalid range. localstack returns an incorrect error code RequestedRangeNotSatisfiable instead of InvalidRange

Expected behavior

(from S3) Err: InvalidRange: The requested range is not satisfiable

Actual behavior

(from localstack) RequestedRangeNotSatisfiable

Client code (AWS SDK code snippet, or sequence of "awslocal" commands)

	// Test S3 error code when invalid byte range is specified
	// Note: the file should be smaller than the range specified
	contentLength := 1000000
	rangeByteHeader := "bytes=1000-1000000"
        fmt.Printf("    range:  %v\n", rangeByteHeader)

	getObjectInput := &s3.GetObjectInput{
		Bucket: aws.String(bucket),
		Key:    aws.String(key),
		Range:  aws.String(rangeByteHeader),
	}
	buffer := aws.NewWriteAtBuffer(make([]byte, contentLength))

	fmt.Println("downloader.Download ...")
	size, err := downloader.Download(buffer, getObjectInput)
	_ = size
	fmt.Printf("Err: %+v\n", err)

created time in 16 days

issue commenttonybaloney/pycharm-security

Update insecure.json regularly

Good to know. Thanks @tonybaloney

The URL option works for me. I am happy with this solution so feel free to close the ticket if appropriate.

charleswhchan

comment created time in 18 days

issue commentlocalstack/localstack

Simulate S3 BadDigest and InvalidDigest errors

Hi @whummer, thanks for looking into this.

What I am trying to do is write a test to write to S3 with localstack and produce conditions that would lead to BadDigest and InvalidDigest errors. This is so that I can check the error handling is working correctly in my code.

Based on my understanding of the AWS REST API. In the code you listed above, when "actual != expected", localstack should return BadDigest as the checksum does not match. Whereas if the content of "Context-MD5" is not a base64 encoded MD5, then the API should return InvalidDigest.

Does this help?

charleswhchan

comment created time in 19 days

issue openedtonybaloney/pycharm-security

Update insecure.json regularly

It looks like safety-db updates their database once a month. https://github.com/pyupio/safety-db/commits/master/data/insecure.json

Is it possible for this project to follow the same cadence? There were some false warnings that was fixed after Apr 14th release.

created time in 20 days

issue openedlocalstack/localstack

Simulate S3 BadDigest and InvalidDigest errors

Type of request: This is a ...

[ ] bug report [x] feature request

Detailed description

As a Developer, I want to be able to simulate BadDigest and InvalidDigest errors, So that I can test the error handling logic

Expected behavior

https://docs.aws.amazon.com/AmazonS3/latest/API/ErrorResponses.html BadDigest: The Content-MD5 you specified did not match what we received. InvalidDigest: The Content-MD5 you specified is not valid.

Actual behavior

localstack returns InvalidDigest when MD5 does not match payload.

created time in 20 days

delete branch charleswhchan/machinery

delete branch : fix-typo-in-config.go

delete time in a month

delete branch charleswhchan/machinery

delete branch : typo-task-status

delete time in a month

push eventcharleswhchan/machinery

Charles Chan

commit sha 3450bb4c17bc38569a9909fc9818f224446a4075

Fix typo in config.goRe-apply 342ef9fe

view details

push time in a month

PR opened RichardKnop/machinery

Fix typo in config.go

Re-apply 342ef9fe

+1 -1

0 comment

1 changed file

pr created time in a month

create barnchcharleswhchan/machinery

branch : fix-typo-in-config.go

created branch time in a month

issue commentRichardKnop/machinery

[Question] Calling Task without server

I see what you mean. Interesting concept. Unfortunately it is not something I had experience with.

Comparing the JSON format in the README against tasks.Signature, I can see newer changes in the struct. I suspect the README might not be the most up to date.

You might want to have a look at https://github.com/RichardKnop/machinery/blob/master/v1/brokers/amqp/amqp.go#L186. I can see it call json.Marshal(signature), so you could write a small golang server to publish a small task to print out the exact message + format being published to the queue.

tarunwadhwa13

comment created time in a month

create barnchcharleswhchan/machinery

branch : typo-task-status

created branch time in a month

issue commentRichardKnop/machinery

Question: I need to get timestamp for task completion time.

The easiest solution is to log a message in your task, or use TaskResult

siredwin

comment created time in a month

issue commentRichardKnop/machinery

Parameter passing between chain tasks does not match the document

Docs updated. See #339.

WisperDin

comment created time in a month

issue commentRichardKnop/machinery

[Question] Calling Task without server

If I understand your question: you want to be able to test your task without invoking queue/worker?

Since tasks are just defined as regular functions, they call be called directly as well to test/debug the implementation. This might help you isolate issues with Registration, Signatures, or Sending Tasks.

Using Add task as an example:

func Add(args ...int64) (int64, error) {
  sum := int64(0)
  for _, arg := range args {
    sum += arg
  }
  return sum, nil
}

You can call it directly:

result, err := Add(1, 2)
tarunwadhwa13

comment created time in a month

pull request commentdwavesystems/dwave-cloud-client

Add support for client-side certificate auth

Thanks! LGTM. You can leave this as a branch and we will test and leave a comment.

No rush to merge this PR until we know it's working e2e.

randomir

comment created time in a month

issue commentcelery/django-celery-beat

Multiple instances

You can add a lock to prevent multiple instances of beat from running in parallel.

Also see https://blog.heroku.com/redbeat-celery-beat-scheduler:

Finally, we added a simple lock that prevents multiple Beat daemons from running concurrently. This can sometimes be a problem for Heroku customers when they scale up from a single worker or during development.

eikebartels

comment created time in a month

issue commentRichardKnop/machinery

The log output should be customizable

@MerlinFeng : Did you see how to specify a custom logger in the README? https://github.com/RichardKnop/machinery#custom-logger

MerlinFeng

comment created time in a month

pull request commentRichardKnop/machinery

Fix 413 - (Redis) Simplify TTL

@RichardKnop : can you please close #413 as well? Thanks.

charleswhchan

comment created time in a month

delete branch charleswhchan/machinery

delete branch : fix-413

delete time in a month

delete branch charleswhchan/machinery

delete branch : fix-399

delete time in a month

PR opened RichardKnop/machinery

Fix 413 - (Redis) Simplify TTL

Set TTL for the key at the same time when setting the value

+26 -33

0 comment

2 changed files

pr created time in a month

create barnchcharleswhchan/machinery

branch : fix-413

created branch time in a month

PR opened RichardKnop/machinery

Fix #339 - Document argument passing behavior with chains

Clarify example in README

+6 -3

0 comment

1 changed file

pr created time in a month

push eventcharleswhchan/machinery

Charles Chan

commit sha be459e8bb3548e6f2ad9a006b91a8ccac9399a12

Fix #339 - Document argument passing behavior with chains

view details

push time in a month

push eventcharleswhchan/machinery

Charles Chan

commit sha a45fabae999215921518b8141ecbf503c40f929f

Fix #339 - Document argument passing behavior with chains

view details

push time in a month

create barnchcharleswhchan/machinery

branch : fix-399

created branch time in a month

push eventcharleswhchan/machinery

Julian Wachholz

commit sha d56568990fd1f6a98f6edb85ad30196bbd800ba0

Add AMQP queue declaration configuration options

view details

Julian Wachholz

commit sha aaf643692c8ed30e09702c8118a27aed91debc4f

Add priority field to tasks.Signature

view details

Patrik Cyvoct

commit sha 956b8d06941f0753c4fef86466f6e9351e719686

Add multiple amqp brokers Signed-off-by: Patrik Cyvoct <patrik@ptrk.io>

view details

Patrik Cyvoct

commit sha dc01f0e93c30b5529277648b1de7e35dda89283f

Add test for multiple amqp brokers Signed-off-by: Patrik Cyvoct <patrik@ptrk.io>

view details

mjarkk

commit sha e825abf93d526c7505ba3ec4e9cf8d8ebb6ca693

Added IgnoreWhenTaskNotRegistered option

view details

Wing Gao

commit sha b5602ccb7660e797bea8f06713f9c6197d32be10

feat: use go-redis

view details

Wing Gao

commit sha c4bfb83d3eb21ca1db5f0e95bc93548ceb994474

feat(redis): add config DelayedTasksKey

view details

Wing Gao

commit sha 2e61561703453a7f360d4bf475f2c6b238a67a20

feat(redis): add backend

view details

Wing Gao

commit sha e946a7371676719453d2f5091258f67287a664f0

feat(redis): add test for cluster

view details

Uzair Khan

commit sha 422c46e392c7a1179b25ebb44f110dd35a514c3a

add option to stop task deletion useful when we want to sqs to send messages to dlq on repeated consumption and processing failure of erroneous messages

view details

Charles Chan

commit sha 35ad717e24d2b90082e3e6eae5c92c73366539ec

Update Retry Tasks description

view details

Richard Knop

commit sha edf30ef54891777f86a10dc4a098ff4c3dbc5e06

Merge pull request #459 from WingGao/go-redis support redis cluster

view details

Richard Knop

commit sha 60cb100280f2d4004ab6cc4d4ee7fc3952a9d0a1

Merge branch 'master' into tread-no-handeler-as-failed-task

view details

Richard Knop

commit sha 3b244ca307d79354671ce09b35d0594693a5d3dc

fix: fixed a bug in redis backend factory

view details

Richard Knop

commit sha 2bcf3d9600540022e5a61de732951620e7b9aa12

Merge pull request #456 from mjarkk/tread-no-handeler-as-failed-task Proposal to add IgnoreWhenTaskNotRegistered option

view details

Richard Knop

commit sha 41b91b22e49439e8ca193b3247f39297237f6884

Merge branch 'master' into add-option-to-stop-task-deletion

view details

Richard Knop

commit sha 61b341685baf8c6566b2ab6e1681fbe3cb280c89

Merge pull request #472 from Uzair614/add-option-to-stop-task-deletion add option to stop task deletion

view details

Richard Knop

commit sha 985720f53d8d13548659095e57876d34f86f2749

Merge branch 'master' into multiple_aqmp

view details

Richard Knop

commit sha ee2b5403862d8b1f14b4de38f197b511a1f44640

Merge pull request #470 from charleswhchan/patch-1 Update Retry Tasks description

view details

Richard Knop

commit sha 0d7c6f56e875a6b2e995826aaccb2ac8c8f86bd6

Merge pull request #428 from julianwachholz/feature/queuedeclareargs Add AMQP queue declaration configuration options

view details

push time in a month

issue commentRichardKnop/machinery

Celery compatibility

No - Machinery is NOT compatible with Celery.

sashker

comment created time in a month

issue commentRichardKnop/machinery

Secret Information is logged when new worker is launched.

As the PR had been merged. Perhaps this ticket can be closed?

yolossn

comment created time in a month

issue commentRichardKnop/machinery

release v1.7.5 with better dynamodb support

Hi @roopakv, I see v1.7.5 had been released on Apr 20: https://github.com/RichardKnop/machinery/releases/tag/v1.7.5

Could you close this issue? :smiley:

roopakv

comment created time in a month

delete branch charleswhchan/machinery

delete branch : issue-529-redis-default-concurrency

delete time in a month

delete branch charleswhchan/machinery

delete branch : issue-548

delete time in a month

create barnchcharleswhchan/machinery

branch : issue-548

created branch time in a month

issue commentRichardKnop/machinery

Is it support periodic task?

Related to #521

lannyMa

comment created time in a month

issue commentRichardKnop/machinery

Support for periodic task

Also consider https://github.com/jasonlvhit/gocron for scheduling periodic tasks.

CatchZeng

comment created time in a month

PR opened RichardKnop/machinery

Fix #529 - Set Redis default concurrency when value is 0

According to the README:

Use the second parameter of server.NewWorker to limit the number of concurrently running worker.Process() calls (per worker). Example: 1 will serialize task execution while 0 makes the number of concurrently executed tasks unlimited (default).

This change will set the concurrency to 2 * num CPUs when concurrency = 0. While it isn't "unlimited", it is neverthe a practical choice for most workloads (mixed CPU, I/O) and better than 1. This change should make the behavior more consistent with the doc.

Should also fix #535

+4 -2

0 comment

2 changed files

pr created time in a month

push eventcharleswhchan/machinery

Charles Chan

commit sha 499547fabfc25ed1eba649fe754db0c02508c9f1

Fix #529 - Set Redis default concurrency when value is 0 According to the README: > Use the second parameter of `server.NewWorker` to limit the number of concurrently running > worker.Process() calls (per worker). Example: 1 will serialize task execution while 0 makes > the number of concurrently executed tasks unlimited (default). This change will set the concurrency to 2 * num CPUs when concurrency = 0. While it isn't "unlimited", it is neverthe a practical choice for most workloads (mixed CPU, I/O) and better than 1. This change should make the behavior more consistent with the doc. Should also fix #535

view details

push time in a month

push eventcharleswhchan/machinery

Charles Chan

commit sha 1aaf7098c62ddbfe59e9d8661ed4151d11cd0a7f

Fix #529 - Set Redis default concurrency when value is 0 According to the README: > Use the second parameter of `server.NewWorker` to limit the number of concurrently running > worker.Process() calls (per worker). Example: 1 will serialize task execution while 0 makes > the number of concurrently executed tasks unlimited (default). This change will set the concurrency to 2 * num CPUs when concurrency = 2. While it isn't "unlimited", it is neverthe a practical choice for most workloads (mixed CPU, I/O) and better than 1. This change should make the behavior more consistent with the doc. Should also fix #535

view details

push time in a month

create barnchcharleswhchan/machinery

branch : issue-529-redis-default-concurrency

created branch time in a month

delete branch charleswhchan/machinery

delete branch : improve-readme

delete time in 2 months

pull request commentRichardKnop/machinery

Improve README

Thank you. 👍

charleswhchan

comment created time in 2 months

delete branch charleswhchan/machinery

delete branch : rename-ErrCouldNotUnmarshaTaskSignature

delete time in 2 months

delete branch charleswhchan/machinery

delete branch : fix-typos-in-redis-comments

delete time in 2 months

PR opened RichardKnop/machinery

Improve README

Minor improvement to add hyperlink to the main configuration sections.

(Probably need to split the README into multiple files later)

+25 -4

0 comment

1 changed file

pr created time in 2 months

create barnchcharleswhchan/machinery

branch : improve-readme

created branch time in 2 months

create barnchcharleswhchan/machinery

branch : rename-ErrCouldNotUnmarshaTaskSignature

created branch time in 2 months

create barnchcharleswhchan/machinery

branch : fix-typos-in-redis-comments

created branch time in 2 months

issue openedRichardKnop/machinery

[Bug] RedisConfig.NormalTasksPollPeriod does not pick up tasks if value is less than 1 second

RedisConfig.NormalTasksPollPeriod is specified as milliseconds:

// NormalTasksPollPeriod specifies the period in milliseconds when polling redis for normal tasks
// Default: 1000
NormalTasksPollPeriod int `yaml:"normal_tasks_poll_period" envconfig:"REDIS_NORMAL_TASKS_POLL_PERIOD"`

I noticed when the value is set to <1 second, Machinery does not pick up any tasks. eg. when set to 20ms, using Redis WATCH command, I notice machinery will call `blpop "machinery_tasks" "0.02".

Reading the documentation for BLPOP:

The timeout argument is interpreted as an integer value specifying the maximum number of seconds to block. A timeout of zero can be used to block indefinitely.

I did not look into where the problem is, but I suspect it is either:

  1. Redis is expecting whole integer in seconds and is returning an error with "0.02"? (see goredis:go#L327, or redis.go#L346)
  2. Problem with the go-redis library?

Workaround: Set NormalTasksPollPeriod to 1000 ms.

If sub-second timeout is not NormalTasksPollPeriod supported, then there should be a clear error message when starting machinery worker.

created time in 2 months

startedbuger/jsonparser

started time in 2 months

startedvektra/mockery

started time in 2 months

starteddghubble/sling

started time in 2 months

issue openedRichardKnop/machinery

Document Eager backend?

I searching for a way write e2e tests with Machinery tasks, I came across the eager backend: https://github.com/RichardKnop/machinery/tree/master/v1/backends/eager

Tickets:

  • https://github.com/RichardKnop/machinery/pull/29
  • https://github.com/RichardKnop/machinery/pull/32
  • https://github.com/RichardKnop/machinery/pull/33

I think this is a great feature. However, it is not documented in the README file. So this is a request to add relevant information for users. 😃

created time in 2 months

startedpython/mypy

started time in 2 months

startedPyCQA/prospector

started time in 2 months

startedPyCQA/bandit

started time in 2 months

startedpsf/black

started time in 2 months

startedgoogle/yapf

started time in 2 months

startedhhatto/autopep8

started time in 2 months

delete branch charleswhchan/awesome-go

delete branch : add-goCron

delete time in 2 months

PR closed avelino/awesome-go

Add goCron to list of Job Schedulers pending-submitter-response quality standard

Please check if what you want to add to awesome-go list meets quality standards before sending pull request. Thanks!

Please provide package links to:

  • github.com repo: https://github.com/go-co-op/gocron
  • pkg.go.dev: https://pkg.go.dev/github.com/go-co-op/gocron?tab=doc
  • goreportcard.com: https://goreportcard.com/report/github.com/go-co-op/gocron
  • coverage service link (codecov, coveralls, gocover etc.)

Very good coverage

Note: that new categories can be added only when there are 3 packages or more.

Make sure that you've checked the boxes below before you submit PR:

  • [x] I have added my package in alphabetical order.
  • [x] I have an appropriate description with correct grammar.
  • [x] I know that this package was not listed before.
  • [x] I have added pkg.go.dev link to the repo and to my pull request.
  • [x] I have added coverage service link to the repo and to my pull request.
  • [x] I have added goreportcard link to the repo and to my pull request.
  • [x] I have read Contribution guidelines, maintainers note and Quality standard.

Thanks for your PR, you're awesome! :+1:

+1 -0

1 comment

1 changed file

charleswhchan

pr closed time in 2 months

startedtektoncd/pipeline

started time in 2 months

startedhibiken/asynq

started time in 2 months

issue commentgrpc/grpc

E0505 17:00:34.036852668 47929 chttp2_transport.cc:2880] ipv6:[::1]:50051: Keepalive watchdog fired. Closing transport.

For anyone having the same problem. I can confirm the settings in https://github.com/grpc/grpc-go/issues/2202 does not result in keepalive watchdog fired

charleswhchan

comment created time in 2 months

issue commentgrpc/grpc

E0505 17:00:34.036852668 47929 chttp2_transport.cc:2880] ipv6:[::1]:50051: Keepalive watchdog fired. Closing transport.

Also, could you please clarify the difference between the following options:

GRPC_ARG_KEEPALIVE_TIME_MS This channel argument controls the period (in milliseconds) after which a keepalive ping is sent on the transport.

and

GRPC_ARG_HTTP2_MIN_SENT_PING_INTERVAL_WITHOUT_DATA_MS If there are no data frames being received on the transport, this channel argument controls the minimum time (in milliseconds) gRPC Core will wait between successive pings.

Does GRPC_ARG_KEEPALIVE_TIME_MS send a keepalive ping regardless if there is data provided GRPC_ARG_HTTP2_MAX_PINGS_WITHOUT_DATA = true? Does that mean GRPC_ARG_HTTP2_MIN_SENT_PING_INTERVAL_WITHOUT_DATA_MS is only applicable when GRPC_ARG_HTTP2_MAX_PINGS_WITHOUT_DATA = false?

Thanks

charleswhchan

comment created time in 2 months

issue openedgrpc/grpc

E0505 17:00:34.036852668 47929 chttp2_transport.cc:2880] ipv6:[::1]:50051: Keepalive watchdog fired. Closing transport.

What version of gRPC and what language are you using?

Go (server): google.golang.org/grpc v1.27.1
Python (client): grpcio==1.27.2 

What operating system (Linux, Windows,...) and version?

Linux Ubuntu 18.04

# uname -a
4.15.0-99-generic #100-Ubuntu SMP Wed Apr 22 20:32:56 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

What did you do?

Go (server)

// MaxMessageSizeBytes 2GB to support large data size
const MaxMessageSizeBytes = 2097152000

// KeepalivePolicyMinTime The minimum time between keepalive pings. Client pings can not be more frequent than this.
const KeepalivePolicyMinTime = 5 * time.Second

// KeepalivePolicyPermitWithoutStream Permit keepalive pings even with no inflight RPCs.
const KeepalivePolicyPermitWithoutStream = true

// KeepaliveParamTime If the server doesn't see any activity after this time, it
// pings the client to see if the transport is still alive.
const KeepaliveParamTime = 5 * time.Second

// KeepaliveParamTimeOut After having pinged for keepalive check, the server waits for this duration and
// the connection is closed if no activity is seen after that.
const KeepaliveParamTimeOut = 5 * time.Second
...
...
	grpcListener, err := net.Listen(network, address)
	if err != nil {
		return nil, err
	}

	keepaliveParameters := keepalive.ServerParameters{
		Time:    KeepaliveParamTime,
		Timeout: KeepaliveParamTimeOut,
	}

	keepalivePolicy := keepalive.EnforcementPolicy{
		MinTime:             KeepalivePolicyMinTime,
		PermitWithoutStream: KeepalivePolicyPermitWithoutStream,
	}

	var (
		recoveryFunc grpc_recovery.RecoveryHandlerFunc
	)

	opts := []grpc_recovery.Option{
		grpc_recovery.WithRecoveryHandler(recoveryFunc),
	}

	grpcServer := grpc.NewServer(
		grpc_middleware.WithUnaryServerChain(
			grpc_recovery.UnaryServerInterceptor(opts...),
		),
		grpc_middleware.WithStreamServerChain(
			grpc_recovery.StreamServerInterceptor(opts...),
		),
		grpc.MaxRecvMsgSize(MaxMessageSizeBytes),
		grpc.KeepaliveEnforcementPolicy(keepalivePolicy),
		grpc.KeepaliveParams(keepaliveParameters),
	)

// followed by grpcServer.Serve

Python (client)

import grpc
import time

from my_pb
from my_grpc

GRPC_CHANNEL_OPTIONS = [
                        ('grpc.keepalive_time_ms', 5000),
                        ('grpc.keepalive_timeout_ms', 5000),
                        ('grpc.keepalive_permit_without_calls', True),
                        ('grpc.http2.max_pings_without_data', 0),
                        ('grpc.http2.min_time_between_pings_ms', 5000),
                        ('grpc.http2.min_ping_interval_without_data_ms', 5000),
                        ('grpc.max_send_message_length', 2000 * 1024 * 1024),
                        ('grpc.max_receive_message_length', 2000 * 1024 * 1024)]

if __name__ == "__main__":
    url = "localhost:50051"
    my_channels = [grpc.insecure_channel(url, options=GRPC_CHANNEL_OPTIONS) for i in range(10)]
    my_stubs = [my_grpc.MyStub(my_channels[i]) for i in range(10)]
    print(f"Setup grpc channel + stubs... {len(my_channels)}")

    # make a request on each connection
    for my_stub in my_stubs:
        try:
            request = my_pb.GetRequest()
            request.id = 'abc'
            response = my_stub.Get(request)
            print(response)
        except Exception as e:
            # print(e)
            pass

    # test: grpc should keepalive with the connection
    while True:
        print("Sleeping ...")
        time.sleep(10)

What did you expect to see?

Keepalive maintains the connection when no request/response flows through the channel. And I also do not expect to see network/connection/watchdog error received on the client.

What did you see instead?

Errors seen in the client. (No error on server.)

Setup grpc channel + stubs... 10
Sleeping ...
Sleeping ...
E0505 16:54:08.762435018   47441 chttp2_transport.cc:2893]   keepalive_ping_end state error: 0 (expect: 1)
Sleeping ...
E0505 16:54:18.762541095   47452 chttp2_transport.cc:2893]   keepalive_ping_end state error: 0 (expect: 1)
Sleeping ...
E0505 16:54:28.766357886   47452 chttp2_transport.cc:2893]   keepalive_ping_end state error: 0 (expect: 1)
Sleeping ...
E0505 16:54:38.765747109   47441 chttp2_transport.cc:2893]   keepalive_ping_end state error: 0 (expect: 1)
Sleeping ...
E0505 16:54:48.766456333   47452 chttp2_transport.cc:2880]   ipv6:[::1]:50051: Keepalive watchdog fired. Closing transport.
Sleeping ...
Sleeping ...
Setup grpc channel + stubs... 10
Sleeping ...
Sleeping ...
Sleeping ...
E0505 17:00:34.036852668   47929 chttp2_transport.cc:2880]   ipv6:[::1]:50051: Keepalive watchdog fired. Closing transport.
Sleeping ...
Sleeping ...
Sleeping ...
Sleeping ...

I've read https://github.com/grpc/grpc-go/issues/2202 and tried other configuration. However, I am still seeing the same error.

Go (server)

// KeepalivePolicyMinTime The minimum time between keepalive pings. Client pings can not be more frequent than this.
// const KeepalivePolicyMinTime = 5 * time.Second
const KeepalivePolicyMinTime = 10 * time.Second

// KeepalivePolicyPermitWithoutStream Permit keepalive pings even with no inflight RPCs.
const KeepalivePolicyPermitWithoutStream = true

// KeepaliveParamTime If the server doesn't see any activity after this time, it
// pings the client to see if the transport is still alive.
// const KeepaliveParamTime = 5 * time.Second
const KeepaliveParamTime = 10 * time.Second

// KeepaliveParamTimeOut After having pinged for keepalive check, the server waits for this duration and
// the connection is closed if no activity is seen after that.
const KeepaliveParamTimeOut = 5 * time.Second

Python (client)

GRPC_CHANNEL_OPTIONS = [
                        ('grpc.keepalive_time_ms', 10000),
                        ('grpc.keepalive_timeout_ms', 5000),
                        ('grpc.keepalive_permit_without_calls', True),
                        ('grpc.http2.max_pings_without_data', 0),
                        ('grpc.http2.min_time_between_pings_ms', 10000),
                        # ('grpc.http2.min_ping_interval_without_data_ms', 10000),  # N/A on client side
                        ('grpc.max_send_message_length', 2000 * 1024 * 1024),
                        ('grpc.max_receive_message_length', 2000 * 1024 * 1024)]

Could you please help me understand what is the correct way to configure these options so that I would not encounter the Keepalive watchdog fired. Closing transport error?

created time in 2 months

startedminio/minio

started time in 2 months

startedboto/boto3

started time in 2 months

startedBachmann1234/diff_cover

started time in 2 months

PR opened boto/boto3

Fix doc for S3 download_fileobj()

FileObj comes after Bucket & Key in the list of parameters. Reorder doc to match the same ordery.

+4 -4

0 comment

1 changed file

pr created time in 2 months

push eventcharleswhchan/boto3

Charles Chan

commit sha 73887e59661d40456dc3cb3b28cbf7357b0938ce

Fix doc for S3 download_fileobj() FileObj comes after Bucket & Key in the list of parameters. Reorder doc to match the same ordery.

view details

push time in 2 months

push eventcharleswhchan/boto3

Charles Chan

commit sha 866dd19f528aa57fe205d2b706ecdd6e58265dd6

Fix doc for S3 download_fileobj() FileObj comes after Bucket & Key in the list of parameters. Reorder doc to match the same ordery.

view details

push time in 2 months

create barnchcharleswhchan/boto3

branch : fix-s3-download_fileobj-doc

created branch time in 2 months

more