profile
viewpoint
If you are wondering where the data of this site comes from, please visit https://api.github.com/users/michaelklishin/events. GitMemory does not store any data, but only uses NGINX to cache data for a period of time. The idea behind GitMemory is simply to give users a better reading experience.
Michael Klishin michaelklishin novemberain.com Infrastructure, data services, distributed systems, and traveling the world.

jondot/sneakers 2128

A fast background processing framework for Ruby and RabbitMQ

aweber/rabbitmq-autocluster 338

This project is now maintained by the RabbitMQ Team, visit the official repo @

metrics-clojure/metrics-clojure 333

A thin façade around Coda Hale's metrics library.

jhalterman/lyra 264

High availability RabbitMQ client

michaelklishin/cassandra-chef-cookbook 162

Chef cookbook for Apache Cassandra, DataStax Enterprise (DSE) and DataStax agent

gotthardp/rabbitmq-email 141

SMTP Gateway Plugin for RabbitMQ

gildegoma/chef-android-sdk 27

Development repository for Android SDK Chef Cookbook

michaelklishin/chash 24

A yet another consistent hashing library for Clojure

gotthardp/rabbitmq-auth-backend-ip-range 18

RabbitMQ client authorization based on source IP address

Kyorai/cuttlefish 15

Erlang library to read sysctl-like configuration files

pull request commentrabbitmq/rabbitmq-server

Obfuscate credentials in shovel worker states to avoid plaintext pass…

Backported to v3.8.x.

thuandb

comment created time in a few seconds

pull request commentrabbitmq/rabbitmq-server

Obfuscate credentials in shovel worker states to avoid plaintext pass…

Backported to v3.9.x.

thuandb

comment created time in a few seconds

push eventrabbitmq/rabbitmq-server

Michael Klishin

commit sha 70b5917c834c903db1042fab6cc1e51a225f57a5

Merge branch 'shovel-obfuscatepassword' (cherry picked from commit 879e49ca7a3ac34687effcd56f783b11c06162e1)

view details

push time in 2 minutes

push eventrabbitmq/rabbitmq-server

Michael Klishin

commit sha 39c4d6c3ad52ac8070ea39a981546021f80ad691

Merge branch 'shovel-obfuscatepassword' (cherry picked from commit 879e49ca7a3ac34687effcd56f783b11c06162e1)

view details

push time in 2 minutes

delete branch rabbitmq/rabbitmq-server

delete branch : shovel-obfuscatepassword

delete time in 4 minutes

push eventrabbitmq/rabbitmq-server

Thuan Duong Ba

commit sha 6dbdc991c3111aa4ffa12a150b1402cf5c5e798e

Obfuscate credentials in shovel worker states to avoid plaintext passwords being logged on crashes

view details

Thuan Duong Ba

commit sha 7aedc45d1fe674e262fd6dda53c2530fe59be8cb

convert property map to proplists and vice versa for uri obfucation in shovel config

view details

Michael Klishin

commit sha 1c0904053386db3fc35c0e9a52d510d3896a1b48

Use rabbit_data_coercion functions here

view details

Michael Klishin

commit sha 59c4327f0b4285743fcce717173e260338fb15f4

Merge branch 'master' into shovel-obfuscatepassword

view details

Michael Klishin

commit sha 1cabd3ccd4c8ec9bad4cdf4dbb56e4d4a10afca5

Naming

view details

Michael Klishin

commit sha 879e49ca7a3ac34687effcd56f783b11c06162e1

Merge branch 'shovel-obfuscatepassword'

view details

push time in 4 minutes

PR merged rabbitmq/rabbitmq-server

Obfuscate credentials in shovel worker states to avoid plaintext pass…

…words being logged on crashes

When a shovel is configured incorrectly (either with incorrect username/password, or the targeted broker is down or unreachable), rabbit_shovel_worker crashes with its full state including URIs with plain text passwords being logged in the crash log, and sometimes also in the default log.

  • This is an example of a crash log having password logged in plaintext:
2021-08-04 02:22:23 =SUPERVISOR REPORT====
     Supervisor: {<0.740.0>,rabbit_shovel_dyn_worker_sup}
     Context:    child_terminated
     Reason:     shutdown
     Offender:   [{pid,<0.860.0>},{id,{<<"/">>,<<"notWorkingShovel">>}},{mfargs,{rabbit_shovel_worker,start_link,[dynamic,{<<"/">>,<<"notWorkingShovel">>},[{<<"ack-mode">>,<<"on-confirm">>},{<<"dest-add-forward-headers">>,false},{<<"dest-protocol">>,<<"amqp091">>},{<<"dest-uri">>,<<"amqp://">>},{<<"src-delete-after">>,<<"never">>},{<<"src-protocol">>,<<"amqp091">>},{<<"src-queue">>,<<"test-shovel-queue">>},{<<"src-uri">>,<<"amqp://user:pass@wronghost.com:5671">>}]]}},{restart_type,{permanent,5}},{shutdown,4294967295},{child_type,worker}]
  • When node is restarted, plain text password get logged in default log too:
2021-08-04 02:22:07.857 [info] <0.740.0> supervisor: {<0.740.0>,rabbit_shovel_dyn_worker_sup}, errorContext: child_terminated, reason: shutdown, offender: [{pid,<0.741.0>},{id,{<<"/">>,<<"notWorkingShovel">>}},{mfargs,{rabbit_shovel_worker,start_link,[dynamic,{<<"/">>,<<"notWorkingShovel">>},[{<<"ack-mode">>,<<"on-confirm">>},{<<"dest-add-forward-headers">>,false},{<<"dest-protocol">>,<<"amqp091">>},{<<"dest-uri">>,<<"amqp://">>},{<<"src-delete-after">>,<<"never">>},{<<"src-protocol">>,<<"amqp091">>},{<<"src-queue">>,<<"test-shovel-queue">>},{<<"src-uri">>,<<"amqp://user:pass@wronghost.com:5671">>}]]}},{restart_type,{permanent,5}},{shutdown,4294967295},{child_type,worker}]

The issue was reported in the past: https://github.com/rabbitmq/rabbitmq-server/issues/2709

Proposed Changes

The following change is an implementation to avoid plain text passwords being logged with the shovel workers' states upon crashing. Specifically, the change is to only store obfuscated URIs in the shovel workers' states and deobfuscate them when accessed. As a result, when shovel workers crash, the passwords will not be logged in plain text. The error logs from the shovel plugin will tell users what went wrong.

For example, when the targeted broker's DNS is not correct, the error message in default log indicates what the problem is:

2021-07-31 17:25:29.294574+00:00 [erro] <0.22741.5> Shovel 'notWorkingShovel' failed to connect (URI: amqp://wronghost.com:5671): unknown host (failed to resolve hostname)
2021-07-31 17:25:29.294677+00:00 [erro] <0.22741.5> Shovel 'notWorkingShovel' has no more URIs to try for connection
2021-07-31 17:25:29.294727+00:00 [erro] <0.22741.5> Shovel 'notWorkingShovel' could not connect to source

The crash log will not show the password in plain text:

2021-07-31 17:25:29.294843+00:00 [erro] <0.22740.5>     supervisor: {<0.22740.5>,rabbit_shovel_dyn_worker_sup}
2021-07-31 17:25:29.294843+00:00 [erro] <0.22740.5>     errorContext: child_terminated
2021-07-31 17:25:29.294843+00:00 [erro] <0.22740.5>     reason: shutdown
2021-07-31 17:25:29.294843+00:00 [erro] <0.22740.5>     offender: [{pid,<0.22741.5>},
2021-07-31 17:25:29.294843+00:00 [erro] <0.22740.5>                {id,{<<"/">>,<<"notWorkingShovel">>}},
2021-07-31 17:25:29.294843+00:00 [erro] <0.22740.5>                {mfargs,
2021-07-31 17:25:29.294843+00:00 [erro] <0.22740.5>                    {rabbit_shovel_worker,start_link,
2021-07-31 17:25:29.294843+00:00 [erro] <0.22740.5>                        [dynamic,
2021-07-31 17:25:29.294843+00:00 [erro] <0.22740.5>                         {<<"/">>,<<"notWorkingShovel">>},
2021-07-31 17:25:29.294843+00:00 [erro] <0.22740.5>                         [{<<"dest-uri">>,
2021-07-31 17:25:29.294843+00:00 [erro] <0.22740.5>                           [{encrypted,
2021-07-31 17:25:29.294843+00:00 [erro] <0.22740.5>                                <<"HtDNAVY31TtCO2I1UByk1OWwXn5AfSl/zouMBki3NG1nnAWxF3WpfEu7lmz//btl">>}]},
2021-07-31 17:25:29.294843+00:00 [erro] <0.22740.5>                          {<<"src-uri">>,
2021-07-31 17:25:29.294843+00:00 [erro] <0.22740.5>                           [{encrypted,
2021-07-31 17:25:29.294843+00:00 [erro] <0.22740.5>                                <<"m1B4OoxBTldD2Xo5VuDepCsfcmALH/mM61IuATxMBvS+MPJqxfUVfCtLh+ZCikouPmdGX1CkoOgVh+UIlmFN05ByuYsM3GmvcxjMjAvIvRo=">>}]},
2021-07-31 17:25:29.294843+00:00 [erro] <0.22740.5>                          {<<"ack-mode">>,<<"on-confirm">>},
2021-07-31 17:25:29.294843+00:00 [erro] <0.22740.5>                          {<<"dest-add-forward-headers">>,false},
2021-07-31 17:25:29.294843+00:00 [erro] <0.22740.5>                          {<<"dest-protocol">>,<<"amqp091">>},
2021-07-31 17:25:29.294843+00:00 [erro] <0.22740.5>                          {<<"src-delete-after">>,<<"never">>},
2021-07-31 17:25:29.294843+00:00 [erro] <0.22740.5>                          {<<"src-protocol">>,<<"amqp091">>},
2021-07-31 17:25:29.294843+00:00 [erro] <0.22740.5>                          {<<"src-queue">>,<<"test-shovel-queue">>}]]}},
2021-07-31 17:25:29.294843+00:00 [erro] <0.22740.5>                {restart_type,{permanent,5}},
2021-07-31 17:25:29.294843+00:00 [erro] <0.22740.5>                {shutdown,4294967295},
2021-07-31 17:25:29.294843+00:00 [erro] <0.22740.5>                {child_type,worker}]

Types of Changes

What types of changes does your code introduce to this project? Put an x in the boxes that apply

  • [x] Bug fix (non-breaking change which fixes issue #NNNN)
  • [ ] New feature (non-breaking change which adds functionality)
  • [ ] Breaking change (fix or feature that would cause an observable behavior change in existing systems)
  • [ ] Documentation improvements (corrections, new content, etc)
  • [ ] Cosmetic change (whitespace, formatting, etc)
  • [ ] Build system and/or CI

Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask on the mailing list. We're here to help! This is simply a reminder of what we are going to look for before merging your code.

  • [x] I have read the CONTRIBUTING.md document
  • [x] I have signed the CA (see https://cla.pivotal.io/sign/rabbitmq)
  • [x] I have added tests that prove my fix is effective or that my feature works
  • [x] All tests pass locally with my changes
  • [ ] If relevant, I have added necessary documentation to https://github.com/rabbitmq/rabbitmq-website
  • [ ] If relevant, I have added this change to the first version(s) in release-notes that I expect to introduce it

Further Comments

If this is a relatively large or complex change, kick off the discussion by explaining why you chose the solution you did and what alternatives you considered, etc.

+49 -13

0 comment

3 changed files

thuandb

pr closed time in 4 minutes

delete branch rabbitmq/rabbitmq-server

delete branch : mergify/bp/v3.9.x/pr-3474

delete time in 40 minutes

push eventrabbitmq/rabbitmq-server

Philip Kuryloski

commit sha 02f40d5db4b3f2a836356105aa48c12192c34741

Use 3.8.22 in 3.8/3.9 mixed version testing (cherry picked from commit d2b032ff60b361bfbb6019d5ada733fcc9af2eb5)

view details

Michael Klishin

commit sha c8778b64bf9b148fac4e2f3c146d8a479131e784

Merge pull request #3475 from rabbitmq/mergify/bp/v3.9.x/pr-3474 Use 3.8.22 in 3.8/3.9 mixed version testing (backport #3474)

view details

push time in 40 minutes

PR merged rabbitmq/rabbitmq-server

Use 3.8.22 in 3.8/3.9 mixed version testing (backport #3474) bazel

This is an automatic backport of pull request #3474 done by Mergify.


<details> <summary>Mergify commands and options</summary>

<br />

More conditions and actions can be found in the documentation.

You can also trigger Mergify actions by commenting on this pull request:

  • @Mergifyio refresh will re-evaluate the rules
  • @Mergifyio rebase will rebase this PR on its base branch
  • @Mergifyio update will merge the base branch into this PR
  • @Mergifyio backport <destination> will backport this PR on <destination> branch

Additionally, on Mergify dashboard you can:

  • look at your merge queues
  • generate the Mergify configuration with the config editor.

Finally, you can contact us on https://mergify.io/ </details>

+5 -5

0 comment

2 changed files

mergify[bot]

pr closed time in 40 minutes

create barnchrabbitmq/rabbitmq-server

branch : shovel-obfuscatepassword

created branch time in 41 minutes

delete branch rabbitmq/rabbitmq-server

delete branch : use-latest-release-in-mixed-version-testing

delete time in 43 minutes

push eventrabbitmq/rabbitmq-server

Philip Kuryloski

commit sha d2b032ff60b361bfbb6019d5ada733fcc9af2eb5

Use 3.8.22 in 3.8/3.9 mixed version testing

view details

Michael Klishin

commit sha 0d54e71e8e1ef1f5fcac577fc49f5aaad0c5edb1

Merge pull request #3474 from rabbitmq/use-latest-release-in-mixed-version-testing Use 3.8.22 in 3.8/3.9 mixed version testing

view details

push time in 43 minutes

PR merged rabbitmq/rabbitmq-server

Use 3.8.22 in 3.8/3.9 mixed version testing bazel backport-v3.9.x

Updates the patch version of 3.8 used when testing mixed version clusters

+5 -5

2 comments

2 changed files

pjk25

pr closed time in 44 minutes

pull request commentrabbitmq/rabbitmq-website

Add cluster size recommendations for performance and throughput

Yup, it was a typo, sorry. We cannot accept it.

dreiucker

comment created time in 44 minutes

push eventrabbitmq/rabbitmq-website

Michael Klishin

commit sha 3ea74992879ce62ab1b2ea81d87661d66f5f3741

A typo

view details

push time in an hour

push eventrabbitmq/rabbitmq-website

Michael Klishin

commit sha 490e6237bed1b55be22e8a1826cf6bc01bcd8e0d

3.9.6 is out

view details

Michael Klishin

commit sha b8aa31697945167cc8897231cdbcf07863dd68ca

Merge branch 'stable' into live

view details

push time in 2 hours

push eventrabbitmq/rabbitmq-website

Michael Klishin

commit sha 490e6237bed1b55be22e8a1826cf6bc01bcd8e0d

3.9.6 is out

view details

push time in 2 hours

push eventrabbitmq/rabbitmq-website

Aitor Perez Cedres

commit sha 7b17546e5cac791ab931371fec85e028510bdec7

Document x-namespace feature in Topology Operator Signed-off-by: Aitor Perez Cedres <acedres@vmware.com>

view details

Martin Markó

commit sha dad726e5f22a33683b0ad6b8f293cbd5d23e71bb

Changed link to a more relevant Wikipedia article

view details

David Ansari

commit sha 0371023d070d350b12a3e16413d0011e212b51d8

Require field publishNotReadyAddresses to be set in headless service for K8s peer discovery to work. Example: apiVersion: v1 kind: Service spec: publishNotReadyAddresses: true If NOT set, K8s peer discovery will determine the peers correctly (i.e. both ready and not-ready pods in https://github.com/rabbitmq/rabbitmq-server/blob/513a643e9123c7bb3b3059a127a103ad0d99c205/deps/rabbitmq_peer_discovery_k8s/src/rabbit_peer_discovery_k8s.erl#L79 ). However, when setting the lock in https://github.com/rabbitmq/rabbitmq-server/blob/513a643e9123c7bb3b3059a127a103ad0d99c205/deps/rabbitmq_peer_discovery_k8s/src/rabbit_peer_discovery_k8s.erl#L82 the headless service will route traffic to only ready pods resulting in the locking mechanism not to work with podManagementPolicy set to parallel. See also https://groups.google.com/g/rabbitmq-users/c/HgiBFyRqEks As specified in the K8s API field publishNotReadyAddresses must be set "to propagate SRV DNS records for its Pods for the purpose of peer discovery".

view details

Michael Klishin

commit sha 63ecf848ffae4bfb72a6e396dbc761ad6fcb6e9c

Merge pull request #1268 from rabbitmq/k8s-peer-discovery Require field publishNotReadyAddresses to be set

view details

Michael Klishin

commit sha 88e8e8fc679e6f0dbe8586102df8f92172084b20

Merge pull request #1267 from martin-marko/patch-1 Changed link to a more relevant Wikipedia article

view details

Michael Klishin

commit sha 0f31b78f2d181419cd5a11ca46d0ffaf2f979e8b

Merge pull request #1264 from Zerpet/top-op-x-namespace Document x-namespace feature in Topology Operator

view details

Michael Klishin

commit sha 5a50557203ce53af8bdc76171c06db6006bcd5b4

Edits to the (two) Stream guides

view details

GitHubPang

commit sha 39a80e37ef5acb86a14e4ece49dbdcc856a8f70b

Correct code example in .NET/C# Client API Guide See https://github.com/rabbitmq/rabbitmq-dotnet-client/blob/5ab37c795d8e333d84d245ed0e25e310c26feadb/projects/RabbitMQ.Client/client/api/ConnectionFactory.cs#L323

view details

Michael Klishin

commit sha 867b396a4d7a5b55826a319b8dab5365c6754e45

Merge pull request #1270 from GitHubPang/live Correct code example in .NET/C# Client API Guide

view details

Michael Klishin

commit sha 92c784847a04f23178b869c6f981ba4e96fbc943

Mention streams at the top of the Queues guide

view details

GitHubPang

commit sha 7ce41db3446ff799efd147c9e49af74dc90bb08c

Fix a few typos

view details

Michael Klishin

commit sha 43d2c43804e4f5ce268515371ced823ac6cb3895

Merge pull request #1271 from GitHubPang/live Fix a few typos

view details

push time in 2 hours

PR closed rabbitmq/rabbitmq-website

Add cluster size recommendations for performance and throughput

Purpose of this PR

Add specific cluster size recommendations regards performance and throughput look like

Reason for such a change

We are having lengthy discussions with stakeholders how a proper RabbitMQ cluster topology looks like. Since we assume that it would help our service as well as other RabbitMQ users if RabbitMQ could specific that area in the official RabbitMQ documentation instead of having it indirectly documented via several blog posts or deeper understanding of the cluster architecture.

Why a PR?

This is just a recommendation what we would add. And since it was mentioned in slack that PR > Issue I followed that guideline.

Slack discussion

https://rabbitmq.slack.com/archives/C1EDN83PA/p1630339373040200

+5 -0

1 comment

1 changed file

dreiucker

pr closed time in 3 hours

pull request commentrabbitmq/rabbitmq-website

Add cluster size recommendations for performance and throughput

Thank you for taking the time to contribute! My discussion with the team suggests we can accept this particular wording. There are aspects in which having double digit nodes would be problematic until we replace the schema data store. In other ways, it can be a valid approach.

We'd really want to avoid blanket recommendations on this very broad topic in general.

dreiucker

comment created time in 3 hours

startedrabbitmq/rabbitmq-server

started time in 4 hours

push eventrabbitmq/rabbitmq-server

Michael Klishin

commit sha ae4d84f403aeb1331e00ce50104d031c729ed1f1

(cherry picked from commit 7ce2a0dac810211015012354a5fd1cce4fb16afe) Conflicts: release-notes/3.9.6.md

view details

push time in 4 hours

pull request commentrabbitmq/rabbitmq-server

Increase classic queue shutdown timeout

Backported to v3.8.x.

lhoguin

comment created time in 4 hours

push eventrabbitmq/rabbitmq-server

Michael Klishin

commit sha 0e3adc574bacc5a067ef087b9584314173fbc5aa

Merge pull request #3409 from rabbitmq/lh-increase-queue-shutdown-timeout Increase classic queue shutdown timeout (cherry picked from commit 5fb118e8ef062b5169f139365cb0b036e9baf30e)

view details

push time in 4 hours

pull request commentrabbitmq/rabbitmq-server

Increase classic queue shutdown timeout

Backported to v3.9.x.

lhoguin

comment created time in 4 hours

delete branch rabbitmq/rabbitmq-server

delete branch : mergify/bp/v3.9.x/pr-3409

delete time in 4 hours

push eventrabbitmq/rabbitmq-server

Michael Klishin

commit sha 7ce2a0dac810211015012354a5fd1cce4fb16afe

#3409 was postponed for 3.9.7

view details

push time in 4 hours

PR closed rabbitmq/rabbitmq-server

Increase classic queue shutdown timeout (backport #3409) conflicts backport-v3.8.x

This is an automatic backport of pull request #3409 done by Mergify. Cherry-pick of 180ce5623a9a9a77bc93e41ebeed848eb6a96de9 has failed:

On branch mergify/bp/v3.9.x/pr-3409
Your branch is ahead of 'origin/v3.9.x' by 1 commit.
  (use "git push" to publish your local commits)

You are currently cherry-picking commit 180ce5623a.
  (fix conflicts and run "git cherry-pick --continue")
  (use "git cherry-pick --skip" to skip this patch)
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

Changes to be committed:
	modified:   release-notes/3.9.6.md

Unmerged paths:
  (use "git add/rm <file>..." as appropriate to mark resolution)
	deleted by us:   release-notes/3.8.23.md

To fix up this pull request, you can check it out locally. See documentation: https://docs.github.com/en/github/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/checking-out-pull-requests-locally


<details> <summary>Mergify commands and options</summary>

<br />

More conditions and actions can be found in the documentation.

You can also trigger Mergify actions by commenting on this pull request:

  • @Mergifyio refresh will re-evaluate the rules
  • @Mergifyio rebase will rebase this PR on its base branch
  • @Mergifyio update will merge the base branch into this PR
  • @Mergifyio backport <destination> will backport this PR on <destination> branch

Additionally, on Mergify dashboard you can:

  • look at your merge queues
  • generate the Mergify configuration with the config editor.

Finally, you can contact us on https://mergify.io/ </details>

+91 -7

0 comment

5 changed files

mergify[bot]

pr closed time in 4 hours

delete branch rabbitmq/rabbitmq-server

delete branch : lh-increase-queue-shutdown-timeout

delete time in 4 hours