profile
viewpoint
Sotirios Mantziaris mantzas Athens, Greece http://blog.mantziaris.eu https://gr.linkedin.com/in/sotirismantziaris

Pull request review commentbeatlabs/patron

HTTP Component Builder

 func (c *Component) createHTTPServer() *http.Server { 		Handler:      routerAfterMiddleware, 	} }++// propSetMsg is used to log successes when using+// the HTTP Builder setters, in a uniform way+const propSetMsg = "Setting property '%v' for '%v'"++// Builder gathers all required and optional properties, in order+// to construct an HTTP component+type Builder struct {+	Component

We have 2 options here:

  • Leave it as is, which allows the developer to construct the component without using the builder, which defeats the purpose to have a builder
  • Return the component interface and make the component unexported, which violates the "accept interfaces, return structs" guideline I am ok with going with the first option...
tpaschalis

comment created time in 11 hours

Pull request review commentbeatlabs/patron

HTTP Component Builder

 func (c *Component) createHTTPServer() *http.Server { 		Handler:      routerAfterMiddleware, 	} }++// propSetMsg is used to log successes when using+// the HTTP Builder setters, in a uniform way+const propSetMsg = "Setting property '%v' for '%v'"++// Builder gathers all required and optional properties, in order+// to construct an HTTP component

All comments should end with a punctuation. https://github.com/golang/go/wiki/CodeReviewComments

tpaschalis

comment created time in 2 days

Pull request review commentbeatlabs/patron

HTTP Component Builder

+package http++import (+	"time"++	"github.com/beatlabs/patron/errors"+)++// Data used for testing the HTTP Component builder++var httpBuilderNoErrors = []error{}+var httpBuilderAllErrors = []error{+	errors.New("Nil AliveCheckFunc was provided"),+	errors.New("Nil ReadyCheckFunc provided"),+	errors.New("Invalid HTTP Port provided"),+	errors.New("Negative or zero read timeout provided"),+	errors.New("Negative or zero write timeout provided"),+	errors.New("Empty Routes slice provided"),+	errors.New("Empty list of middlewares provided"),+	errors.New("Invalid cert or key provided"),+}++type builderTestcase struct {

Check out the std way of doing table driven tests... e.g. https://github.com/taxibeat/rems/blob/develop/pkg/microrule/microrule_test.go

tpaschalis

comment created time in 2 days

Pull request review commentbeatlabs/patron

HTTP Component Builder

 func (c *Component) createHTTPServer() *http.Server { 		Handler:      routerAfterMiddleware, 	} }++// propSetMsg is used to log successes when using

Maybe use a more descriptive name in order to avoid the comment.

tpaschalis

comment created time in 2 days

Pull request review commentbeatlabs/patron

HTTP Component Builder

 func (c *Component) createHTTPServer() *http.Server { 		Handler:      routerAfterMiddleware, 	} }++// propSetMsg is used to log successes when using+// the HTTP Builder setters, in a uniform way+const propSetMsg = "Setting property '%v' for '%v'"++// Builder gathers all required and optional properties, in order+// to construct an HTTP component+type Builder struct {+	Component

Since we have a builder now we can unexport the component. I would not use structure embedding here since the builder "inherits" all fields and functions...

tpaschalis

comment created time in 2 days

push eventbeatlabs/patron

George Papadopoulos

commit sha 9b7a1fcd0a993c448f6205231e83b120de3c23a7

Introduce logging on HTTP requests (#81) Signed-off-by: George Papadopoulos <g.papadopoulos@thebeat.co>

view details

Sotirios Mantziaris

commit sha 5110a9f77f086bf4ae662d22b0c7fbc95d885959

User context logger in async failure Signed-off-by: Sotirios Mantziaris <smantziaris@gmail.com>

view details

Sotirios Mantziaris

commit sha 39c060c7a67268a47eada67d216c774191c4f21b

User context logger Signed-off-by: Sotirios Mantziaris <smantziaris@gmail.com>

view details

Sotirios Mantziaris

commit sha 2342683d30535be76898dd2b533e8167fc9e5597

Added correlation const Signed-off-by: Sotirios Mantziaris <smantziaris@gmail.com>

view details

Sotirios Mantziaris

commit sha 8d4f624c35c648810ac02a59d743d4867efd5f33

Added correlation to http header Signed-off-by: Sotirios Mantziaris <smantziaris@gmail.com>

view details

push time in 2 days

push eventbeatlabs/patron

George Papadopoulos

commit sha 9b7a1fcd0a993c448f6205231e83b120de3c23a7

Introduce logging on HTTP requests (#81) Signed-off-by: George Papadopoulos <g.papadopoulos@thebeat.co>

view details

push time in 3 days

PR merged beatlabs/patron

Reviewers
Introduce logging on HTTP requests

<!-- Thanks for taking precious time for making a PR.

Before creating a pull request, please make sure:

  • Your PR solves one problem for which a issue exist and a solution has been discussed
  • You have read the guide for contributing
    • See https://github.com/beatlabs/patron/blob/master/CONTRIBUTING.md
  • You signed all your commits (otherwise we won't be able to merge the PR)
    • See https://github.com/beatlabs/patron/blob/master/CONTRIBUTING.md#sign-your-work
  • You added unit tests for the new functionality
  • You mention in the PR description which issue it is addressing, e.g. "Resolves #123" -->

Which problem is this PR solving?

This PR closes #80

Short description of the changes

  • Introduce log.Enabled()
  • Rename NewTracingMiddleware to NewLoggingTracingMiddleware
  • Log HTTP requests with the help of responseWriter in a form usable by ELK.
  • Log requests only when debugging log level is enabled for performance reasons.
+117 -9

1 comment

7 changed files

gpapadopg

pr closed time in 3 days

issue closedbeatlabs/patron

Logging of HTTP request/response

<!-- Welcome to the patron framework project.

  • Please search for existing issues to avoid creating duplicate bugs/feature requests.
  • Please be respectful and considerate of others when commenting on issues.
  • Please provide as much information as possible so we all understand the issue.

-->

Is your feature request related to a problem? Please describe

<!-- REQUIRED A clear and concise description of what the problem is. --> Patron does not log HTTP requests like web servers do. Think of something like Apache or Nginx access.log where each HTTP request is logged in a standard log format.

Is this a bug? Please provide steps to reproduce, a failing test etc.

<!-- REQUIRED A clear and concise way to reproduce the bug. --> This is not a bug but a nice to have when developing an application.

Describe the solution

<!-- OPTIONAL A clear and concise description of what you want to happen. --> A solution would be to log each HTTP request/response information (for example Client IP, Time of Request, HTTP Status Code, Bytes Sent, etc) from some middleware. It seems this functionality can be incorporated in NewTracingMiddleware.

Additional context

<!-- OPTIONAL Add any other context or screenshots about the feature request here. -->

closed time in 3 days

gpapadopg

pull request commentbeatlabs/patron

HTTP Component Builder

Things that I'm not 100% clear on, and might require a quick chat.

1. Is the current testing scope wide enough? Or should I include more test cases, where the component is actually being used, and acts like it should?

2. Should the builder have some logic, where it aborts under some conditions? Or is this part of the application logic later on?

3. On the other hand, should I include some default values eg. a default `httpReadTimeout` in case that its builder function has not been used, or is used without arguments. (Personally, not really a fan).
  1. Testing should be as close to 100% coverage in unit tests
  2. All the logic should live in the component that the builder creates.
  3. We should use the existing defaults but provider overrrides with the builder.

Hope this helps...

tpaschalis

comment created time in 3 days

push eventbeatlabs/patron

Sotirios Mantziaris

commit sha 05073e06e3b19d211a52c6ec7c8c72cd45c56cd4

Added correlation to http header Signed-off-by: Sotirios Mantziaris <smantziaris@gmail.com>

view details

push time in 4 days

push eventbeatlabs/patron

Sotirios Mantziaris

commit sha 5dfd66080ed27f4b8bb0cbaadae72226be7e10c5

Added correlation const Signed-off-by: Sotirios Mantziaris <smantziaris@gmail.com>

view details

push time in 5 days

push eventbeatlabs/patron

Sotirios Mantziaris

commit sha 455908544216baa61eed439ead481833d6460995

User context logger Signed-off-by: Sotirios Mantziaris <smantziaris@gmail.com>

view details

push time in 6 days

PR opened beatlabs/patron

Reviewers
Using context logger

Which problem is this PR solving?

This PR closes #113

Short description of the changes

  • Replaced usages of logger with context logger where appropriate.
+1 -1

0 comment

1 changed file

pr created time in 6 days

push eventbeatlabs/patron

Sotirios Mantziaris

commit sha d0d5387f59226821628d962670725064a0b9d5ef

User context logger in async failure Signed-off-by: Sotirios Mantziaris <smantziaris@gmail.com>

view details

push time in 6 days

create barnchbeatlabs/patron

branch : use-context-logger

created branch time in 6 days

issue openedbeatlabs/patron

Logging does not use the context logger

Is your feature request related to a problem? Please describe

We introduced correlation ids in the logger to make debugging easier.

Is this a bug? Please provide steps to reproduce, a failing test etc.

Async component.go:152 uses a logger without using it from context. As a result there is no correlation id in the log message that can help.

Describe the solution

Use the context logger inside the component to have the correlation id in the logs.

created time in 6 days

issue openedbeatlabs/patron

Sql Client tracing does not populate the db name and the user

Is your feature request related to a problem? Please describe

When opening a DB we do not populate the user and db name. We should get from the DSN the username and the db name and create the proper DB. Check out sql.go line 114 and 123.

Is this a bug? Please provide steps to reproduce, a failing test etc.

Just use the traced sql client.

Describe the solution

Extract the user and db name.

created time in 11 days

Pull request review commentbeatlabs/patron

Async comp builder

 type Component struct { 	retryWait    time.Duration } -// New returns a new async component. The default behavior is to return a error of failure.-// Use options to change the default behavior.-func New(name string, p ProcessorFunc, cf ConsumerFactory, oo ...OptionFunc) (*Component, error) {+// Builder gathers all required properties in order to construct a component+type Builder struct {+	errors       []error+	name         string+	proc         ProcessorFunc+	failStrategy FailStrategy+	cf           ConsumerFactory+	retries      uint+	retryWait    time.Duration+} +// New initializes a new builder for a component with the given name+// by default the failStrategy will be NackExitStrategy+func New(name string) *Builder {+	var errs []error 	if name == "" {-		return nil, errors.New("name is required")+		errs = append(errs, errors.New("name is required")) 	}+	return &Builder{+		name:   name,+		errors: errs,+	}+} -	if p == nil {-		return nil, errors.New("work processor is required")+// WithProcessor sets the processing logic for the component+// it will append an error to the builder if the processor is nil+func (cb *Builder) WithProcessor(proc ProcessorFunc) *Builder {

Since processor and consumerfactory are required, they should move to the constructor. These are bare minimum things in order to create a async component.

drakos74

comment created time in 14 days

Pull request review commentbeatlabs/patron

Async comp builder

 type Component struct { 	retryWait    time.Duration } -// New returns a new async component. The default behavior is to return a error of failure.-// Use options to change the default behavior.-func New(name string, p ProcessorFunc, cf ConsumerFactory, oo ...OptionFunc) (*Component, error) {+// Builder gathers all required properties in order to construct a component+type Builder struct {+	errors       []error+	name         string+	proc         ProcessorFunc+	failStrategy FailStrategy+	cf           ConsumerFactory+	retries      uint+	retryWait    time.Duration+} +// New initializes a new builder for a component with the given name+// by default the failStrategy will be NackExitStrategy+func New(name string) *Builder {+	var errs []error 	if name == "" {-		return nil, errors.New("name is required")+		errs = append(errs, errors.New("name is required")) 	}+	return &Builder{+		name:   name,+		errors: errs,+	}+} -	if p == nil {-		return nil, errors.New("work processor is required")+// WithProcessor sets the processing logic for the component+// it will append an error to the builder if the processor is nil+func (cb *Builder) WithProcessor(proc ProcessorFunc) *Builder {+	if proc == nil {+		cb.errors = append(cb.errors, errors.New("work processor is required"))+	} else {+		log.Infof(propSetMSG, "processor func", cb.name)+		cb.proc = proc 	}+	return cb+} +// WithConsumerFactory defines the consumer factory to be used in order to create the appropriate consumer+// it will append an error to the builder if the factory is nil+func (cb *Builder) WithConsumerFactory(cf ConsumerFactory) *Builder { 	if cf == nil {-		return nil, errors.New("consumer is required")+		cb.errors = append(cb.errors, errors.New("consumer is required"))+	} else {+		log.Infof(propSetMSG, "consumer factory", cb.name)+		cb.cf = cf 	}+	return cb+} -	c := &Component{-		name:         name,-		proc:         p,-		cf:           cf,-		failStrategy: NackExitStrategy,-		retries:      0,-		retryWait:    0,+// WithFailureStrategy defines the failure strategy to be used+// default value is NackExitStrategy+// it will append an error to the builder if the strategy is not one of the pre-defined ones+func (cb *Builder) WithFailureStrategy(fs FailStrategy) *Builder {+	if _, ok := failStrategyValues[fs]; ok {

As I understand it, we are using the map to check if we support the failurestrategy or if and invalid value has been passed in. If that is the reason we should do a simple switch statement to see if the value is acceptable or not, and avoid having an map allocated

drakos74

comment created time in 14 days

Pull request review commentbeatlabs/patron

Async comp builder

 package async  import ( 	"context"+	"fmt" 	"time"  	"github.com/beatlabs/patron/errors" 	"github.com/beatlabs/patron/log" 	"github.com/prometheus/client_golang/prometheus" ) +var propSetMSG = "property '%s' set for '%s'"

could this be a const?

drakos74

comment created time in 14 days

Pull request review commentbeatlabs/patron

Async comp builder

 func (c *Component) processMessage(msg Message, ch chan error) { 	} } +var errInvalidFS = errors.New("invalid failure strategy")+var failureStrategyErrorMSG = "%s failed when executing failure strategy"

could this be a constant?

drakos74

comment created time in 14 days

Pull request review commentbeatlabs/patron

Async comp builder

 type Component struct { 	retryWait    time.Duration } -// New returns a new async component. The default behavior is to return a error of failure.-// Use options to change the default behavior.-func New(name string, p ProcessorFunc, cf ConsumerFactory, oo ...OptionFunc) (*Component, error) {+// Builder gathers all required properties in order to construct a component+type Builder struct {+	errors       []error+	name         string+	proc         ProcessorFunc+	failStrategy FailStrategy+	cf           ConsumerFactory+	retries      uint+	retryWait    time.Duration+} +// New initializes a new builder for a component with the given name+// by default the failStrategy will be NackExitStrategy+func New(name string) *Builder {+	var errs []error 	if name == "" {-		return nil, errors.New("name is required")+		errs = append(errs, errors.New("name is required")) 	}+	return &Builder{+		name:   name,+		errors: errs,+	}+} -	if p == nil {-		return nil, errors.New("work processor is required")+// WithProcessor sets the processing logic for the component+// it will append an error to the builder if the processor is nil+func (cb *Builder) WithProcessor(proc ProcessorFunc) *Builder {+	if proc == nil {+		cb.errors = append(cb.errors, errors.New("work processor is required"))+	} else {+		log.Infof(propSetMSG, "processor func", cb.name)+		cb.proc = proc 	}+	return cb+} +// WithConsumerFactory defines the consumer factory to be used in order to create the appropriate consumer+// it will append an error to the builder if the factory is nil+func (cb *Builder) WithConsumerFactory(cf ConsumerFactory) *Builder { 	if cf == nil {-		return nil, errors.New("consumer is required")+		cb.errors = append(cb.errors, errors.New("consumer is required"))+	} else {+		log.Infof(propSetMSG, "consumer factory", cb.name)+		cb.cf = cf 	}+	return cb+} -	c := &Component{-		name:         name,-		proc:         p,-		cf:           cf,-		failStrategy: NackExitStrategy,-		retries:      0,-		retryWait:    0,+// WithFailureStrategy defines the failure strategy to be used+// default value is NackExitStrategy+// it will append an error to the builder if the strategy is not one of the pre-defined ones+func (cb *Builder) WithFailureStrategy(fs FailStrategy) *Builder {+	if _, ok := failStrategyValues[fs]; ok {+		log.Infof(propSetMSG, "failure strategy", cb.name)+		cb.failStrategy = fs+	} else {+		cb.errors = append(cb.errors, errors.New("invalid strategy provided")) 	}+	return cb+} -	for _, o := range oo {-		err := o(c)-		if err != nil {-			return nil, err-		}+// WithRetries specifies the retry events number for the component+// default value is '0'+func (cb *Builder) WithRetries(retries uint) *Builder {+	log.Infof(propSetMSG, "retries", cb.name)+	cb.retries = retries+	return cb+}++// WithRetryWait specifies the duration for the component to wait between retries+// default value is '0'+// it will append an error to the builder if the value is smaller than '0'
// it will append an error to the builder if the value is smaller than '0'.
drakos74

comment created time in 14 days

Pull request review commentbeatlabs/patron

Async comp builder

 type Component struct { 	retryWait    time.Duration } -// New returns a new async component. The default behavior is to return a error of failure.-// Use options to change the default behavior.-func New(name string, p ProcessorFunc, cf ConsumerFactory, oo ...OptionFunc) (*Component, error) {+// Builder gathers all required properties in order to construct a component+type Builder struct {+	errors       []error+	name         string+	proc         ProcessorFunc+	failStrategy FailStrategy+	cf           ConsumerFactory+	retries      uint+	retryWait    time.Duration+} +// New initializes a new builder for a component with the given name+// by default the failStrategy will be NackExitStrategy+func New(name string) *Builder {+	var errs []error 	if name == "" {-		return nil, errors.New("name is required")+		errs = append(errs, errors.New("name is required")) 	}+	return &Builder{+		name:   name,+		errors: errs,+	}+} -	if p == nil {-		return nil, errors.New("work processor is required")+// WithProcessor sets the processing logic for the component+// it will append an error to the builder if the processor is nil+func (cb *Builder) WithProcessor(proc ProcessorFunc) *Builder {+	if proc == nil {+		cb.errors = append(cb.errors, errors.New("work processor is required"))+	} else {+		log.Infof(propSetMSG, "processor func", cb.name)+		cb.proc = proc 	}+	return cb+} +// WithConsumerFactory defines the consumer factory to be used in order to create the appropriate consumer+// it will append an error to the builder if the factory is nil+func (cb *Builder) WithConsumerFactory(cf ConsumerFactory) *Builder { 	if cf == nil {-		return nil, errors.New("consumer is required")+		cb.errors = append(cb.errors, errors.New("consumer is required"))+	} else {+		log.Infof(propSetMSG, "consumer factory", cb.name)+		cb.cf = cf 	}+	return cb+} -	c := &Component{-		name:         name,-		proc:         p,-		cf:           cf,-		failStrategy: NackExitStrategy,-		retries:      0,-		retryWait:    0,+// WithFailureStrategy defines the failure strategy to be used+// default value is NackExitStrategy+// it will append an error to the builder if the strategy is not one of the pre-defined ones+func (cb *Builder) WithFailureStrategy(fs FailStrategy) *Builder {+	if _, ok := failStrategyValues[fs]; ok {+		log.Infof(propSetMSG, "failure strategy", cb.name)+		cb.failStrategy = fs+	} else {+		cb.errors = append(cb.errors, errors.New("invalid strategy provided")) 	}+	return cb+} -	for _, o := range oo {-		err := o(c)-		if err != nil {-			return nil, err-		}+// WithRetries specifies the retry events number for the component+// default value is '0'
// default value is '0'.
drakos74

comment created time in 14 days

Pull request review commentbeatlabs/patron

Async comp builder

 type Component struct { 	retryWait    time.Duration } -// New returns a new async component. The default behavior is to return a error of failure.-// Use options to change the default behavior.-func New(name string, p ProcessorFunc, cf ConsumerFactory, oo ...OptionFunc) (*Component, error) {+// Builder gathers all required properties in order to construct a component+type Builder struct {+	errors       []error+	name         string+	proc         ProcessorFunc+	failStrategy FailStrategy+	cf           ConsumerFactory+	retries      uint+	retryWait    time.Duration+} +// New initializes a new builder for a component with the given name+// by default the failStrategy will be NackExitStrategy+func New(name string) *Builder {+	var errs []error 	if name == "" {-		return nil, errors.New("name is required")+		errs = append(errs, errors.New("name is required")) 	}+	return &Builder{+		name:   name,+		errors: errs,+	}+} -	if p == nil {-		return nil, errors.New("work processor is required")+// WithProcessor sets the processing logic for the component+// it will append an error to the builder if the processor is nil+func (cb *Builder) WithProcessor(proc ProcessorFunc) *Builder {+	if proc == nil {+		cb.errors = append(cb.errors, errors.New("work processor is required"))+	} else {+		log.Infof(propSetMSG, "processor func", cb.name)+		cb.proc = proc 	}+	return cb+} +// WithConsumerFactory defines the consumer factory to be used in order to create the appropriate consumer+// it will append an error to the builder if the factory is nil+func (cb *Builder) WithConsumerFactory(cf ConsumerFactory) *Builder { 	if cf == nil {-		return nil, errors.New("consumer is required")+		cb.errors = append(cb.errors, errors.New("consumer is required"))+	} else {+		log.Infof(propSetMSG, "consumer factory", cb.name)+		cb.cf = cf 	}+	return cb+} -	c := &Component{-		name:         name,-		proc:         p,-		cf:           cf,-		failStrategy: NackExitStrategy,-		retries:      0,-		retryWait:    0,+// WithFailureStrategy defines the failure strategy to be used+// default value is NackExitStrategy+// it will append an error to the builder if the strategy is not one of the pre-defined ones
// it will append an error to the builder if the strategy is not one of the pre-defined ones.
drakos74

comment created time in 14 days

Pull request review commentbeatlabs/patron

Async comp builder

 type Component struct { 	retryWait    time.Duration } -// New returns a new async component. The default behavior is to return a error of failure.-// Use options to change the default behavior.-func New(name string, p ProcessorFunc, cf ConsumerFactory, oo ...OptionFunc) (*Component, error) {+// Builder gathers all required properties in order to construct a component+type Builder struct {+	errors       []error+	name         string+	proc         ProcessorFunc+	failStrategy FailStrategy+	cf           ConsumerFactory+	retries      uint+	retryWait    time.Duration+} +// New initializes a new builder for a component with the given name+// by default the failStrategy will be NackExitStrategy+func New(name string) *Builder {+	var errs []error 	if name == "" {-		return nil, errors.New("name is required")+		errs = append(errs, errors.New("name is required")) 	}+	return &Builder{+		name:   name,+		errors: errs,+	}+} -	if p == nil {-		return nil, errors.New("work processor is required")+// WithProcessor sets the processing logic for the component+// it will append an error to the builder if the processor is nil+func (cb *Builder) WithProcessor(proc ProcessorFunc) *Builder {+	if proc == nil {+		cb.errors = append(cb.errors, errors.New("work processor is required"))+	} else {+		log.Infof(propSetMSG, "processor func", cb.name)+		cb.proc = proc 	}+	return cb+} +// WithConsumerFactory defines the consumer factory to be used in order to create the appropriate consumer+// it will append an error to the builder if the factory is nil+func (cb *Builder) WithConsumerFactory(cf ConsumerFactory) *Builder {

Like mentioned above this should go to the constructor.

drakos74

comment created time in 14 days

Pull request review commentbeatlabs/patron

Async comp builder

 type Component struct { 	retryWait    time.Duration } -// New returns a new async component. The default behavior is to return a error of failure.-// Use options to change the default behavior.-func New(name string, p ProcessorFunc, cf ConsumerFactory, oo ...OptionFunc) (*Component, error) {+// Builder gathers all required properties in order to construct a component+type Builder struct {+	errors       []error+	name         string+	proc         ProcessorFunc+	failStrategy FailStrategy+	cf           ConsumerFactory+	retries      uint+	retryWait    time.Duration+} +// New initializes a new builder for a component with the given name+// by default the failStrategy will be NackExitStrategy+func New(name string) *Builder {+	var errs []error 	if name == "" {-		return nil, errors.New("name is required")+		errs = append(errs, errors.New("name is required")) 	}+	return &Builder{+		name:   name,+		errors: errs,+	}+} -	if p == nil {-		return nil, errors.New("work processor is required")+// WithProcessor sets the processing logic for the component+// it will append an error to the builder if the processor is nil

all comments should end with a period.

// it will append an error to the builder if the processor is nil.
drakos74

comment created time in 14 days

push eventbeatlabs/patron

Sotirios Mantziaris

commit sha 564c950341184aaed79ddaa54fbe74b09bca2b64

CR fixes Signed-off-by: Sotirios Mantziaris <smantziaris@gmail.com>

view details

push time in 14 days

Pull request review commentbeatlabs/patron

Service Builder

 import ( 	"context"

we could!

mantzas

comment created time in 14 days

push eventdrakos74/patron

Sotirios Mantziaris

commit sha 6206b85cd641c3673379bfeb32791bb177bda901

Logging correlation support (#105) Signed-off-by: Sotirios Mantziaris <smantziaris@gmail.com>

view details

Sotirios Mantziaris

commit sha b090fbaa4a069803ab89b88d3406c22b43e32a81

Fixed tracing documentation (#96)

view details

Sotirios Mantziaris

commit sha f523ff659873925d29adf8e8e86152798510ee55

Introduce Make file and linting (#108) Signed-off-by: Sotirios Mantziaris <smantziaris@gmail.com>

view details

William

commit sha 9b450e83ff92f213aad69f1b936e27084001cbf7

Update doc to add more info about the liveness and readiness HTTP endpoints (#99) Signed-off-by: William Claude <w.claude@thebeat.co>

view details

Sotirios Mantziaris

commit sha efb7203856377efa3178d77a11770cd6fcb5d3f3

Merge branch 'master' into async-comp-builder

view details

push time in 16 days

push eventgpapadopg/patron

Christos Petropoulos

commit sha 02bb6c078c139a491c75cab340301d82ba6f2350

Update Producer Message to include key, closes #85 (#86)

view details

Vassilis Boliastis

commit sha 0d2010ee306f73e7ec92946f7c1c70e156f15df6

Add elasticsearch integration. (#82)

view details

Mohamed Osama

commit sha b0179d71755264a79e1dffb6bd3672f98bef7b12

Fix: Kafka initializer in patron cli (#89) Signed-off-by: Mohamed Osama <mohamed.o.alnagdy@gmail.com>

view details

George Kechagias

commit sha 097f979b840ad6dc21ee8c6a6f966150e120f45e

Readme Fixes (#87) Signed-off-by: gkech <geo.kechagias@gmail.com>

view details

Alexey Zarakovskiy

commit sha 233cb239232bc54b5101f3e6d099a95421609be5

Patron CLI: Added generation of a .gitignore (#90)

view details

Christos Papidas

commit sha ab07780ff1fcfbd35e1417b299fdb40d067b39b8

Remove concurrency from consumers on message arrived (#95)

view details

Angelos Roussakis

commit sha ef20a4bacad0c65e15c7a36ac7ea060a242b1390

Introducing Alive/Ready endpoint (#93) Signed-off-by: Angelos Roussakis <aggelos.roussakis@gmail.com>

view details

Vangelis Katikaridis

commit sha cbfecb514b43be9495e516e21d771df5069cffe3

Decoder injection integration (#101)

view details

Sotirios Mantziaris

commit sha 15d01ae3785a159383beed79561bb79ae23c4121

Update CODEOWNERS

view details

Constantinos Christofilos

commit sha f01673ea06ce020afffdd36450704f0be1e0184f

Inject context in service Run() (#103) Signed-off-by: Constantinos Christofilos <k.christofilos@thebeat.co>

view details

Sotirios Mantziaris

commit sha 6206b85cd641c3673379bfeb32791bb177bda901

Logging correlation support (#105) Signed-off-by: Sotirios Mantziaris <smantziaris@gmail.com>

view details

Sotirios Mantziaris

commit sha b090fbaa4a069803ab89b88d3406c22b43e32a81

Fixed tracing documentation (#96)

view details

Sotirios Mantziaris

commit sha f523ff659873925d29adf8e8e86152798510ee55

Introduce Make file and linting (#108) Signed-off-by: Sotirios Mantziaris <smantziaris@gmail.com>

view details

William

commit sha 9b450e83ff92f213aad69f1b936e27084001cbf7

Update doc to add more info about the liveness and readiness HTTP endpoints (#99) Signed-off-by: William Claude <w.claude@thebeat.co>

view details

Sotirios Mantziaris

commit sha b0ebe52ae8f69cf0a3bdcbc7f330dad7ffd1290f

Merge branch 'master' into http-logging

view details

push time in 16 days

push eventbeatlabs/patron

William

commit sha 9b450e83ff92f213aad69f1b936e27084001cbf7

Update doc to add more info about the liveness and readiness HTTP endpoints (#99) Signed-off-by: William Claude <w.claude@thebeat.co>

view details

push time in 16 days

PR merged beatlabs/patron

Reviewers
Update readiness & liveness doc

Which problem is this PR solving?

Resolves #98

Short description of the changes

Add doc about liveness and readiness HTTP endpoints into README.md

+16 -0

0 comment

1 changed file

aziule

pr closed time in 16 days

issue closedbeatlabs/patron

Add doc about the new alive and ready routes

Is your feature request related to a problem? Please describe

The doc does not tell what are the routes for the default endpoints created by Patron (readiness, liveness, ...). We have to dive into the code to find the routes (/alive, /ready, ...)

Describe the solution

Add some doc to README.md:

  • Routes to the different endpoints
  • Status codes returned and their meaning

closed time in 16 days

aziule

push eventaziule/patron

Vangelis Katikaridis

commit sha cbfecb514b43be9495e516e21d771df5069cffe3

Decoder injection integration (#101)

view details

Sotirios Mantziaris

commit sha 15d01ae3785a159383beed79561bb79ae23c4121

Update CODEOWNERS

view details

Constantinos Christofilos

commit sha f01673ea06ce020afffdd36450704f0be1e0184f

Inject context in service Run() (#103) Signed-off-by: Constantinos Christofilos <k.christofilos@thebeat.co>

view details

Sotirios Mantziaris

commit sha 6206b85cd641c3673379bfeb32791bb177bda901

Logging correlation support (#105) Signed-off-by: Sotirios Mantziaris <smantziaris@gmail.com>

view details

Sotirios Mantziaris

commit sha b090fbaa4a069803ab89b88d3406c22b43e32a81

Fixed tracing documentation (#96)

view details

Sotirios Mantziaris

commit sha f523ff659873925d29adf8e8e86152798510ee55

Introduce Make file and linting (#108) Signed-off-by: Sotirios Mantziaris <smantziaris@gmail.com>

view details

Sotirios Mantziaris

commit sha fcce35f5e805cf2caf0d79d817c0ebf6c6603215

Merge branch 'master' into document-default-endpoints

view details

push time in 16 days

delete branch beatlabs/patron

delete branch : make-file

delete time in 16 days

push eventbeatlabs/patron

Sotirios Mantziaris

commit sha f523ff659873925d29adf8e8e86152798510ee55

Introduce Make file and linting (#108) Signed-off-by: Sotirios Mantziaris <smantziaris@gmail.com>

view details

push time in 16 days

PR merged beatlabs/patron

Reviewers
Introduce Make file and linting

Which problem is this PR solving?

This PR closes #106.

Short description of the changes

  • Added make file
  • Added supporting script
  • Linted all
+708 -35

0 comment

14 changed files

mantzas

pr closed time in 16 days

issue closedbeatlabs/patron

Introduce Make file

Is your feature request related to a problem? Please describe

There is a need for creating a make file to handle common console tasks like testing, integration tests, CI, etc.

Describe the solution

Introduce Make file.

closed time in 16 days

mantzas

push eventbeatlabs/patron

Sotirios Mantziaris

commit sha 0941a9bac5cf495979782683a455452817f4c7ed

Linting Signed-off-by: Sotirios Mantziaris <smantziaris@gmail.com>

view details

push time in 16 days

push eventbeatlabs/patron

Sotirios Mantziaris

commit sha 6206b85cd641c3673379bfeb32791bb177bda901

Logging correlation support (#105) Signed-off-by: Sotirios Mantziaris <smantziaris@gmail.com>

view details

Sotirios Mantziaris

commit sha b090fbaa4a069803ab89b88d3406c22b43e32a81

Fixed tracing documentation (#96)

view details

Sotirios Mantziaris

commit sha 2e870a96c18f650e0d4e4415d71777961547e694

Merge branch 'master' into make-file

view details

push time in 16 days

delete branch beatlabs/patron

delete branch : FixDocumentation

delete time in 16 days

push eventbeatlabs/patron

Sotirios Mantziaris

commit sha b090fbaa4a069803ab89b88d3406c22b43e32a81

Fixed tracing documentation (#96)

view details

push time in 16 days

PR merged beatlabs/patron

Reviewers
Fixed tracing documentation

<!-- Thanks for taking precious time for making a PR.

Before creating a pull request, please make sure:

  • Your PR solves one problem for which a issue exist and a solution has been discussed
  • You have read the guide for contributing
    • See https://github.com/beatlabs/patron/blob/master/CONTRIBUTING.md
  • You signed all your commits (otherwise we won't be able to merge the PR)
    • See https://github.com/beatlabs/patron/blob/master/CONTRIBUTING.md#sign-your-work
  • You added unit tests for the new functionality
  • You mention in the PR description which issue it is addressing, e.g. "Resolves #123" -->

Which problem is this PR solving?

The tracing documentation stated that tracing is enabled by default but it is not.

Short description of the changes

Fixed documentation and sampler parameter.

+1 -1

0 comment

1 changed file

mantzas

pr closed time in 16 days

push eventbeatlabs/patron

Vangelis Katikaridis

commit sha cbfecb514b43be9495e516e21d771df5069cffe3

Decoder injection integration (#101)

view details

Sotirios Mantziaris

commit sha 15d01ae3785a159383beed79561bb79ae23c4121

Update CODEOWNERS

view details

Constantinos Christofilos

commit sha f01673ea06ce020afffdd36450704f0be1e0184f

Inject context in service Run() (#103) Signed-off-by: Constantinos Christofilos <k.christofilos@thebeat.co>

view details

Sotirios Mantziaris

commit sha 6206b85cd641c3673379bfeb32791bb177bda901

Logging correlation support (#105) Signed-off-by: Sotirios Mantziaris <smantziaris@gmail.com>

view details

Sotirios Mantziaris

commit sha a7e6e27b9b11986f54819725cee56c6111521ee8

Merge branch 'master' into FixDocumentation

view details

push time in 16 days

push eventbeatlabs/patron

Sotirios Mantziaris

commit sha b2c047b70344a889e983a57b1c0f56059625413b

Minor fix Signed-off-by: Sotirios Mantziaris <smantziaris@gmail.com>

view details

push time in 16 days

created tagbeatlabs/patron

tagv0.30.0

Microservice framework following best cloud practices with a focus on productivity.

created time in 17 days

release beatlabs/patron

v0.30.0

released time in 17 days

push eventbeatlabs/patron

Sotirios Mantziaris

commit sha 6206b85cd641c3673379bfeb32791bb177bda901

Logging correlation support (#105) Signed-off-by: Sotirios Mantziaris <smantziaris@gmail.com>

view details

push time in 17 days

PR merged beatlabs/patron

Reviewers
Logging correlation support

Which problem is this PR solving?

This PR closes #104

Short description of the changes

  • Added a new correlation packacge
  • Added code on incoming and outgoing points (components and clients)
+306 -66

0 comment

19 changed files

mantzas

pr closed time in 17 days

issue closedbeatlabs/patron

Logging correlation support

Is your feature request related to a problem? Please describe

In order to have logs that we can correlate through various microservice invocations, we could propagate a correlation it, like the tracing id.

Describe the solution

Introduce a new HEADER in all supported components: HTTP, Kafka, SNS/SQS, RabbitMQ, etc. The name of the header should be "X-CORRELATION-ID".

Additional context

We use logger propagation already via the context.Context so we could enhance this logger with another correlation id field.

closed time in 17 days

mantzas

push eventbeatlabs/patron

Sotirios Mantziaris

commit sha 15d01ae3785a159383beed79561bb79ae23c4121

Update CODEOWNERS

view details

Constantinos Christofilos

commit sha f01673ea06ce020afffdd36450704f0be1e0184f

Inject context in service Run() (#103) Signed-off-by: Constantinos Christofilos <k.christofilos@thebeat.co>

view details

Sotirios Mantziaris

commit sha 806bf6a8ef4a8907e8b87518561ce82df25172e5

Merge branch 'master' into make-file

view details

push time in 17 days

push eventbeatlabs/patron

Sotirios Mantziaris

commit sha 15d01ae3785a159383beed79561bb79ae23c4121

Update CODEOWNERS

view details

Constantinos Christofilos

commit sha f01673ea06ce020afffdd36450704f0be1e0184f

Inject context in service Run() (#103) Signed-off-by: Constantinos Christofilos <k.christofilos@thebeat.co>

view details

Sotirios Mantziaris

commit sha 1e109cbb61fa82f4d83f2041c60cc82a6fe8fce1

Merge branch 'master' into logging-correlation

view details

push time in 20 days

push eventbeatlabs/patron

Sotirios Mantziaris

commit sha a324c92cd9674c1746aa215456ca91bfe77dcee6

Fixed the returned logger from context Signed-off-by: Sotirios Mantziaris <smantziaris@gmail.com>

view details

push time in 20 days

push eventbeatlabs/patron

Constantinos Christofilos

commit sha f01673ea06ce020afffdd36450704f0be1e0184f

Inject context in service Run() (#103) Signed-off-by: Constantinos Christofilos <k.christofilos@thebeat.co>

view details

push time in 20 days

PR merged beatlabs/patron

Reviewers
Inject context in service Run()

<!-- Thanks for taking precious time for making a PR.

Before creating a pull request, please make sure:

  • Your PR solves one problem for which a issue exist and a solution has been discussed
  • You have read the guide for contributing
    • See https://github.com/beatlabs/patron/blob/master/CONTRIBUTING.md
  • You signed all your commits (otherwise we won't be able to merge the PR)
    • See https://github.com/beatlabs/patron/blob/master/CONTRIBUTING.md#sign-your-work
  • You added unit tests for the new functionality
  • You mention in the PR description which issue it is addressing, e.g. "Resolves #123" -->

Which problem is this PR solving?

This PR solves #102 issue <!-- REQUIRED -->

Short description of the changes

Run() signature changes to Run(ctx context.Context) in order to pass context into service. <!-- REQUIRED -->

+29 -17

0 comment

8 changed files

c0nstantx

pr closed time in 20 days

push eventbeatlabs/patron

Sotirios Mantziaris

commit sha 15d01ae3785a159383beed79561bb79ae23c4121

Update CODEOWNERS

view details

push time in 20 days

push eventbeatlabs/patron

Sotirios Mantziaris

commit sha ce6b43a7494f000045417ce7c8ac5c7c613efe6f

Improved circleci file Signed-off-by: Sotirios Mantziaris <smantziaris@gmail.com>

view details

push time in 25 days

push eventbeatlabs/patron

Sotirios Mantziaris

commit sha 7aab01f9be6bbb2cdce105950e58f37f7cb3365c

Improved circleci file Signed-off-by: Sotirios Mantziaris <smantziaris@gmail.com>

view details

push time in 25 days

push eventbeatlabs/patron

Sotirios Mantziaris

commit sha 716813a2eca169a7e38a8bfb640333f2dbdd18fc

Improved circleci file Signed-off-by: Sotirios Mantziaris <smantziaris@gmail.com>

view details

push time in 25 days

push eventbeatlabs/patron

Sotirios Mantziaris

commit sha 0300f9796a6440ded0e5efdb200e96c7d4ea2795

Improved circleci file Signed-off-by: Sotirios Mantziaris <smantziaris@gmail.com>

view details

push time in 25 days

push eventbeatlabs/patron

Sotirios Mantziaris

commit sha 115fd7b2eb4a73b158f2b61ae416a5b18e7425f2

Improved circleci file Signed-off-by: Sotirios Mantziaris <smantziaris@gmail.com>

view details

push time in 25 days

push eventbeatlabs/patron

Sotirios Mantziaris

commit sha f7ecef0fd8ab3ea6f6dc956e3f450fa31c8d00b0

Improved circleci file Signed-off-by: Sotirios Mantziaris <smantziaris@gmail.com>

view details

push time in 25 days

push eventbeatlabs/patron

Sotirios Mantziaris

commit sha 14c28b45b93ff55f730f150089ff22402560949c

Improved circleci file Signed-off-by: Sotirios Mantziaris <smantziaris@gmail.com>

view details

push time in 25 days

push eventbeatlabs/patron

Sotirios Mantziaris

commit sha 5be3d36835339e10b3e48aaa3c4dca462eb5f82b

Improved circleci file Signed-off-by: Sotirios Mantziaris <smantziaris@gmail.com>

view details

push time in 25 days

push eventbeatlabs/patron

Sotirios Mantziaris

commit sha 043fce4aab8778a15316f574814f7bcc7997b79b

Improved circleci file Signed-off-by: Sotirios Mantziaris <smantziaris@gmail.com>

view details

push time in 25 days

push eventbeatlabs/patron

Sotirios Mantziaris

commit sha 43e151f98c388223b49115c4d93a98d433d3ca74

Improved circleci file Signed-off-by: Sotirios Mantziaris <smantziaris@gmail.com>

view details

push time in 25 days

push eventbeatlabs/patron

Sotirios Mantziaris

commit sha a2a8508a09b9efbc878acdf1881c80b9cd5ae624

Improved circleci file Signed-off-by: Sotirios Mantziaris <smantziaris@gmail.com>

view details

push time in 25 days

PR opened beatlabs/patron

Reviewers
WIP Introduce Make file

Which problem is this PR solving?

This PR closes #106.

Short description of the changes

  • Added make file
  • Added supporting script
  • Linted all
+74 -15

0 comment

12 changed files

pr created time in 25 days

create barnchbeatlabs/patron

branch : make-file

created branch time in 25 days

issue closedbeatlabs/patron

Default health check implementation

Is your feature request related to a problem? Please describe

There is a route in Patron to handle health check but there is no implementation of a composite health check worker that will monitor multiple health checks at once.

Describe the solution

Composite health check accepts N functions, each reporting about the health status of some particular part of the system. If any of them start to fail, the composite also fails the health check.

This composite checker provides a single function that is fed to Patron for a default http healthcheck endpoint.

Additional context

The idea of how it could be done is here: https://github.com/taxibeat/canary-service/tree/master/internal/health https://github.com/taxibeat/canary-service/blob/master/internal/health/health.go#L71

I'd only migrate timer-based implementation to channel-based so that functions can recover if needed, and the decision whether to crash a service will be delegated to a client.

closed time in 25 days

azarakovskiy

issue closedbeatlabs/patron

Remove concurrency from consumers on message arrived.

Is your feature request related to a problem? Please describe

Consumers should handle the messages synchronously one after another. Right now this is not happing when a message arrived the consumer runs on goroutine.

Is this a bug? Please provide steps to reproduce, a failing test etc.

Create a demo application with the following code in consumer handler:

fmt.Println('message arrived')
time.Sleep(5 * time.Second)
fmt.Println("message finished")
return nil

send multiple messages to the consumer

the output will be

message arrived message arrived message arrived message finished message finished message finished

should be

message arrived message finished message arrived message finished message arrived message finished

Describe the solution

We should remove the concurrency on the message that arrived.

closed time in 25 days

cpapidas

push eventbeatlabs/patron

Sotirios Mantziaris

commit sha 687359000c8426a2a6c8e6cada399f37b5e18761

Added readme Signed-off-by: Sotirios Mantziaris <smantziaris@gmail.com>

view details

push time in 25 days

push eventbeatlabs/patron

Sotirios Mantziaris

commit sha eba7605b2c968ca1170e8641244e82e989ce11d9

Minor fix Signed-off-by: Sotirios Mantziaris <smantziaris@gmail.com>

view details

push time in 25 days

push eventbeatlabs/patron

Sotirios Mantziaris

commit sha feed7227070d4c3f7e00c746f47b130973536cf6

Added AMQP and HTTP client support Signed-off-by: Sotirios Mantziaris <smantziaris@gmail.com>

view details

push time in 25 days

push eventbeatlabs/patron

Sotirios Mantziaris

commit sha 1f1e05e886c777c159104aeb2067b21093988866

Added SNS and Kafka client support Signed-off-by: Sotirios Mantziaris <smantziaris@gmail.com>

view details

push time in 25 days

push eventbeatlabs/patron

Sotirios Mantziaris

commit sha c7be4fec924669cce26669fa52c84669e4586ca8

Linter fixes Signed-off-by: Sotirios Mantziaris <smantziaris@gmail.com>

view details

push time in 25 days

issue openedbeatlabs/patron

Move trace clients in a client package

Is your feature request related to a problem? Please describe

The clients in the trace package introduced tracing, but we also introduced correlation id propagation so they are not tracing specific anymore.

Describe the solution

A better package could be client which is generic enough for this purpose.

created time in 25 days

issue openedbeatlabs/patron

Introduce Make file

Is your feature request related to a problem? Please describe

There is a need for creating a make file to handle common console tasks like testing, integration tests, CI, etc.

Describe the solution

Introduce Make file.

created time in 25 days

push eventbeatlabs/patron

Sotirios Mantziaris

commit sha 04009c8737e9a6fd13f4c5c6f34ded57bf0a0be5

Added HTTP support Signed-off-by: Sotirios Mantziaris <smantziaris@gmail.com>

view details

push time in 25 days

push eventbeatlabs/patron

Sotirios Mantziaris

commit sha fe5aa40b3058403719042f078f664c3a0e09faa9

Added Kafka,AMQP support Signed-off-by: Sotirios Mantziaris <smantziaris@gmail.com>

view details

push time in 25 days

push eventbeatlabs/patron

Sotirios Mantziaris

commit sha ff756ff0ce6f563d0a156005dccf2d99dfdb2144

Added SQS consumer support Signed-off-by: Sotirios Mantziaris <smantziaris@gmail.com>

view details

push time in 25 days

PR opened beatlabs/patron

Reviewers
WIP Logging correlation support

Which problem is this PR solving?

This PR closes #104

Short description of the changes

  • Added a new correlation packacge
  • Added code on incoming and outgoing points (components and clients)
+58 -0

0 comment

2 changed files

pr created time in 25 days

create barnchbeatlabs/patron

branch : logging-correlation

created branch time in 25 days

issue openedbeatlabs/patron

Logging correlation support

Is your feature request related to a problem? Please describe

In order to have logs that we can correlate through various microservice invocations, we could propagate a correlation it, like the tracing id.

Describe the solution

Introduce a new HEADER in all supported components: HTTP, Kafka, SNS/SQS, RabbitMQ, etc. The name of the header should be "X-CORRELATION-ID".

Additional context

We use logger propagation already via the context.Context so we could enhance this logger with another correlation id field.

created time in 25 days

created tagbeatlabs/patron

tagv0.29.0

Microservice framework following best cloud practices with a focus on productivity.

created time in a month

release beatlabs/patron

v0.29.0

released time in a month

push eventbeatlabs/patron

Vangelis Katikaridis

commit sha cbfecb514b43be9495e516e21d771df5069cffe3

Decoder injection integration (#101)

view details

push time in a month

PR merged beatlabs/patron

Decoder injection integration

Which problem is this PR solving?

Enable injecting of decoder for kafka message consumption which closes #100.

Short description of the changes

  • decorated the kafka consumer factory with a builder pattern, which enables users to inject a custom decoder
  • decoder is aware of the contextType (from the message of the consumer set up)
  • used the previous decoding approach as the default implementation
  • updated go version to 1.13
  • added some doc to exported methods for the async message implementation for kafka
+433 -68

0 comment

6 changed files

drakos74

pr closed time in a month

issue closedbeatlabs/patron

Inject decoder/encoder for kafka integration package

Problem Statement

  • When interacting with kafka the decoding options are currently hardcoded to be either json or protobuf.
  • In the general case , where somebody wants to implement some custom (potentially fast serialization protocol) or use other industry standards (i.e. avro) the framework does not support that out of the box.

Solution Proposal

steps to implement the solution :

  • inject an encoder / decoder implementation tied to the producer / consumer this can be easily achieved with the OptionFunc arguments
  • pass over the encoder / decoder to the message interface needs to be added as an enhancement
  • if nothing is provided fall back to the current approach as a default keep backwards compatibility

optional addition to the patron library

  • implement an avro encoder/decoder implementation to be injected by choice easily and ship out of the box with patron proposed avro implementation to be used : https://github.com/linkedin/goavro

closed time in a month

drakos74

Pull request review commentbeatlabs/patron

Decoder injection integration

 func Start(offset int64) OptionFunc { 		return nil 	} }++// Decoder option for injecting a specific decoder implementation+func Decoder(dec encoding.DecodeRawFunc) OptionFunc {+	return func(c *consumer) error {+		c.dec = dec

I agree that we check it later, but the user intends to set up a decoder and we should warn him if the decoder is nil. If we don't the code will actually try the content type in order to find out the decoder.

drakos74

comment created time in a month

Pull request review commentbeatlabs/patron

Decoder injection integration

 package main  import ( 	"fmt"+	"github.com/beatlabs/patron/encoding/json"

Formatting...

drakos74

comment created time in a month

Pull request review commentbeatlabs/patron

Decoder injection integration

 func Start(offset int64) OptionFunc { 		return nil 	} }++// Decoder option for injecting a specific decoder implementation+func Decoder(dec encoding.DecodeRawFunc) OptionFunc {+	return func(c *consumer) error {+		c.dec = dec

We should check if the decoder is nil and return an error and add the appropriate tests.

drakos74

comment created time in a month

Pull request review commentbeatlabs/patron

Decoder injection integration

 func (m *message) Nack() error { // Factory definition of a consumer factory. type Factory struct { 	name    string-	ct      string 	topic   string 	group   string 	brokers []string+	dec     encoding.DecodeRawFunc 	oo      []OptionFunc }  // New constructor.-func New(name, ct, topic, group string, brokers []string, oo ...OptionFunc) (*Factory, error) {+// if a nil dec is used, it will be replaced by a contentType aware decoder logic+// Not very nice to handle nil arguments, a builder would be nicer, but leaving this for later as part of another issue

We can remove the comments now that we have improved the code.

drakos74

comment created time in a month

Pull request review commentbeatlabs/patron

Decoder injection integration

 func TestConsumer_ConsumeWithoutGroup(t *testing.T) {  	ctx.Done() }++type decodingTestData struct {+	counter                eventCounter+	msgs                   []*sarama.ConsumerMessage+	decoder                encoding.DecodeRawFunc+	dmsgs                  [][]string+	combinedDecoderVersion int32+}++type eventCounter struct {+	messageCount int+	decodingErr  int+	resultErr    int+	claimErr     int+}++func Test_DecodingMessage(t *testing.T) {

Is this not a nice candidate for table-driven tests? We can levewrage the comments as sub test names.

drakos74

comment created time in a month

Pull request review commentbeatlabs/patron

Decoder injection integration

 package kafka  import (+	"github.com/beatlabs/patron/encoding"

Your IDE is not properly setup so it does not eun formating on imports... Goland->Settinngs->Tools->Filewatchers->add goimports

drakos74

comment created time in a month

Pull request review commentbeatlabs/patron

Decoder injection integration

 func TestConsumer_ConsumeWithoutGroup(t *testing.T) {  	ctx.Done() }++type decodingTestData struct {+	counter                eventCounter+	msgs                   []*sarama.ConsumerMessage+	decoder                encoding.DecodeRawFunc+	dmsgs                  [][]string+	combinedDecoderVersion int32+}++type eventCounter struct {+	messageCount int+	decodingErr  int+	resultErr    int+	claimErr     int+}++func Test_DecodingMessage(t *testing.T) {++	testingdata := []decodingTestData{+		// expect a decoding error , as we are injecting an erroring decoder implementation+		{+			counter: eventCounter{+				decodingErr: 1,+			},+			msgs: []*sarama.ConsumerMessage{+				saramaConsumerMessage("[\"value\"]", &sarama.RecordHeader{}),+			},+			decoder: erroringDecoder,+		},+		// the json decoder is not compatible with the message raw string format+		{+			counter: eventCounter{+				decodingErr: 1,+			},+			msgs: []*sarama.ConsumerMessage{+				saramaConsumerMessage("value", &sarama.RecordHeader{}),+			},+			decoder: go_json.Unmarshal,+		},+		// correctly set up value for the jsonDecoder+		{+			counter: eventCounter{+				messageCount: 1,+			},+			msgs: []*sarama.ConsumerMessage{+				saramaConsumerMessage("[\"value\",\"key\"]", &sarama.RecordHeader{}),+			},+			dmsgs:   [][]string{{"value", "key"}},+			decoder: go_json.Unmarshal,+		},+		// verify positive use of the json as a default decoder+		{+			counter: eventCounter{+				messageCount: 1,+			},+			msgs: []*sarama.ConsumerMessage{+				saramaConsumerMessage("[\"value\",\"key\"]", &sarama.RecordHeader{}),+			},+			dmsgs: [][]string{{"value", "key"}},+		},+		// verify negative use of the json as a default decoder+		{+			counter: eventCounter{+				decodingErr: 1,+			},+			msgs: []*sarama.ConsumerMessage{+				saramaConsumerMessage("key value", &sarama.RecordHeader{}),+			},+		},+		// use of the jsonDecoder with multiple messages+		{+			counter: eventCounter{+				decodingErr:  1,+				messageCount: 2,+			},+			msgs: []*sarama.ConsumerMessage{+				saramaConsumerMessage("[\"key\"]", &sarama.RecordHeader{}),+				saramaConsumerMessage("wrong json", &sarama.RecordHeader{}),+				saramaConsumerMessage("[\"value\"]", &sarama.RecordHeader{}),+			},+			dmsgs:   [][]string{{"key"}, {"value"}},+			decoder: go_json.Unmarshal,+		},+		// correctly set up content type for the hardcoded json decoder with message header contentType+		{+			counter: eventCounter{+				messageCount: 1,+			},+			msgs: []*sarama.ConsumerMessage{+				saramaConsumerMessage("[\"value\",\"key\"]", &sarama.RecordHeader{+					Key:   []byte(encoding.ContentTypeHeader),+					Value: []byte(json.Type),+				}),+			},+			dmsgs: [][]string{{"value", "key"}},+		},+		// correctly set up custom string decoder+		{+			counter: eventCounter{+				messageCount: 1,+			},+			msgs: []*sarama.ConsumerMessage{+				saramaConsumerMessage("key value", &sarama.RecordHeader{}),+			},+			dmsgs:   [][]string{{"key", "value"}},+			decoder: stringToSliceDecoder,+		},+		// exotic decoder implementation as an example of consuming messages with different schema+		// the approach is an avro like one, where the first byte will point to the right decoder implementation+		{+			counter: eventCounter{+				messageCount: 3,+				resultErr:    1,+				decodingErr:  2,+			},+			msgs: []*sarama.ConsumerMessage{+				// will use json decoder based on the message header but fail due to bad json+				versionedConsumerMessage("\"key\" \"value\"]", &sarama.RecordHeader{}, 1),+				// will use json decoder based on the message header+				versionedConsumerMessage("[\"key\",\"value\"]", &sarama.RecordHeader{}, 1),+				// will fail at the result level due to the wrong message header, string instead of json+				versionedConsumerMessage("[\"key\",\"value\"]", &sarama.RecordHeader{}, 2),+				// will use void decoder because there is no message header+				versionedConsumerMessage("any string ... ", &sarama.RecordHeader{}, 99),+				// will produce error due to the content type invoking the erroringDecoder+				versionedConsumerMessage("[\"key\",\"value\"]", &sarama.RecordHeader{}, 9),+				// will use string decoder based on the message header+				versionedConsumerMessage("key value", &sarama.RecordHeader{}, 2),+			},+			dmsgs:   [][]string{{"key", "value"}, {}, {"key", "value"}},+			decoder: combinedDecoder,+		},+	}++	for _, testdata := range testingdata {+		testMessageClaim(t, testdata)+	}++}++func saramaConsumerMessage(value string, header *sarama.RecordHeader) *sarama.ConsumerMessage {+	return versionedConsumerMessage(value, header, 0)+}++func versionedConsumerMessage(value string, header *sarama.RecordHeader, version uint8) *sarama.ConsumerMessage {++	bytes := []byte(value)++	if version > 0 {+		bytes = append([]byte{version}, bytes...)+	}++	return &sarama.ConsumerMessage{+		Topic:          "TEST_TOPIC",+		Partition:      0,+		Key:            []byte("key"),+		Value:          bytes,+		Offset:         0,+		Timestamp:      time.Now(),+		BlockTimestamp: time.Now(),+		Headers:        []*sarama.RecordHeader{header},+	}+}++func testMessageClaim(t *testing.T, data decodingTestData) {++	ctx := context.Background()++	counter := eventCounter{}++	factory, err := New("name", "topic", "group", []string{"0.0.0.0:9092"}, data.decoder)++	assert.NoError(t, err, "Could not create factory")++	c, err := factory.Create()++	assert.NoError(t, err, "Could not create component")++	// do a dirty cast for the sake of facilitating the test+	kc := reflect.ValueOf(c).Elem().Interface().(consumer)++	// claim and process the messages and update the counters accordingly+	for _, km := range data.msgs {++		if data.combinedDecoderVersion != 0 {+			km.Value = append([]byte{byte(data.combinedDecoderVersion)}, km.Value...)++		}++		msg, err := claimMessage(ctx, &kc, km)++		if err != nil {+			counter.claimErr+++			continue+		}++		err = process(&counter, &data)(msg)+		if err != nil {+			println(fmt.Sprintf("Could not process message %v : %v", msg, err))+		}+	}++	assert.Equal(t, data.counter, counter)++}++// some naive decoder implementations for testing++func erroringDecoder(data []byte, v interface{}) error {+	return fmt.Errorf("Predefined Decoder Error for message %s", string(data))+}++func VoidDecoder(data []byte, v interface{}) error {

If it is not exported then ...

func voidDecoder(data []byte, v interface{}) error {
drakos74

comment created time in a month

Pull request review commentbeatlabs/patron

Decoder injection integration

 func newKafkaComponent(name, broker, topic, group, amqpURL, amqpExc string) (*ka  	kafkaCmp := kafkaComponent{} -	cf, err := kafka.New(name, json.Type, topic, group, []string{broker})+	cf, err := kafka.New(name, topic, group, []string{broker}, nil)
	cf, err := kafka.New(name, topic, group, []string{broker}, json.DecodeRawFunc)
drakos74

comment created time in a month

Pull request review commentbeatlabs/patron

Decoder injection integration

 package kafka  import (+	"bytes" 	"context"+	"encoding/binary" 	go_json "encoding/json"

std packages should not be aliased. in case of colision alias the non-std ones.

drakos74

comment created time in a month

Pull request review commentbeatlabs/patron

Decoder injection integration

 package kafka  import (+	"bytes" 	"context"+	"encoding/binary"+	go_json "encoding/json"+	"fmt"+	"github.com/beatlabs/patron/async"

It seems that your IDE might not be setup correctly because the ordering of the imports is not ideal...

drakos74

comment created time in a month

more