profile
viewpoint

cockroachdb/cockroach 17830

CockroachDB - the open source, cloud-native SQL database.

kardianos/govendor 5012

Go vendor tool that works with the standard vendor file.

denisenkom/go-mssqldb 1148

Microsoft SQL server driver written in go language

golang-sql/civil 3

Civil Date and Time package for Go

kardianos/garage 2

Garage Door Opener

kardianos/gohttptun 2

A tool to tunnel TCP over HTTP, written in Go

kardianos/go-smtpd 1

SMTP server library for Go

issue commentkubernetes/kubernetes

Strategy to deal with transitive depencencies on client-go <1.18

@mikedanese (I'm author of govendor tool and I now use go modules.) Honest question, why doesn't client-go issue a v2, v3 on obviously breaking changes? I understand you want to keep it in line with k8s releases, but still, there must be a way to do both at some level. Then clients could incrementally adopt versions.

ibuildthecloud

comment created time in an hour

push eventsolidcoredata/databus

Daniel Theophanes

commit sha cd824a088f871e5edc6f1312a15e14f96a90f282

v2: spec out {} vs () notation differences

view details

push time in 3 days

startedcontainous/yaegi

started time in 5 days

starteddosco/super-graph

started time in 8 days

startedpydio/cells

started time in 9 days

push eventsolidcoredata/databus

Daniel Theophanes

commit sha 473c0cda01a4952eb70d63bdf4b2763c34629f0f

v2/parse: start child parsing

view details

push time in 12 days

startedevanw/esbuild

started time in 12 days

pull request commentdenisenkom/go-mssqldb

Add documentation for SetQueryNotification

@chris-rossi Then maybe the signature would be:

type Notification struct {...}
func (n *Notification) QueryContext(ctx context.Context, query string, ...interface{}) (*sql.Rows, error)
func (n *Notification) Cancel(ctx context.Context) error
func (n *Notification) ID() string
func SetQueryNotification(conn *sql.Conn, opts QueryNotificationOptions) (*Notification, error)

?

chris-rossi

comment created time in 15 days

issue closedgolang/go

database/sql: a potential circular wait between db.openerCh and db.mu.Lock()

What version of Go are you using (go version)?

I believe this issue exists since go1.10 to the latest master branch <pre> $ go version go version go1.12.9 darwin/amd64 </pre>

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

I believe this issue is not relevant to OS or processor. <details><summary><code>go env</code> Output</summary><br><pre> $ go env GOARCH="amd64" GOBIN="" GOCACHE="/Users/me/Library/Caches/go-build" GOEXE="" GOFLAGS="" GOHOSTARCH="amd64" GOHOSTOS="darwin" GOOS="darwin" GOPATH="/Users/me/warehouse/myfork/go_proj_Gerrit" GOPROXY="" GORACE="" GOROOT="/usr/local/go" GOTMPDIR="" GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64" GCCGO="gccgo" CC="clang" CXX="clang++" CGO_ENABLED="1" GOMOD="" CGO_CFLAGS="-g -O2" CGO_CPPFLAGS="" CGO_CXXFLAGS="-g -O2" CGO_FFLAGS="-g -O2" CGO_LDFLAGS="-g -O2" PKG_CONFIG="pkg-config" GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/3j/2hnb78gj2clc98lqrhd1s7_r0000gn/T/go-build966675892=/tmp/go-build -gno-record-gcc-switches -fno-common" </pre></details>

What did you see?

In /src/database/sql/sql.go, db.openerCh is used in the following way:

// This is the size of the connectionOpener request chan (DB.openerCh).
// This value should be larger than the maximum typical value
// used for db.maxOpen. If maxOpen is significantly larger than
// connectionRequestQueueSize then it is possible for ALL calls into the *DB
// to block until the connectionOpener can satisfy the backlog of requests.
var connectionRequestQueueSize = 1000000
db.openerCh := make(chan struct{}, connectionRequestQueueSize)

// Goroutine 1
db.mu.Lock()
for ... {
    db.openerCh <- struct{}{} // Block
}()
db.mu.Lock()

// Goroutine 2, a.k.a. connectionOpener()
for {
    select {
    case <-ctx.Done():
        return
    case <- db.openerCh:
        db.mu.Lock() // Block
        ...
        db.mu.Unlock()
        ...
        continue
}

When the buffer is full, db.openerCh <- struct{}{} can be blocked. In the annotation above, it is believed that this send will be unblocked when "the connectionOpener can satisfy the backlog of requests". However, connectionOpener (a.k.a. Goroutine 2) can also be blocked trying to acquire db.mu.Lock(), which is held by Goroutine 1. So a circular wait presents.

I suppose this problem is not noticed because the buffer size is big and hard to be full. However, as the annotation says, send can still be blocking. I am not a security guy, but maybe DOS attack can force this happen? I found an issue mentioning related code, but it seems that they are not talking about circular wait: #10886

To fix this problem, I have written a patch, which can make these goroutines unblockable no matter what happens, but it is quite ugly... Do you think this is a problem? If so, shall I open a PR?

closed time in 16 days

lzhfromustc

issue commentgolang/go

database/sql: a potential circular wait between db.openerCh and db.mu.Lock()

I don't think this is a problem. A connection pool with a million connections, or even a million active goroutines seems unlikely. If someone actually hits this in practice, or is even able to hit this issue in a contrived test case with a real database, let me know.

lzhfromustc

comment created time in 16 days

startedpusher/oauth2_proxy

started time in 16 days

startedfree/sql_exporter

started time in 16 days

startedzeebo/blake3

started time in 16 days

issue commentdenisenkom/go-mssqldb

MSSQL connection started failing on 2/6/20

Okay, I think I'm getting a picture for what happened. Thanks again for creating the issue and using CI/CD.

Piror to https://github.com/denisenkom/go-mssqldb/commit/3f988484a57ee992b0febb2257ca939c58abfd2a the driver only supported NTLM(v1). At that commit NTLMv2 is negotiated now. However, NTLMv2 may enforce stricter security and system requirements. Does the "User ID" have a domain part currently? If not, maybe that can be set?

@sboutzen Can we force use of NTLM rather then NTLMv2?

arjitj2

comment created time in 20 days

issue commentdenisenkom/go-mssqldb

MSSQL connection started failing on 2/6/20

@arjitj2 Can I get a few details from you?

  1. What OS is your client and server on?
  2. What version of Go?
  3. Do you use Windows Authentication or SQL Server authentication?
arjitj2

comment created time in 20 days

issue commentdenisenkom/go-mssqldb

MSSQL connection started failing on 2/6/20

Thank you for reporting this. We'll look into it right away.

arjitj2

comment created time in 20 days

PR closed denisenkom/go-mssqldb

Add a Github workflow for running unit tests

This allows unit tests to be run via Github actions with every push.

Hmmm, this might not be necessary given that there's appveyor.

+20 -0

2 comments

1 changed file

doodlesbykumbi

pr closed time in 21 days

pull request commentdenisenkom/go-mssqldb

Add a Github workflow for running unit tests

So this is a database driver... that tests against SQL Server.

doodlesbykumbi

comment created time in 21 days

issue commentgolang/go

proposal: civil time package

For reference, here is a comment where the timezone causes ambiguity: https://github.com/denisenkom/go-mssqldb/pull/396#issuecomment-397818621 This is the SQL Server driver, looking for the best way to unmarshal datetime and datetime2 (both timestamp without zone types) into time.Time.

jba

comment created time in 21 days

pull request commentdenisenkom/go-mssqldb

Add documentation for SetQueryNotification

@chris-rossi and @yukiwongky Thank you for this documentation.

Upon examining this feature and how it is used, I would strongly like to alter the current implementation unless I'm totally missing something.

Recent database/sql package has a Conn.Raw function, which is our key to functionality like this. https://pkg.go.dev/database/sql?tab=doc#Conn.Raw

I would like to come up with an alternate API for using query notifications that do not involve bypassing the connection pool. First, is it required to set it on a Stmt? In SQL Server, a query batch is first class, not a Stmt. Second, I would like an API like:

package mssql

type QueryNotificationOptions {
    Message string
    Options string // map[string]string?
    Timeout time.Duration
}

// SetQueryNotification may be used on a *sql.Conn prior to a query to receive notifications
// on that query until the given timeout.
//
//    conn, err := db.Conn(ctx)
//    err = mssql.SetQueryNotifications(conn, opts)
//    rows, err := conn.QueryContext(ctx, `select * from notifications;`)
func SetQueryNotification(conn *sql.Conn, opts QueryNotificationOptions) error {
    return conn.Raw(func(ci interface{}) error {
        msc, ok := ci.(*mssql.Conn)
        if !ok {
            return fmt.Errorf("expected connection to be of type *mssql.Conn, got %T", ci)
        }
        msc.SetQueryNotification(opts)
    })
}
chris-rossi

comment created time in 21 days

push eventdenisenkom/go-mssqldb

Chris Hamilton

commit sha bbfc9a55622eeb8e47808e0f7270efcf5ae3dce2

change query notification timeout from seconds to milliseconds (#535) mssql: query notification timeout in milliseconds, not seconds

view details

push time in 21 days

PR merged denisenkom/go-mssqldb

change query notification timeout from seconds to milliseconds

Addresses issue #526 .

@jackwakefield, if you had a fix in the works let me know and I'll remove this PR.

Tested timeout using the query select * from sys.dm_qn_subscriptions and found that the timeout value was equal to the value set by the driver.

+4 -1

1 comment

1 changed file

chris-rossi

pr closed time in 21 days

push eventchris-rossi/go-mssqldb

Daniel Theophanes

commit sha c60a9f1b76af81d28c3dd72fb678a5adc75768fe

Update mssql.go

view details

push time in 21 days

push eventdenisenkom/go-mssqldb

Simon Boje Outzen

commit sha 3f988484a57ee992b0febb2257ca939c58abfd2a

Add support for NTLMv2 (#551) mssql: add support for NTLMv2

view details

push time in 21 days

PR merged denisenkom/go-mssqldb

Add support for NTLMv2

Fix for https://github.com/denisenkom/go-mssqldb/issues/146

+207 -26

7 comments

2 changed files

sboutzen

pr closed time in 21 days

issue commentgolang/go

database/sql: way to execute the SQL directly without prepare for IBM/DB2

It looks like that above driver uses ODBC3 to implement connections. There may be a limitation there. Furthermore, it looks like it has a dedicated API call to create or drop tables.

The issue isn't with database/sql, but likely with your driver.

akhilravuri1

comment created time in 22 days

issue closedgolang/go

database/sql: way to execute the SQL directly without prepare for IBM/DB2

<!-- Please answer these questions before submitting your issue. Thanks! For questions please use one of our forums: https://github.com/golang/go/wiki/Questions -->

What version of Go are you using (go version)?

<pre> $ go version go version go1.13.4 windows/amd64 </pre>

Does this issue reproduce with the latest release?

NO

What operating system and processor architecture are you using (go env)?

<details><summary><code>go env</code> Output</summary><br><pre> $ go env set GO111MODULE= set GOARCH=amd64 set GOBIN= set GOCACHE=C:\Users\rakhil\AppData\Local\go-build set GOENV=C:\Users\rakhil\AppData\Roaming\go\env set GOEXE=.exe set GOFLAGS= set GOHOSTARCH=amd64 set GOHOSTOS=windows set GONOPROXY= set GONOSUMDB= set GOOS=windows set GOPATH=C:\Users\rakhil\go set GOPRIVATE= set GOPROXY=https://proxy.golang.org,direct set GOROOT=c:\go set GOSUMDB=sum.golang.org set GOTMPDIR= set GOTOOLDIR=c:\go\pkg\tool\windows_amd64 set GCCGO=gccgo set AR=ar set CC=gcc set CXX=g++ set CGO_ENABLED=1 set GOMOD= set CGO_CFLAGS=-g -O2 set CGO_CPPFLAGS= set CGO_CXXFLAGS=-g -O2 set CGO_FFLAGS=-g -O2 set CGO_LDFLAGS=-g -O2 set PKG_CONFIG=pkg-config set GOGCCFLAGS=-m64 -mthreads -fmessage-length=0 -fdebug-prefix-map=C:\Users\rakhil\AppData\Local\Temp\go-build747585790=/tmp/go-build -gno-record-gcc-switches </pre></details>

What did you do?

I was trying to create a view in the BIGSQL database by IBM using the go_ibm_db package which is based on your database/sql package but it was not created because your database/sql uses prepare and query API's even if we call exec. But for creating a view and altering table BIGSQL throws an error if we use prepare and query instead of execdirect.

What did you expect to see?

view is created and the table is altered successfully.

What did you see instead?

BIGSQL returns statement not supported.

Do you have any plans for supporting ExecDirect api?

Thanks, Akhil

closed time in 22 days

akhilravuri1

issue commentgolang/go

database/sql: way to execute the SQL directly without prepare for IBM/DB2

Looking at https://github.com/ibmdb/go_ibm_db it looks like the driver doesn't implement the driver.Queryer interface. Once that is implemented, it will operate as desired.

akhilravuri1

comment created time in 22 days

issue commentgolang/go

proposal: civil time package

@rsc I like your proposal for time.Day.

I feel like time.Clock would be useful, but the proposal presented isn't quite right I think. I would want a Clock to either: 1) be easy to convert to and from time.Duration, or 2) have some of these methods on time.Duration directly (Duration.Format, then retrieve days, hours, min, secs, nanos as integers rather then floats).

These types don't have to be in the standard library for drivers to use them.

While it isn't strictly necessary, I agree with @jba that having a DateTime (time.Time without zone) would be useful as it enhances code clarity; I've had bugs in Go today where the zone was incorrect going into or out of a database with a DateTime field that resulted in an incorrect value. The problem (as much as it is) is round tripping DateTime field values. If you put a DateTime into the database, you want to ensure that the same value is selected out. So if you put in a time.Time with a local timezone and just use the face (no zone) value, but then the sql scan returns a time.Time with a UTC timezone, it can get you in trouble. Should the scan as a localtime zone might be better, but the problem remains, just reversed UTC zone in, local out). This, can be fine, until you hit some other marshal function that has different assumptions (ie uses zone when you assume it should not).

If a next step were to be taken, would it to decide where such a package would reside (stdlib/time, golang.org/x/time/civil)?

jba

comment created time in 23 days

pull request commentdenisenkom/go-mssqldb

Add support for NTLMv2

@sboutzen I will not gate the PR on resolving this, as you say, it may happen on master as well.

sboutzen

comment created time in 23 days

pull request commentdenisenkom/go-mssqldb

Added support for federated authentication to enable Azure AD authentication

@wrosenuance Thank you for the updates. There are a number of changes going on and it will take me a few days to review. Thank you for being patient with me and my requests so far.

wrosenuance

comment created time in 23 days

issue commentdenisenkom/go-mssqldb

SQL date type is always scanned as time.Time which conflicts with civil.Date destination type

Are you using the rows.Scan? Or are you using some other type of wrapper? Can you make a minimally complete, runnable example that demonstrates this?

Maxiride

comment created time in 23 days

startedgothinkster/realworld

started time in 24 days

pull request commentdenisenkom/go-mssqldb

Add support for NTLMv2

Try running test with go test -v -race -cpu 4 -count 10 locally, ensuring you have a connection env var dsn defined.

sboutzen

comment created time in 24 days

startedevilsocket/opensnitch

started time in a month

Pull request review commentdenisenkom/go-mssqldb

Added support for federated authentication to enable Azure AD authentication

+package mssql++import (+	"context"+	"crypto/rsa"+	"crypto/x509"+	"errors"+	"fmt"+	"io/ioutil"++	"github.com/Azure/go-autorest/autorest/adal"+	"golang.org/x/crypto/pkcs12"+)++const (+	activeDirectoryEndpoint = "https://login.microsoftonline.com/"+	azureSQLResource        = "https://database.windows.net/"+	driverClientID          = "7f98cb04-cd1e-40df-9140-3bf7e2cea4db"+)++func fedAuthGetClientCertificate(clientCertPath, clientCertPassword string) (*x509.Certificate, *rsa.PrivateKey, error) {+	pkcs, err := ioutil.ReadFile(clientCertPath)+	if err != nil {+		return nil, nil, fmt.Errorf("Failed to read the AD client certificate from path %s: %v", clientCertPath, err)+	}++	privateKey, certificate, err := pkcs12.Decode(pkcs, clientCertPassword)+	if err != nil {+		return nil, nil, fmt.Errorf("Failed to read the AD client certificate from path %s: %v", clientCertPath, err)+	}++	rsaPrivateKey, isRsaKey := privateKey.(*rsa.PrivateKey)+	if !isRsaKey {+		return nil, nil, fmt.Errorf("AD client certificate at path %s must contain an RSA private key", clientCertPath)+	}++	return certificate, rsaPrivateKey, nil+}++func fedAuthGetAccessToken(ctx context.Context, resource, tenantID string, p connectParams, log optionalLogger) (accessToken string, err error) {+	// The activeDirectoryEndpoint URL is used as a base against which the+	// tenant ID is resolved. When the workflow provides a complete endpoint+	// URL for the tenant, the URL resolution just returns that endpoint.+	oauthConfig, err := adal.NewOAuthConfig(activeDirectoryEndpoint, tenantID)+	if err != nil {+		log.Printf("Failed to obtain OAuth configuration for endpoint %s and tenant %s: %v", activeDirectoryEndpoint, tenantID, err)

You must not log like this. Rather, you can do return "", fmt.Errorf("Failed to obtain... %v", err) but no log to console.

wrosenuance

comment created time in a month

Pull request review commentdenisenkom/go-mssqldb

Added support for federated authentication to enable Azure AD authentication

+package mssql++import (+	"context"+	"crypto/rsa"+	"crypto/x509"+	"errors"+	"fmt"+	"io/ioutil"++	"github.com/Azure/go-autorest/autorest/adal"+	"golang.org/x/crypto/pkcs12"+)++const (+	activeDirectoryEndpoint = "https://login.microsoftonline.com/"+	azureSQLResource        = "https://database.windows.net/"+	driverClientID          = "7f98cb04-cd1e-40df-9140-3bf7e2cea4db"+)++func fedAuthGetClientCertificate(clientCertPath, clientCertPassword string) (*x509.Certificate, *rsa.PrivateKey, error) {+	pkcs, err := ioutil.ReadFile(clientCertPath)+	if err != nil {+		return nil, nil, fmt.Errorf("Failed to read the AD client certificate from path %s: %v", clientCertPath, err)+	}++	privateKey, certificate, err := pkcs12.Decode(pkcs, clientCertPassword)+	if err != nil {+		return nil, nil, fmt.Errorf("Failed to read the AD client certificate from path %s: %v", clientCertPath, err)+	}++	rsaPrivateKey, isRsaKey := privateKey.(*rsa.PrivateKey)+	if !isRsaKey {+		return nil, nil, fmt.Errorf("AD client certificate at path %s must contain an RSA private key", clientCertPath)+	}++	return certificate, rsaPrivateKey, nil+}++func fedAuthGetAccessToken(ctx context.Context, resource, tenantID string, p connectParams, log optionalLogger) (accessToken string, err error) {+	// The activeDirectoryEndpoint URL is used as a base against which the+	// tenant ID is resolved. When the workflow provides a complete endpoint+	// URL for the tenant, the URL resolution just returns that endpoint.+	oauthConfig, err := adal.NewOAuthConfig(activeDirectoryEndpoint, tenantID)+	if err != nil {+		log.Printf("Failed to obtain OAuth configuration for endpoint %s and tenant %s: %v", activeDirectoryEndpoint, tenantID, err)+		return "", err+	}++	var token *adal.ServicePrincipalToken+	if p.fedAuthLibrary == fedAuthLibrarySecurityToken {

Try to get rid of these if/else sequences and nested if/elses. Use switch and early returns or initial value and overrdie.

wrosenuance

comment created time in a month

Pull request review commentdenisenkom/go-mssqldb

Added support for federated authentication to enable Azure AD authentication

 initiate_connection: 	}  	login := login{-		TDSVersion:   verTDS74,-		PacketSize:   uint32(outbuf.PackageSize()),-		Database:     p.database,-		OptionFlags2: fODBC, // to get unlimited TEXTSIZE-		HostName:     p.workstation,-		ServerName:   p.host,-		AppName:      p.appname,-		TypeFlags:    p.typeFlags,+		TDSVersion:          verTDS74,+		PacketSize:          uint32(outbuf.PackageSize()),+		Database:            p.database,+		OptionFlags2:        fODBC, // to get unlimited TEXTSIZE+		HostName:            p.workstation,+		ServerName:          p.host,+		AppName:             p.appname,+		TypeFlags:           p.typeFlags,+		FedAuthLibrary:      p.fedAuthLibrary,+		FedAuthEcho:         fedAuthEcho,+		FedAuthADALWorkflow: p.fedAuthADALWorkflow, 	} 	auth, auth_ok := getAuth(p.user, p.password, p.serverSPN, p.workstation)-	if auth_ok {+	if login.FedAuthLibrary == fedAuthLibraryReserved && auth_ok {

This section +30 lines of the nested else/if needs to be cleaned up. using switch may help, and maybe try to get rid of nested ifs with early returns.

wrosenuance

comment created time in a month

Pull request review commentdenisenkom/go-mssqldb

Added support for federated authentication to enable Azure AD authentication

+package mssql++import (+	"context"+	"crypto/rsa"+	"crypto/x509"+	"errors"+	"fmt"+	"io/ioutil"++	"github.com/Azure/go-autorest/autorest/adal"+	"golang.org/x/crypto/pkcs12"+)++const (+	activeDirectoryEndpoint = "https://login.microsoftonline.com/"+	azureSQLResource        = "https://database.windows.net/"+	driverClientID          = "7f98cb04-cd1e-40df-9140-3bf7e2cea4db"+)++func fedAuthGetClientCertificate(clientCertPath, clientCertPassword string) (*x509.Certificate, *rsa.PrivateKey, error) {+	pkcs, err := ioutil.ReadFile(clientCertPath)+	if err != nil {+		return nil, nil, fmt.Errorf("Failed to read the AD client certificate from path %s: %v", clientCertPath, err)+	}++	privateKey, certificate, err := pkcs12.Decode(pkcs, clientCertPassword)+	if err != nil {+		return nil, nil, fmt.Errorf("Failed to read the AD client certificate from path %s: %v", clientCertPath, err)+	}++	rsaPrivateKey, isRsaKey := privateKey.(*rsa.PrivateKey)+	if !isRsaKey {+		return nil, nil, fmt.Errorf("AD client certificate at path %s must contain an RSA private key", clientCertPath)+	}++	return certificate, rsaPrivateKey, nil+}++func fedAuthGetAccessToken(ctx context.Context, resource, tenantID string, p connectParams, log optionalLogger) (accessToken string, err error) {

I'd really like to see this function isolated in a sub package at a minimum. Maybe we can still directly reference it, but I would really like it injected somehow.

wrosenuance

comment created time in a month

pull request commentdenisenkom/go-mssqldb

Added support for federated authentication to enable Azure AD authentication

I've merged #546, please rebase on master and I'll review this closely.

wrosenuance

comment created time in a month

push eventdenisenkom/go-mssqldb

Paul Meyer

commit sha 0f454e2ecd6ad8fb4691cdbf10e399e05ca03784

AAD Authentication using AccessToken (#546) mssql: add Azure Active Directory token based login Create a new connector that allows refreshing the access token based on an external identifier.

view details

push time in a month

PR merged denisenkom/go-mssqldb

AAD Authentication using AccessToken

Although it leaves room for improvement with respect to convenience from the DSN, this PR implements AAD authentication through access tokens. To test this implementation, create a SQL Azure database and set yourself as the AAD admin for the database. Then run

az login # if not already logged in
at=$(az account get-access-token --resource https://database.windows.net/ --q accessToken -o tsv)
export SQLSERVER_DSN="sqlserver://srv.database.windows.net?database=db&accesstoken=$at"
go test -v

To use a service principal, grant the service principal access to the database as described in the docs.

At least partially fixes #446

+519 -39

3 comments

11 changed files

paulmey

pr closed time in a month

issue closeddenisenkom/go-mssqldb

Azure Active Directory Authentication is not supported?

Hi,

I am able to connect to Azure database using SQL authentication but when I use Azure AD credentials I receive TLS handshake error when pinging the database: Cannot read handshake packet: read tcp: wsarecv: An existing connection was forcibly closed by the remote host.

Connection String: sqlserver://user:pass@database.windows.net:1433?app+name=MyAppName&database=dbname&encrypt=true&hostNameInCertificate=%2A.database.windows.net&trustservercertificate=true

Is the Azure AD supported or not?

Thanks

closed time in a month

zbynekbotlo

push eventsolidcoredata/databus

Daniel Theophanes

commit sha ce55125e865e507dbd87edf999139f4b7ad1800c

v2/parse: move out parse v4

view details

push time in a month

issue commentschmorrison/Zoho

License

You could put in a different branch, but just changing the license and tagging a new release would be enough.

BSD and MIT are both popular choices. The problem with GPL is it is viral and Go is statically linked.

On Fri, Jan 31, 2020, 06:19 Sam Morrison notifications@github.com wrote:

Thanks for your comment, and duly noted. What are your suggestions for license? Are there any considerations I should make when changing the lisense; like versioning the current license to a different branch?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/schmorrison/Zoho/issues/20?email_source=notifications&email_token=AAFYLMMWYJXG2DJZDMX3GF3RAQXPDA5CNFSM4KN6IMVKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEKOZDCY#issuecomment-580751755, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAFYLMKQM2DFTUDSRV2PDE3RAQXPDANCNFSM4KN6IMVA .

kardianos

comment created time in a month

issue openedschmorrison/Zoho

License

This looks like a nice start for integration to Zoho.

Please be aware, it is doubtful many people will be able to use a GPL3 licensed package. It is likely almost everyone who is interested in using both Go and Zoho will be unable to use this package.

created time in a month

pull request commentdenisenkom/go-mssqldb

Added support for federated authentication to enable Azure AD authentication

@wrosenuance I just sent a review #546 . I'm going to wait for that PR to merge before doing a review on this one, but doing a skim on this one I would request the following general changes:

  • If possible, inject the adal functionality if possible (so this module doesn't direclty depend on it. I like how #546 does it. If not possible, that might be fine, but I'll need to understand why. It looks like adal package is only used in a single function, so it may be possible to inject that in the connector or similar.
  • Please replace if/else chains with either early returns or switch statements.
  • We will want to add a go.mod file under the example folder, #546 may do that.
wrosenuance

comment created time in a month

Pull request review commentdenisenkom/go-mssqldb

AAD Authentication using AccessToken

 func TestSendLogin(t *testing.T) { 	} } +func TestSendLoginWithFeatureExt(t *testing.T) {+	memBuf := new(MockTransport)+	buf := newTdsBuffer(1024, memBuf)+	login := login{+		TDSVersion:     verTDS74,+		PacketSize:     0x1000,+		ClientProgVer:  0x01060100,+		ClientPID:      100,+		ClientTimeZone: -4 * 60,+		ClientID:       [6]byte{0x12, 0x34, 0x56, 0x78, 0x90, 0xab},+		OptionFlags1:   0xe0,+		OptionFlags3:   8,+		HostName:       "subdev1",+		AppName:        "appname",+		ServerName:     "servername",+		CtlIntName:     "library",+		Language:       "en",+		Database:       "database",+		ClientLCID:     0x204,+	}+	login.FeatureExt.Add(&featureExtFedAuthSTS{+		FedAuthToken: "fedauthtoken",+	})+	err := sendLogin(buf, login)+	if err != nil {+		t.Error("sendLogin should succeed")

I suspect you want t.Fatal

paulmey

comment created time in a month

Pull request review commentdenisenkom/go-mssqldb

AAD Authentication using AccessToken

 func parseLoginAck(r *tdsBuffer) loginAckStruct { 	return res } +func parseFeatureExtAck(r *tdsBuffer) {+	for {+		featureID := r.byte() // FeatureID+		if featureID == 0xff {+			break

Just "return" please.

paulmey

comment created time in a month

Pull request review commentdenisenkom/go-mssqldb

AAD Authentication using AccessToken

 func TestSendLogin(t *testing.T) { 	} } +func TestSendLoginWithFeatureExt(t *testing.T) {+	memBuf := new(MockTransport)+	buf := newTdsBuffer(1024, memBuf)+	login := login{+		TDSVersion:     verTDS74,+		PacketSize:     0x1000,+		ClientProgVer:  0x01060100,+		ClientPID:      100,+		ClientTimeZone: -4 * 60,+		ClientID:       [6]byte{0x12, 0x34, 0x56, 0x78, 0x90, 0xab},+		OptionFlags1:   0xe0,+		OptionFlags3:   8,+		HostName:       "subdev1",+		AppName:        "appname",+		ServerName:     "servername",+		CtlIntName:     "library",+		Language:       "en",+		Database:       "database",+		ClientLCID:     0x204,+	}+	login.FeatureExt.Add(&featureExtFedAuthSTS{+		FedAuthToken: "fedauthtoken",+	})+	err := sendLogin(buf, login)+	if err != nil {+		t.Error("sendLogin should succeed")+	}+	ref := []byte{+		16, 1, 0, 223, 0, 0, 1, 0, 215, 0, 0, 0, 4, 0, 0, 116, 0, 16, 0, 0, 0, 1,+		6, 1, 100, 0, 0, 0, 0, 0, 0, 0, 224, 0, 0, 24, 16, 255, 255, 255, 4, 2, 0,+		0, 94, 0, 7, 0, 108, 0, 0, 0, 108, 0, 0, 0, 108, 0, 7, 0, 122, 0, 10, 0, 176,+		0, 4, 0, 142, 0, 7, 0, 156, 0, 2, 0, 160, 0, 8, 0, 18, 52, 86, 120, 144, 171,+		176, 0, 0, 0, 176, 0, 0, 0, 176, 0, 0, 0, 0, 0, 0, 0, 115, 0, 117, 0, 98,+		0, 100, 0, 101, 0, 118, 0, 49, 0, 97, 0, 112, 0, 112, 0, 110, 0, 97, 0,+		109, 0, 101, 0, 115, 0, 101, 0, 114, 0, 118, 0, 101, 0, 114, 0, 110, 0, 97,+		0, 109, 0, 101, 0, 108, 0, 105, 0, 98, 0, 114, 0, 97, 0, 114, 0, 121, 0, 101,+		0, 110, 0, 100, 0, 97, 0, 116, 0, 97, 0, 98, 0, 97, 0, 115, 0, 101, 0, 180, 0,+		0, 0, 2, 29, 0, 0, 0, 2, 24, 0, 0, 0, 102, 0, 101, 0, 100, 0, 97, 0, 117, 0,+		116, 0, 104, 0, 116, 0, 111, 0, 107, 0, 101, 0, 110, 0, 255}+	out := memBuf.Bytes()+	if !bytes.Equal(ref, out) {+		fmt.Println("Expected:")+		fmt.Print(hex.Dump(ref))+		fmt.Println("Returned:")+		fmt.Print(hex.Dump(out))+		t.Error("input output don't match")

I suspect you want t.Fatal here.

paulmey

comment created time in a month

Pull request review commentdenisenkom/go-mssqldb

AAD Authentication using AccessToken

 func parseLoginAck(r *tdsBuffer) loginAckStruct { 	return res } +func parseFeatureExtAck(r *tdsBuffer) {+	for {

I don't like unbounded loops when parsing inputs.

Can we put some ridiculous large constraint here like 1000 or even just 10? for i := 0; i < 10; i++ { ... } panic("parsed more then X and we are worried!")

paulmey

comment created time in a month

Pull request review commentdenisenkom/go-mssqldb

AAD Authentication using AccessToken

 func TestSendLogin(t *testing.T) { 	} } +func TestSendLoginWithFeatureExt(t *testing.T) {+	memBuf := new(MockTransport)+	buf := newTdsBuffer(1024, memBuf)+	login := login{+		TDSVersion:     verTDS74,+		PacketSize:     0x1000,+		ClientProgVer:  0x01060100,+		ClientPID:      100,+		ClientTimeZone: -4 * 60,+		ClientID:       [6]byte{0x12, 0x34, 0x56, 0x78, 0x90, 0xab},+		OptionFlags1:   0xe0,+		OptionFlags3:   8,+		HostName:       "subdev1",+		AppName:        "appname",+		ServerName:     "servername",+		CtlIntName:     "library",+		Language:       "en",+		Database:       "database",+		ClientLCID:     0x204,+	}+	login.FeatureExt.Add(&featureExtFedAuthSTS{+		FedAuthToken: "fedauthtoken",+	})+	err := sendLogin(buf, login)+	if err != nil {+		t.Error("sendLogin should succeed")+	}+	ref := []byte{+		16, 1, 0, 223, 0, 0, 1, 0, 215, 0, 0, 0, 4, 0, 0, 116, 0, 16, 0, 0, 0, 1,+		6, 1, 100, 0, 0, 0, 0, 0, 0, 0, 224, 0, 0, 24, 16, 255, 255, 255, 4, 2, 0,+		0, 94, 0, 7, 0, 108, 0, 0, 0, 108, 0, 0, 0, 108, 0, 7, 0, 122, 0, 10, 0, 176,+		0, 4, 0, 142, 0, 7, 0, 156, 0, 2, 0, 160, 0, 8, 0, 18, 52, 86, 120, 144, 171,+		176, 0, 0, 0, 176, 0, 0, 0, 176, 0, 0, 0, 0, 0, 0, 0, 115, 0, 117, 0, 98,+		0, 100, 0, 101, 0, 118, 0, 49, 0, 97, 0, 112, 0, 112, 0, 110, 0, 97, 0,+		109, 0, 101, 0, 115, 0, 101, 0, 114, 0, 118, 0, 101, 0, 114, 0, 110, 0, 97,+		0, 109, 0, 101, 0, 108, 0, 105, 0, 98, 0, 114, 0, 97, 0, 114, 0, 121, 0, 101,+		0, 110, 0, 100, 0, 97, 0, 116, 0, 97, 0, 98, 0, 97, 0, 115, 0, 101, 0, 180, 0,+		0, 0, 2, 29, 0, 0, 0, 2, 24, 0, 0, 0, 102, 0, 101, 0, 100, 0, 97, 0, 117, 0,+		116, 0, 104, 0, 116, 0, 111, 0, 107, 0, 101, 0, 110, 0, 255}+	out := memBuf.Bytes()+	if !bytes.Equal(ref, out) {+		fmt.Println("Expected:")

Let's use t.Logf here please.

paulmey

comment created time in a month

Pull request review commentdenisenkom/go-mssqldb

AAD Authentication using AccessToken

 initiate_connection: 		AppName:      p.appname, 		TypeFlags:    p.typeFlags, 	}-	auth, auth_ok := getAuth(p.user, p.password, p.serverSPN, p.workstation)-	if auth_ok {+	auth, authOk := getAuth(p.user, p.password, p.serverSPN, p.workstation)+	if p.accessToken != "" { // accesstoken ignores user/password

Please change chained if/else into a switch. switch { case len(p.accessToken) > 0: ... case authOk: ... default: ... }

paulmey

comment created time in a month

Pull request review commentdenisenkom/go-mssqldb

AAD Authentication using AccessToken

 initiate_connection: 		AppName:      p.appname, 		TypeFlags:    p.typeFlags, 	}-	auth, auth_ok := getAuth(p.user, p.password, p.serverSPN, p.workstation)-	if auth_ok {+	auth, authOk := getAuth(p.user, p.password, p.serverSPN, p.workstation)+	if p.accessToken != "" { // accesstoken ignores user/password+		featurext := &featureExtFedAuthSTS{+			FedAuthEcho:  fields[preloginFEDAUTHREQUIRED] != nil && fields[preloginFEDAUTHREQUIRED][0] == 1,

Can we be sure fields[preloginFEDAUTHREQUIRED][0] len > 0?

paulmey

comment created time in a month

Pull request review commentdenisenkom/go-mssqldb

AAD Authentication using AccessToken

+package main++import (+	"database/sql"+	"flag"+	"fmt"+	"log"++	"github.com/Azure/go-autorest/autorest/adal"+	mssql "github.com/denisenkom/go-mssqldb"+)++var (+	debug    = flag.Bool("debug", false, "enable debugging")+	server   = flag.String("server", "", "the database server")+	database = flag.String("database", "", "the database")+)++func main() {+	flag.Parse()++	if *debug {+		fmt.Printf(" server:%s\n", *server)+		fmt.Printf(" database:%s\n", *database)+	}++	if *server == "" {+		log.Fatal("Server name cannot be left empty")+	}++	if *database == "" {+		log.Fatal("Database name cannot be left empty")+	}++	connString := fmt.Sprintf("Server=%s;Database=%s", *server, *database)+	if *debug {+		fmt.Printf(" connString:%s\n", connString)+	}++	tokenProvider, err := getMSITokenProvider()+	if err != nil {+		log.Fatal("Error creating token provider for system assigned Azure Managed Identity:", err.Error())+	}++	connector, err := mssql.NewAccessTokenConnector(+		connString, tokenProvider)+	if err != nil {+		log.Fatal("Connector creation failed:", err.Error())+	}+	conn := sql.OpenDB(connector)+	defer conn.Close()++	stmt, err := conn.Prepare("select 1, 'abc'")

This is fine as-is, but SQL Server really doesn't need prepared statements in 99.9% of the time. I'd prefer examples not use prepared statements. (my own nit, fine to leave as is).

paulmey

comment created time in a month

Pull request review commentdenisenkom/go-mssqldb

AAD Authentication using AccessToken

+// +build go1.10++package mssql++import (+	"context"+	"database/sql/driver"+	"errors"+	"fmt"+	"strings"+	"testing"+)++func TestNewAccessTokenConnector(t *testing.T) {+	dsn := "Server=server.database.windows.net;Database=db"+	tp := func() (string, error) { return "token", nil }+	type args struct {+		dsn           string+		tokenProvider func() (string, error)+	}+	tests := []struct {+		name    string+		args    args+		want    func(driver.Connector) error+		wantErr bool+	}{+		{"Happy path",

Please name the fields, my eyes get lost counting where I'm at. I like the table driven test.

paulmey

comment created time in a month

Pull request review commentdenisenkom/go-mssqldb

AAD Authentication using AccessToken

 func sendLogin(w *tdsBuffer, login login) error { 	offset += uint16(len(atchdbfile)) 	hdr.ChangePasswordOffset = offset 	offset += uint16(len(changepassword))-	hdr.Length = uint32(offset)++	featureExtOffset := uint32(0)+	if featureExtLen := len(featureExt); featureExtLen > 0 {+		hdr.OptionFlags3 |= fExtension+		hdr.ExtensionOffset = offset+		hdr.ExtensionLength = 4+		offset += hdr.ExtensionLength // DWORD+		featureExtOffset = uint32(offset)+		hdr.Length = uint32(offset) + uint32(featureExtLen)+	} else {

Remove this else, just set the hdr.Length after the end of this segment hdr.Length = uint32(offset) + uint32(len(featrureExt))

paulmey

comment created time in a month

Pull request review commentdenisenkom/go-mssqldb

AAD Authentication using AccessToken

 Other supported formats are listed below.   * `odbc:server=localhost;user id=sa;password={foo{bar}` // Literal `{`, password is "foo{bar"   * `odbc:server=localhost;user id=sa;password={foo}}bar}` // Escaped `} with `}}`, password is "foo}bar" +### Azure Active Directory authentication - preview++The configuration of functionality might change in the future.++Azure Active Directory (AAD) is supported through the `accesstoken` parameter in the DSN. However,

I really like the access token connector.

What is the use case for hard coding accesstoken in the dsn?

Or rather, I understand that we may have an access token parameter. Do we need to advertise this parameter? Or can we just say " use the NewAccessTokenConnector"?

paulmey

comment created time in a month

pull request commentdenisenkom/go-mssqldb

Azure Active Directory authentication

Okay, I just read through the Issue history. I think I understand the situation now.

I assume we will keep this PR a s a draft and NOT merge it, but rather merge each contribution by itself.

paulmey

comment created time in a month

pull request commentdenisenkom/go-mssqldb

Azure Active Directory authentication

@paulmey What's the current status on this PR? I see it merges two other PRs, one from you and another from @wrosenuance . I'm assuming it isn't ready yet for review due to draft status, correct?

Lastly, how would you propose we test (CI) this code so it doesn't break? SQL Server (hosted) has a free dev license, but this looks like Azure specific?

paulmey

comment created time in a month

push eventsolidcoredata/databus

Daniel Theophanes

commit sha 77db6e5543f623792b39fd9c58ceb5d8708b7c55

v2: more notes and attempts parse in various ways

view details

push time in a month

Pull request review commentdenisenkom/go-mssqldb

Add support for NTLMv2

 package mssql  import (+	"bytes" 	"encoding/hex"+	"github.com/stretchr/testify/assert"

Remove "testify/assert"

sboutzen

comment created time in a month

Pull request review commentdenisenkom/go-mssqldb

Add support for NTLMv2

 func TestNTLMSessionResponse(t *testing.T) { 		t.Errorf("got:\n%sexpected:\n%s", hex.Dump(nt[:]), hex.Dump(val[:])) 	} }++func TestNTLMV2Response(t *testing.T) {+	target := "DOMAIN"+	username := "user"+	password := "SecREt01"+	challenge := [8]byte{ 0x01, 0x23, 0x45, 0x67, 0x89, 0xab, 0xcd, 0xef}+	targetInformationBlock, _ := hex.DecodeString("02000c0044004f004d00410049004e0001000c005300450052005600450052000400140064006f006d00610069006e002e0063006f006d00030022007300650072007600650072002e0064006f006d00610069006e002e0063006f006d0000000000")+	nonceBytes, _ := hex.DecodeString("ffffff0011223344")+	var nonce [8]byte+	copy(nonce[:8], nonceBytes[:])+	timestamp, err := time.Parse(time.RFC3339, "2006-01-02T15:04:05Z")+	if err != nil {+		panic(err)+	}++	expectedNTLMV2Response, _ := hex.DecodeString("5c788afec59c1fef3f90bf6ea419c02501010000000000000fc4a4d5fdf6b200ffffff00112233440000000002000c0044004f004d00410049004e0001000c005300450052005600450052000400140064006f006d00610069006e002e0063006f006d00030022007300650072007600650072002e0064006f006d00610069006e002e0063006f006d000000000000000000")+	ntlmV2Response := getNTLMV2Response(target, username, password, challenge, nonce, targetInformationBlock, timestamp)+	if bytes.Compare(ntlmV2Response, expectedNTLMV2Response) != 0 {+		t.Errorf("got:\n%s\nexpected:\n%s", hex.Dump(ntlmV2Response), hex.Dump(expectedNTLMV2Response))+	}++	expectedLMV2Response, _ := hex.DecodeString("d6e6152ea25d03b7c6ba6629c2d6aaf0ffffff0011223344")+	lmV2Response := getLMV2Response(target, username, password, challenge, nonce)+	if bytes.Compare(lmV2Response, expectedLMV2Response) != 0 {+		t.Errorf("got:\n%s\nexpected:\n%s", hex.Dump(ntlmV2Response), hex.Dump(expectedNTLMV2Response))+	}+}++func TestGetTargetInformationBlock(t *testing.T) {+	bytes, _ := hex.DecodeString("4e544c4d53535000020000000600060038000000058289026999bc21067c77f40000000000000000ac00ac003e0000000a0039380000000f4600570042000200060046005700420001000c00590037004100410041003400040022006000700065002e00610058006e0071006e0070006e00650074002e0063006f006d00030030007900370041004100410034002e006000700065002e00610058006e0071006e0070006e00650074002e0063006f006d00050024006100610058006d002e00610058006e0071006e0070006e00650074002e0063006f006d00070008007d9647e8aed6d50100000000")+	info, target, err := getTargetInformationData(bytes)+	assert.Nil(t, err)

We really really don't need to pull in a new module for testing three variable values.

sboutzen

comment created time in a month

Pull request review commentdenisenkom/go-mssqldb

Add support for NTLMv2

 func (auth *ntlmAuth) NextBytes(bytes []byte) ([]byte, error) { 	var lm, nt []byte 	if (flags & _NEGOTIATE_EXTENDED_SESSIONSECURITY) != 0 { 		nonce := clientChallenge()-		var lm_bytes [24]byte-		copy(lm_bytes[:8], nonce[:])-		lm = lm_bytes[:]-		nt_bytes := ntlmSessionResponse(nonce, challenge, auth.Password)-		nt = nt_bytes[:]+		if (flags & _NEGOTIATE_TARGET_INFO) != 0 {+			timestamp := time.Now()+			targetInformationBytes, target, err := getTargetInformationData(bytes)+			if err != nil {+				return nil, err+			}++			nt = getNTLMV2Response(target, auth.UserName, auth.Password, challenge, nonce, targetInformationBytes, timestamp)

I understand you are trying to encapsulate here, but getNTLMV2Response and getLMV2Response takes many of the same inputs, and does many of the same things.

Wouldn't it make sense to have one function that returned both the NTLM and LM response at the same time?

sboutzen

comment created time in a month

Pull request review commentdenisenkom/go-mssqldb

Add support for NTLMv2

 func ntlmSessionResponse(clientNonce [8]byte, serverChallenge [8]byte, password 	return response(hash, passwordHash) } +func ntlmHashNoPadding(val string) []byte {+	hash := make([]byte, 16)+	h := md4.New()+	h.Write(utf16le(val))+	h.Sum(hash[:0])++	return hash+}++func hmacMD5(passwordHash, data []byte) []byte {+	hmacEntity := hmac.New(md5.New, passwordHash)+	hmacEntity.Write(data)++	return hmacEntity.Sum(nil)+}++func getTargetInformationData(bytes []byte) (infoBytes []byte, target string, err error) {+	type2MessageLength := len(bytes)+	targetNameAllocated := binary.LittleEndian.Uint16(bytes[14:16])+	targetNameOffset := binary.LittleEndian.Uint32(bytes[16:20])+	if type2MessageLength < int(targetNameOffset + uint32(targetNameAllocated)) {+		return nil, target, errorInvalidServerType2Message+	}++	target, err = decodeUTF16le(bytes[targetNameOffset:targetNameOffset+uint32(targetNameAllocated)])+	if err != nil {+		return nil, target, err+	}++	targetInformationAllocated := binary.LittleEndian.Uint16(bytes[42:44])+	targetInformationDataOffset := binary.LittleEndian.Uint32(bytes[44:48])+	if type2MessageLength < int(targetInformationDataOffset + uint32(targetInformationAllocated)) {+		return nil, target, errorInvalidServerType2Message+	}++	targetInformationBytes := make([]byte, targetInformationAllocated)+	copy(targetInformationBytes, bytes[targetInformationDataOffset:targetInformationDataOffset+uint32(targetInformationAllocated)])++	return targetInformationBytes, target, nil+}++func getNTLMV2Response(target, username, password string, challenge, nonce [8]byte, targetInformation []byte, timestamp time.Time) []byte {+	ntlmHash := ntlmHashNoPadding(password)+	usernameAndTargetBytes := utf16le(strings.ToUpper(username) + target)+	ntlmV2Hash := hmacMD5(ntlmHash, usernameAndTargetBytes)+	targetInfoLength := len(targetInformation)+	blob := make([]byte, 32+targetInfoLength)+	binary.BigEndian.PutUint32(blob[:4], 0x01010000)+	binary.BigEndian.PutUint32(blob[4:8], 0x00000000)+	binary.BigEndian.PutUint64(blob[8:16], uint64(timestamp.UnixNano()))+	copy(blob[16:24], nonce[:])+	binary.BigEndian.PutUint32(blob[24:28], 0x00000000)+	copy(blob[28:], targetInformation)+	binary.BigEndian.PutUint32(blob[28+targetInfoLength:], 0x00000000)+	challengeLength := len(challenge)+	blobLength := len(blob)+	challengeAndBlob := make([]byte, challengeLength + blobLength)+	copy(challengeAndBlob[:challengeLength], challenge[:])+	copy(challengeAndBlob[challengeLength:], blob)+	finalKey := hmacMD5(ntlmV2Hash, challengeAndBlob)+	response := append(finalKey, blob...)++	return response+}++func getLMV2Response(target, username, password string, challenge, nonce [8]byte) []byte {

func level doc with reference to spec would be appreciated.

sboutzen

comment created time in a month

Pull request review commentdenisenkom/go-mssqldb

Add support for NTLMv2

 func ntlmSessionResponse(clientNonce [8]byte, serverChallenge [8]byte, password 	return response(hash, passwordHash) } +func ntlmHashNoPadding(val string) []byte {+	hash := make([]byte, 16)+	h := md4.New()+	h.Write(utf16le(val))+	h.Sum(hash[:0])++	return hash+}++func hmacMD5(passwordHash, data []byte) []byte {+	hmacEntity := hmac.New(md5.New, passwordHash)+	hmacEntity.Write(data)++	return hmacEntity.Sum(nil)+}++func getTargetInformationData(bytes []byte) (infoBytes []byte, target string, err error) {+	type2MessageLength := len(bytes)+	targetNameAllocated := binary.LittleEndian.Uint16(bytes[14:16])+	targetNameOffset := binary.LittleEndian.Uint32(bytes[16:20])+	if type2MessageLength < int(targetNameOffset + uint32(targetNameAllocated)) {+		return nil, target, errorInvalidServerType2Message+	}++	target, err = decodeUTF16le(bytes[targetNameOffset:targetNameOffset+uint32(targetNameAllocated)])+	if err != nil {+		return nil, target, err+	}++	targetInformationAllocated := binary.LittleEndian.Uint16(bytes[42:44])+	targetInformationDataOffset := binary.LittleEndian.Uint32(bytes[44:48])+	if type2MessageLength < int(targetInformationDataOffset + uint32(targetInformationAllocated)) {+		return nil, target, errorInvalidServerType2Message

More specific error message please.

sboutzen

comment created time in a month

Pull request review commentdenisenkom/go-mssqldb

Add support for NTLMv2

 func (auth *ntlmAuth) NextBytes(bytes []byte) ([]byte, error) { 	var lm, nt []byte 	if (flags & _NEGOTIATE_EXTENDED_SESSIONSECURITY) != 0 {

Starting here, we see the server wants to neg session security, and we want to populate lm and nt messages.

If we pulled this out into a separate function, we can use a pattern of early returns to remove these nested if/elses, which makes reading harder. Now that you are adding a nested if/else, please try to do so.

sboutzen

comment created time in a month

Pull request review commentdenisenkom/go-mssqldb

Add support for NTLMv2

 func ntlmSessionResponse(clientNonce [8]byte, serverChallenge [8]byte, password 	return response(hash, passwordHash) } +func ntlmHashNoPadding(val string) []byte {+	hash := make([]byte, 16)+	h := md4.New()+	h.Write(utf16le(val))+	h.Sum(hash[:0])++	return hash+}++func hmacMD5(passwordHash, data []byte) []byte {+	hmacEntity := hmac.New(md5.New, passwordHash)+	hmacEntity.Write(data)++	return hmacEntity.Sum(nil)+}++func getTargetInformationData(bytes []byte) (infoBytes []byte, target string, err error) {+	type2MessageLength := len(bytes)+	targetNameAllocated := binary.LittleEndian.Uint16(bytes[14:16])+	targetNameOffset := binary.LittleEndian.Uint32(bytes[16:20])+	if type2MessageLength < int(targetNameOffset + uint32(targetNameAllocated)) {+		return nil, target, errorInvalidServerType2Message+	}++	target, err = decodeUTF16le(bytes[targetNameOffset:targetNameOffset+uint32(targetNameAllocated)])

ucs22str

sboutzen

comment created time in a month

Pull request review commentdenisenkom/go-mssqldb

Add support for NTLMv2

 func ntlmSessionResponse(clientNonce [8]byte, serverChallenge [8]byte, password 	return response(hash, passwordHash) } +func ntlmHashNoPadding(val string) []byte {+	hash := make([]byte, 16)+	h := md4.New()+	h.Write(utf16le(val))+	h.Sum(hash[:0])++	return hash+}++func hmacMD5(passwordHash, data []byte) []byte {+	hmacEntity := hmac.New(md5.New, passwordHash)+	hmacEntity.Write(data)++	return hmacEntity.Sum(nil)+}++func getTargetInformationData(bytes []byte) (infoBytes []byte, target string, err error) {+	type2MessageLength := len(bytes)+	targetNameAllocated := binary.LittleEndian.Uint16(bytes[14:16])+	targetNameOffset := binary.LittleEndian.Uint32(bytes[16:20])+	if type2MessageLength < int(targetNameOffset + uint32(targetNameAllocated)) {+		return nil, target, errorInvalidServerType2Message

I'd prefer you put in a more specific error message here: fmt.Errorf("mssql: NTLMv2 while parsing X message length %d too small for target offset %d", x, y)

sboutzen

comment created time in a month

Pull request review commentdenisenkom/go-mssqldb

Add support for NTLMv2

 func ntlmSessionResponse(clientNonce [8]byte, serverChallenge [8]byte, password 	return response(hash, passwordHash) } +func ntlmHashNoPadding(val string) []byte {+	hash := make([]byte, 16)+	h := md4.New()+	h.Write(utf16le(val))+	h.Sum(hash[:0])++	return hash+}++func hmacMD5(passwordHash, data []byte) []byte {+	hmacEntity := hmac.New(md5.New, passwordHash)+	hmacEntity.Write(data)++	return hmacEntity.Sum(nil)+}++func getTargetInformationData(bytes []byte) (infoBytes []byte, target string, err error) {+	type2MessageLength := len(bytes)+	targetNameAllocated := binary.LittleEndian.Uint16(bytes[14:16])+	targetNameOffset := binary.LittleEndian.Uint32(bytes[16:20])+	if type2MessageLength < int(targetNameOffset + uint32(targetNameAllocated)) {+		return nil, target, errorInvalidServerType2Message+	}++	target, err = decodeUTF16le(bytes[targetNameOffset:targetNameOffset+uint32(targetNameAllocated)])+	if err != nil {+		return nil, target, err+	}++	targetInformationAllocated := binary.LittleEndian.Uint16(bytes[42:44])+	targetInformationDataOffset := binary.LittleEndian.Uint32(bytes[44:48])+	if type2MessageLength < int(targetInformationDataOffset + uint32(targetInformationAllocated)) {+		return nil, target, errorInvalidServerType2Message+	}++	targetInformationBytes := make([]byte, targetInformationAllocated)+	copy(targetInformationBytes, bytes[targetInformationDataOffset:targetInformationDataOffset+uint32(targetInformationAllocated)])++	return targetInformationBytes, target, nil+}++func getNTLMV2Response(target, username, password string, challenge, nonce [8]byte, targetInformation []byte, timestamp time.Time) []byte {

A bit of docs on this function would be great. Especially references to a spec. (no need to add docs within the function, but I have no way to know what it is expected to do right now)

sboutzen

comment created time in a month

Pull request review commentdenisenkom/go-mssqldb

Add support for NTLMv2

 func ntlmSessionResponse(clientNonce [8]byte, serverChallenge [8]byte, password 	return response(hash, passwordHash) } +func ntlmHashNoPadding(val string) []byte {+	hash := make([]byte, 16)+	h := md4.New()+	h.Write(utf16le(val))+	h.Sum(hash[:0])++	return hash+}++func hmacMD5(passwordHash, data []byte) []byte {+	hmacEntity := hmac.New(md5.New, passwordHash)+	hmacEntity.Write(data)++	return hmacEntity.Sum(nil)+}++func getTargetInformationData(bytes []byte) (infoBytes []byte, target string, err error) {

rename func to something more descriptive. No "data" include subject "ntlmv2" or similar or even more specific. Don't make var name "bytes". You could call it payload, or even data, but not bytes.

Don't call that infoBytes, just info would be fine. A comment on some of these messages would be helpful, especially references to a NTLMv2 specification.

sboutzen

comment created time in a month

Pull request review commentdenisenkom/go-mssqldb

Add support for NTLMv2

 func ntlmSessionResponse(clientNonce [8]byte, serverChallenge [8]byte, password 	return response(hash, passwordHash) } +func ntlmHashNoPadding(val string) []byte {+	hash := make([]byte, 16)+	h := md4.New()+	h.Write(utf16le(val))+	h.Sum(hash[:0])++	return hash+}++func hmacMD5(passwordHash, data []byte) []byte {+	hmacEntity := hmac.New(md5.New, passwordHash)+	hmacEntity.Write(data)++	return hmacEntity.Sum(nil)+}++func getTargetInformationData(bytes []byte) (infoBytes []byte, target string, err error) {+	type2MessageLength := len(bytes)+	targetNameAllocated := binary.LittleEndian.Uint16(bytes[14:16])+	targetNameOffset := binary.LittleEndian.Uint32(bytes[16:20])

Please ensure that the type2MessageLength is > 20 before addressing that size.

sboutzen

comment created time in a month

Pull request review commentdenisenkom/go-mssqldb

Add support for NTLMv2

 go 1.11  require ( 	github.com/golang-sql/civil v0.0.0-20190719163853-cb61b32ac6fe+	github.com/stretchr/testify v1.4.0

Please don't add a testing module. Please just use vanilla testing.T.

sboutzen

comment created time in a month

Pull request review commentdenisenkom/go-mssqldb

Add support for NTLMv2

 func utf16le(val string) []byte { 	return v } +func decodeUTF16le(val []byte) (string, error) {

The function you want is ucs22str I think. If this is different, then add a comment explaining the differences.

sboutzen

comment created time in a month

issue commentdenisenkom/go-mssqldb

Reset/handle/mark bad connections

@excavador

Server is SQL Server 2019 (Linux). In queries_go110_test.go I added:


func TestBadConnection(t *testing.T) {
	ctx, cancel := context.WithTimeout(context.Background(), time.Second*30)
	defer cancel()

	db := open(t)
	defer db.Close()

	db.SetMaxIdleConns(1)
	db.SetMaxOpenConns(1)

	ck := func(err error) (done bool) {
		switch err {
		default:
			t.Fatal(err)
			return true
		case nil:
			return false
		case context.Canceled, context.DeadlineExceeded:
			return true
		}
	}

	var err error
	for ctx.Err() == nil {
		func() {
			var tx *sql.Tx
			tx, err = db.BeginTx(ctx, nil)
			if ck(err) {
				return
			}
			defer tx.Rollback()

			// Execute "long" query.
			_, err = tx.ExecContext(ctx, "WAITFOR DELAY '00:00:01';")
			if ck(err) {
				return
			}

			err = tx.Commit()
			if ck(err) {
				return
			}

			// Wait for small delay.
			time.Sleep(time.Millisecond * 3)
		}()
	}
}

But I was unable to reproduce. Can you reproduce with this code or can you modify this snip to reproduce it? To test, you'll need to set the SQLSERVER_DSN env var.

jozuenoon

comment created time in a month

issue commentgolang/go

database/sql: support tracing queries

@tcolgate I'm not a fan of wrapping drivers either. Can someone provide a summery of behavior different wrappers provide? It is obvious there is a need; it isn't obvious the exact outcomes integrated hooks would need to deliver. I can guess as I have above, but I'd prefer to see real code with real examples we can point to.

kardianos

comment created time in a month

issue commentgolang/go

database/sql: nested transaction or save point support

Sounds good. I'm fine leaving BeginTx out, at least of this initial implementation. I agree that you don't want "allow" parallel transactions (most if not all that would be impossible).

If this is implemented, I'm convinced we will need to turn *Tx into some type of wrapper for an inner *txState. Without this, we will be unable to prevent parent queries to be run on "child" transactions.

If this sounds good, we can move over further review to gerrit. But design overview and questions are great to have here first.

cznic

comment created time in a month

issue commentgolang/go

database/sql: nested transaction or save point support

Reviewing here initially is fine. You can also submit the change for review: https://golang.org/doc/contribute.html#sending_a_change_gerrit

  1. I would avoid random, and just name a sequential name like from the save point name index.
  2. I would replace the ExecContext calls with a new interface defined on the tx.txi.
  3. It would be good to use BeginTx too, but perhaps (for now, require TxOptions to be nil), but the first pass could just stick with Begin as that would be simpler.
  4. I think it would be important for parent Tx to not be able to run a query (except rollback and commit) unless the child Tx has finished with rollback or commit.

Possible interface defined in database/sql/driver/driver.go:

type ChildTx interface {
    Savepoint(ctx context.Context, name string) error
    CommitTo(ctx context.Context, name string) error
    RollbackTo(ctx context.Context, name string) error
}

Different DBMSs use different syntax (MS SQL Server and Oracle use different syntax then MySQL.

In order to accomplish (4), many of the fields in *Tx would need to be defined on an internal new *txState struct that is shared between all related *Tx. The *Tx would then hold the current savepoint name (if any), possibly a ctx and release function (unsure), and a *txState that holds the stmts and txi.

cznic

comment created time in a month

issue commentgolang/go

database/sql: nested transaction or save point support

@jonbodner Yes, you will want to work from tip.

I'm a little worried about complicating the Tx struct and embedded stmt handling too much. Also running a transaction on the non-active Tx should return an error.

cznic

comment created time in a month

issue commentdenisenkom/go-mssqldb

Reset/handle/mark bad connections

@excavador If I understand you correctly, the following will reproduce this:

ctx, cancel := context.WithCancel(context.Background())
defer cancel()

db, err := sql.OpenDB(...)
if err != nil {
	return err
}
defer db.Close()

var tx *sql.Tx
if tx, err = db.BeginTx(ctx, nil); err != nil {
	return err
}
return nil

Or is there more to it? Is there a way you can reproduce it in a single file I can run?

jozuenoon

comment created time in a month

startedkoreader/koreader

started time in a month

issue commentgolang/go

database/sql: customize JSON marshalling for nullable fields

Looking at your play link, I see more clearly what you mean. You would like to define the methods on the Nullable structs.

Yeah, that would likely be a backwards in-compatible change.

toqueteos

comment created time in a month

issue closedgolang/go

database/sql: customize JSON marshalling for nullable fields

What version of Go are you using (go version)?

<pre> $ go version go version go1.13.6 windows/amd64 </pre>

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

<details><summary><code>go env</code> Output</summary><br><pre> $ go env set GO111MODULE= set GOARCH=amd64 set GOBIN= set GOCACHE=C:\Users\Carlos\AppData\Local\go-build set GOENV=C:\Users\Carlos\AppData\Roaming\go\env set GOEXE=.exe set GOFLAGS= set GOHOSTARCH=amd64 set GOHOSTOS=windows set GONOPROXY= set GONOSUMDB= set GOOS=windows set GOPATH=D:\code\go set GOPRIVATE= set GOPROXY=direct set GOROOT=c:\go set GOSUMDB=off set GOTMPDIR= set GOTOOLDIR=c:\go\pkg\tool\windows_amd64 set GCCGO=gccgo set AR=ar set CC=gcc set CXX=g++ set CGO_ENABLED=1 set GOMOD=D:\code\sample\go.mod set CGO_CFLAGS=-g -O2 set CGO_CPPFLAGS= set CGO_CXXFLAGS=-g -O2 set CGO_FFLAGS=-g -O2 set CGO_LDFLAGS=-g -O2 set PKG_CONFIG=pkg-config set GOGCCFLAGS=-m64 -mthreads -fmessage-length=0 -fdebug-prefix-map=C:\Users\Carlos\AppData\Local\Temp\go-build116244046=/tmp/go-build -gno-record-gcc-switches </pre></details>

What did you do?

<!-- If possible, provide a recipe for reproducing the error. A complete runnable program is good. A link on play.golang.org is best. -->

Serialize any of the database/sql null fields as JSON using encoding/json.

https://play.golang.org/p/_sFxOknJhkH

What did you expect to see?

I expect to see all the database/sql null fields serialized as nulls when their Valid == false.

{
	"bool": null,
	"float64": null,
	"int32": null,
	"int64": null,
	"string": null,
	"time": null
}

What did you see instead?

The zero value.

{
	"bool": {
		"Bool": false,
		"Valid": false
	},
	"float64": {
		"Float64": 0,
		"Valid": false
	},
	"int32": {
		"Int32": 0,
		"Valid": false
	},
	"int64": {
		"Int64": 0,
		"Valid": false
	},
	"string": {
		"String": "",
		"Valid": false
	},
	"time": {
		"Time": "0001-01-01T00:00:00Z",
		"Valid": false
	}
}

closed time in a month

toqueteos

issue commentgolang/go

database/sql: customize JSON marshalling for nullable fields

We won't add a MarshalJSON method to Rows. I will be closing this issue.

toqueteos

comment created time in a month

issue commentgolang/go

database/sql: customize JSON marshalling for nullable fields

@toothrot

Please observe https://github.com/golang-sql/table . You can get custom table marshaling of arbitrary tables using this method.

toqueteos

comment created time in a month

issue commentgolang/go

database/sql: add support for PutData and GetData

@akhilravuri1 The Db.Conn.Raw method may help you interact with a driver's native capability. However, I do not expect to add this type of functionality into the current database/sql interface.

I suggest we decline/close this request.

akhilravuri1

comment created time in a month

issue commentgolang/go

database/sql: change default SQL connection lifetime to shorter value

No.

Having any connection lifetime to me is... questionable at best. If you need a shorter lifetime, please set the DB pool options.

mcandre

comment created time in a month

issue closedgolang/go

database/sql: change default SQL connection lifetime to shorter value

The default SQL connection lifetime appears to be zero value, with the result that connections often break from (My)SQL server side default configurations of 8 hours.

Can Go change the default connection lifetime to a lesser value, such as 6 hours or 4 hours, in order to integrate better with typical SQL servers with regular batch, idle query patterns?

https://golang.org/src/database/sql/sql.go

closed time in a month

mcandre

issue commentgolang/go

database/sql: prepare command is always sent again when executing a statement on a DB pool with multiple open connections

@nishaad78 I'm glad you choose to move to a dedicated connection.

Of the entire sql package code base, I find the statement sub-pools the most challenging. Your CL may be fine, but I highly doubt I will be able to merge or maintain it.

nishaad78

comment created time in a month

issue commentgolang/go

database/sql: reuses expired connections with strategy= alwaysNewConn

@gwax Please try out this CL: https://go-review.googlesource.com/c/go/+/216197

You'll need both in the series, and you should implement the ConnectionDiscarder on the driver. But it should work.

gwax

comment created time in a month

issue commentgolang/go

database/sql: nested transaction or save point support

@jonbodner This proposal has been accepted, and I would welcome an implementation, esp if the code doesn't complicate the existing implementation. This would be API support for nested transactions; the benefit of being able to create a consistent Queryer interface over both DB and Tx is too large to implement the more "raw" savepoint interface.

I've thought about the benefit/cost working through various line of business applications. I agree, it would be good to have this API.

cznic

comment created time in a month

created taggolang-sql/table

tagv0.0.6

created time in a month

push eventgolang-sql/table

Daniel Theophanes

commit sha b8c5a547d168bc0760c399ba843b81276d2faaad

table: add README.md

view details

push time in a month

created taggolang-sql/table

tagv0.0.5

created time in a month

push eventgolang-sql/table

Daniel Theophanes

commit sha 2f49cfed2cc056386e871536df7e6a39f9457fe3

table: always use IndexError, add Queryer interface, rename Value

view details

push time in a month

push eventsolidcoredata/databus

Daniel Theophanes

commit sha bc931a06aff6a41e785a4aad9b87b8e85e270234

v2: notes about types and tables, start to try an in-memory SQL engine

view details

push time in a month

startedcenkalti/rain

started time in a month

push eventsolidcoredata/databus

Daniel Theophanes

commit sha 6ff1c9f4d7a6dd8ddd236415db156ca98c6a21dc

v2: notes about resolving types

view details

push time in a month

more