profile
viewpoint

google/go-cloud 6828

The Go Cloud Development Kit (Go CDK): A library and tools for open cloud development in Go.

google/wire 3200

Compile-time Dependency Injection for Go

google/codesearch 2129

Fast, indexed regexp search over large file trees

rsc/2fa 1014

Two-factor authentication on the command line

rsc/c2go 477

C to Go translation tool supporting Go toolchain migration

google/licensecheck 191

The licensecheck package classifies license files and heuristically determines how well they correspond to known open source licenses.

rsc/corpus 185

Corpus of interesting Go code

rsc/benchstat 120

Benchstat computes and compares statistics about benchmarks.

rsc/diff 85

Diff(1)

rsc/dbstore 44

Data structure interface to database/sql

push eventgoogle/licensecheck

Russ Cox

commit sha 9580b5fd1d6c7d57f80d9344cb7879d5c62228b3

licensecheck: do not look for URLs in text matches The old code was somehow finding URLs inside text matches. I'm not sure exactly what was wrong but I rewrote the loop header to be simpler.

view details

push time in 5 days

issue closedgoogle/licensecheck

Add license MIT No Attribution (SPDX MIT-0)

Please add MIT No Attribution (MIT-0).

I don't know how prevalent it is in the Go ecosystem, but as a license it is used by e.g. Amazon

Apologies for not making this a pull request but signing a CA for this was too high friction.

SPDX-License-Identifier: MIT-0

Text:

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

closed time in 5 days

golightlyb

issue commentgoogle/licensecheck

Add license MIT No Attribution (SPDX MIT-0)

Fixed in c20d995.

golightlyb

comment created time in 5 days

issue closedgoogle/licensecheck

BSD-3-clause variants no longer match properly

At 5cc4f616f8db, a BSD-3-Clause variant like https://github.com/d0ngw/cache/blob/v0.1.0/LICENSE no longer matches well (we are back to a split BSD-2/BSD-3). I think the difference is "names" in the license text vs. "name" in the template.

closed time in 5 days

jba

issue commentgoogle/licensecheck

BSD-3-clause variants no longer match properly

All fixed. Not worried about the change in AndreyZWorkAccount.

jba

comment created time in 5 days

push eventgoogle/licensecheck

Russ Cox

commit sha e3fefcd4315eea2f7a87ffbfad90f41a6a8cca5f

licensecheck: more wildcards in ISC, BSD licenses (There is a lot more variation in these than you might expect.)

view details

Russ Cox

commit sha 66f8881cb33b2e053f2b5fd0fada691de3bde816

licensecheck: unescape “ and ” in input Otherwise it looks like literal text ldquo/rdquo, which break up the matches.

view details

Russ Cox

commit sha 0555b02d4275577816acdd11833431a46ebdc60c

licensecheck: add NCSA license The NCSA license is an odd hybrid that starts like the MIT license but then inserts the usual BSD-3 clauses. Having it in the corpus avoids mistaken partial matches for MIT and BSD-3-Clause.

view details

Russ Cox

commit sha af09bd954bad0253e9cf8ea0ccb89d7c726b1b4a

licensecheck: add bzip2 license2 The bzip2 license is an odd hybrid that looks like a mix of the BSD and Zlib licenses. Having it in the corpus avoids mistaken partial matches for those.

view details

Russ Cox

commit sha a8f82f7195a67481d8be592f45360dda907e674b

licensecheck: add CommonsClause to set of recognized licenses This gets added to various licenses to prohibit commercial use, so it is important to be able to recognize. See https://commonsclause.com/.

view details

Russ Cox

commit sha a3d19cfa4546de1e23fa1d4fecd0e209e799efc0

licensecheck: apply opts.Threshold as documented As we get more licenses it is important not to report noisy matches. For example it's easy to find about 6% of bzip2 in other licenses that we don't yet recognize.

view details

Russ Cox

commit sha fa6b913826a42256f0464c113aea9a1985641fcd

licensecheck: lower MinLength to help resync after hyphenated word-wrap See testdata/MIT.t3.

view details

Russ Cox

commit sha 04d5c1878608f9f14abd37d6f0060315a10eaa79

licensecheck: recognize header forms of GPL and MIT in addition to Apache 2.0 Renamed Apache-2.0-User to Apache-2.0-Header for consistency. For clarity of results, dropped the truncation of the -User/-Header suffix.

view details

Russ Cox

commit sha c20d9955c2714194f809700ebf75027f0ea07fea

licensecheck: add MIT-0 license This is the MIT license without the one condition.

view details

Russ Cox

commit sha 3d40d1fd9f15a56451319efbe24bb6ea87a9174a

licensecheck: add OpenSSL license Used by OpenSSL alone but copied into other projects that incorporate code. Also important to recognize because it is non-GPL-compatible.

view details

Russ Cox

commit sha 1f5d45128a236a1dd8598fe9dbff0265b4014818

licensecheck: add MPL-2.0-Header

view details

push time in 5 days

fork rsc/compress

Collection of compression related Go packages.

fork in 5 days

push eventgoogle/licensecheck

Russ Cox

commit sha 5cc4f616f8db561b3005f5038e69177910410a21

licensecheck: add a wildcard to final clause of 3- and 4-clause BSD This avoids the license check splitting the match in two and possibly using a different license for the first part (like BSD-2-Clause). testdata/BSD-3-Clause.t2 says "Neither the name of the Project project nor the names of its authors and contributors" instead of the usual "Neither the name of the copyright holder nor the names of its contributors" Without this CL, the match result is: 93.7% BSD-2-Clause 37.6% 0,560 BSD-3-Clause 62.0% 647,$ With this CL, using a blank in that clause, the match result is: 100% BSD-3-Clause 100% 0,$

view details

push time in 5 days

push eventgoogle/licensecheck

Russ Cox

commit sha b1588200237b2835fed6f245ba226ed4bcec3d5b

licensecheck: fix race in race test All the tests were using the final file. Oops.

view details

Russ Cox

commit sha 7388e7b0608d7b4af65274327c2fa5f2e19965cc

licensecheck: recognize ported Creative Commons URLs Recognize http://creativecommons.org/licenses/by-nc-nd/2.0/us as equivalent to http://creativecommons.org/licenses/by-nc-nd/2.0 Fixes #4.

view details

push time in 6 days

issue closedgoogle/licensecheck

Jurisdiction-ported Creative Commons license URLs are not recognised

The CC BY 3.0 license (and possibly others?) have country coded ports indicated by a two-letter country code in the URL path. These are missed by license check. An example of this is the (old) license for ajstarks/svgo:

The contents of this repository are Licensed under 
the Creative Commons Attribution 3.0 license as described in
http://creativecommons.org/licenses/by/3.0/us/

closed time in 6 days

kortschak

push eventgoogle/licensecheck

Russ Cox

commit sha d48f3bd76aca744a9a8fd8a8fbc340a525aeb8fe

licensecheck: add testdata file name to two more test failure messages Noted post commit in https://github.com/google/licensecheck/commit/63e806fe#r37263209

view details

push time in 6 days

push eventgoogle/licensecheck

Russ Cox

commit sha 872702809ddf6dd97cdbc7736058786a4000b59d

licensecheck: make Cover safe for concurrent use again The use of the dictionary during normalize is not safe. Fix it by not adding to the dictionary when normalizing inputs. This is also more efficient, since it avoids adding input words to the dictionary. All the dictionary needs are license words. Fixes #16.

view details

push time in 6 days

issue closedgoogle/licensecheck

Cover no longer thread-safe

Previously, Cover could be invoked concurrently. Now (a18cfb0627bf), thanks to the shared state in Checker, it can't.

The particular error I saw was "concurrent map read and map write", with one offender at normalize.go:57.

This isn't that important since checking is now a lot faster, but it still takes about 30s to run on about 29K licenses.

closed time in 6 days

jba

issue commentgoogle/licensecheck

Cover no longer thread-safe

Will fix.

jba

comment created time in 6 days

push eventgoogle/licensecheck

Russ Cox

commit sha 5b2f5295c170f8eef08081b54b0f2dbbb4de6152

licensecheck: fix backward gap check going too far Looking backward across a gap for the word Copyright is too easy to jump into the previous license. So stop at the first one. There may be a better way to do this by running the license matches in parallel instead of series, which would let us know when we were scanning backward into a previous license, but for now this should be OK. Fixes #15.

view details

push time in 6 days

issue closedgoogle/licensecheck

MIT-BSD-3 concatenation no longer matched

The file https://github.com/philpearl/csv/blob/v0.0.3/LICENSE is more-or-less the concatenation of an MIT and a BSD-3-Clause.

At e43db20c1115, Cover reported that well: 94.5% MIT:100.0% BSD-3-Clause:98.1%

At current master (a18cfb0627bf), it reports: 67.9% BSD-3-Clause:98.1%

closed time in 6 days

jba

Pull request review commentgoogle/licensecheck

add Checker API, make scans 7X faster, more

 func licenseType(name string) Type { 	return Other } +// A phrase is a sequence of words used as a key for startIndexes.+// Empirically, two words are best; more is slower.+type phrase [2]int32+ type license struct { 	typ          Type 	name         string+	text         string 	doc          *document-	startIndexes map[string][]int+	startIndexes map[phrase][]int }  type document struct {-	text    []byte   // Original text.-	words   []string // Normalized words.-	byteOff []int32  // ith byteOff is byte offset of ith word in original text.+	text    []byte  // Original text.+	words   []int32 // Normalized words (indexes into c.words)+	byteOff []int32 // ith byteOff is byte offset of ith word in original text.+}++// A Checker matches a set of known licenses.+type Checker struct {+	licenses []license+	urls     map[string]string+	dict     map[string]int32 // dict maps word to index in words+	words    []string         // list of known words+}++// A License describes a single license that can be recognized.+// At least one of the Text or the URL should be set.+type License struct {

I'll talk to Rob about it next time we meet. We can add a Type field later if we need to I suppose but first we need to figure out if the Type enum makes that much sense at all.

rsc

comment created time in 6 days

push eventgoogle/licensecheck

Russ Cox

commit sha d8f49991db08196e78f150217170dc808f2679b7

licensecheck: add Checker for defining custom license sets Instead of forcing the use of a hard-coded builtin set, this lets clients of the library define their own license sets. There is still a builtin set, but being able to define alternate sets should reduce the pressure to add every last possible license to the built-in set and let users adjust the set without forking the module. Fixes #10.

view details

Russ Cox

commit sha 3fb817561becc2085830c0ea4ccfa6014c42be9c

licensecheck: 2.3X faster scans (0.73 MB/s -> 1.68 MB/s) Represent normalized text as a list of integer word IDs instead of strings. Then the word comparisons are integer comparisons.

view details

Russ Cox

commit sha 10abcc3c2c9697d92d4a4d2435b6bf14f558f3f7

licensecheck: 3X faster scans (1.68 MB/s -> 5.15 MB/s) Use a word pair to look up possible starting matches instead of just one word. Also skip over the word pair when checking the length of the match, since we already know the initial pair matches.

view details

Russ Cox

commit sha ee88a46801e6b3a7a4e69d90d2bfcb6d4346ec33

licensecheck: expand match Start/End to text start/end when appropriate If a match starts at word 0 or ends at the last word, it makes sense to extend the text byte offsets to the start or end of the text, respectively. There is already a special case in the URL matcher to do this at the end of the text, but there wasn't one for the main license text matches. Adding this case makes the returned match offsets cover the whole file for a whole-file match, as seen in the testdata changes. Removed TestMatch because it is redundant with the now many test cases in the testdata directory and was fussy about the offsets changing.

view details

Russ Cox

commit sha 8385fdb7a9db9b9eb2f4c777b5e8a39291408661

licensecheck: add support for wildcards in license text In license text the sequence ___ now denotes a wildcard that can be filled in by up to 70 words. That's long enough for a handful of copyright lines in a row (see testdata/MIT.t2) but short enough that it can't skip over an entire license. (Our shortest two recognized licenses are Apache-2.0-User at 80 and BSD-0-Clause at 99.) This commit updates the licenses to use ___ where appropriate. This fixes #6. This commit also removes removeCopyrightLines (d686698), which turns out to have been removing quite a few lines in actual license text (see diffs in testdata/CC*) and did not handle other cases now handled by the wildcards (for example, testdata/MIT.t2). This fixes #12. Fixes #6. Fixes #12.

view details

Russ Cox

commit sha f91ba25d055f5c5a412828a74a1a078af2657e2a

licensecheck: remove hack in TestMultiCoverage This was needed before the wildcards went in.

view details

Russ Cox

commit sha a18cfb0627bf9bb7c7eb0c787e106ffc967081bc

licensecheck: add BuiltinLicenses to get list of built-in licenses

view details

push time in 6 days

issue closedgoogle/licensecheck

Apache license recognized as ECL-2

The text below, from https://github.com/hyperledger/burrow/blob/v0.21.0/LICENSE.md, is identical to the Apache 2.0 license without appendix.

At the previous head (e43db20c1115), Cover reported a single Match, to Apache-2.0:

{Name:Apache-2.0 Type:Apache Percent:89.40355329949239 Start:33 End:10172 IsURL:false}

At the current head (9240d01bd357), it reports a single match to ECL-2.0:

{Name:ECL-2.0 Type:Other Percent:81.6793893129771 Start:161 End:10172 IsURL:false} 

The text lacks ECL-2.0's preamble and its additional sentence in section 3.

                                 Apache License
                           Version 2.0, January 2004
                        http://www.apache.org/licenses/

   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION

   1. Definitions.

      "License" shall mean the terms and conditions for use, reproduction,
      and distribution as defined by Sections 1 through 9 of this document.

      "Licensor" shall mean the copyright owner or entity authorized by
      the copyright owner that is granting the License.

      "Legal Entity" shall mean the union of the acting entity and all
      other entities that control, are controlled by, or are under common
      control with that entity. For the purposes of this definition,
      "control" means (i) the power, direct or indirect, to cause the
      direction or management of such entity, whether by contract or
      otherwise, or (ii) ownership of fifty percent (50%) or more of the
      outstanding shares, or (iii) beneficial ownership of such entity.

      "You" (or "Your") shall mean an individual or Legal Entity
      exercising permissions granted by this License.

      "Source" form shall mean the preferred form for making modifications,
      including but not limited to software source code, documentation
      source, and configuration files.

      "Object" form shall mean any form resulting from mechanical
      transformation or translation of a Source form, including but
      not limited to compiled object code, generated documentation,
      and conversions to other media types.

      "Work" shall mean the work of authorship, whether in Source or
      Object form, made available under the License, as indicated by a
      copyright notice that is included in or attached to the work
      (an example is provided in the Appendix below).

      "Derivative Works" shall mean any work, whether in Source or Object
      form, that is based on (or derived from) the Work and for which the
      editorial revisions, annotations, elaborations, or other modifications
      represent, as a whole, an original work of authorship. For the purposes
      of this License, Derivative Works shall not include works that remain
      separable from, or merely link (or bind by name) to the interfaces of,
      the Work and Derivative Works thereof.

      "Contribution" shall mean any work of authorship, including
      the original version of the Work and any modifications or additions
      to that Work or Derivative Works thereof, that is intentionally
      submitted to Licensor for inclusion in the Work by the copyright owner
      or by an individual or Legal Entity authorized to submit on behalf of
      the copyright owner. For the purposes of this definition, "submitted"
      means any form of electronic, verbal, or written communication sent
      to the Licensor or its representatives, including but not limited to
      communication on electronic mailing lists, source code control systems,
      and issue tracking systems that are managed by, or on behalf of, the
      Licensor for the purpose of discussing and improving the Work, but
      excluding communication that is conspicuously marked or otherwise
      designated in writing by the copyright owner as "Not a Contribution."

      "Contributor" shall mean Licensor and any individual or Legal Entity
      on behalf of whom a Contribution has been received by Licensor and
      subsequently incorporated within the Work.

   2. Grant of Copyright License. Subject to the terms and conditions of
      this License, each Contributor hereby grants to You a perpetual,
      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
      copyright license to reproduce, prepare Derivative Works of,
      publicly display, publicly perform, sublicense, and distribute the
      Work and such Derivative Works in Source or Object form.

   3. Grant of Patent License. Subject to the terms and conditions of
      this License, each Contributor hereby grants to You a perpetual,
      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
      (except as stated in this section) patent license to make, have made,
      use, offer to sell, sell, import, and otherwise transfer the Work,
      where such license applies only to those patent claims licensable
      by such Contributor that are necessarily infringed by their
      Contribution(s) alone or by combination of their Contribution(s)
      with the Work to which such Contribution(s) was submitted. If You
      institute patent litigation against any entity (including a
      cross-claim or counterclaim in a lawsuit) alleging that the Work
      or a Contribution incorporated within the Work constitutes direct
      or contributory patent infringement, then any patent licenses
      granted to You under this License for that Work shall terminate
      as of the date such litigation is filed.

   4. Redistribution. You may reproduce and distribute copies of the
      Work or Derivative Works thereof in any medium, with or without
      modifications, and in Source or Object form, provided that You
      meet the following conditions:

      (a) You must give any other recipients of the Work or
          Derivative Works a copy of this License; and

      (b) You must cause any modified files to carry prominent notices
          stating that You changed the files; and

      (c) You must retain, in the Source form of any Derivative Works
          that You distribute, all copyright, patent, trademark, and
          attribution notices from the Source form of the Work,
          excluding those notices that do not pertain to any part of
          the Derivative Works; and

      (d) If the Work includes a "NOTICE" text file as part of its
          distribution, then any Derivative Works that You distribute must
          include a readable copy of the attribution notices contained
          within such NOTICE file, excluding those notices that do not
          pertain to any part of the Derivative Works, in at least one
          of the following places: within a NOTICE text file distributed
          as part of the Derivative Works; within the Source form or
          documentation, if provided along with the Derivative Works; or,
          within a display generated by the Derivative Works, if and
          wherever such third-party notices normally appear. The contents
          of the NOTICE file are for informational purposes only and
          do not modify the License. You may add Your own attribution
          notices within Derivative Works that You distribute, alongside
          or as an addendum to the NOTICE text from the Work, provided
          that such additional attribution notices cannot be construed
          as modifying the License.

      You may add Your own copyright statement to Your modifications and
      may provide additional or different license terms and conditions
      for use, reproduction, or distribution of Your modifications, or
      for any such Derivative Works as a whole, provided Your use,
      reproduction, and distribution of the Work otherwise complies with
      the conditions stated in this License.

   5. Submission of Contributions. Unless You explicitly state otherwise,
      any Contribution intentionally submitted for inclusion in the Work
      by You to the Licensor shall be under the terms and conditions of
      this License, without any additional terms or conditions.
      Notwithstanding the above, nothing herein shall supersede or modify
      the terms of any separate license agreement you may have executed
      with Licensor regarding such Contributions.

   6. Trademarks. This License does not grant permission to use the trade
      names, trademarks, service marks, or product names of the Licensor,
      except as required for reasonable and customary use in describing the
      origin of the Work and reproducing the content of the NOTICE file.

   7. Disclaimer of Warranty. Unless required by applicable law or
      agreed to in writing, Licensor provides the Work (and each
      Contributor provides its Contributions) on an "AS IS" BASIS,
      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
      implied, including, without limitation, any warranties or conditions
      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
      PARTICULAR PURPOSE. You are solely responsible for determining the
      appropriateness of using or redistributing the Work and assume any
      risks associated with Your exercise of permissions under this License.

   8. Limitation of Liability. In no event and under no legal theory,
      whether in tort (including negligence), contract, or otherwise,
      unless required by applicable law (such as deliberate and grossly
      negligent acts) or agreed to in writing, shall any Contributor be
      liable to You for damages, including any direct, indirect, special,
      incidental, or consequential damages of any character arising as a
      result of this License or out of the use or inability to use the
      Work (including but not limited to damages for loss of goodwill,
      work stoppage, computer failure or malfunction, or any and all
      other commercial damages or losses), even if such Contributor
      has been advised of the possibility of such damages.

   9. Accepting Warranty or Additional Liability. While redistributing
      the Work or Derivative Works thereof, You may choose to offer,
      and charge a fee for, acceptance of support, warranty, indemnity,
      or other liability obligations and/or rights consistent with this
      License. However, in accepting such obligations, You may act only
      on Your own behalf and on Your sole responsibility, not on behalf
      of any other Contributor, and only if You agree to indemnify,
      defend, and hold each Contributor harmless for any liability
      incurred by, or claims asserted against, such Contributor by reason
      of your accepting any such warranty or additional liability.

   END OF TERMS AND CONDITIONS

closed time in 6 days

jba

issue closedgoogle/licensecheck

Variable sections are not considered in license descriptions

The SPDX license descriptions (available here) include coding for sections of license that are variable (as described here, with additional notes in the Matching Guidelines paragraph here).

The absence of annotation of variable as opposed to required matching imposes the requirement for heuristic detection based on proportion of matching text, leading to both false positives with clearly incorrect classification of malicious license as OSS licenses and false negatives where perfect SPDX-match licenses are not correctly detected.

These issues have been raised at golang-dev and golang-nuts.

closed time in 6 days

kortschak

issue closedgoogle/licensecheck

allow dynamic additions to the corpus

A licensecheck user may want to recognize licenses that are not part of the built-in corpus. It would be nice if they could avail themselves of the text-matching machinery of licensecheck without being limited by the implicit legal judgments it makes in virtue of having a fixed corpus of license texts.

closed time in 6 days

jba

PR merged google/licensecheck

add Checker API, make scans 7X faster, more

See individual commits for descriptions.

  • add Checker API
  • make scans 7X faster by comparing ints instead of strings, looking up word pairs
  • expand match Start/End when appropriate
  • add wildcard support for better matching
+525 -499

1 comment

54 changed files

rsc

pr closed time in 6 days

push eventrsc/licensecheck

Russ Cox

commit sha 87d5bf0699114c4b770252a09ba89e3e36cc8811

licensecheck: add BuiltinLicenses to get list of built-in licenses

view details

push time in 6 days

Pull request review commentgoogle/licensecheck

add Checker API, make scans 7X faster, more

 func licenseType(name string) Type { 	return Other } +// A phrase is a sequence of words used as a key for startIndexes.+// Empirically, two words are best; more is slower.+type phrase [2]int32+ type license struct { 	typ          Type 	name         string+	text         string 	doc          *document-	startIndexes map[string][]int+	startIndexes map[phrase][]int }  type document struct {-	text    []byte   // Original text.-	words   []string // Normalized words.-	byteOff []int32  // ith byteOff is byte offset of ith word in original text.+	text    []byte  // Original text.+	words   []int32 // Normalized words (indexes into c.words)+	byteOff []int32 // ith byteOff is byte offset of ith word in original text.+}++// A Checker matches a set of known licenses.+type Checker struct {+	licenses []license+	urls     map[string]string+	dict     map[string]int32 // dict maps word to index in words+	words    []string         // list of known words+}++// A License describes a single license that can be recognized.+// At least one of the Text or the URL should be set.+type License struct {+	Name string+	Text string+	URL  string+}++// New returns a new Checker that recognizes the given list of licenses.+func New(licenses []License) *Checker {

I thought some more about this and I think you're right, especially if the set here grows to be useful enough that most people only need one or two special cases. Added func Default.

rsc

comment created time in 6 days

push eventrsc/licensecheck

Russ Cox

commit sha 8b2857fd1403e7e852891780c1ee6a254c9af17e

licensecheck: remove hack in TestMultiCoverage This was needed before the wildcards went in.

view details

push time in 6 days

Pull request review commentgoogle/licensecheck

add Checker API, make scans 7X faster, more

 func licenseType(name string) Type { 	return Other } +// A phrase is a sequence of words used as a key for startIndexes.+// Empirically, two words are best; more is slower.+type phrase [2]int32+ type license struct { 	typ          Type 	name         string+	text         string 	doc          *document-	startIndexes map[string][]int+	startIndexes map[phrase][]int }  type document struct {-	text    []byte   // Original text.-	words   []string // Normalized words.-	byteOff []int32  // ith byteOff is byte offset of ith word in original text.+	text    []byte  // Original text.+	words   []int32 // Normalized words (indexes into c.words)+	byteOff []int32 // ith byteOff is byte offset of ith word in original text.+}++// A Checker matches a set of known licenses.+type Checker struct {+	licenses []license+	urls     map[string]string+	dict     map[string]int32 // dict maps word to index in words+	words    []string         // list of known words+}++// A License describes a single license that can be recognized.+// At least one of the Text or the URL should be set.+type License struct {+	Name string+	Text string+	URL  string+}++// New returns a new Checker that recognizes the given list of licenses.+func New(licenses []License) *Checker {

I don't want to export the built-in list. You should use the set you need and thereby end up isolated from any changes to the built-in list.

rsc

comment created time in 6 days

Pull request review commentgoogle/licensecheck

add Checker API, make scans 7X faster, more

 func licenseType(name string) Type { 	return Other } +// A phrase is a sequence of words used as a key for startIndexes.+// Empirically, two words are best; more is slower.+type phrase [2]int32+ type license struct { 	typ          Type 	name         string+	text         string 	doc          *document-	startIndexes map[string][]int+	startIndexes map[phrase][]int }  type document struct {-	text    []byte   // Original text.-	words   []string // Normalized words.-	byteOff []int32  // ith byteOff is byte offset of ith word in original text.+	text    []byte  // Original text.+	words   []int32 // Normalized words (indexes into c.words)+	byteOff []int32 // ith byteOff is byte offset of ith word in original text.+}++// A Checker matches a set of known licenses.+type Checker struct {+	licenses []license+	urls     map[string]string+	dict     map[string]int32 // dict maps word to index in words+	words    []string         // list of known words+}++// A License describes a single license that can be recognized.+// At least one of the Text or the URL should be set.+type License struct {

I would suggest just not using the type at all, honestly.

rsc

comment created time in 6 days

Pull request review commentgoogle/licensecheck

add Checker API, make scans 7X faster, more

 func checkMatch(t *testing.T, m Match, name string, prevEnd int) { 		return 	} 	lic := findLicense(name)-	// Skip leading white space, almost certainly trimmed copyright lines. 	length := len(lic.doc.text)-	for _, c := range lic.doc.text {-		if c != ' ' && c != '\n' {-			break-		}-		length---	} 	// There is some fudge factor in the match lengths because of terminal spaces, so be forgiving. 	min, max := length-5, length+	if name == "BSD-2-Clause" {

It was needed before I added the proper wildcard support. Removed.

rsc

comment created time in 6 days

issue commentgolang/go

proposal: review meeting minutes

2020-02-12 / @rsc, @bradfitz, @andybons

  • #25348 allow && and || operators and parentheses in build tags
    • commented
  • #36606 cmd/compile: make 64-bit fields be 64-bit aligned on 32-bit systems, add //go:packed directive on structs
    • discussion ongoing
  • #34409 cmd/doc: add "// Unstable:" prefix convention
    • discussion ongoing
  • #36875 cmd/go: refine the default language version based on context
    • put on hold
  • #35667 cmd/link: Include build meta information
    • commented
  • #36681 cmd/vet: flag accidental integer division inside float casts
    • no change in consensus; declined
  • #34594 crypto/cipher: Specify nonce and tag sizes for GCM
    • commented
  • #34648 crypto/elliptic: automatically upgrade CurveParams for known curves and deprecate custom ones
    • discussion ongoing
  • #34105 crypto/elliptic: handle compressed formats in new MarshalCompressed, UnmarshalCompressed
    • commented
  • #35499 crypto/tls expose names for CurveID and SignatureScheme
    • commented
  • #31773 crypto/tls: Authority Information Access support
    • likely decline; last call for comments ⏳
  • #36337 crypto/tls: Export list of Supported TLS Client Extensions
    • commented
  • #22836 crypto/tls: add GetConfigForServer callback to *tls.Config
    • discussion ongoing
  • #35428 crypto/x509: add API to create RFC-compliant CRLs
    • commented
  • #23282 crypto/x509: add support to get SSL context from certificate store on windows
    • likely decline; last call for comments ⏳
  • #33888 crypto/x509: implement TextMarshaler for *Certificate
    • commented
  • #37172 crypto/x509: parse additional fields in CertificateRequest
    • commented
  • #21704 crypto: add Equal(PublicKey) bool method to PublicKey implementations
    • no change in consensus; accepted 🎉
  • #33430 crypto: implement Hash.String
    • commented
  • #5901 encoding/json: allow per-Encoder/per-Decoder registration of marshal/unmarshal functions
    • no change in consensus; accepted 🎉
  • #33457 image: add generic metadata support for jpeg, gif, png
    • discussion ongoing
  • #34624 io, net: implement WriterTo for pipes
    • likely decline; last call for comments ⏳
  • #35562 net/http: add new Context taking/suffixed helper functions for {Get, Head, Post, PostForm}
    • commented
  • #37168 new cryptographic assembly policy
    • commented
  • #37166 package tools
    • likely decline; last call for comments ⏳
  • #36189 panic based error handling
    • no change in consensus; declined
  • #37023 runtime: panicmem should expose the address it's panicking about.
    • commented
  • #37033 runtime: provide centralized facility for managing (c)go pointer handles
    • commented
  • #33688 testing: define naming convention for test functions
    • no change in consensus; declined
  • #31107 text/template,html/template: add ExecuteContext methods
    • commented
  • #35480 time: allow a user callback for dropped ticks
    • no change in consensus; declined
  • #29540 x/crypto/acme/autocert: Enable custom port / IP binding
    • commented
  • #35481 x/crypto/argon2: support secret value and associated data
    • commented
  • #36646 x/crypto/poly1305: deprecate public package
    • likely accept; last call for comments ⏳
  • #31044 x/crypto/ssh/terminal: move all APIs to x/term and deprecate
    • likely accept; last call for comments ⏳
  • #37132 **x/crypto/ssh: add methods to marshal private keys to the OpenSSH format **
    • commented
rsc

comment created time in 6 days

IssuesEvent

issue commentgolang/go

proposal: io, net: implement WriterTo for pipes

Sorry, I got confused by the closed proposal and set the mode wrong. We'll leave this open for one more week just to make sure it ends up in the minutes. But thanks for closing.

Stebalien

comment created time in 6 days

issue commentgolang/go

proposal: net/http: add new Context taking/suffixed helper functions for {Get, Head, Post, PostForm}

I think there is plenty of short-lived command-line use for which these functions are just fine. I do agree that in a long-running program or server or a library that might be used in one of those, the helpers are not appropriate. Let's just document that more clearly. @bradfitz volunteered to write a CL.

rittneje

comment created time in 6 days

issue commentgolang/go

proposal: runtime: provide centralized facility for managing (c)go pointer handles

And you could easily publish an package outside the standard library and see how much use it gets. It's unclear that this must be in the standard library.

oakad

comment created time in 6 days

issue commentgolang/go

proposal: runtime: panicmem should expose the address it's panicking about.

Adding just the address does help with PanicOnFault, so I could see maybe doing this. Thoughts, @aclements, @ianlancetaylor, @randall77?

seebs

comment created time in 6 days

issue commentgolang/go

Proposal: shrink types in sync package

@networkimprov, I don't believe we've decided that TryRLock is a good idea to add at all, so it seems not necessary to condition this proposal on support for it.

This is not a proposal in the sense that it does not create new API. It definitely needs a discussion and decision from the owners of the package (me, @dvyukov, @aclements is a good set).

Removing from the proposal process and changing to NeedsDecision.

balasanjay

comment created time in 6 days

issue commentgolang/go

proposal: package tools

This is definitely a general problem we are aware of. See https://research.swtch.com/deps for example. Custom checks can be built on top of the go command output of things like go list -m all or go mod graph. If you run a custom module proxy, like Athens, you can restrict the set of modules available. Or you can download them and then use go list -m all to decide if you like the set. (Note that downloading never runs code from the modules, unlike in other systems.)

As for "need a way to determine if some set of libs could potentially communicate outside their sandbox", there's really no sandbox at all, so this unfortunately isn't applicable.

In general this is not really a proposal - nothing concrete is being proposed. If you have a more specific proposal, please let us know, but this seems like a likely decline.

rbucker

comment created time in 6 days

issue commentgolang/go

proposal: cmd/go: refine the default language version based on context

I suggest we put this on hold until we are ready to make a backwards-incompatible language change. We'll know more then.

bcmills

comment created time in 6 days

issue commentgolang/go

proposal: cmd/link: Include build meta information

Note that there is a parser today at https://rsc.io/goversion/version. We could move that somewhere in golang.org/x. Many programs will be just fine shelling out to go version -m <file>. We can also add a -json flag to that command to print JSON instead.

Yes, the idea is that we'd add relevant go command line flags. We could look into relevant go environment variables too, but we don't want to break reproducible builds, so we shouldn't write down things like GOPATH, which might mention temporary directories.

michael-obermueller

comment created time in 6 days

issue commentgolang/go

proposal: crypto/tls: Authority Information Access support

Based on the discussion above (including the fact that it's so easily implemented outside the standard library), this seems like a likely decline.

ConradIrwin

comment created time in 6 days

issue commentgolang/go

proposal: crypto/x509: add support to get SSL context from certificate store on windows

Based on the discussion above and the lack of activity, this seems like a likely decline.

tsaridas

comment created time in 6 days

issue commentgolang/go

proposal: text/template,html/template: add ExecuteContext methods

I think context is far less special than error and doesn't deserve the same special-case treatment. If the special case is essential, then it sounds like maybe we don't really have a consensus here and should just leave this alone.

empijei

comment created time in 6 days

issue commentgolang/go

proposal: image: add generic metadata support for jpeg, gif, png

@nigeltao, do you think there's anything we should do here? If so, what? Thanks.

drswork

comment created time in 6 days

issue commentgolang/go

proposal: x/crypto/poly1305: deprecate public package

The reactions to https://github.com/golang/go/issues/36646#issuecomment-579366285 seem positive. This seems like a likely accept.

FiloSottile

comment created time in 6 days

issue commentgolang/go

proposal: x/crypto/ssh/terminal: move all APIs to x/term and deprecate

This seems OK. We can't delete the old code but we can deprecate it. I've seen no objections, so this seems like a likely accept.

tandr

comment created time in 6 days

issue commentgolang/go

proposal: io, net: implement WriterTo for pipes

It sounds like there is not a compelling reason to add this. Having extra interfaces and abstractions that amount to no-ops is not a great thing to do. In this case, instead of making the extra abstraction zero-copy, users should probably reach for removing an io.Pipe, as I mentioned above.

This seems like a likely decline.

Stebalien

comment created time in 6 days

issue commentgolang/go

proposal: allow && and || operators and parentheses in build tags

For the record, I'm coming around to the idea of doing this, but I want to work through the transition plan. I'm not ignoring this issue. :-)

mundaym

comment created time in 6 days

issue commentgolang/go

x/crypto/acme/autocert: Enable custom port / IP binding

Talked to @bradfitz about this (we designed this package).

It seems odd to have both Listen and Listener (maybe Listener should have been named Listen from the start but oh well). For >99% of use cases Listener is doing the right thing.

We could imagine adding a field Addr string to Manager, defaulting to ":443", but then what if you want to change the keepalives that listener does?

As it is, listener (the unexported type) is a very small wrapper around Manager itself. It probably makes sense to just copy it out. All you really need access to is m.TLSConf(), which you have.

axxelG

comment created time in 6 days

issue commentgolang/go

proposal: new cryptographic assembly policy

I seem to remember a discussion when we wrote https://github.com/golang/go/wiki/AssemblyPolicy (only 18 months ago). Does anyone know if there's a link somewhere to that discussion?

FiloSottile

comment created time in 6 days

issue commentgolang/go

proposal: crypto/x509: add API to create RFC-compliant CRLs

Adding to proposal minutes for wider visibility. Seems OK.

jefferai

comment created time in 6 days

issue commentgolang/go

proposal: crypto/x509: implement TextMarshaler for *Certificate

Adding to proposal minutes but seems headed for likely decline.

kevinburkemeter

comment created time in 6 days

issue commentgolang/go

proposal: crypto/cipher: Specify nonce and tag sizes for GCM

Adding to proposal minutes, seems headed for likely decline.

defund

comment created time in 6 days

issue commentgolang/go

proposal: x/crypto/argon2: support secret value and associated data

Adding to proposal minutes.

except

comment created time in 6 days

issue commentgolang/go

proposal: crypto/tls: Export list of Supported TLS Client Extensions

Adding to proposal minutes, seems headed for likely decline.

fishboy25uk

comment created time in 6 days

issue commentgolang/go

proposal: crypto/tls expose names for CurveID and SignatureScheme

Adding to proposal minutes, seems headed for likely accept.

izolight

comment created time in 6 days

issue commentgolang/go

crypto/elliptic: handle compressed formats in new MarshalCompressed, UnmarshalCompressed

Adding to proposal minutes, seems headed for likely accept.

roman-khimov

comment created time in 6 days

issue commentgolang/go

proposal: crypto/elliptic: automatically upgrade CurveParams for known curves and deprecate custom ones

Adding to proposal minutes, more discussion welcome.

FiloSottile

comment created time in 6 days

issue commentgolang/go

proposal: crypto: implement Hash.String

Added to proposal minutes, seems headed for likely accept.

Ma124

comment created time in 6 days

issue commentgolang/go

proposal: crypto/x509: parse additional fields in CertificateRequest

Adding to proposal minutes, seems headed for likely accept.

chauncyc

comment created time in 6 days

issue commentgolang/go

proposal: x/crypto/ssh: add methods to marshal private keys to the OpenSSH format

Will add to the proposal minutes. Seems headed for likely accept.

maraino

comment created time in 6 days

issue commentgolang/go

proposal: crypto: add Equal(PublicKey) bool method to PublicKey implementations

No change in consensus, so accepted.

erahn

comment created time in 6 days

issue commentgolang/go

proposal: encoding/json: allow per-Encoder/per-Decoder registration of marshal/unmarshal functions

No change in consensus, so accepting.

rsc

comment created time in 6 days

issue commentgolang/go

proposal: encoding/json: allow per-Encoder/per-Decoder registration of marshal/unmarshal functions

@dsnet, only that the method on Decoder/Encoder be called Register. It will be clear at call sites that a func is being passed; it doesn't add much to call it RegisterFunc.

rsc

comment created time in 6 days

issue closedgolang/go

proposal: testing: define naming convention for test functions

This proposal is for eliciting discussion around the current absent naming convention for test functions. This should not involve any changes to the language specification itself but instead serve as a guidepost for community tooling and best practices.

Background

There is precedent within the community and within the Go packages themselves to name test functions in a specific way based on their definitions. Two primary cases demonstrate this.

gotests

The popular gotests tool defines an informal specification for naming tests and uses it to auto-generate table driven tests from a given function definition. It appears to be widely accepted and used among the IDE community.

If accepted into the main specification, the IDE integrations using this tool could be further developed with additional features such as "Jump to Test for Function" / "Jump to Function for Test". Without a specification these features would be based upon an informal specification and subject to potential breakage if the community shifts to another upstream test generation tool.

(Disclaimer -- this issue came from a discussion on gotests: https://github.com/cweill/gotests/issues/106)

testing examples

In the go source for the testing package a naming convention has already been defined for "Example" test cases. Extracting the content from those docs for clarity:

The naming convention to declare examples for the package, a function F, a type T and method M on type T are:

func Example() { ... }
func ExampleF() { ... }
func ExampleT() { ... }
func ExampleT_M() { ... }

Multiple example functions for a package/type/function/method may be provided by appending a distinct suffix to the name. The suffix must start with a lower-case letter.

func Example_suffix() { ... }
func ExampleF_suffix() { ... }
func ExampleT_suffix() { ... }
func ExampleT_M_suffix() { ... }

Having a naming convention for Example test functions gives a solid starting point if we decide to move forward for a standard Test function.

Proposal

I would stop short of officially recommending a naming convention for Go to adopt, but encourage that the discussion start based on the existing examples we are seeing in gotests and in the Example function naming conventions. Looking forward to what everyone thinks!

closed time in 6 days

taiidani

issue commentgolang/go

proposal: testing: define naming convention for test functions

No change in consensus, so declining.

taiidani

comment created time in 6 days

issue closedgolang/go

proposal: panic based error handling

Since "there is really nothing wrong with using panic within a single package", as discussed in #35093, I thought I would use this to propose improving error handling. My apologies if this duplicates other proposals, and feel free to merge/close as appropriate.

I suggest adding two new functions to the errors package:

Because these primitives can be implemented in pure go1, I have built a sample implementation:

import "git.sr.ht/~urandom/errors"

func Set(v interface{}) (ret error) {
        defer errors.Handlef(&ret, "setting: %v", v)
        _, err := set(v)
        errors.Check(err)
        return nil
}

This is proposal borrows exact syntax heavily from the try proposal, however panic could be used differently:

package errors

func Check(ret *error, check error, msg string)

closed time in 6 days

arnottcr

issue commentgolang/go

proposal: panic based error handling

There's no change in consensus, so declining.

arnottcr

comment created time in 6 days

issue closedgolang/go

proposal: time: allow a user callback for dropped ticks

Hi there!

There is a known time.Ticker behavior when ticks are being dropped if user can't read from time.Ticker.C channel in time.

For some kind of "real-time" applications it may be useful to know that ticks are dropped to react somehow on such events.

I suppose it could be allowed just by extending time.Ticker struct with an optional exported callback field (and little tweaks of time.sendTime use):

const interval = time.Second

t := time.NewTicker(interval)
t.OnTickDrop = func() {
    log.Printf("warning: dropping %s tick", interval)
}

Without such feature users need to implement their own tick scheduler with very similar implementation to time's internals.

closed time in 6 days

gobwas

issue commentgolang/go

proposal: time: allow a user callback for dropped ticks

No change in consensus, so declining.

gobwas

comment created time in 6 days

issue closedgolang/go

proposal: cmd/vet: flag accidental integer division inside float casts

We have caught bugs of the form seen in https://github.com/kubernetes/kubernetes/pull/83132#pullrequestreview-346205014 a number of times in Kubernetes — namely, code that writes float64(some_int / another_int) when what is clearly meant is float64(some_int) / another_int.

It strikes me that the likelihood that anyone would ever intentionally lose precision with an integer divison and then cast the known-to-be-integral result to float64 is not necessarily exactly zero, but very low. In the rare cases where that is what's desired, it's easy to replace f := float64(a/b) with t := a/b; f := float64(t) and doing so more clearly communicates the intentional truncation.

The proposal is to have go vet flag expressions of the form float32(x / y) or float64(x / y) where x and y are expressions yielding a result of integral type.

closed time in 6 days

jktomer

issue commentgolang/go

proposal: cmd/vet: flag accidental integer division inside float casts

Thanks for taking the time to dig up numbers. No change in consensus, so declining.

jktomer

comment created time in 6 days

push eventrsc/licensecheck

Russ Cox

commit sha ab3bc95258ae640743e595b81896c8a3bfa82e59

licensecheck: add Checker for defining custom license sets Instead of forcing the use of a hard-coded builtin set, this lets clients of the library define their own license sets. There is still a builtin set, but being able to define alternate sets should reduce the pressure to add every last possible license to the built-in set and let users adjust the set without forking the module. Fixes #10.

view details

Russ Cox

commit sha 7b6489c58e374fea4cc9e9992e2fc2bd65d69870

licensecheck: 2.3X faster scans (0.73 MB/s -> 1.68 MB/s) Represent normalized text as a list of integer word IDs instead of strings. Then the word comparisons are integer comparisons.

view details

Russ Cox

commit sha 5a2d44eed29fa7c520301ef0e3f497ac91cb426c

licensecheck: 3X faster scans (1.68 MB/s -> 5.15 MB/s) Use a word pair to look up possible starting matches instead of just one word. Also skip over the word pair when checking the length of the match, since we already know the initial pair matches.

view details

Russ Cox

commit sha b2c98bdc5ea5ca3c2c1e1a38aa1353d99f33b520

licensecheck: expand match Start/End to text start/end when appropriate If a match starts at word 0 or ends at the last word, it makes sense to extend the text byte offsets to the start or end of the text, respectively. There is already a special case in the URL matcher to do this at the end of the text, but there wasn't one for the main license text matches. Adding this case makes the returned match offsets cover the whole file for a whole-file match, as seen in the testdata changes. Removed TestMatch because it is redundant with the now many test cases in the testdata directory and was fussy about the offsets changing.

view details

Russ Cox

commit sha 8466e1d974e1903295ce6509b4402ce75c343f5f

licensecheck: add support for wildcards in license text In license text the sequence ___ now denotes a wildcard that can be filled in by up to 70 words. That's long enough for a handful of copyright lines in a row (see testdata/MIT.t2) but short enough that it can't skip over an entire license. (Our shortest two recognized licenses are Apache-2.0-User at 80 and BSD-0-Clause at 99.) This commit updates the licenses to use ___ where appropriate. This fixes #6. This commit also removes removeCopyrightLines (d686698), which turns out to have been removing quite a few lines in actual license text (see diffs in testdata/CC*) and did not handle other cases now handled by the wildcards (for example, testdata/MIT.t2). This fixes #12. Fixes #6. Fixes #12.

view details

push time in 6 days

push eventrsc/licensecheck

Russ Cox

commit sha 463d502b5dea197a78ff96e067ba8a08a8e6429c

licensecheck: add support for wildcards in license text In license text the sequence ___ now denotes a wildcard that can be filled in by up to 70 words. That's long enough for a handful of copyright lines in a row (see testdata/MIT.t2) but short enough that it can't skip over an entire license. (Our shortest two recognized licenses are Apache-2.0-User at 80 and BSD-0-Clause at 99.) This commit updates the licenses to use ___ where appropriate. This fixes #6. This commit also removes removeCopyrightLines (d686698), which turns out to have been removing quite a few lines in actual license text (see diffs in testdata/CC*) and did not handle other cases now handled by the wildcards (for example, testdata/MIT.t2). This fixes #12. Fixes #6. Fixes #12.

view details

push time in 6 days

issue commentgoogle/licensecheck

Apache license recognized as ECL-2

I take it back. This is fixed by #6 in the sense that #6 lets us remove removeCopyright and removeCopyright is the root cause of this bug.

jba

comment created time in 6 days

push eventrsc/licensecheck

Russ Cox

commit sha 9efdb3f4d4adca8f600699f8ab66e9dcb5d0d33d

licensecheck: expand match Start/End to text start/end when appropriate If a match starts at word 0 or ends at the last word, it makes sense to extend the text byte offsets to the start or end of the text, respectively. There is already a special case in the URL matcher to do this at the end of the text, but there wasn't one for the main license text matches. Adding this case makes the returned match offsets cover the whole file for a whole-file match, as seen in the testdata changes. Removed TestMatch because it is redundant with the now many test cases in the testdata directory and was fussy about the offsets changing.

view details

push time in 7 days

pull request commentgoogle/licensecheck

add Checker API, make scans 7X faster

One interesting question: should we drop the ok bool from the Cover method result? It seems to be redundant with len(cov.Match) == 0 or cov.Percent == 0.

rsc

comment created time in 7 days

PR opened google/licensecheck

add Checker API, make scans 7X faster

See individual commits for descriptions.

  • add Checker API
  • make scans 7X faster by comparing ints instead of strings, looking up word pairs
+303 -219

0 comment

8 changed files

pr created time in 7 days

push eventrsc/licensecheck

Russ Cox

commit sha 63e806fe99e47b10acfbc3730664ec098cb954a0

licensecheck: add testdata-based test The testdata format is specified in testdata/README. Only single-license test cases for now, generated by running the licenses themselves through fmt to wrap lines. Even so, some bugs to be fixed are noted in the test files.

view details

Russ Cox

commit sha c7612ce4df0ddd2b16058d609a49f1d89a575923

licensecheck: add Checker for defining custom license sets Instead of forcing the use of a hard-coded builtin set, this lets clients of the library define their own license sets. There is still a builtin set, but being able to define alternate sets should reduce the pressure to add every last possible license to the built-in set and let users adjust the set without forking the module.

view details

Russ Cox

commit sha d5c31dcb6694012a539cd8b64362e9e751783112

licensecheck: 2.3X faster scans (0.73 MB/s -> 1.68 MB/s) Represent normalized text as a list of integer word IDs instead of strings. Then the word comparisons are integer comparisons.

view details

Russ Cox

commit sha 555b9d6e28ace84adce1c687f5d7d0342667cb41

licensecheck: 3X faster scans (1.68 MB/s -> 5.15 MB/s) Use a word pair to look up possible starting matches instead of just one word. Also skip over the word pair when checking the length of the match, since we already know the initial pair matches.

view details

push time in 7 days

push eventgoogle/licensecheck

Russ Cox

commit sha 63e806fe99e47b10acfbc3730664ec098cb954a0

licensecheck: add testdata-based test The testdata format is specified in testdata/README. Only single-license test cases for now, generated by running the licenses themselves through fmt to wrap lines. Even so, some bugs to be fixed are noted in the test files.

view details

push time in 7 days

fork rsc/licensecheck

The licensecheck package classifies license files and heuristically determines how well they correspond to known open source licenses.

fork in 7 days

issue commentgoogle/licensecheck

Apache license recognized as ECL-2

I've reproduced this and will send a fix. I don't believe it is related to #6.

jba

comment created time in 7 days

issue commentgolang/go

x/exp: reconsider using single module for all unrelated experimental packages of various maturity levels

Nothing should depend on x/exp. If lots of things need cmd/apidiff, it sounds like it should move.

In general, x/ repos should not depend on non-golang repos. x/exp is an exception because it's not supposed to be important for people (see previous paragraph).

It would be fine to put a go.mod in x/exp/shiny today to contain the dependencies and not pollute the rest of x/exp that people should not be depending on.

In the longer term we need to work out the plan for x/ repos generally. I'm OK with a band-aid in x/exp because, again, people should not be depending on it.

x/tools is a bit of a dumping ground but maybe is the right home for cmd/apidiff at this point. It at least tends dependencies better.

The right fix for today is to put a go.mod in x/exp/shiny.

In the longer term, lazy loading of go.mods will make this situation not bother looking at shiny, because it doesn't matter for the build at hand.

dmitshur

comment created time in 7 days

issue commentgolang/go

proposal: crypto/x509: implement TextMarshaler for *Certificate

A text dump is not a marshalled encoding. That would imply we needed to parse it during Unmarshal as well. A helper in x509 might be fine but not a TextMarshal method but it would require x509 taking on the OID -> Name mapping. This really seems like something to put in an external package.

kevinburkemeter

comment created time in 7 days

PR closed 9fans/plan9port

acme: ^S -> Putall
+5 -0

4 comments

2 changed files

dexen

pr closed time in 8 days

pull request comment9fans/plan9port

acme: ^S -> Putall

Sorry but we're not adding any more keybindings, especially not for Control sequences. The Kcmd actions were added to make editing nicer on single-button Mac trackpads but I think that's where the line stops. Maybe they were a mistake, but so be it.

Note that we can't even do Control equivalents on non-Mac because ^C too often needs to be interrupt.

dexen

comment created time in 8 days

issue commentgolang/go

proposal: cmd/doc: add "// Unstable:" prefix convention

But the whole point of unstable is that it's unstable. By using them you are accepting breakage. Better that than not realizing there's any breakage in the first place because that fact - which runs counter to all conventions in the ecosystem - is buried in a comment that you're likely not to notice without special tooling. And there can be an arbitrarily long transition in which the "final" Unstable version wraps the stable ones if the module author so chooses.

arnottcr

comment created time in 11 days

issue commentgolang/go

proposal: cmd/doc: add "// Unstable:" prefix convention

I am very uncomfortable with this proposal as written, for the reasons I outlined in November (https://github.com/golang/go/issues/34409#issuecomment-553546180). This is too big a thing to bury in a comment. I still believe it should be something else, most likely an "Unstable" prefix on the name itself.

arnottcr

comment created time in 11 days

issue commentgolang/go

cmd/go: make go.mod exclude directives deterministic

Yes, we should always write down a choice if we decide we have to make a choice.

jayconrod

comment created time in 12 days

issue closedgolang/go

go.dev: display of licenses for included packages

On behalf of @kortschak, from https://github.com/golang/go/issues/36758#issuecomment-580915914:

I also strongly disagree that you are doing the correct thing with vendored code. You are displaying host license on the vendored code package, which is irrelevant and not displaying vendored code licenses on the host module. This is facing exactly the wrong direction.

closed time in 13 days

jba

issue commentgolang/go

go.dev: display of licenses for included packages

I'm sorry we weren't able to reach agreement about how the site should behave when reporting the overall Kubernetes license. It sounds like we can close this issue.

jba

comment created time in 13 days

issue commentgolang/go

proposal: review meeting minutes

2020-02-05 / @rsc, @ianlancetaylor, @griesemer, @andybons

  • #36606 cmd/compile: make 64-bit fields be 64-bit aligned on 32-bit systems, add //go:packed directive on structs
    • active discussion
  • #35544 cmd/doc: module documentation
    • commented
  • #36875 cmd/go: refine the default language version based on context
    • active discussion
  • #35667 cmd/link: Include build meta information
    • commented
  • #36681 cmd/vet: flag accidental integer division inside float casts
    • likely decline; last call for comments ⏳
  • #31773 crypto/tls: Authority Information Access support
    • commented
  • #22836 crypto/tls: add GetConfigForServer callback to *tls.Config
    • commented
  • #23282 crypto/x509: add support to get SSL context from certificate store on windows
    • commented
  • #21704 crypto: add Equal(PublicKey) bool method to PublicKey implementations
    • likely accept; last call for comments ⏳
  • #5901 encoding/json: allow per-Encoder/per-Decoder registration of marshal/unmarshal functions
    • likely accept; last call for comments ⏳
  • #33457 image: add generic metadata support for jpeg, gif, png
    • active discussion
  • #34624 io, net: implement WriterTo for pipes
    • commented
  • #36189 panic based error handling
    • likely decline; last call for comments ⏳
  • #33136 reflect: optimize v.Set(reflect.Zero(v.Type())) to not allocate
    • no change in consensus; accepted 🎉
  • #29982 sizeof: new package with constants for Int, Uint, Uintptr, Int8, Float64, etc
    • no change in consensus; accepted 🎉
  • #36290 strings: add (*Reader).String()
    • retracted
    • no change in consensus; declined
  • #33688 testing: define naming convention for test functions
    • likely decline; last call for comments ⏳
  • #34306 testing: flag show skipped
    • no change in consensus; declined
  • #31107 text/template,html/template: add ExecuteContext methods
    • commented
  • #35480 time: allow a user callback for dropped ticks
    • likely decline; last call for comments ⏳
  • #29289 wiki: CodeReviewComments change: Discourage duplicate imports with different names
    • no change in consensus; declined
  • #36646 x/crypto/poly1305: deprecate public package
    • commented
  • #31044 x/crypto/ssh/terminal: move all APIs to x/term and deprecate
    • commented
  • #35947 x/tools/cmd/godoc: GORDO enriched Go documentation format.
    • no change in consensus; declined
rsc

comment created time in 13 days

issue commentgolang/go

proposal: cmd/doc: module documentation

It's still very unclear what it means to "document a module". This is why we didn't move forward with #35546.

I understand your point that pkg.go.dev is making this unclear as well. (For example, https://pkg.go.dev/golang.org/x/crypto/ssh/terminal?tab=overview shows the module-level README but isn't really about the terminal package despite the framing.) It's all still up in the air.

I'm not sure there's more information needed or available right now to move forward, but I've moved this into the active proposal list so we keep looking at it.

carnott-snap

comment created time in 13 days

issue commentgolang/go

proposal: text/template,html/template: add ExecuteContext methods

What we've learned since https://github.com/golang/go/issues/31107#issuecomment-501007297 is that the security arguments for context are likely spurious, and that nonces should be done differently. But other things may want context.

I am still very skeptical of magically filling in an argument to a function based on invisible state. That would be the first time we've done that, as I said before. But we could still do what I mentioned above, namely:

  1. Add an Execute method variant that takes an explicit context arg.
  2. Make the context available from a builtin "context" function, so that templates can pass it to functions that need it, explicitly.

(If you use the context template function without ExecuteContext, you'd get context.Background, presumably.)

@rogpeppe, would that satisfy your need for context support?

empijei

comment created time in 13 days

issue commentgolang/go

proposal: testing: define naming convention for test functions

Benchmarks have no strict convention either. Only Examples do, because we need to know which function to show them next to in the godoc output.

How could a tool (or go test itself) determine which method is the target of the test

This is not really a well-formed question. Many tests test multiple functions. Only the simplest tests focus on just one function. To the extent that that happens, it is often TestF to test the function F. But it's not a "defined" or required convention.

This seems like a likely decline (infeasible).

taiidani

comment created time in 13 days

issue commentgolang/go

proposal: panic based error handling

This doesn't need to be in the standard library, and we've decided not to add more error handling changes for quite some time. Thanks for publishing the idea as a separate package others can already import.

This seems like a likely decline.

arnottcr

comment created time in 13 days

issue commentgolang/go

proposal: errors: configurable Handle(error) function

Moving to Go2 to be with all the other error handling proposals.

mvndaai

comment created time in 13 days

issue commentgolang/go

proposal: time: allow a user callback for dropped ticks

Based on the discussion above, this seems like a likely decline.

gobwas

comment created time in 13 days

issue commentgolang/go

proposal: cmd/go: refine the default language version based on context

Very important to note that, while at some point in the future we plan to make a backwards-incompatible version of Go, that is still aways off. Among other things it cannot happen until GOPATH is no longer used at all. So while you are right that we need to consider this before then, Go 1.15 is not going to be the "first break".

bcmills

comment created time in 13 days

issue commentgolang/go

proposal: cmd/link: Include build meta information

OK, so to summarize, the proposal was to put the following in the binary in a new section:

  • Go build version
  • Build information (modules and versions)
  • Compiler options (build mode, compiler, gcflags, ldflags)
  • User-defined data

The first two are already present in the binary and don't need to be duplicated. (They can be extracted by parsing the binary or by running go version <binary> or at runtime by using debug.ReadBuildInfo.) The last (user-defined data) is apparently not needed, per previous comment ("adding custom information is e.g. has no importance for our use cases").

That leaves compiler information (basically, the go command-line options). It seems like we could plausibly write that down too. I've wanted it at least once in the last month. That would just be a single new line in the existing format, not a whole new section.

If we refocus this discussion to be adding go command command-line flags, does that address the initial request well enough?

michael-obermueller

comment created time in 13 days

issue commentgolang/go

proposal: encoding/json: allow per-Encoder/per-Decoder registration of marshal/unmarshal functions

Let's use json.Register like gob.Register (different kind of argument but same idea). Otherwise this seems like a likely accept.

rsc

comment created time in 13 days

issue commentgolang/go

proposal: io, net: implement WriterTo for pipes

I understand the panic now, having looked more closely at the code. But I don't really understand why this is an important optimization. The extra code is very subtle. It avoids the intermediate copy, but how often does io.Pipe get used in performance-sensitive contexts?

It's weird, because io.Pipe already is zero-copy between the read and write ends. This optimization only applies when you are using io.Copy(w, pr) where pr is an io.Pipe read side, meaning there's another write end pw somewhere else, and there is a call pw.Write happening, and the effect of all this is to end up forwarding that directly to w.Write. But in that case, why have the io.Pipe at all?

Stebalien

comment created time in 13 days

more