profile
viewpoint
Danny Thomas DanielThomas @Netflix Los Gatos, California Developer Productivity @ Netflix

DanielThomas/ec2gaming 100

EC2 Gaming on macOS with Steam In-Home Streaming

DanielThomas/dotfiles 13

Opinionated dotfiles for Mac OS with zsh

DanielThomas/groundhog 8

Simple, high performance HTTP capture and replay

DanielThomas/advent 0

Advent of Code

DanielThomas/artifactory-user-plugins 0

Sample Artifactory User Plugins

DanielThomas/clouddriver 0

read and write operations across cloud providers

DanielThomas/deck 0

Management UI for Spinnaker

DanielThomas/frigga 0

Utilities for working with Asgard named objects

DanielThomas/genie 0

Federated Job Execution Engine

DanielThomas/go-left-pad 0

String left pad

Pull request review commentnebula-plugins/gradle-info-plugin

InfoJarManifestPlugin: introduce support for configuring MetaInfNormalization

 class InfoJarManifestPlugin implements Plugin<Project>, InfoReporterPlugin {                     jarTask.manifest.attributes.putAll(attrs)                 }             }++            project.afterEvaluate {+                if(GradleKt.versionGreaterThan(project.gradle, "6.6-rc-1")) {+                    configureMetaInfNormalization(project)+                }+            }+         }     }++    @CompileDynamic+    private void configureMetaInfNormalization(Project project) {+        InfoBrokerPluginExtension extension = project.extensions.getByName('infoBroker') as InfoBrokerPluginExtension+        project.normalization.runtimeClasspath.metaInf(new Action<MetaInfNormalization>() {+            @Override+            void execute(MetaInfNormalization metaInfNormalization) {+                if(extension.ignoreManifestForNormalization.isPresent() && extension.ignoreManifestForNormalization.get()) {+                    metaInfNormalization.ignoreManifest()+                }+                if(extension.ignoreNormalizationCompletely.isPresent() && extension.ignoreNormalizationCompletely.get()) {+                    metaInfNormalization.ignoreCompletely()+                }+                if(extension.ignoredPropertiesForNormalization.isPresent()) {+                    extension.ignoredPropertiesForNormalization.get().each { manifestProperty ->+                        metaInfNormalization.ignoreProperty(manifestProperty)+                    }+                }+                if(extension.ignoredManifestAttributesForNormalization.isPresent()) {+                    extension.ignoredManifestAttributesForNormalization.get().each { manifestAttribute ->

Same here.

rpalcolea

comment created time in 19 days

Pull request review commentnebula-plugins/gradle-info-plugin

InfoJarManifestPlugin: introduce support for configuring MetaInfNormalization

 class InfoJarManifestPlugin implements Plugin<Project>, InfoReporterPlugin {                     jarTask.manifest.attributes.putAll(attrs)                 }             }++            project.afterEvaluate {+                if(GradleKt.versionGreaterThan(project.gradle, "6.6-rc-1")) {+                    configureMetaInfNormalization(project)+                }+            }+         }     }++    @CompileDynamic+    private void configureMetaInfNormalization(Project project) {+        InfoBrokerPluginExtension extension = project.extensions.getByName('infoBroker') as InfoBrokerPluginExtension+        project.normalization.runtimeClasspath.metaInf(new Action<MetaInfNormalization>() {+            @Override+            void execute(MetaInfNormalization metaInfNormalization) {+                if(extension.ignoreManifestForNormalization.isPresent() && extension.ignoreManifestForNormalization.get()) {+                    metaInfNormalization.ignoreManifest()+                }+                if(extension.ignoreNormalizationCompletely.isPresent() && extension.ignoreNormalizationCompletely.get()) {+                    metaInfNormalization.ignoreCompletely()+                }+                if(extension.ignoredPropertiesForNormalization.isPresent()) {+                    extension.ignoredPropertiesForNormalization.get().each { manifestProperty ->

We know what the default changing properties are here - think we should configure those explicitly rather than exposing general purpose normalization extensions.

rpalcolea

comment created time in 19 days

Pull request review commentnebula-plugins/gradle-info-plugin

InfoJarManifestPlugin: introduce support for configuring MetaInfNormalization

 class InfoJarManifestPlugin implements Plugin<Project>, InfoReporterPlugin {                     jarTask.manifest.attributes.putAll(attrs)                 }             }++            project.afterEvaluate {+                if(GradleKt.versionGreaterThan(project.gradle, "6.6-rc-1")) {+                    configureMetaInfNormalization(project)+                }+            }+         }     }++    @CompileDynamic+    private void configureMetaInfNormalization(Project project) {+        InfoBrokerPluginExtension extension = project.extensions.getByName('infoBroker') as InfoBrokerPluginExtension+        project.normalization.runtimeClasspath.metaInf(new Action<MetaInfNormalization>() {+            @Override+            void execute(MetaInfNormalization metaInfNormalization) {+                if(extension.ignoreManifestForNormalization.isPresent() && extension.ignoreManifestForNormalization.get()) {

I'm not sure I'd expose this here - that's something you'd want to configure directly, no?

rpalcolea

comment created time in 19 days

issue commentgradle/gradle

Component metadata rules with injected services leak project references

Yes it is. We turned off caching again because of the exclude issues we saw, but it's intended to be cacheable.

DanielThomas

comment created time in a month

issue commentgradle/gradle

Component metadata rules with injected services leak project references

Oh, maybe I misread the path. I gather valuesForPreviousSession is discarded on a the new run? DefaultCrossBuildInMemoryCache threw me.

DanielThomas

comment created time in a month

issue commentgradle/gradle

JUnit 5 Dynamic tests not showing with their displayName

This problem is worse with frameworks like https://github.com/dmcg/minutest which depend entirely on the display names to provide useful information:

minutest-delegate

johnzielke

comment created time in a month

issue openedgradle/gradle

Improve ability to limit build cache upload size

We limit build cache uploads to 10MB, however the build cache implementation doesn't appear to deal well with the upload being short-circuited:

Could not store entry 2d499ed2c28b7f6f832b5cf09651add2 in remote build cache
org.gradle.caching.BuildCacheException: Broken pipe (Write failed)
        at org.gradle.caching.http.internal.HttpBuildCacheService.wrap(HttpBuildCacheService.java:190)
        at org.gradle.caching.http.internal.HttpBuildCacheService.store(HttpBuildCacheService.java:176)
        at org.gradle.caching.internal.controller.service.BaseBuildCacheServiceHandle.storeInner(BaseBuildCacheServiceHandle.java:93)
        at org.gradle.caching.internal.controller.service.OpFiringBuildCacheServiceHandle.access$001(OpFiringBuildCacheServiceHandle.java:35)
        at org.gradle.caching.internal.controller.service.OpFiringBuildCacheServiceHandle$2.run(OpFiringBuildCacheServiceHandle.java:71)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:395)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:387)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor$1.execute(DefaultBuildOperationExecutor.java:157)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:242)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:150)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:84)
        at org.gradle.internal.operations.DelegatingBuildOperationExecutor.run(DelegatingBuildOperationExecutor.java:31)
        at org.gradle.caching.internal.controller.service.OpFiringBuildCacheServiceHandle.storeInner(OpFiringBuildCacheServiceHandle.java:68)
        at org.gradle.caching.internal.controller.service.BaseBuildCacheServiceHandle.store(BaseBuildCacheServiceHandle.java:86)
        at org.gradle.caching.internal.controller.DefaultBuildCacheController.lambda$store$1(DefaultBuildCacheController.java:182)
        at org.gradle.caching.local.internal.DefaultBuildCacheTempFileStore.withTempFile(DefaultBuildCacheTempFileStore.java:46)
        at org.gradle.caching.internal.controller.DefaultBuildCacheController.store(DefaultBuildCacheController.java:178)
        at org.gradle.internal.execution.steps.CacheStep.store(CacheStep.java:145)
        at org.gradle.internal.execution.steps.CacheStep.lambda$executeAndStoreInCache$6(CacheStep.java:137)
        at org.gradle.internal.Try$Success.ifSuccessfulOrElse(Try.java:187)
        at org.gradle.internal.execution.steps.CacheStep.executeAndStoreInCache(CacheStep.java:136)
        at org.gradle.internal.execution.steps.CacheStep.lambda$executeWithCache$2(CacheStep.java:112)
        at java.util.Optional.orElseGet(Optional.java:267)
        at org.gradle.internal.execution.steps.CacheStep.lambda$executeWithCache$3(CacheStep.java:112)
        at org.gradle.internal.Try$Success.map(Try.java:162)
        at org.gradle.internal.execution.steps.CacheStep.executeWithCache(CacheStep.java:81)
        at org.gradle.internal.execution.steps.CacheStep.execute(CacheStep.java:71)
        at org.gradle.internal.execution.steps.CacheStep.execute(CacheStep.java:43)
        at org.gradle.internal.execution.steps.StoreExecutionStateStep.execute(StoreExecutionStateStep.java:44)
        at org.gradle.internal.execution.steps.StoreExecutionStateStep.execute(StoreExecutionStateStep.java:33)
        at org.gradle.internal.execution.steps.RecordOutputsStep.execute(RecordOutputsStep.java:38)
        at org.gradle.internal.execution.steps.RecordOutputsStep.execute(RecordOutputsStep.java:24)
        at org.gradle.internal.execution.steps.SkipUpToDateStep.executeBecause(SkipUpToDateStep.java:92)
        at org.gradle.internal.execution.steps.SkipUpToDateStep.lambda$execute$0(SkipUpToDateStep.java:85)
        at java.util.Optional.map(Optional.java:215)
        at org.gradle.internal.execution.steps.SkipUpToDateStep.execute(SkipUpToDateStep.java:55)
        at org.gradle.internal.execution.steps.SkipUpToDateStep.execute(SkipUpToDateStep.java:39)
        at org.gradle.internal.execution.steps.ResolveChangesStep.execute(ResolveChangesStep.java:76)
        at org.gradle.internal.execution.steps.ResolveChangesStep.execute(ResolveChangesStep.java:37)
        at org.gradle.internal.execution.steps.legacy.MarkSnapshottingInputsFinishedStep.execute(MarkSnapshottingInputsFinishedStep.java:36)
        at org.gradle.internal.execution.steps.legacy.MarkSnapshottingInputsFinishedStep.execute(MarkSnapshottingInputsFinishedStep.java:26)
        at org.gradle.internal.execution.steps.ResolveCachingStateStep.execute(ResolveCachingStateStep.java:94)
        at org.gradle.internal.execution.steps.ResolveCachingStateStep.execute(ResolveCachingStateStep.java:49)
        at org.gradle.internal.execution.steps.CaptureStateBeforeExecutionStep.execute(CaptureStateBeforeExecutionStep.java:79)
        at org.gradle.internal.execution.steps.CaptureStateBeforeExecutionStep.execute(CaptureStateBeforeExecutionStep.java:53)
        at org.gradle.internal.execution.steps.ValidateStep.execute(ValidateStep.java:74)
        at org.gradle.internal.execution.steps.SkipEmptyWorkStep.lambda$execute$2(SkipEmptyWorkStep.java:78)
        at java.util.Optional.orElseGet(Optional.java:267)
        at org.gradle.internal.execution.steps.SkipEmptyWorkStep.execute(SkipEmptyWorkStep.java:78)
        at org.gradle.internal.execution.steps.SkipEmptyWorkStep.execute(SkipEmptyWorkStep.java:34)
        at org.gradle.internal.execution.steps.legacy.MarkSnapshottingInputsStartedStep.execute(MarkSnapshottingInputsStartedStep.java:39)
        at org.gradle.internal.execution.steps.LoadExecutionStateStep.execute(LoadExecutionStateStep.java:40)
        at org.gradle.internal.execution.steps.LoadExecutionStateStep.execute(LoadExecutionStateStep.java:28)
        at org.gradle.internal.execution.impl.DefaultWorkExecutor.execute(DefaultWorkExecutor.java:33)
        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeIfValid(ExecuteActionsTaskExecuter.java:194)
        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.execute(ExecuteActionsTaskExecuter.java:186)
        at org.gradle.api.internal.tasks.execution.CleanupStaleOutputsExecuter.execute(CleanupStaleOutputsExecuter.java:114)
        at org.gradle.api.internal.tasks.execution.FinalizePropertiesTaskExecuter.execute(FinalizePropertiesTaskExecuter.java:46)
        at org.gradle.api.internal.tasks.execution.ResolveTaskExecutionModeExecuter.execute(ResolveTaskExecutionModeExecuter.java:62)
        at org.gradle.api.internal.tasks.execution.SkipTaskWithNoActionsExecuter.execute(SkipTaskWithNoActionsExecuter.java:57)
        at org.gradle.api.internal.tasks.execution.SkipOnlyIfTaskExecuter.execute(SkipOnlyIfTaskExecuter.java:56)
        at org.gradle.api.internal.tasks.execution.CatchExceptionTaskExecuter.execute(CatchExceptionTaskExecuter.java:36)
        at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.executeTask(EventFiringTaskExecuter.java:77)
        at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.call(EventFiringTaskExecuter.java:55)
        at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.call(EventFiringTaskExecuter.java:52)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor$CallableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:409)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor$CallableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:399)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor$1.execute(DefaultBuildOperationExecutor.java:157)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:242)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:150)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor.call(DefaultBuildOperationExecutor.java:94)
        at org.gradle.internal.operations.DelegatingBuildOperationExecutor.call(DelegatingBuildOperationExecutor.java:36)
        at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter.execute(EventFiringTaskExecuter.java:52)
        at org.gradle.execution.plan.LocalTaskNodeExecutor.execute(LocalTaskNodeExecutor.java:41)
        at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$InvokeNodeExecutorsAction.execute(DefaultTaskExecutionGraph.java:356)
        at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$InvokeNodeExecutorsAction.execute(DefaultTaskExecutionGraph.java:343)
        at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$BuildOperationAwareExecutionAction.execute(DefaultTaskExecutionGraph.java:336)
        at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$BuildOperationAwareExecutionAction.execute(DefaultTaskExecutionGraph.java:322)
        at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.lambda$run$0(DefaultPlanExecutor.java:127)
        at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.execute(DefaultPlanExecutor.java:191)
        at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.executeNextNode(DefaultPlanExecutor.java:182)
        at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.run(DefaultPlanExecutor.java:124)
        at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
        at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:48)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:56)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.SocketException: Broken pipe (Write failed)
        at java.net.SocketOutputStream.socketWrite0(Native Method)
        at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:111)
        at java.net.SocketOutputStream.write(SocketOutputStream.java:155)
        at sun.security.ssl.OutputRecord.writeBuffer(OutputRecord.java:431)
        at sun.security.ssl.OutputRecord.write(OutputRecord.java:417)
        at sun.security.ssl.SSLSocketImpl.writeRecordInternal(SSLSocketImpl.java:894)
        at sun.security.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:865)
        at sun.security.ssl.AppOutputStream.write(AppOutputStream.java:123)
        at org.apache.http.impl.io.SessionOutputBufferImpl.streamWrite(SessionOutputBufferImpl.java:124)
        at org.apache.http.impl.io.SessionOutputBufferImpl.flushBuffer(SessionOutputBufferImpl.java:136)
        at org.apache.http.impl.io.SessionOutputBufferImpl.write(SessionOutputBufferImpl.java:167)
        at org.apache.http.impl.io.ContentLengthOutputStream.write(ContentLengthOutputStream.java:113)
        at com.google.common.io.ByteStreams.copy(ByteStreams.java:112)
        at com.google.common.io.ByteSource.copyTo(ByteSource.java:246)
        at org.gradle.caching.internal.controller.service.StoreTarget.writeTo(StoreTarget.java:45)
        at org.gradle.caching.http.internal.HttpBuildCacheService$1.writeTo(HttpBuildCacheService.java:151)
        at org.apache.http.impl.execchain.RequestEntityProxy.writeTo(RequestEntityProxy.java:121)
        at org.apache.http.impl.DefaultBHttpClientConnection.sendRequestEntity(DefaultBHttpClientConnection.java:156)
        at org.apache.http.impl.conn.CPoolProxy.sendRequestEntity(CPoolProxy.java:152)
        at org.apache.http.protocol.HttpRequestExecutor.doSendRequest(HttpRequestExecutor.java:238)
        at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:123)
        at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
        at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186)
        at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
        at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
        at org.gradle.internal.resource.transport.http.HttpClientHelper.performHttpRequest(HttpClientHelper.java:141)
        at org.gradle.internal.resource.transport.http.HttpClientHelper.performHttpRequest(HttpClientHelper.java:117)
        at org.gradle.caching.http.internal.HttpBuildCacheService.store(HttpBuildCacheService.java:159)
        ... 86 more
        Suppressed: javax.net.ssl.SSLException: Connection has been shutdown: javax.net.ssl.SSLException: java.net.SocketException: Broken pipe (Write failed)
                at sun.security.ssl.SSLSocketImpl.checkEOF(SSLSocketImpl.java:1554)
                at sun.security.ssl.SSLSocketImpl.checkWrite(SSLSocketImpl.java:1566)
                at sun.security.ssl.AppOutputStream.write(AppOutputStream.java:71)
                at org.apache.http.impl.io.SessionOutputBufferImpl.streamWrite(SessionOutputBufferImpl.java:124)
                at org.apache.http.impl.io.SessionOutputBufferImpl.flushBuffer(SessionOutputBufferImpl.java:136)
                at org.apache.http.impl.io.SessionOutputBufferImpl.flush(SessionOutputBufferImpl.java:144)
                at org.apache.http.impl.io.ContentLengthOutputStream.close(ContentLengthOutputStream.java:93)
                at com.google.common.io.Closer.close(Closer.java:214)
                at org.gradle.caching.internal.controller.service.StoreTarget.writeTo(StoreTarget.java:49)
                ... 100 more
        Caused by: javax.net.ssl.SSLException: java.net.SocketException: Broken pipe (Write failed)
                at sun.security.ssl.Alerts.getSSLException(Alerts.java:214)
                at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1967)
                at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1924)
                at sun.security.ssl.SSLSocketImpl.handleException(SSLSocketImpl.java:1888)
                at sun.security.ssl.SSLSocketImpl.handleException(SSLSocketImpl.java:1833)
                at sun.security.ssl.AppOutputStream.write(AppOutputStream.java:128)
                at org.apache.http.impl.io.SessionOutputBufferImpl.streamWrite(SessionOutputBufferImpl.java:124)
                at org.apache.http.impl.io.SessionOutputBufferImpl.flushBuffer(SessionOutputBufferImpl.java:136)
                at org.apache.http.impl.io.SessionOutputBufferImpl.write(SessionOutputBufferImpl.java:167)
                at org.apache.http.impl.io.ContentLengthOutputStream.write(ContentLengthOutputStream.java:113)
                at com.google.common.io.ByteStreams.copy(ByteStreams.java:112)
                at com.google.common.io.ByteSource.copyTo(ByteSource.java:246)
                at org.gradle.caching.internal.controller.service.StoreTarget.writeTo(StoreTarget.java:45)
                ... 100 more
        [CIRCULAR REFERENCE:java.net.SocketException: Broken pipe (Write failed)]

Unclear if there's a supported contract for this, but the Enterprise implementation does appear to support limiting the upload size - but I don't know how to make our implemention behave better than this.

Expected Behavior

It probably makes sense for this configuration to be client side too but also wouldn't hurt to enable expect/continue support to allow a build cache to validate PUT before continuing:

https://hc.apache.org/httpclient-3.x/performance.html#Expect-continue_handshake

Current Behavior

Limiting upload size appears to confuse Gradle.

Context

Gradle 6.5

created time in 2 months

push eventDanielThomas/dotfiles

Danny Thomas

commit sha c980b8129abcf41db1b23f0f1c8cca52ce4d3cd8

Switch to powerlevel10k

view details

push time in 2 months

created tagnebula-plugins/nebula-release-plugin

tagv15.0.3

Release opinions based around gradle-git

created time in 2 months

release nebula-plugins/nebula-release-plugin

v15.0.3

released time in 2 months

push eventnebula-plugins/nebula-release-plugin

Danny Thomas

commit sha 7f8e88d1888b3bdf555b938f34a91e1bba60db7c

Avoid git status call for strategy selection unless required

view details

Danny Thomas

commit sha 1b4976e93a0a8402af68abf8c4f68436163fec6c

Merge pull request #193 from nebula-plugins/avoid-git-status Avoid git status call for strategy selection unless required

view details

push time in 2 months

PR merged nebula-plugins/nebula-release-plugin

Avoid git status call for strategy selection unless required

Git status can be really slow on repositories with deep history.

I wonder if we could use an alternative for checking dirty?

https://stackoverflow.com/a/2658301/364206

For instance on gradle/gradle:

git status --porcelain  0.12s user 0.40s system 154% cpu 0.342 total
git diff --shortstat  0.08s user 0.21s system 304% cpu 0.093 total

Looking at StatusCommand it appears it's just doing a diff anyway. This looks even better:

git diff-index --quiet HEAD  0.03s user 0.19s system 618% cpu 0.035 total

But it looks like those alternatives only catch changed files, not new files not in the index.

+1 -2

1 comment

1 changed file

DanielThomas

pr closed time in 2 months

create barnchnebula-plugins/nebula-release-plugin

branch : avoid-git-status

created branch time in 2 months

issue openedgradle/gradle

PluginDescriptor.getImplementationClassName overhead due to ineffective caching

It appears that there are multiple instances of DefaultPluginRegistry and DefaultPluginManager created in a build, the init script, each project which means the expense of calls to PluginDescriptor.getImplementationClassName() can add up due to file system operations (when there's no other classloading going on due to the classloader being cached).

Screen Shot 2020-06-24 at 1 54 43 PM

In fact, a breakpoint breaking in the classloader indicates that in one project with 55 projects we saw 1024 calls to the descriptor to load properties:

  61 nebula.release
  59 org.springframework.boot
  59 org.gradle.nebula-versioning
  59 org.gradle.nebula-release
  59 org.gradle.maven-publish
  59 org.gradle.ivy-publish
  59 nebula.nebula-bintray
  59 nebula.info-broker
  59 nebula-versioning
  59 nebula-release
  59 com.jfrog.bintray
  59 com.jfrog.artifactory
  58 org.gradle.java-platform
  58 org.gradle.java
  58 nebula.contacts-base
  58 com.github.johnrengelman.shadow
   5 com.google.protobuf
   4 org.gradle.android-library
   4 org.gradle.android
   4 netflix.nebula
   4 netflix.lombok
   4 netflix.bom-publish
   4 nebula.integtest
   4 io.spring.javaformat
   4 com.android.test
   4 com.android.library
   4 com.android.instantapp
   4 com.android.feature
   4 com.android.base
   4 com.android.application
   4 android-library
   4 android
   3 org.ajoberstar.grgit
   3 nebula.facet
   3 io.franzbecker.gradle-lombok
   1 org.gradle.wrapper
   1 org.gradle.idea
   1 org.gradle.help-tasks
   1 org.gradle.groovy
   1 org.gradle.eclipse-wtp
   1 org.gradle.eclipse
   1 org.gradle.build-init

Expected Behavior

Plugin descriptors loading should be globally cached.

Current Behavior

It's per-registry/manager.

Context

Noticed while orofiling project evaluation performance.

Steps to Reproduce

Evaluate any project repeatedly and note the property loading overhead.

Your Environment

Gradle 6.5

created time in 2 months

created tagnebula-plugins/gradle-resolution-rules-plugin

tagv7.7.6

Gradle plugin for providing reusable dependency resolution rules.

created time in 2 months

created tagnebula-plugins/gradle-resolution-rules-plugin

tagv7.7.5

Gradle plugin for providing reusable dependency resolution rules.

created time in 2 months

push eventnebula-plugins/gradle-resolution-rules-plugin

Danny Thomas

commit sha 8aceeaea480b716d04d19dcf9a8ef62b4fbdad7e

Make object mapper a companion singleton

view details

push time in 2 months

created tagnebula-plugins/gradle-resolution-rules-plugin

tagv7.7.4

Gradle plugin for providing reusable dependency resolution rules.

created time in 2 months

push eventnebula-plugins/gradle-resolution-rules-plugin

Danny Thomas

commit sha 39545375bb22930aef3f29db9a664be79b3120bc

Turns out that this breaks the interaction with locking Temporarily ignore the warnings so we can get a release out.

view details

push time in 2 months

created tagnebula-plugins/gradle-resolution-rules-plugin

tagv7.7.3

Gradle plugin for providing reusable dependency resolution rules.

created time in 2 months

push eventnebula-plugins/gradle-resolution-rules-plugin

Danny Thomas

commit sha 599a7ca98150908b2f727aad49c255acbe569096

Avoid thread locals and use a ConcurrentHashMap instead

view details

push time in 2 months

push eventnebula-plugins/gradle-resolution-rules-plugin

Danny Thomas

commit sha 744df818e7b9b2c3743dfe5cf83772abbaad1650

Add parallel tests and avoid cross-project mutation

view details

Danny Thomas

commit sha cc055bb21be9eb51384ad0903dc65194bbc96cfa

Avoid matcher thread safety issues and improve alignment matching performance

view details

push time in 2 months

push eventDanielThomas/dotfiles

Danny Thomas

commit sha c1c468e44211618012daa3efbdafdec721c9d976

Update profiler functions for latest YourKit

view details

push time in 2 months

issue commentnebula-plugins/gradle-info-plugin

error using nebula.plugin.info.InfoPlugin with Gradle 6.0+ versions

See https://github.com/nebula-plugins/gradle-info-plugin/releases

ramakrishnakaushik97

comment created time in 2 months

issue commentnebula-plugins/gradle-info-plugin

error using nebula.plugin.info.InfoPlugin with Gradle 6.0+ versions

That's looks like info 8.+ - you might be getting upgraded via conflict resolution. Check buildEnvironment and make sure you're at least Gradle 6.1 if you're picking up 8.0.0 and later or downgrade.

ramakrishnakaushik97

comment created time in 2 months

pull request commentnebula-plugins/gradle-resolution-rules-plugin

Resolution rules should not be applied after the project is evaluated to honor dependency locks

Oooh, I should have noticed that afterEvaluate, I specifically added the onExecute handler to take care of that. Why was that needed in the first place?

https://github.com/nebula-plugins/gradle-resolution-rules-plugin/commit/e6ad9277c9a2e1c80f98fae1a098a2f51c6b11dc

rpalcolea

comment created time in 2 months

created tagnebula-plugins/gradle-resolution-rules-plugin

tagv7.7.0

Gradle plugin for providing reusable dependency resolution rules.

created time in 2 months

release nebula-plugins/gradle-resolution-rules-plugin

v7.7.0

released time in 2 months

created tagnebula-plugins/gradle-dependency-lock-plugin

tagv9.2.0

A plugin to allow people using dynamic dependency versions to lock them to specific versions.

created time in 2 months

release nebula-plugins/gradle-dependency-lock-plugin

v9.2.0

released time in 2 months

push eventnebula-plugins/gradle-resolution-rules-plugin

Danny Thomas

commit sha 9c1b6313b1bebced5373c4f451ec79b33b019625

Fix one last test failure

view details

push time in 2 months

push eventnebula-plugins/gradle-dependency-lock-plugin

Danny Thomas

commit sha c4ec1c08c5e7b3a616e3a90e719ee3c0baa4bcd5

Avoid expense of filtering configuration collections and use findByName instead

view details

push time in 2 months

push eventnebula-plugins/gradle-resolution-rules-plugin

Danny Thomas

commit sha 38e64b9937811ef59aa0fabf8a80ce3a2f8eed91

Additional message assertions

view details

push time in 2 months

push eventnebula-plugins/gradle-resolution-rules-plugin

Danny Thomas

commit sha 60f0b46ccf15a4307ee8984ae93c55f3d6f48dbd

Fix error message change

view details

push time in 2 months

push eventnebula-plugins/gradle-resolution-rules-plugin

Danny Thomas

commit sha 264b9c6c3b9a91b563228932b04199f31a57512b

Avoid notation parsing and interpolation on hot paths

view details

Danny Thomas

commit sha 5d2ee058bda956090b9c27b0eb6051330b93571b

Only parse rules once and filter independently for each project This avoids Jackson ObjectMapper and zip file reading overhead for each project.

view details

push time in 2 months

Pull request review commentnebula-plugins/gradle-resolution-rules-plugin

SubstituteRule: introduce memoize for version selectors

 class SubstituteRuleMissingVersionException(moduleId: ModuleVersionIdentifier, r fun Configuration.exclude(group: String, module: String) {     exclude(mapOf("group" to group, "module" to module)) }++class Memoize<in T, out R>(val f: (T) -> R) : (T) -> R {+    private val values = ConcurrentHashMap<T, R>()+    override fun invoke(x: T): R {+        return values.getOrPut(x, { f(x) })

getOrPut isn't guarenteed to be atomic - computeIfAbsent is probably what you're after.

rpalcolea

comment created time in 2 months

issue openedgradle/gradle

Component metadata rules with injected services leak project references

We inject an ObjectFactory in a ComponentMetadataRule which causes an injected field to remain on the instance for the factory:

    static class IvyVariantDerivationRule implements ComponentMetadataRule {

        @Inject
        ObjectFactory getObjects() { }

However that leaves an __objects__ field on the $Inject class causing project reference leaks in the daemon:

Screen Shot 2020-06-17 at 2 29 00 PM

We're configuring this by class, as it's intended to be a cachable rule (which I guess would also leak this reference if it weren't on this path):

project.dependencies.components.all(IvyVariantDerivationRule)

Expected Behavior

Class-based component metadata rules provide a safe way of using project-scoped services without leaking references to them.

Current Behavior

Project references are retained causing memory leaks.

Steps to Reproduce

  • Configure a class based rule, injecting a project scoped service that holds a reference to the project, such as ObjectFactory
  • Run a build
  • Take a heap dump and notice that there are live references to DefaultProject_Decorated instances

Your Environment

Gradle 6.5

created time in 2 months

issue openedgradle/gradle

Gradle 6.5 allocated large numbers of 'jar transforms' threads

Looks like these eventually get freed but wanted to check in that such large numbers of allocated threads weren't a surprise:

{noformat} jstack 32507 | grep 'jar transforms' | wc -l 223 {noformat}

Expected Behavior

Not sure. That's why I'm asking.

Current Behavior

Large number of transform threads are allocated.

Context

N/A

Steps to Reproduce

Ran a build for the first time on Gradle 6.5.

Your Environment

Gradle 6.5

created time in 2 months

push eventDanielThomas/dotfiles

Danny Thomas

commit sha 2282271cce1d8626b6d346fee04deaf3fbb66357

Configure ff only

view details

push time in 2 months

startedrobzienert/ZFTutorials

started time in 2 months

issue commentnebula-plugins/gradle-info-plugin

Build Error java.lang.NoSuchMethodError: org.gradle.api.provider.ProviderFactory.systemProperty(Ljava/lang/String;)Lorg/gradle/api/provider/Provider;

A long time ago now Gradle broke plugin publishing in a way that meant that it was publishing the requested dynamic constraints for a plugin, rather than the exact version. If you were using one of the affected plugin versions and it bundles info it could have upgraded you.

You can use the buildEnvironment task to check.

Bwvolleyball

comment created time in 2 months

issue closednebula-plugins/gradle-info-plugin

Build Error java.lang.NoSuchMethodError: org.gradle.api.provider.ProviderFactory.systemProperty(Ljava/lang/String;)Lorg/gradle/api/provider/Provider;

We recently started seeing this error in our builds:

Gradle version 5.6.4.

./gradlew build --stacktrace

//...

* Exception is:

java.lang.NoSuchMethodError: org.gradle.api.provider.ProviderFactory.systemProperty(Ljava/lang/String;)Lorg/gradle/api/provider/Provider;
	at nebula.plugin.info.basic.BasicInfoPlugin$_apply_closure1.doCall(BasicInfoPlugin.groovy:76)
	at org.gradle.util.ClosureBackedAction.execute(ClosureBackedAction.java:71)
	at org.gradle.util.ConfigureUtil.configureTarget(ConfigureUtil.java:154)
	at org.gradle.util.ConfigureUtil.configure(ConfigureUtil.java:105)

I'm not sure if this line is the root cause, but it seems to be new? https://github.com/nebula-plugins/gradle-info-plugin/blob/master/src/main/groovy/nebula/plugin/info/basic/BasicInfoPlugin.groovy#L76

closed time in 2 months

Bwvolleyball

issue commentnebula-plugins/gradle-info-plugin

Build Error java.lang.NoSuchMethodError: org.gradle.api.provider.ProviderFactory.systemProperty(Ljava/lang/String;)Lorg/gradle/api/provider/Provider;

You picked up a major release. See https://github.com/nebula-plugins/gradle-info-plugin/releases/tag/v8.0.0

Bwvolleyball

comment created time in 2 months

push eventnebula-plugins/nebula-kotlin-plugin

Danny Thomas

commit sha 97a8ff4acca6f3d98a07f3274a82402ab6aa759d

Update README.md

view details

push time in 3 months

issue closednebula-plugins/nebula-release-plugin

Tag support for v2020-05-07-1 format

Currently we are trying to adopt the v2020-05-07-1 format (last digit stands for build number for the build day)for tag, and by reading through the documentation and trying, looks like it's not supportable:

  1. v2020.5.7 is workable, but it strips out all the leading zero. Behind the scene, everything is still treated as a number.
  2. v2020.5.7-rc.1 is kind of a work-around, but not quite what we want, and it only works with candidate task, not final task.

Is it configurable to add the tag format of v2020-05-07-1? If yes, will it be treated as valid date (year, month, day), or still just number?

Thanks

closed time in 3 months

kzhu2012

issue commentnebula-plugins/nebula-release-plugin

Tag support for v2020-05-07-1 format

The plugin supports semver and is about managing release lines and release promotion, which isn't what you'd be doing with that kind of scheme. If you want <timestamp>.<build> that feels something suited for continuous releases - use your own logic to set the version and always publish, and treat every build as a release candidate.

kzhu2012

comment created time in 3 months

issue commentjetty-project/jetty-alpn

jetty-alpn incompatible with OpenJDK 8u251

Certainly is in the Zulu distribution:

/Library/Java/JavaVirtualMachines/zulu-8.jdk/Contents/Home $ unzip -l src.zip | grep sun.security.ssl
     8529  04-07-2020 00:49   sun/security/ssl/Alerts.java
     6059  04-07-2020 00:49   sun/security/ssl/ALPNExtension.java
     4952  09-19-2018 10:02   sun/security/ssl/AppInputStream.java
     5413  09-19-2018 10:02   sun/security/ssl/AppOutputStream.java
     5824  09-19-2018 10:02   sun/security/ssl/Authenticator.java
    18181  09-19-2018 10:02   sun/security/ssl/BaseSSLSocketImpl.java
     5080  09-19-2018 10:02   sun/security/ssl/ByteBufferInputStream.java
    40283  09-19-2018 10:02   sun/security/ssl/CipherBox.java
    83282  04-07-2020 00:49   sun/security/ssl/CipherSuite.java
     5750  09-19-2018 10:02   sun/security/ssl/CipherSuiteList.java
    75957  04-07-2020 00:49   sun/security/ssl/ClientHandshaker.java
     6779  09-19-2018 10:02   sun/security/ssl/Debug.java
     3437  09-19-2018 10:02   sun/security/ssl/DHClientKeyExchange.java
    25857  03-02-2020 03:31   sun/security/ssl/DHCrypt.java
     2729  09-19-2018 10:02   sun/security/ssl/ECDHClientKeyExchange.java
     5800  09-19-2018 10:02   sun/security/ssl/ECDHCrypt.java
    14518  03-02-2020 03:31   sun/security/ssl/EllipticCurvesExtension.java
     3483  09-19-2018 10:02   sun/security/ssl/EllipticPointFormatsExtension.java
     7608  09-19-2018 10:02   sun/security/ssl/EngineArgs.java
    14947  09-19-2018 10:02   sun/security/ssl/EngineInputRecord.java
    11453  09-19-2018 10:02   sun/security/ssl/EngineOutputRecord.java
     7751  09-19-2018 10:02   sun/security/ssl/EngineWriter.java
     3958  09-19-2018 10:02   sun/security/ssl/EphemeralKeyManager.java
     2443  04-07-2020 00:19   sun/security/ssl/ExtendedMasterSecretExtension.java
     4395  04-07-2020 00:49   sun/security/ssl/ExtensionType.java
    14555  09-19-2018 10:02   sun/security/ssl/HandshakeHash.java
     6748  09-19-2018 10:02   sun/security/ssl/HandshakeInStream.java
    71091  04-07-2020 00:49   sun/security/ssl/HandshakeMessage.java
     7346  09-19-2018 10:02   sun/security/ssl/HandshakeOutStream.java
    58810  04-07-2020 00:49   sun/security/ssl/Handshaker.java
    33905  10-03-2018 12:35   sun/security/ssl/HandshakeStateManager.java
     1635  09-19-2018 10:02   sun/security/ssl/HelloExtension.java
     5794  04-07-2020 00:49   sun/security/ssl/HelloExtensions.java
    30620  09-19-2018 10:02   sun/security/ssl/InputRecord.java
    15554  09-19-2018 10:02   sun/security/ssl/JsseJce.java
     5249  09-19-2018 10:02   sun/security/ssl/KerberosClientKeyExchange.java
     4897  09-19-2018 10:02   sun/security/ssl/KeyManagerFactoryImpl.java
    18219  09-19-2018 10:02   sun/security/ssl/krb5/KerberosClientKeyExchangeImpl.java
    10672  09-19-2018 10:02   sun/security/ssl/krb5/KerberosPreMasterSecret.java
     4081  09-19-2018 10:02   sun/security/ssl/krb5/Krb5ProxyImpl.java
     4578  09-19-2018 10:02   sun/security/ssl/Krb5Helper.java
     2724  09-19-2018 10:02   sun/security/ssl/Krb5Proxy.java
     5242  09-19-2018 10:02   sun/security/ssl/MAC.java
    21353  09-19-2018 10:02   sun/security/ssl/OutputRecord.java
     5261  09-19-2018 10:02   sun/security/ssl/ProtocolList.java
     6762  09-19-2018 10:02   sun/security/ssl/ProtocolVersion.java
     2952  09-19-2018 10:02   sun/security/ssl/RandomCookie.java
     5391  09-19-2018 10:02   sun/security/ssl/Record.java
     4188  09-19-2018 10:02   sun/security/ssl/RenegotiationInfoExtension.java
    10679  09-19-2018 10:02   sun/security/ssl/RSAClientKeyExchange.java
     7263  04-07-2020 00:49   sun/security/ssl/RSASignature.java
    85665  04-07-2020 00:49   sun/security/ssl/ServerHandshaker.java
    10921  09-19-2018 10:02   sun/security/ssl/ServerNameExtension.java
     4324  07-10-2019 10:59   sun/security/ssl/SessionId.java
     4751  09-19-2018 10:02   sun/security/ssl/SignatureAlgorithmsExtension.java
    18714  03-02-2020 03:31   sun/security/ssl/SignatureAndHashAlgorithm.java
     9494  09-19-2018 10:02   sun/security/ssl/SSLAlgorithmConstraints.java
     8625  06-28-2019 07:16   sun/security/ssl/SSLAlgorithmDecomposer.java
    54791  07-10-2019 10:59   sun/security/ssl/SSLContextImpl.java
    78305  04-07-2020 00:49   sun/security/ssl/SSLEngineImpl.java
     4212  09-19-2018 10:02   sun/security/ssl/SSLServerSocketFactoryImpl.java
    12533  04-07-2020 00:49   sun/security/ssl/SSLServerSocketImpl.java
     8125  07-10-2019 10:59   sun/security/ssl/SSLSessionContextImpl.java
    29295  04-04-2020 01:12   sun/security/ssl/SSLSessionImpl.java
     7317  09-19-2018 10:02   sun/security/ssl/SSLSocketFactoryImpl.java
   104269  04-07-2020 00:49   sun/security/ssl/SSLSocketImpl.java
     9316  04-07-2020 00:49   sun/security/ssl/SunJSSE.java
    15501  09-19-2018 10:02   sun/security/ssl/SunX509KeyManagerImpl.java
     6731  07-10-2019 10:59   sun/security/ssl/TrustManagerFactoryImpl.java
    14664  07-10-2019 10:59   sun/security/ssl/TrustStoreManager.java
     1988  09-19-2018 10:02   sun/security/ssl/UnknownExtension.java
     4363  09-19-2018 10:02   sun/security/ssl/Utilities.java
    34094  09-19-2018 10:02   sun/security/ssl/X509KeyManagerImpl.java
    18148  03-02-2020 03:31   sun/security/ssl/X509TrustManagerImpl.java
DanielThomas

comment created time in 3 months

more