Ask questionsrecommended preset is too strict
I fear this may be too strict for real production sites. The PWA stuff being included makes it pretty opinionated too.
(I'd like paulirish.com to pass our recommended preset but a service worker doesnt make sense there. ) Right now I think there's only but a few showcase PWA sites that will actually pass this preset. :/
The strictness here matters a lot more since assertion with recommended is on by default in autorun. I think it's rough that most folks that try autorun for the first time will have failures to deal with before they can see a report or a diff.
In summary: I think any default for autorun should be MUCH more forgiving (if we're recommending new users start with autorun). no PWA checks, allowing color contrast failures etc. And probably this will have to be called
forgiving, etc instead of
also only slightly related but what's happening here?
seems like its not using any assertions if they're being explicitly provided? i probably am not reading this right :)
Answer questions paulirish
tl;dr - I agree, it's very strict, but that's the whole value of a preset.
a preset yes, but not a preset that's a default in our recommended/quickstart path.
I think its cool for us to have a "recommended" preset that's quite strict.. it seems spiritually equivalent to 90s everywhere. (Even then, we'd have to talk about if PWA is part of that. :)
I think if an assertion preset is part of autorun, then the preset has to be significantly more lenient. (like 95% of users would pass it or as you would put it.... "worthless" :) maybe
Add a wizard flow for bootstrapping a config file that flips all their currently failing audits to warn
my idea here is that a wizard flow could craft a new set of assertions based on their run, creating a hold-the-line assertion config. (with warn/error behavior TBD). And we'd do that rather than relax an existing preset.
Tweak docs to encourage lhci autorun --rc-overrides.upload.target=temporary-public-storage as the initial setup.
love it. just to confirm, upload isn't a default because it's creepy and data is sensitive, etc etc, right? i like it remaining opt-in but heavily recommended. 👍
(Note that if upload is configured, autorun does not do recommended by default)
whoa i didn't know or expect that.
speaking as someone who's very conservative on assertion breaking the build... i'm fine with this 😛 . but speaking as a more general interested party... i'm surprised. how come?
seems like its not using any assertions if they're being explicitly provided
then we don't add any arguments to the assert command, so it just uses the configuration from the rc file.
got it. makes sense, cool. :)