problems from https://adventofcode.com/2019
Sourcegraph blog, feature announcements, and website (about.sourcegraph.com)
Design Patterns Project #1
Dockerfile for ADR Tools
A FileSystem Abstraction System for Go
Prometheus Alertmanager
This repo contains the code for allReady, an open-source solution focused on increasing awareness, efficiency and impact of preparedness campaigns as they are delivered by humanitarian and disaster response organizations in local communities.
One framework. Mobile & desktop.
issue commentopen-policy-agent/conftest
Exception is outputting "No policies found" instead of registering an exception
This has been resolved in the latest, unreleased version of Conftest.
If you fetch latest, build, and use that version against the example exception-- it should work for you.
comment created time in 9 hours
issue commentopen-policy-agent/conftest
`warn_something` can't be ignored with an exception (with caveat)
Resolved via https://github.com/open-policy-agent/conftest/pull/416
comment created time in 9 hours
push eventjpreese/conftest
commit sha f2cb3473d29787ee9d4af451438ad0d5555d40bf
Remove warn prefix when evaluating exceptions Signed-off-by: John Reese <[email protected]>
push time in 9 hours
issue commentopen-policy-agent/conftest
The docs were updated to reflect the new --combine behavior which hasn't been released yet, but will be in 0.22.0.
The previous docs for 0.21.0 that show the combine behavior can be found here
package main
deny[msg] {
deployment := input["deployment.yaml"]["spec"]["selector"]["matchLabels"]["app"]
service := input["service.yaml"]["spec"]["selector"]["app"]
deployment != service
msg = sprintf("Expected these values to be the same but received %v for deployment and %v for service", [deployment, service])
}
comment created time in 9 hours
issue commentopen-policy-agent/conftest
That is what that policy should do. It will print out the annotations that exist but do not have an associated nginx.ingress.kubernetes.io annotation. If the rule has a non-empty msg, the policy has failed.
comment created time in 9 hours
push eventjpreese/conftest
commit sha a40248a95afad01ebbc55ad3c98d821a3758bca5
Localhost pathing Signed-off-by: John Reese <[email protected]>
push time in 9 hours
push eventjpreese/conftest
commit sha 5748d63e273a2f1a3a45560ef0c03ce51da80452
Localhost pathing Signed-off-by: John Reese <[email protected]>
push time in 9 hours
push eventjpreese/conftest
commit sha e10ac19bd98940f01ce738d64c39c9602489721e
Localhost pathing Signed-off-by: John Reese <[email protected]>
push time in 9 hours
push eventjpreese/conftest
commit sha 126e0f6307710c72f6f7719bf0061e08b2efeb3a
Localhost pathing Signed-off-by: John Reese <[email protected]>
push time in 9 hours
push eventjpreese/conftest
commit sha 19aff67282e1b818ac03f2db539ca3aa3c5c544b
Localhost pathing Signed-off-by: John Reese <[email protected]>
push time in 9 hours
push eventjpreese/conftest
commit sha 0ef90e0287ed7698bbdf9df11fc65a31ee544a6a
Localhost pathing Signed-off-by: John Reese <[email protected]>
push time in 10 hours
push eventjpreese/conftest
commit sha 3879a1ec92323c281856a2c678c9e5bdaafa8388
Localhost pathing Signed-off-by: John Reese <[email protected]>
push time in 10 hours
push eventjpreese/conftest
commit sha 63cb814ba037e97a0f53dbce93edd94cd6e1cefc
Localhost pathing Signed-off-by: John Reese <[email protected]>
push time in 10 hours
PR opened open-policy-agent/conftest
Changes of note:
-
Fix more pathing issues on Windows for policies/data docs during push and pull.
-
Fix an inconsistency where it was possible to push to localhost, but could not pull from localhost.
-
Fix an issue where pushing to a URL that did not have a tag, but had a port, would not push as latest (e.g. localhost:5000/repo)
-
Add a
--policyflag to the push command to be consistent with the other commands. Previously the path to push to the registry was supplied as an argument. -
Remove the Loader{} type in favor of just Load(), not real reason to have to create a struct. Additionally split Load() into two. One for loading just policies, and one for loading policies and data. Noticed this when updating the tests and was being forced to pass in empty slices for the data path.
-
Added a test for push/pull using the registry image
pr created time in 10 hours
push eventjpreese/conftest
commit sha fab84b70f4b3edc4e2b5b0ed09116e7eb91f982d
Localhost pathing Signed-off-by: John Reese <[email protected]>
push time in 10 hours
issue openedhashicorp/go-getter
localhost not considered a valid scheme
When the URL is localhost:80, the detector errors with "localhost is not a valid scheme". However, 127.0.0.1:80 is considered valid.
created time in 10 hours
push eventjpreese/conftest
commit sha bb53a2e3f52eaa3280e726a92e7b5c071e77bc78
Localhost pathing Signed-off-by: John Reese <[email protected]>
push time in 10 hours
push eventjpreese/conftest
commit sha e0f686ae59d14fdbb4994569d5936c10a1d33d8e
Localhost pathing Signed-off-by: John Reese <[email protected]>
push time in 11 hours
push eventjpreese/conftest
commit sha da47bc3141934af024daeaed78c02bd9daa9d388
Localhost pathing Signed-off-by: John Reese <[email protected]>
push time in 11 hours
push eventjpreese/conftest
commit sha 1ac8d2c2ecd13edd233f639d00eedecc5069f2ed
Localhost pathing Signed-off-by: John Reese <[email protected]>
push time in 12 hours
push eventjpreese/conftest
commit sha 5c16e93cd07e21555d36c6b9065194569f1956f2
Localhost pathing Signed-off-by: John Reese <[email protected]>
push time in 12 hours
push eventplexsystems/sinker
commit sha 59218eed2e2686c3102fb2ee239a9276dc7c29c0
pull.go: log on stderr instead of returning error on a per image basis (#37)
push time in 13 hours
PR merged plexsystems/sinker
This PR makes the pull image loop less sensitive to registry connection error - basically returns a message on stderr but does not exit sinker when the pull operation on one image is failing, which is more adapted for CI usage. Example:
INFO[0025] Pulling docker.elastic.co/kibana/kibana-oss:7.4.2-SNAPSHOT
INFO[0026] Unable to pull docker.elastic.co/kibana/kibana-oss:7.4.2-SNAPSHOT (Retrying #1)
INFO[0032] Unable to pull docker.elastic.co/kibana/kibana-oss:7.4.2-SNAPSHOT (Retrying #2)
ERRO[0032] pull image and wait: retry: All attempts fail:
#1: try pull image: pull image: Error response from daemon: Get https://docker.elastic.co/v2/kibana/kibana-oss/manifests/7.4.2-SNAPSHOT: unauthorized: authentication required
#2: try pull image: pull image: Error response from daemon: Get https://docker.elastic.co/v2/kibana/kibana-oss/manifests/7.4.2-SNAPSHOT: unauthorized: authentication required
INFO[0032] All images have been pulled!
pr closed time in 13 hours
pull request commentplexsystems/sinker
pull.go: log on stderr instead of returning error on a per image basis
From #38, going to check for the existence of the image and validate auth before starting to pull with sinker pull.
Approving and merging this, and we'll work off of that state! Thanks so much @ravens !
comment created time in 13 hours
issue commentplexsystems/sinker
That is a fair point, theres no way to fully guarantee success due to network errors. That is the hope with Retry, to mitigate those (and they would hopefully be rare!).
But this would prevent issues relating to authorization, which looks like was the problem from the error message in the PR
7.4.2-SNAPSHOT: unauthorized: authentication required
comment created time in 14 hours
push eventjpreese/conftest
commit sha 53fe2fa6aff749739231f415886c74b81bfa4a11
initial commit Signed-off-by: John Reese <[email protected]>
push time in a day
issue commentplexsystems/sinker
Thanks for opening this issue @ravens. The scenario definitely isn't ideal.
Rather than logging an error during the pull operation, what are your thoughts on first verifying that you can successfully pull all images in the manifest? We could just call ImageExistsAtRemote on all of the images, and only pull them if all of the images are there. This would cover the case of invalid auth as well as if the image just didn't exist.
comment created time in a day
push eventjpreese/conftest
commit sha 14bb7af5553ba2f2ce2abd1e8d60733d39e57469
Push image on release Signed-off-by: John Reese <[email protected]>
push time in a day
push eventjpreese/conftest
commit sha 4d199aa22ccffcfcd213e0eac7d3d8a7dd6b46a5
Push image on release Signed-off-by: John Reese <[email protected]>
push time in a day
push eventjpreese/conftest
commit sha 163ee5b5a577ae4e8929824fe691c0ae02b6264b
Push image on release Signed-off-by: John Reese <[email protected]>
push time in a day
issue commentopen-policy-agent/conftest
blocks with same name no longer testable
@sarcasticadmin Awesome, much appreciated!
comment created time in a day
push eventjpreese/conftest
commit sha 0a8d957f6d51a006b695a5d9ab3d1b742b5d158b
Push image on release Signed-off-by: John Reese <[email protected]>
push time in a day
push eventjpreese/conftest
commit sha b59d60a094fede0544d580d205c939f392af4d1b
Push image on release Signed-off-by: John Reese <[email protected]>
push time in a day
push eventjpreese/conftest
commit sha 827849ac41f802cddc6ae7f6cfcd98812b975349
Push image on release Signed-off-by: John Reese <[email protected]>
push time in a day
push eventjpreese/conftest
commit sha 450751557d12cd9fc7ae44def01183885d97fb8f
Push image on release Signed-off-by: John Reese <[email protected]>
push time in a day
push eventjpreese/conftest
commit sha 37db8c87ded253b08713feafa7894cc707a4e395
Push image on release Signed-off-by: John Reese <[email protected]>
push time in a day
push eventjpreese/conftest
commit sha fd1fc283dc5cd6e0235a2d51c562c0015c0dea6b
Push image on release Signed-off-by: John Reese <[email protected]>
push time in a day
push eventjpreese/conftest
commit sha 46b55d5ff7b509be725b09bb0c6ae6cb9cdf44ba
Push image on release Signed-off-by: John Reese <[email protected]>
push time in a day
push eventjpreese/conftest
commit sha 118fd3fb49ab4a38b2252d954430c334ea172cd5
Push image on release Signed-off-by: John Reese <[email protected]>
push time in a day
push eventjpreese/conftest
commit sha 3299fea86af10ce49b0175f2c4171191852c2f5f
Push image on release Signed-off-by: John Reese <[email protected]>
push time in a day
push eventjpreese/conftest
commit sha 27af9f43952f08287e99cdbeebc2690c4bd742e8
Push image on release Signed-off-by: John Reese <[email protected]>
push time in a day
push eventjpreese/conftest
commit sha b5bad18fdd56c2e8e27908609d0337bd8218ec9c
Change input flag to parser (#410) Signed-off-by: John Reese <[email protected]>
push time in 2 days
push eventopen-policy-agent/conftest
commit sha b5bad18fdd56c2e8e27908609d0337bd8218ec9c
Change input flag to parser (#410) Signed-off-by: John Reese <[email protected]>
push time in 2 days
PR merged open-policy-agent/conftest
This PR aims to change the --input flag to instead, be named --parser.
Reasoning:
The parser package exposes a number of different parsers, and these parsers are responsible for parsing different file formats. I feel its currently a little confusing to have conftest test --input tf, under the hood this just means to use the hcl2 parser, which we've done a mapping for. Instead, I think it makes more sense to say conftest test --parser hcl1 or conftest test --parser hcl2.
I do think theres value in being able to figure out which Parser to use, based on a file, which NewFromPath accomplishes.
This also keeps the language more consistent with what is really happening under the hood, figuring out which parser to use with the given input. As well as keeps the parser package focused on its parsers rather than specific implementations that use the language.
pr closed time in 2 days
pull request commentopen-policy-agent/conftest
Thanks for the review! I had the same concern as well, but being that I suspect this is a low-usage flag and being pre-v1, I didn't think it worth it to maintain backwards compatibility.
comment created time in 2 days
push eventjpreese/conftest
commit sha 01ab5c8cbb931b77fbe874f0c7769ca185cd1ff0
Fix plugins on Windows (#400) * Fix plugins on Windows Signed-off-by: John Reese <[email protected]> * Run tidy Signed-off-by: John Reese <[email protected]>
push time in 2 days
pull request commentopen-policy-agent/conftest
Combine configurations into struct when using combine
There is not currently plans to include this in non-combined mode. To get path data, #65 has been suggested. I'm not a fan of always needing index[] for policies.
comment created time in 2 days
pull request commentopen-policy-agent/conftest
Combine configurations into struct when using combine
@benjamin-bergia we knew this would be a breaking change, but feel it's the better approach moving forward. That said, I'll look into this to see if there's anything that can be done to support both approaches in a way that makes sense.
Do you have any specific concerns with the new syntax? I definitely understand fear of rewriting policies.
comment created time in 2 days
issue commentopen-policy-agent/conftest
Create a Windows test environment
Shouldn't be too big of a hurdle. CircleCI has matrix jobs which would let us run CI against our current environment as well as a Windows-based one.
comment created time in 2 days
push eventjpreese/conftest
commit sha f2723729789776640a79c8994864df0ec8a8d957
Conflicts Signed-off-by: John Reese <[email protected]>
push time in 2 days
push eventjpreese/conftest
commit sha 67f5f4e54bc3196a11b618e144d27c7fc1598697
Add maintainers field (#408) Signed-off-by: John Reese <[email protected]>
push time in 2 days
issue commentopen-policy-agent/conftest
Support XDG Specification for Plugin Directory
#1 sounds good to me. I initially hesitated to add it into the same command, because always printing the path seemed noisy to me. The plugin dir should never really change unless you're messing around with variables.
Adding some sort of flag to list to show more information about the found plugins, including its location, sounds like a good approach.
comment created time in 2 days
push eventjpreese/conftest
commit sha 037ecfae5d3f4c330e674faa7ecd2ce170d895dc
Conflicts Signed-off-by: John Reese <[email protected]>
push time in 2 days
push eventjpreese/conftest
commit sha d31f0c8961fd4d1bc554dc2e0a48fc42323c7f95
Initial commit (#402) Signed-off-by: John Reese <[email protected]>
commit sha b314e48cb146be31d6927d10349bd06fd13a4062
Conflicts Signed-off-by: John Reese <[email protected]>
push time in 2 days
push eventjpreese/conftest
commit sha d31f0c8961fd4d1bc554dc2e0a48fc42323c7f95
Initial commit (#402) Signed-off-by: John Reese <[email protected]>
push time in 2 days
push eventjpreese/conftest
commit sha c148932abeba56426884736ce1e0854e99f10e05
Change input flag to parser Signed-off-by: John Reese <[email protected]>
push time in 2 days
Pull request review commentopen-policy-agent/conftest
cloud.google.com/go v0.44.2/go.mod h1:60680Gw3Yr4ikxnPRS/oxxkBccT6SA1yMk63TGekxK cloud.google.com/go v0.45.1/go.mod h1:RpBamKRgapWJb87xiFSdk4g1CME7QZg3uwTez+TSTjc= cloud.google.com/go v0.46.3 h1:AVXDdKsrtX33oR9fbCMu/+c1o8Ofjq6Ku/MInaLVg5Y= cloud.google.com/go v0.46.3/go.mod h1:a6bKKbmY7er1mI7TEI4lsAkts/mkhTSZK8w33B4RAg0=+cloud.google.com/go/bigquery v1.0.1 h1:hL+ycaJpVE9M7nLoiXb/Pn10ENE2u+oddxbD8uu0ZVU=
Not really sure, I've always considered go.sum more or less an artifact of the build and usually don't pay much attention to the results. Lots of drift can occur just based on upstream dependency changes and such!
I ran go mod tidy on the branch and pushed the resulting commit.
comment created time in 2 days
push eventjpreese/conftest
commit sha 9d1e428b92f384f5c42b07123fb22e7bde646587
Run tidy Signed-off-by: John Reese <[email protected]>
push time in 2 days
PR opened open-policy-agent/conftest
This PR aims to change the --input flag to instead, be named --parser.
Reasoning:
The parser package exposes a number of different parsers, and these parsers are responsible for parsing different file formats. I feel its currently a little confusing to have conftest test --input tf, under the hood this just means to use the hcl2 parser, which we've done a mapping for. Instead, I think it makes more sense to say conftest test --parser hcl1 or conftest test --parser hcl2.
I do think theres value in being able to figure out which Parser to use, based on a file, which NewFromPath accomplishes.
This also keeps the language more consistent with what is really happening under the hood, figuring out which parser to use with the given input. As well as keeps the parser package focused on its parsers rather than specific implementations that use the language.
pr created time in 2 days
push eventjpreese/conftest
commit sha 5f4be6af10319ca4e92c33e28783f995768f7361
Change input flag to parser Signed-off-by: John Reese <[email protected]>
push time in 2 days
push eventjpreese/conftest
commit sha 3a772ae02ccb4f288fe75effff7c10566ee54670
Update docs to reflect newest releases (#407) * Update documentation to reflect newest changes Signed-off-by: John Reese <[email protected]> * Add more details around reading about policies Signed-off-by: John Reese <[email protected]>
commit sha 3e5e03c9371768cb96233e84581008ae5dd70ccc
Remove custom HCL2 conversion (#403) Signed-off-by: John Reese <[email protected]>
push time in 2 days
issue commentopen-policy-agent/conftest
Support XDG Specification for Plugin Directory
I'd add that with the priority order and behavior if
$XDG_XYZis set but the directory doesn't actually exist it's probably best to log an error for the user & fall back to the next one?
In this context, is the directory the plugin directory or the directory defined in $XDG_XYZ? In the event that the plugin directory does not exist, to me, that would just mean no plugins are installed. plugin install would add the directory.
If the $XDG_XYZ directory itself does not exist, I would vote for hard erroring. If the data in that variable isn't good, then I would pressure the user to remove or fix it.
I'd say maybe it'd be better named as conftest plugin location as it avoids the confusion between $XDG_CACHE_HOME and $XDG_DATA_HOME & also reading it without context could look like a command to somehow cache your plugins?
I think that's a fair argument. Location does make sense to me. I'll look at some other tools that support this behavior and see what they end up calling it.
comment created time in 2 days
issue commentopen-policy-agent/conftest
New `.conftest` dir for plugins doesn't follow XDG Base Dir spec
@Blokje5 @06kellyjac
After https://github.com/open-policy-agent/conftest/pull/400 is completed, was going to take a look at this next.
I think we should be able to detect if $XDG_DATA_HOME or $XDG_DATA_DIRS, and if so, use that as the location to store the plugins.
$XDG_DATA_HOME seems the most appropriate for where plugins should be downloaded to. While we refer to the plugin dir as a cache, it feels more like an installation folder.
[ Reference discussion on $XDG_DATA_HOME ] : https://github.com/helm/helm/issues/7206
Additional consideration would be the $XDG_DATA_DIRS variable. If $XDG_DATA_HOME is not set, but XDG_DATA_DIRS is, we would parse that variable.
[ Reference discussion on $XDG_DATA_DIRS and priority ] : https://github.com/KhronosGroup/Vulkan-Loader/issues/245
This would result in the following priority order and behavior:
-
If
$XDG_DATA_HOMEis set, the plugins are put in$XDG_DATA_HOME/conftest/plugins -
If
$XDG_DATA_HOMEis not set, but$XDG_DATA_DIRSis set, iterate through$XDG_DATA_DIRSin the order they appear for$dir/conftest/plugins -
If neither
$XDG_DATA_HOMEor$XDG_DATA_DIRSis set, plugins will be at$HOME/conftest/plugins
I would also like to introduce new commands:
-
conftest plugin cache- Print the location of the plugin cache that is currently being looked at. This value would change based on how theXDG_variables are set and would be incredibly useful for troubleshooting. -
conftest plugin list- List the currently installed plugins.
I also think it makes sense to update our plugin configuration YAML to include the source URL (or path on disk) it came from for when we start talking about needing to re-download plugins automatically and checking for version updates.
Does all of this sound OK? Before this issue I have honestly not heard of the XDG_ spec so this is all how I currently understand the spec to work, but I may be misunderstanding.
comment created time in 2 days
issue closedopen-policy-agent/conftest
dpkg-query: warning: parsing file '/var/lib/dpkg/status' .... missing 'Maintainer' field
When installing conftest from deb file, it puts the following content into /var/lib/dpkg/status:
Package: conftest
Status: install ok installed
Installed-Size: 33708
Architecture: amd64
Version: 0.21.0
Description: Test your configuration using Open Policy Agent
Homepage: https://github.com/open-policy-agent/conftest
On any subsequent apt operation, the following warning is displayed:
dpkg-query: warning: parsing file '/var/lib/dpkg/status' near line 2067 package 'conftest':
missing 'Maintainer' field
closed time in 3 days
eyalzek
issue commentopen-policy-agent/conftest
dpkg-query: warning: parsing file '/var/lib/dpkg/status' .... missing 'Maintainer' field
Thanks for reporting this @eyalzek! Resolved in #408
comment created time in 3 days
push eventjpreese/conftest
commit sha 6bdab3bd8c348d02268b6821f315273e6dbfec5c
Add maintainers field Signed-off-by: John Reese <[email protected]>
push time in 3 days
PR opened open-policy-agent/conftest
Resolves #399
This field appears to be required by dpkg and .. does not support multiple aliases?
pr created time in 3 days
PR closed open-policy-agent/conftest
Signed-off-by: Arvind Naidu [email protected]
pr closed time in 3 days
pull request commentopen-policy-agent/conftest
[DRAFT] feat(conftest): policy results as github comments and status checks
Looks like activity has stalled out on this, and @jalseth has created this functionality over at https://github.com/YubicoLabs/action-conftest if we want to route users looking for this functionality there.
comment created time in 3 days
PR closed open-policy-agent/conftest
Fixes: #266
But I'm not sure that we should merge that one because it's not backward-compatible. Folks, who have already tests that written for conftest, will have to refactor the tests
Need thoughts on that
pr closed time in 3 days
pull request commentopen-policy-agent/conftest
Closing in favor of the PR in the upstream HCL2JSON implementation https://github.com/tmccombs/hcl2json/pull/20
comment created time in 3 days
pull request commentopen-policy-agent/conftest
I think the examples will stay in bats, those have to be files on disk so it makes sense for those to live there.
There might be testdata files that are also used in bats, if the case that were testing doesn't make sense to have the scenario with inline Go. But the objective would be to reduce the reliance on it.
For steps forward, I don't mind just merging this in and we can adjust as we go. Just waiting on the other PRs to see what kind of state the repository is in after that.
comment created time in 3 days
Pull request review commentopen-policy-agent/conftest
func TestConversion(t *testing.T) { } for name, tc := range testTable { bytes := []byte(tc.input)- conf, diags := hclsyntax.ParseConfig(bytes, "test", hcl.Pos{Byte: 0, Line: 1, Column: 1})- if diags.HasErrors() {- t.Errorf("Failed to parse config: %v", diags)- }- converted, err := convertFile(conf)-- if err != nil {- t.Errorf("Unable to convert from hcl: %v", err)- } - jb, err := json.MarshalIndent(converted, "", "\t")+ json, err := convert.Bytes(bytes, "", convert.Options{})
It probably makes sense to do so. My intent here was to not change the tests to prove the upstream was backwards compatible with what we already had.
I'll be updating this again soon after we figure out https://github.com/tmccombs/hcl2json/pull/20 which will introduce breaking changes anyway. At that point, these tests matter less and we can call the Parser directly.
Thoughts?
comment created time in 3 days
Pull request review commenttmccombs/hcl2json
Always use sets for block values
func File(file *hcl.File, options Options) ([]byte, error) { type jsonObj map[string]interface{} -func convertFile(file *hcl.File, options Options) (jsonObj, error) {- c := converter{bytes: file.Bytes, options: options}- body := file.Body.(*hclsyntax.Body)-- return c.convertBody(body)-}- type converter struct { bytes []byte options Options } -func (c *converter) rangeSource(r hcl.Range) string {- // for some reason the range doesn't include the ending paren, so- // check if the next character is an ending paren, and include it if it is.- end := r.End.Byte- if c.bytes[end] == ')' {- end+++func convertFile(file *hcl.File, options Options) (jsonObj, error) {+ body, ok := file.Body.(*hclsyntax.Body)+ if !ok {+ return nil, fmt.Errorf("convert file body to body type") }- return string(c.bytes[r.Start.Byte:end])++ c := converter{+ bytes: file.Bytes,+ options: options,+ }++ out, err := c.convertBody(body)+ if err != nil {+ return nil, fmt.Errorf("convert body: %w", err)+ }++ return out, nil } func (c *converter) convertBody(body *hclsyntax.Body) (jsonObj, error) {- var err error out := make(jsonObj)- for key, value := range body.Attributes {- out[key], err = c.convertExpression(value.Expr)- if err != nil {- return nil, err++ for _, block := range body.Blocks {+ if err := c.convertBlock(block, out); err != nil {+ return nil, fmt.Errorf("convert block: %w", err) } } - for _, block := range body.Blocks {- err = c.convertBlock(block, out)+ var err error+ for key, value := range body.Attributes {+ out[key], err = c.convertExpression(value.Expr) if err != nil {- return nil, err+ return nil, fmt.Errorf("convert expression: %w", err) } } return out, nil } -func (c *converter) convertBlock(block *hclsyntax.Block, out jsonObj) error {- var key string = block.Type-- value, err := c.convertBody(block.Body)- if err != nil {- return err+func (c *converter) rangeSource(r hcl.Range) string {+ // for some reason the range doesn't include the ending paren, so+ // check if the next character is an ending paren, and include it if it is.+ end := r.End.Byte+ if c.bytes[end] == ')' {+ end++ }+ return string(c.bytes[r.Start.Byte:end])+} +func (c *converter) convertBlock(block *hclsyntax.Block, out jsonObj) error {+ key := block.Type for _, label := range block.Labels {- if inner, exists := out[key]; exists {++ // Labels represented in HCL are defined as quoted strings after the name of the block:+ // block "label_one" "label_two"+ //+ // Labels represtend in JSON are nested one after the other:+ // "label_one": {+ // "label_two": {}+ // }+ //+ // To create the JSON representation, check to see if the label exists in the current output:+ //+ // When the label exists, move onto the next label reference.+ // When a label does not exist, create the label in the output and set that as the next label reference+ // in order to append (potential) labels to it.+ if _, exists := out[key]; exists { var ok bool- out, ok = inner.(jsonObj)+ out, ok = out[key].(jsonObj) if !ok {- // TODO: better diagnostics return fmt.Errorf("Unable to convert Block to JSON: %v.%v", block.Type, strings.Join(block.Labels, ".")) } } else {- obj := make(jsonObj)- out[key] = obj- out = obj+ out[key] = make(jsonObj)+ out = out[key].(jsonObj)
For both of these, this tripped me up for quite a bit.
When using out[key] =, we're actually changing the value passed in by the user.
When using out =, we're just setting the "next" out value to get the next label.
All of this makes sense in hindsight, but creating the single obj and setting it to both out[key] as well as out wasn't immediately clear.
This feels more natural in that we're creating a new value at out[key] and then setting out to that value that we just created.
I'm more than happy to revert any of this. It really was just trying to organize things in my head, but I don't want to make it confusing for you if the old way made more sense to you.
comment created time in 3 days
Pull request review commenttmccombs/hcl2json
Always use sets for block values
func File(file *hcl.File, options Options) ([]byte, error) { type jsonObj map[string]interface{} -func convertFile(file *hcl.File, options Options) (jsonObj, error) {- c := converter{bytes: file.Bytes, options: options}- body := file.Body.(*hclsyntax.Body)-- return c.convertBody(body)-}- type converter struct { bytes []byte options Options } -func (c *converter) rangeSource(r hcl.Range) string {- // for some reason the range doesn't include the ending paren, so- // check if the next character is an ending paren, and include it if it is.- end := r.End.Byte- if c.bytes[end] == ')' {- end+++func convertFile(file *hcl.File, options Options) (jsonObj, error) {+ body, ok := file.Body.(*hclsyntax.Body)+ if !ok {+ return nil, fmt.Errorf("convert file body to body type") }- return string(c.bytes[r.Start.Byte:end])++ c := converter{+ bytes: file.Bytes,+ options: options,+ }++ out, err := c.convertBody(body)+ if err != nil {+ return nil, fmt.Errorf("convert body: %w", err)+ }++ return out, nil } func (c *converter) convertBody(body *hclsyntax.Body) (jsonObj, error) {- var err error out := make(jsonObj)- for key, value := range body.Attributes {- out[key], err = c.convertExpression(value.Expr)- if err != nil {- return nil, err++ for _, block := range body.Blocks {
It does not, no. This must have just gotten moved as I was trying to make sense of the codebase and how the HCL parser bucketed each part.
comment created time in 3 days
issue commentopen-policy-agent/conftest
blocks with same name no longer testable
@drewmullen @sarcasticadmin
PR to set-i-fy all blocks here. Additional discussions on the related issue.
https://github.com/tmccombs/hcl2json/pull/20
comment created time in 3 days
PR opened tmccombs/hcl2json
For https://github.com/tmccombs/hcl2json/issues/18
Disclaimer: 99.9% of the change is just always wrapping value on line 125 (https://github.com/tmccombs/hcl2json/blob/master/convert/convert.go#L125) inside of []interface{}.
The other diffs are mostly from me just trying to understand and document the behavior.
Without going into Terraform-specific checks, this feels like the path forward. For Terraform, it might be a little silly for some blocks, like variables:
variable "my_var" {
type = string
}
will produce
"variable": {
"my_var": [
{
"type": "${string}"
}
]
}
Even though its impossible for it to have more than one. But in HCL, a block is a block.
pr created time in 3 days
push eventjpreese/hcl2json
commit sha c269182a1356d3e85b1f6b64250c8a4a77cc6364
Always use sets for block types
push time in 3 days
delete branch plexsystems/plexsystems.github.io
delete branch : versionless-graphql
delete time in 3 days
Pull request review commentopen-policy-agent/conftest
Update docs to reflect newest releases
# Conftest -Conftest is a utility to help you write tests against structured configuration data. For instance you could write tests for your Kubernetes configurations, or Tekton pipeline definitions, Terraform code, Serverless configs or any other structured data.+Conftest is a utility to help you write tests against structured configuration data. For instance, you could write tests for your Kubernetes configurations, Tekton pipeline definitions, Terraform code, Serverless configs or any other structured data. Conftest relies on the Rego language from [Open Policy Agent](https://www.openpolicyagent.org/) for writing the assertions. You can read more about Rego in [How do I write policies](https://www.openpolicyagent.org/docs/how-do-i-write-policies.html) in the Open Policy Agent documentation. ## Usage -Conftest allows you to write policies using Open Policy Agent/rego and apply them to one or-more configuration files. Policies by default should be placed in a directory called `policy` but this can be overridden.+Policies by default should be placed in a directory called `policy`, but this can be overridden with the `--policy` flag.
Good catch, I do prefer the consistency to keep calling them policies. I also updated the link to the documentation site.
comment created time in 3 days
push eventjpreese/conftest
commit sha 2127984f981813cf7c271ef063d23b30e13d3fdc
Add more details around reading about policies Signed-off-by: John Reese <[email protected]>
push time in 3 days
Pull request review commentopen-policy-agent/conftest
Update docs to reflect newest releases
Before submitting large changes, please open an issue on GitHub outlining: - Detailed description of what your changes would entail. - Alternative solutions or approaches if applicable. -Use your judgement about what constitutes a large change. If you aren't sure, send a message to the `#conftest` channel in the OPA slack or submit an issue on GitHub.+Use your judgment about what constitutes a large change. If you aren't sure, send a message to the `#conftest` channel in the OPA slack or submit an issue on GitHub.
LOL I actually Google'd this last night. Judgment appears to be the "more correct" approach according to what I'm seeing, but in the end both are acceptable and didn't necessarily warrant a change. I just didn't bother going back and undoing it.
comment created time in 3 days
push eventjpreese/conftest
commit sha 336af58e51c52c9555d1c1f99b595b80cb21fe81
Add git to Dockerfile (#404) Signed-off-by: John Reese <[email protected]>
commit sha cf13dffb7061aba3ef46b4d8811f4e67c7fc006d
Remove runtime check on YAML parser (#406) Signed-off-by: John Reese <[email protected]>
commit sha 1d94894027dbde1d813862a4402fc6311cad8f44
Sort combined results by file path (#405) Signed-off-by: John Reese <[email protected]>
push time in 3 days
push eventjpreese/hcl2json
commit sha cc821717a9997100183e8b3ff85f984f0faea191
Remove indentation from package
commit sha 92a7ef6bb3b9acedcb9e46a0bb3601866e0b1fd6
Merge pull request #19 from jpreese/remove-indent Remove indentation from package
push time in 4 days
pull request commentopen-policy-agent/conftest
I agree with the idea of splitting up examples from testdata, but I'm not sure which files actually need to be moved and which could be removed in favor of inline Rego in a Go test. I have a lot of other PRs that touch on tests, and don't necessarily want to cause conflicts when its unknown if the data needs to move.
I planned on going through and combing through the current test suite and making sure that everything is covered appropriately, but my hope is that testdata would be incredibly small, maybe not even needed.
comment created time in 4 days
push eventjpreese/conftest
commit sha 0a5d15321a37ca17e33d8008bd4fa846dfa4e2d8
Update documentation to reflect newest changes Signed-off-by: John Reese <[email protected]>
push time in 4 days