profile
viewpoint
Torin Sandall tsandall Styra, Inc. twitter.com/sometorin Software Engineer at Styra. We're hiring!

helm/charts 14424

Curated applications for Kubernetes

fugue/fregot 134

Fugue Rego Toolkit

open-policy-agent/vscode-opa 55

An extension for VS Code which provides support for OPA

open-policy-agent/gatekeeper-library 22

The OPA Gatekeeper policy library.

tsandall/admission-webhook-demo 6

Examples, code, and manifests for demonstrating admission controller webhooks.

tsandall/bundle-roots-example 2

A small example that shows how to configure OPA bundle roots

tsandall/dacp 2

Examples of OPA admission control policies for enforcement and audit

tsandall/akka-cluster-exp 0

Experimenting with akka-cluster examples.

tsandall/akka-persistence-kafka 0

A replicated Akka Persistence journal backed by Apache Kafka

PullRequestReviewEvent

issue commentopen-policy-agent/opa

Extending OPA's runtime/server

First of all, thanks for filing a well-written, detailed issue! I'll reply to a few of your comments/questions that I think are important and then leave some thoughts at the end.

Authenticate JWTs that require a signing key to be retrieved from an external API, based on a key ID in the JWT.

You can accomplish this either with http.send() or by compiling your own custom built-in functions into OPA. You mentioned below that http.send() would not work because the service doesn't set caching headers. How will you handle cache invalidation? The approach that OPA implements for http.send() is fairly standard and should work well with endpoints that serve JWKS or the like. If you absolutely cannot modify the service to conform to HTTP caching standards, the custom built-in route would be the way to go.

I don't see why io.jwt.decode and the related verification functions (e.g,. io.jwt.verify_hs256, etc.) cannot be used if you have a custom built-in function fetching keys. Can you elaborate on why io.jwt.decode (and the related suite of verification functions) is insufficient?

Inject data into the request input sent to OPA, before any policy evaluation starts.

Is there a reason the input data sent to OPA has to be mutated? Normally, the input data sent to OPA would contain a JWT token and the policy would implement rules that verify the token and then expose the claims inside the token for the rest of the policy. This avoids hardcoding logic into the server that can otherwise be specified in the policy itself.

Control/modify the structure of all logs generated by OPA to fit our desired schema.

Can you elaborate a little bit on what you're looking to do here? We've been thinking about hiding the low-level access logs for a little while now. If we made that change, the only logs that would remain would be primarily for integration purposes (e.g., decision logs, status logs, etc.) We could look at providing more fine-grained config to control logging levels (currently it's one-size-fits-all).

Hide/disable many of the OPA server routes/features (require HTTPS, only allow requests to evaluate named policy, etc.)

The listener type and address is configuration given to OPA on startup. What routes are accessible can be controlled via authorization.

Add a route that can do named policy evaluation in bulk

If you wanted to return multiple policy decisions in a single query you could write a rule that produces those decisions in a single JSON document.

Return "obligations" from rego policy evaluation (similar to the xacml concept), so that apps can prompt end users to perform actions to get access (e.g. re-authenticate).

The answer I'd give here is similar to the last point--rules can generate non-boolean values (e..g, maps, lists, etc.) that represent concepts like obligations. Those can be composed and returned just like any other value generated by your rules.

After pouring over OPA's runtime/server code the last few days, this is the approach we're taking initially: [...]

Given your requirements to customize the CLI, listeners, routes, logging, metrics, etc. I'm wondering what value you're going to receive from deploying the OPA runtime as-is. The runtime exposes the OPA API and has a few opinionated choices re: configuration, metrics, logging, etc. If those do not work in your environment you can build your own runtime around the OPA components.

Here's an example that embeds OPA as a library but goes beyond just using the rego package: https://github.com/open-policy-agent/example-api-authz-go. I think that might be useful if you want to go down the path you outlined.

We don't want to maintain a long-lived fork of OPA, but we are very interested in contributing back to OPA.

We're happy to work with folks that want to contribute back. Ideally, IMO, the different components that make up OPA can be reused outside the OPA runtime for custom use cases like this. I recommend looking at that example above to see if it would suit your needs.

gshively11

comment created time in 11 hours

Pull request review commentopen-policy-agent/opa

Add net.cidr_merge function to produce smallest possible list of CIDRs

 var NetCIDRContainsMatches = &Builtin{ 	), } +// NetCIDRMerge merges IP addresses and subnets into the smallest possible list of CIDRs.+var NetCIDRMerge = &Builtin{+	Name: "net.cidr_merge",+	Decl: types.NewFunction(+		types.Args(+			types.NewArray(nil, types.NewAny(types.S)),

Is there any reason we can't accept a set of strings as well?

ashutosh-narkar

comment created time in 13 hours

Pull request review commentopen-policy-agent/opa

Add net.cidr_merge function to produce smallest possible list of CIDRs

+cases:+  - note: netcidrmerge/cidr single subnet+    modules:+      - |+        package generated

It would be worthwhile to add a test case that provides IPs as inputs (not CIDRs) as the function should support both.

ashutosh-narkar

comment created time in 13 hours

Pull request review commentopen-policy-agent/opa

Add net.cidr_merge function to produce smallest possible list of CIDRs

+cases:+  - note: netcidrmerge/cidr single subnet+    modules:+      - |+        package generated

The generated package was just used for migrating the existing tests. We can just use something a bit more human-friendly for new tests write (e.g., package test).

ashutosh-narkar

comment created time in 13 hours

PullRequestReviewEvent
PullRequestReviewEvent

Pull request review commentopen-policy-agent/opa

plugins/bundle: Support for saving and reading bundles from disk

 bundles:   authz:     service: acmecorp     resource: somedir/bundle.tar.gz+    persistence_file_location: /tmp/example

I'd say yes, but to start I don't think we'll need the persistence_file_location.

ashutosh-narkar

comment created time in 13 hours

PullRequestReviewEvent

issue commentopen-policy-agent/opa

Support gRPC body in OPA Istio plugin

Would you imagine this like providing support for another content-type (like application/protobuf), where the provided set of descriptors would allow the handler to translate the input before passing it on into eval? We'd need some way to convey the message type then... 🤔 (In gRPC, that's done via a header, see PROTOCOL-HTTP2.md)

Yes, exactly. If PB encoded messages do not even have a type hint that indicates the message type/class then it would have to be sent out-of-band. Without having given it too much thought, what I'd like to see is opa build extended to accept a set of proto descriptors so they could be easily packaged into the bundle and distributed with the rest of the policy and data. When OPA activates the bundle, the server handler would have everything it needs to deserialize incoming PB messages assuming they contain enough type info (in the message or out-of-band). Perhaps the server would be gRPC-based; I'm not sure.

PBs can be made self-describing, too, but I don't think this is used much...

My conclusion was the same; self-describing PBs are not widely used and the client libraries don't support it (e.g,. last time I looked, the golang package didn't support them.)

ledor473

comment created time in 13 hours

issue commentopen-policy-agent/opa

Restrict data output - Question

@Jeyakumardevarajulu I don't understand your question. If you're asking how to supply multiple policy files you can either do so explicitly with -d (e.g,. opa eval -d p1.rego -d p2.rego ...) or you can pass entire directories (which will be loaded recursively!) If you structure your workspace to follow the Bundle format, you can easily pass the entire workspace with -b.

As a side note, one thing I noticed is your policy refers to input.usergroups. Is that intentional? It should be data.usergroups from what I can gather from the rest of your comment.

Jeyakumardevarajulu

comment created time in 14 hours

issue commentopen-policy-agent/opa

Add support for days and weeks in parse_duration_ns

If we assume that days, weeks, and years are always 24h, 7d, and 365d respectively, this is a relatively easy change to make. The problem is that this is not always true--but maybe that's not important for the majority of use cases.

freeseacher

comment created time in 14 hours

PR opened open-policy-agent/opa

ast: Fix panic during local var rewriting

This commit fixes an issue similar to e88579bfdae8fd6edd940915a2fb259ebdd21429: when a comprehension is nested inside of a set or used as an object key, the rewriting needs to be careful to make a copy of the set/object to avoid mutating the elemenet/key in-place.

Fixes #2720

Signed-off-by: Torin Sandall torinsandall@gmail.com

<!--

Thanks for submitting a PR to OPA!

Before pressing 'Create pull request' please read the checklist below.

  • All code changes should be accompanied with tests. If you are not modifying any tests, just provide a short explanation of why updates to tests are not necessary. In addition to helping catch bugs, tests are extremely helpful in providing context that explains how your changes can be used.

  • All changes to public APIs must be accompanied with docs. Examples of public APIs include built-in functions, config fields, and of course, exported Go types/functions/constants/etc.

  • Commit messages should explain why you made the changes, not what you changed. Use active voice. Keep the subject line under 50 characters or so.

  • All commits must be signed off by the author. If you are not familiar with signing off, see CONTRIBUTING.md below.

For more information on contributing to OPA see:

-->

+100 -27

0 comment

2 changed files

pr created time in 14 hours

create barnchtsandall/opa

branch : fix-2720

created branch time in 14 hours

issue closedopen-policy-agent/opa

Restrict data output - Question

<!-- Thanks for opening an issue to request a feature or file a bug! If you provide some basic information it helps us address problems faster. -->

When I execute below command and it returns complete data.json along with my result "true" or false" from rego policy. How to restrict it?

opa eval -i input.json -d testPolicy.rego -d data.json "data"

testpolicy.rego

is_user_owner_or_viewer = false { true }

Expected Behavior

Actual Behavior

Steps to Reproduce the Problem

<!-- If this is a bug report please provide as much detail as possible so that we can reproduce the problem. Examples:

  • OPA version
  • Example query, input, data, and policy that OPA was given
  • Example output that OPA returned
  • For server and CLI, the flags/configuration that you provided to OPA
  • For server, any relevant log messages from OPA
  • For Go and Wasm, the arguments you invoked OPA with -->

Additional Info

<!-- Any additional information you think might be helpful. Examples include the environment where OPA was running (e.g., if inside Kubernetes, what resource limits did you configure OPA with?), how long OPA had been running for, what was happening around the time when you identified the problem, etc. -->

closed time in 14 hours

Jeyakumardevarajulu

issue commentopen-policy-agent/opa

Restrict data output - Question

Don't query for "data". Just query for the value you're interested in: data.<package>.<rule-name>, e.g., data.foo.bar.is_user_owner_or_viewer.

Jeyakumardevarajulu

comment created time in 14 hours

issue openedopen-policy-agent/opa

fuzz: Panic during local variable rewriting stage

package d c({({0|t:=0}):0})
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
        panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x89cbda]

goroutine 1 [running]:
github.com/open-policy-agent/opa/ast.(*Compiler).compile.func1()
        /home/torin/src/opa/ast/compile.go:881 +0x67
panic(0xbfc020, 0x11f7350)
        /usr/local/go/src/runtime/panic.go:969 +0x175
github.com/open-policy-agent/opa/ast.(*Compiler).rewriteLocalVars.func1.1(0x0, 0x250eef6a57e40609)
        /home/torin/src/opa/ast/compile.go:1120 +0x15a
github.com/open-policy-agent/opa/ast.WalkTerms.func1(0xc8ce80, 0x0, 0xc000320ba0)
        /home/torin/src/opa/ast/visit.go:181 +0x42
github.com/open-policy-agent/opa/ast.(*GenericVisitor).Walk(0xc0003060c0, 0xc8ce80, 0x0)
        /home/torin/src/opa/ast/visit.go:270 +0x58
github.com/open-policy-agent/opa/ast.(*GenericVisitor).Walk.func1(0xc00031e960, 0xc00031eaa0)
        /home/torin/src/opa/ast/visit.go:340 +0x97
github.com/open-policy-agent/opa/ast.(*object).Foreach.func1(0xc00031e960, 0xc00031eaa0, 0x40e218, 0x30)
        /home/torin/src/opa/ast/term.go:1810 +0x39
github.com/open-policy-agent/opa/ast.(*object).Iter(0xc000320bd0, 0xc000305ca8, 0xc000305d90, 0x89ccd5)
        /home/torin/src/opa/ast/term.go:1788 +0x7a
github.com/open-policy-agent/opa/ast.(*object).Foreach(0xc000320bd0, 0xc000305d98)
        /home/torin/src/opa/ast/term.go:1809 +0x53
github.com/open-policy-agent/opa/ast.(*GenericVisitor).Walk(0xc0003060c0, 0xcb3cc0, 0xc000320bd0)
        /home/torin/src/opa/ast/visit.go:338 +0x994
github.com/open-policy-agent/opa/ast.(*GenericVisitor).Walk(0xc0003060c0, 0xc8ce80, 0xc00031e940)
        /home/torin/src/opa/ast/visit.go:332 +0xbc7
github.com/open-policy-agent/opa/ast.(*GenericVisitor).Walk(0xc0003060c0, 0xc3f580, 0xc00031eee0)
        /home/torin/src/opa/ast/visit.go:312 +0xc50
github.com/open-policy-agent/opa/ast.(*GenericVisitor).Walk(0xc0003060c0, 0xc73260, 0xc0002165a0)
        /home/torin/src/opa/ast/visit.go:299 +0x8a5
github.com/open-policy-agent/opa/ast.WalkTerms(0xc73260, 0xc0002165a0, 0xc000306210)
        /home/torin/src/opa/ast/visit.go:185 +0x70
github.com/open-policy-agent/opa/ast.(*Compiler).rewriteLocalVars.func1(0xc000215480, 0x1249301)
        /home/torin/src/opa/ast/compile.go:1117 +0xce
github.com/open-policy-agent/opa/ast.WalkRules.func1(0xc73340, 0xc000215480, 0xc00031eec0)
        /home/torin/src/opa/ast/visit.go:229 +0x56
github.com/open-policy-agent/opa/ast.(*GenericVisitor).Walk(0xc0003064d0, 0xc73340, 0xc000215480)
        /home/torin/src/opa/ast/visit.go:270 +0x58
github.com/open-policy-agent/opa/ast.(*GenericVisitor).Walk(0xc0003064d0, 0xc4a0a0, 0xc000216550)
        /home/torin/src/opa/ast/visit.go:281 +0x59c
github.com/open-policy-agent/opa/ast.WalkRules(0xc4a0a0, 0xc000216550, 0xc000306538)
        /home/torin/src/opa/ast/visit.go:238 +0x70
github.com/open-policy-agent/opa/ast.(*Compiler).rewriteLocalVars(0xc0003120f0)
        /home/torin/src/opa/ast/compile.go:1103 +0xe5
github.com/open-policy-agent/opa/ast.(*Compiler).runStage(0xc0003120f0, 0xcf48e2, 0x20, 0xc0002091f0)
        /home/torin/src/opa/ast/compile.go:866 +0x4f
github.com/open-policy-agent/opa/ast.(*Compiler).compile(0xc0003120f0)
        /home/torin/src/opa/ast/compile.go:886 +0x11d
github.com/open-policy-agent/opa/ast.(*Compiler).Compile(0xc0003120f0, 0xc000306888)
        /home/torin/src/opa/ast/compile.go:351 +0x2c5
github.com/open-policy-agent/opa/bundle.writeModules(0xdd6340, 0xc000138010, 0xddcfa0, 0xc000214d00, 0xdc4a60, 0xc000308450, 0xc0003120f0, 0xddb540, 0xc00027caa0, 0x0, ...)
        /home/torin/src/opa/bundle/store.go:397 +0x5ed
github.com/open-policy-agent/opa/bundle.activateBundles(0xc000307068, 0xc00027caa0, 0xddcfa0)
        /home/torin/src/opa/bundle/store.go:243 +0x866
github.com/open-policy-agent/opa/bundle.Activate(...)
        /home/torin/src/opa/bundle/store.go:161
github.com/open-policy-agent/opa/internal/runtime/init.InsertAndCompile(0xdd6340, 0xc000138010, 0xddcfa0, 0xc000214d00, 0xdc4a60, 0xc000308450, 0xc000205b00, 0xc000205b30, 0x0, 0x0, ...)
        /home/torin/src/opa/internal/runtime/init/init.go:67 +0x313
github.com/open-policy-agent/opa/plugins.(*Manager).Init.func1(0xdc4a60, 0xc000308450, 0xc000138010, 0xc000209190)
        /home/torin/src/opa/plugins/plugins.go:255 +0x158
github.com/open-policy-agent/opa/storage.Txn(0xdd6340, 0xc000138010, 0xddcfa0, 0xc000214d00, 0x1, 0xc00020a340, 0xc000307328, 0x0, 0x0)
        /home/torin/src/opa/storage/storage.go:95 +0xe9
github.com/open-policy-agent/opa/plugins.(*Manager).Init(0xc000312000, 0xdd6340, 0xc000138010, 0xc00024e030, 0x24)
        /home/torin/src/opa/plugins/plugins.go:253 +0x105
github.com/open-policy-agent/opa/runtime.NewRuntime(0xdd6340, 0xc000138010, 0xc00024e030, 0x24, 0xc000206560, 0xc000206580, 0x0, 0x0, 0x0, 0x0, ...)
        /home/torin/src/opa/runtime/runtime.go:251 +0x408
github.com/open-policy-agent/opa/cmd.initRuntime(0xdd6340, 0xc000138010, 0x0, 0x0, 0xc000206560, 0xc000206580, 0x0, 0x0, 0x0, 0x0, ...)
        /home/torin/src/opa/cmd/run.go:258 +0x545
github.com/open-policy-agent/opa/cmd.init.8.func1(0xc00021d180, 0xc0002084e0, 0x1, 0x1)
        /home/torin/src/opa/cmd/run.go:156 +0xea
github.com/spf13/cobra.(*Command).execute(0xc00021d180, 0xc0002084b0, 0x1, 0x1, 0xc00021d180, 0xc0002084b0)
        /home/torin/src/opa/vendor/github.com/spf13/cobra/command.go:766 +0x2c2
github.com/spf13/cobra.(*Command).ExecuteC(0x1203480, 0xc000068778, 0xc000113f78, 0x4062a5)
        /home/torin/src/opa/vendor/github.com/spf13/cobra/command.go:852 +0x2fe
github.com/spf13/cobra.(*Command).Execute(...)
        /home/torin/src/opa/vendor/github.com/spf13/cobra/command.go:800
main.main()
        /home/torin/src/opa/main.go:15 +0x31

created time in 15 hours

PR opened open-policy-agent/opa

Treat built-in function errors as false/undefined instead of halting evaluation

This PR modifies how built-in function errors are handled. See the changelog update for a high-level description.

This change balances the need to recover from built-in function errors in the policy without adding explicit error handling or exceptions (which would significantly complicate policy authoring or perhaps have other unintended consequences.) Overall, I think this is the right default behaviour, but I want to gather feedback from various folks before moving forward.

A few other changes could also be made:

  1. Return all errors caught during evaluation alongside the result when strict mode is disabled
  2. Add support for pragmas so that policies could opt-in to strict or relaxed behaviour

(1) would give callers more flexibility to decide whether error conditions are significant (or not). From my perspective, this would be difficult for developers to consume so I'm not sure how useful it would be in practice.

(2) needs more thought and could require more extensive changes however I don't want to exclude it.

cc @timothyhinrichs @jaspervdj-luminal @maxsmythe @srenatus @patrick-east @ashutosh-narkar

+279 -56

0 comment

20 changed files

pr created time in a day

create barnchtsandall/opa

branch : treat-builtin-errors-as-undefined

created branch time in a day

issue closedopen-policy-agent/opa

Decisions logs response: Log upload skipped.

Expected Behavior

The decisions_logs are enabled and the gzip file is expected to be sent.

Actual Behavior

The decisions_logs are enabled but it does not show any type of error in the logs only information. Only gets this message in logs even if we set the log in debug mode:

{ "level": "info", "msg": "Log upload skipped.", "plugin": "decision_logs", "time": "2020-09-17T19:09:16Z" }

Steps to Reproduce the Problem

  • OPA version = 0.23.2-envoy
  • OPA config = services: controller: url: https://url for bundle allow_insecure_tls: true logstash: url: https://IP:PORT <--- URI for decisions logs allow_insecure_tls: true

bundles: opa-test: service: controller resource: Bundle URL polling: min_delay_seconds: 60 max_delay_seconds: 120

decision_logs: partition_name: / service: logstash reporting: min_delay_seconds: 60 max_delay_seconds: 120

  • Example output that OPA returned

{ "level": "info", "msg": "Log upload skipped.", "plugin": "decision_logs", "time": "2020-09-17T19:09:16Z" }

Additional Info

OPA server over AKS 1.16.13

closed time in a day

nestajah

issue commentopen-policy-agent/opa

Decisions logs response: Log upload skipped.

Yes, the URL that logs are sent to will be <SERVICE_BASE_URL>/logs. If a partition name is configured, it will be appended. Clsoing this issue because there doesn't appear to be any bug in OPA.

nestajah

comment created time in a day

issue commentopen-policy-agent/opa

Upgrade gorilla/mux to at least 1.7.4

After reading the release notes for gorilla/mux, I agree with the others that it's unclear whether upgrading will have meaningful impact on performance. At the same time, it's easy to test out.

Without more information on the underlying issue (e.g., the rules and data set in use, example query inputs, resource limits on the deployment, etc.) it's difficult to provide more guidance.

If we don't hear back in the next week or so, let's close this.

loudmouth

comment created time in a day

issue closedopen-policy-agent/opa

opa-envoy-plugin(branch:service-updater_2020-07-09-1815.332) not work on istio v1.4.8

I am trying to implement sample provided by https://github.com/open-policy-agent/opa-istio-plugin Env information:

#docker version
Client:
 Version:           18.06.0-ce
 API version:       1.38
 Go version:        go1.10.3
 Git commit:        0ffa825
 Built:             Wed Jul 18 19:08:18 2018
 OS/Arch:           linux/amd64
 Experimental:      false
#kubectl version
Client Version: version.Info{Major:"1", Minor:"17", GitVersion:"v1.17.9", GitCommit:"4fb7ed12476d57b8437ada90b4f93b17ffaeed99", GitTreeState:"clean", BuildDate:"2020-07-15T16:18:16Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"}`

istio version : v1.4.8

opa-envoy-plugin create step:

step-1:

 #kubectl apply -f https://github.com/open-policy-agent/opa-envoy-plugin/blob/service-updater_2020-07-09-1815.332/quick_start.yaml
#kubectl -n istio-system get envoyfilter ext-authz
NAME        AGE
ext-authz   19m
# kubectl -n opa-istio get pods
NAME                                    READY   STATUS    RESTARTS   AGE
admission-controller-5966f9bf4c-zpc7k   1/1     Running   0          38m

step-2:

#kubectl create ns opa-test
#kubectl label namespace opa-test opa-istio-injection="enabled"
#kubectl label namespace opa-test istio-injection="enabled"

step-3:

 vi opa-configmap.yaml 
apiVersion: v1
kind: ConfigMap
metadata:
  name: opa-istio-config
data:
  config.yaml: |
    plugins:
      envoy_ext_authz_grpc:
        addr: :9191
        path: istio/authz/allow
    decision_logs:
      console: true

apiVersion: v1
kind: ConfigMap
metadata:
  name: opa-policy
data:
  policy.rego: |
    package istio.authz
    import input.attributes.request.http as http_request
    default allow = false
    allow {
        roles_for_user[r]
        required_roles[r]
    }
    roles_for_user[r] {
        r := user_roles[user_name][_]
    }
    required_roles[r] {
        perm := role_perms[r][_]
        perm.method = http_request.method
        perm.path = http_request.path
    }
    user_name = parsed {
        [_, encoded] := split(http_request.headers.authorization, " ")
        [parsed, _] := split(base64url.decode(encoded), ":")
    }
    user_roles = {
        "alice": ["guest"],
        "bob": ["admin"]
    }
    role_perms = {
        "guest": [
            {"method": "GET",  "path": "/productpage"},
        ],
        "admin": [
            {"method": "GET",  "path": "/productpage"},
            {"method": "GET",  "path": "/api/v1/products"},
        ],
    }
#kubectl -n opa-test apply -f opa-configmap.yaml 
 #kubectl -n opa-test get configmap
  NAME               DATA   AGE
  opa-istio-config   1      14m
  opa-policy         1      14m

step-4

#kubectl -n opa-test apply -f https://raw.githubusercontent.com/istio/istio/master/samples/bookinfo/platform/kube/bookinfo.yaml
#kubectl -n opa-test apply -f https://raw.githubusercontent.com/istio/istio/master/samples/bookinfo/networking/bookinfo-gateway.yaml
# kubectl -n opa-test get pods
NAME                              READY   STATUS    RESTARTS   AGE
details-v1-5974b67c8-dt78p        2/2     Running   0          31m
productpage-v1-64794f5db4-blvfn   2/2     Running   0          31m
ratings-v1-c6cdf8d98-rpzbp        2/2     Running   0          31m
reviews-v1-7f6558b974-jmwdt       2/2     Running   0          31m
reviews-v2-6cb6ccd848-xhslp       2/2     Running   0          31m
reviews-v3-cc56b578-svbjx         2/2     Running   0          31m
# kubectl -n opa-test get gateway
NAME               AGE
bookinfo-gateway   28m
# kubectl -n opa-test get vs
NAME       GATEWAYS             HOSTS   AGE
bookinfo   [bookinfo-gateway]   [*]     28m

# kubectl -n opa-test describe pod/productpage-v1-64794f5db4-blvfn
Name:         productpage-v1-64794f5db4-blvfn
Namespace:    opa-test
Priority:     0
Node:         master001
Start Time:   Wed, 09 Sep 2020 08:51:42 +0800
Labels:       app=productpage
              pod-template-hash=64794f5db4
              version=v1
Annotations:  cni.projectcalico.org/podIP: 10.233.77.50/32
              cni.projectcalico.org/podIPs: 10.233.77.50/32
Status:       Running
IP:           10.233.77.50
IPs:
  IP:           10.233.77.50
Controlled By:  ReplicaSet/productpage-v1-64794f5db4
Containers:
  productpage:
    Container ID:   docker://5e94157cb36caabffb893cd35b0b80952ef575f218bc9d3830b13a33b66eec9a
    Image:          docker.io/istio/examples-bookinfo-productpage-v1:1.16.2
    Image ID:       docker-pullable://istio/examples-bookinfo-productpage-v1@sha256:63ac3b4fb6c3ba395f5d044b0e10bae513afb34b9b7d862b3a7c3de7e0686667
    Port:           9080/TCP
    Host Port:      0/TCP
    State:          Running
      Started:      Wed, 09 Sep 2020 08:51:56 +0800
    Ready:          True
    Restart Count:  0
    Environment:    <none>
    Mounts:
      /tmp from tmp (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from bookinfo-productpage-token-gh5lw (ro)
  opa-istio:
    Container ID:  docker://dcd05c5ec04f018a9b8abe07051553f78d345c1e1dd0ba8dab7cf67bf528f13c
    Image:         openpolicyagent/opa:0.21.1-istio
    Image ID:      docker-pullable://openpolicyagent/opa@sha256:a39afd00fdca6e4dc2f7edeac2d3e48f1272ec3708d0aaceb86c158903a4f914
    Port:          <none>
    Host Port:     <none>
    Args:
      run
      --server
      --config-file=/config/config.yaml
      --addr=localhost:8181
      --diagnostic-addr=0.0.0.0:8282
      /policy/policy.rego
    State:          Running
      Started:      Wed, 09 Sep 2020 08:53:51 +0800
    Ready:          True
    Restart Count:  0
    Liveness:       http-get http://:8282/health%3Fplugins delay=0s timeout=1s period=10s #success=1 #failure=3
    Readiness:      http-get http://:8282/health%3Fplugins delay=0s timeout=1s period=10s #success=1 #failure=3
    Environment:    <none>
    Mounts:
      /config from opa-istio-config (rw)
      /policy from opa-policy (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from bookinfo-productpage-token-gh5lw (ro)
Conditions:
  Type              Status
  Initialized       True 
  Ready             True 
  ContainersReady   True 
  PodScheduled      True 
Volumes:
  tmp:
    Type:       EmptyDir (a temporary directory that shares a pod's lifetime)
    Medium:     
    SizeLimit:  <unset>
  bookinfo-productpage-token-gh5lw:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  bookinfo-productpage-token-gh5lw
    Optional:    false
  opa-istio-config:
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      opa-istio-config
    Optional:  false
  opa-policy:
    Type:        ConfigMap (a volume populated by a ConfigMap)
    Name:        opa-policy
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type    Reason     Age        From                Message
  ----    ------     ----       ----                -------
  Normal  Scheduled  <unknown>  default-scheduler   Successfully assigned opa-test/productpage-v1-64794f5db4-blvfn to master001
  Normal  Pulled     32m        kubelet, master001  Container image "docker.io/istio/examples-bookinfo-productpage-v1:1.16.2" already present on machine
  Normal  Created    32m        kubelet, master001  Created container productpage
  Normal  Started    32m        kubelet, master001  Started container productpage
  Normal  Pulling    32m        kubelet, master001  Pulling image "openpolicyagent/opa:0.21.1-istio"
  Normal  Pulled     30m        kubelet, master001  Successfully pulled image "openpolicyagent/opa:0.21.1-istio"
  Normal  Created    30m        kubelet, master001  Created container opa-istio
  Normal  Started    30m        kubelet, master001  Started container opa-istio

step-5

#curl --user alice:password -i http://$GATEWAY_URL/productpage
HTTP/1.1 200 OK
content-type: text/html; charset=utf-8
content-length: 5183
server: istio-envoy
date: Wed, 09 Sep 2020 01:07:11 GMT
x-envoy-upstream-service-time: 39

#curl --user alice:password -i http://$GATEWAY_URL/api/v1/products
HTTP/1.1 200 OK
content-type: application/json
content-length: 395
server: istio-envoy
date: Wed, 09 Sep 2020 01:07:31 GMT
x-envoy-upstream-service-time: 3

closed time in 2 days

cnicy

issue commentopen-policy-agent/opa

opa-envoy-plugin(branch:service-updater_2020-07-09-1815.332) not work on istio v1.4.8

I'm closing this issue because we only support v1.5.0 and later.

cnicy

comment created time in 2 days

issue closedopen-policy-agent/opa

Query evaluation taking more time when opa is used as envoy authorization filter

I am running OPA as a authorization filter in envoy. I have written a simple RBAC policy. I have executed policy in REGO playground. Its taking microseconds to evaluate the policy. But if I provide same policy to OPA which is running as an authorization filter its taking milliseconds to evaluate the policy, so which is not effective for us.

Would you please tell me why this is happening?

closed time in 2 days

PriyaKatkade

issue commentopen-policy-agent/opa

Query evaluation taking more time when opa is used as envoy authorization filter

I'm going to close this issue because there hasn't been a response and it's unclear whether this was just an environmental issue. We can re-open if needed.

PriyaKatkade

comment created time in 2 days

PR opened open-policy-agent/opa

watch: Remove deprecated watch package and server feature

This commit removes the deprecated watch package and server feature. This feature was never adopted and since the implementation was not incremental, it would inevitably encounter performance/scalability issues.

Fixes #2265

Signed-off-by: Torin Sandall torinsandall@gmail.com

<!--

Thanks for submitting a PR to OPA!

Before pressing 'Create pull request' please read the checklist below.

  • All code changes should be accompanied with tests. If you are not modifying any tests, just provide a short explanation of why updates to tests are not necessary. In addition to helping catch bugs, tests are extremely helpful in providing context that explains how your changes can be used.

  • All changes to public APIs must be accompanied with docs. Examples of public APIs include built-in functions, config fields, and of course, exported Go types/functions/constants/etc.

  • Commit messages should explain why you made the changes, not what you changed. Use active voice. Keep the subject line under 50 characters or so.

  • All commits must be signed off by the author. If you are not familiar with signing off, see CONTRIBUTING.md below.

For more information on contributing to OPA see:

-->

+0 -1932

0 comment

5 changed files

pr created time in 2 days

create barnchtsandall/opa

branch : deprecate-watch-option

created branch time in 2 days

PR opened open-policy-agent/opa

ast: Fix panic in parser post-processing of expressions

This commit fixes a panic caught in the fuzzer due to misuse of operands returned by expr.Operand().

Fixes #2714

Signed-off-by: Torin Sandall torinsandall@gmail.com

<!--

Thanks for submitting a PR to OPA!

Before pressing 'Create pull request' please read the checklist below.

  • All code changes should be accompanied with tests. If you are not modifying any tests, just provide a short explanation of why updates to tests are not necessary. In addition to helping catch bugs, tests are extremely helpful in providing context that explains how your changes can be used.

  • All changes to public APIs must be accompanied with docs. Examples of public APIs include built-in functions, config fields, and of course, exported Go types/functions/constants/etc.

  • Commit messages should explain why you made the changes, not what you changed. Use active voice. Keep the subject line under 50 characters or so.

  • All commits must be signed off by the author. If you are not familiar with signing off, see CONTRIBUTING.md below.

For more information on contributing to OPA see:

-->

+28 -0

0 comment

2 changed files

pr created time in 2 days

push eventtsandall/opa

Torin Sandall

commit sha 685b49e5c2aca56fb74a31d792f2bf2e2bb58dfe

ast: Remove dead parser code This post-processing code was used by old the PEG parser but since the parser was rewritten by hand, we no longer need this. Signed-off-by: Torin Sandall <torinsandall@gmail.com>

view details

Torin Sandall

commit sha fda417c4be62f997650ff53725861991426e74f8

ast: Fix output var analysis to accept refs with non-var heads Previously the output var analysis would panic if it encountered refs with non-var head terms. This assumption was invalidated in v0.17 with the introduction of indirect refs. Fixes #2678 Signed-off-by: Torin Sandall <torinsandall@gmail.com>

view details

Torin Sandall

commit sha 428219c92264c809b5dc9b1c0af5ab7d0df2e7fc

ast: Fix object corruption during safety reordering The safety check was corrupting object and set values that contained comprehension as object keys or set elements because the comprehension values themselves were mutated in place. This change fixes the issue by copying object/set values like we do in other places. This change also removes the setExprIndices function which was also mutating values inside of a visitor. Signed-off-by: Torin Sandall <torinsandall@gmail.com>

view details

Torin Sandall

commit sha fb9d16530abeab54f0df2f7780b91622e1375ecf

ast: Fix panic in parser post-processing of expressions This commit fixes a panic caught in the fuzzer due to misuse of operands returned by expr.Operand(). Fixes #2714 Signed-off-by: Torin Sandall <torinsandall@gmail.com>

view details

push time in 2 days

issue openedopen-policy-agent/opa

fuzz: Panic found during parser post-processing

Test input:

0{{}}assign()
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x887237]

goroutine 1 [running]:
github.com/open-policy-agent/opa/ast.(*Term).Hash(...)
        /home/torin/src/opa/ast/term.go:361
github.com/open-policy-agent/opa/ast.(*set).get(0xc0000fdbc0, 0x0, 0x7fad69154aa8)
        /home/torin/src/opa/ast/term.go:1540 +0x37
github.com/open-policy-agent/opa/ast.(*set).Contains(0xc0000fdbc0, 0x0, 0xc0002baa40)
        /home/torin/src/opa/ast/term.go:1452 +0x35
github.com/open-policy-agent/opa/ast.ParseCompleteDocRuleFromEqExpr(0xc00012c870, 0x0, 0x0, 0x1, 0xce30bf, 0x6)
        /home/torin/src/opa/ast/parser_ext.go:241 +0x54
github.com/open-policy-agent/opa/ast.ParseCompleteDocRuleFromAssignmentExpr(0xc00012c870, 0x0, 0x0, 0x1, 0xc000127280, 0x7fad69154800)
        /home/torin/src/opa/ast/parser_ext.go:225 +0x45
github.com/open-policy-agent/opa/ast.ParseRuleFromExpr(0xc00012c870, 0xc00012c820, 0x8, 0x40, 0xc71820)
        /home/torin/src/opa/ast/parser_ext.go:167 +0x2d1
github.com/open-policy-agent/opa/ast.ParseRuleFromBody(0xc00012c870, 0xc00011e2e0, 0x1, 0x1, 0x2, 0xc00011d060, 0x1)
        /home/torin/src/opa/ast/parser_ext.go:130 +0x98
github.com/open-policy-agent/opa/ast.parseModule(0x7fff10ba972c, 0x6, 0xc000127200, 0x3, 0x4, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
        /home/torin/src/opa/ast/parser_ext.go:600 +0x30d
github.com/open-policy-agent/opa/ast.ParseModule(0x7fff10ba972c, 0x6, 0xc0002bad38, 0xd, 0xd, 0x46ae85, 0xc000000180)
        /home/torin/src/opa/ast/parser_ext.go:417 +0xe5
github.com/open-policy-agent/opa/loader.loadRego(0x7fff10ba972c, 0x6, 0xc0002be000, 0xd, 0x20d, 0xde0f00, 0xc000117b60, 0x0, 0x0, 0x0)
        /home/torin/src/opa/loader/loader.go:503 +0xbe
github.com/open-policy-agent/opa/loader.loadKnownTypes(0x7fff10ba972c, 0x6, 0xc0002be000, 0xd, 0x20d, 0xde0f00, 0xc000117b60, 0x0, 0x0, 0x7fff10ba972c, ...)
        /home/torin/src/opa/loader/loader.go:464 +0x50a
github.com/open-policy-agent/opa/loader.fileLoader.Filtered.func1(0xc0002ae6f0, 0x7fff10ba972c, 0x6, 0x0, 0x0, 0x0)
        /home/torin/src/opa/loader/loader.go:151 +0xad
github.com/open-policy-agent/opa/loader.allRec(0x7fff10ba972c, 0x6, 0xc000117540, 0xc0002bb138, 0xc0002ae6f0, 0x0, 0xc0002bb1a0)
        /home/torin/src/opa/loader/loader.go:436 +0x456
github.com/open-policy-agent/opa/loader.all(0xc00011cb40, 0x1, 0x1, 0xc000117540, 0xc0002bb1a0, 0x203000, 0x203000, 0x40d930)
        /home/torin/src/opa/loader/loader.go:407 +0x239
github.com/open-policy-agent/opa/loader.fileLoader.Filtered(0xde0f00, 0xc000117b60, 0x0, 0x0, 0x0, 0x0, 0x0, 0xc0002ae6c0, 0xc00011cb40, 0x1, ...)
        /home/torin/src/opa/loader/loader.go:144 +0xd1
github.com/open-policy-agent/opa/internal/runtime/init.LoadPaths(0xc00011cb40, 0x1, 0x1, 0xc000117540, 0x0, 0x0, 0x0, 0x0, 0xc0001229f8, 0x3)
        /home/torin/src/opa/internal/runtime/init/init.go:131 +0x3c5
github.com/open-policy-agent/opa/runtime.NewRuntime(0xddbca0, 0xc0000220c0, 0xc000290030, 0x24, 0xc000117460, 0xc000117480, 0x0, 0x0, 0x0, 0x0, ...)
        /home/torin/src/opa/runtime/runtime.go:236 +0x15b
github.com/open-policy-agent/opa/cmd.initRuntime(0xddbca0, 0xc0000220c0, 0x0, 0x0, 0xc000117460, 0xc000117480, 0x0, 0x0, 0x0, 0x0, ...)
        /home/torin/src/opa/cmd/run.go:258 +0x545
github.com/open-policy-agent/opa/cmd.init.8.func1(0xc000177400, 0xc00011cb40, 0x1, 0x1)
        /home/torin/src/opa/cmd/run.go:156 +0xea
github.com/spf13/cobra.(*Command).execute(0xc000177400, 0xc00011cb10, 0x1, 0x1, 0xc000177400, 0xc00011cb10)
        /home/torin/src/opa/vendor/github.com/spf13/cobra/command.go:766 +0x2c2
github.com/spf13/cobra.(*Command).ExecuteC(0x120a580, 0xc000068778, 0xc000079f78, 0x4062a5)
        /home/torin/src/opa/vendor/github.com/spf13/cobra/command.go:852 +0x2fe
github.com/spf13/cobra.(*Command).Execute(...)
        /home/torin/src/opa/vendor/github.com/spf13/cobra/command.go:800
main.main()
        /home/torin/src/opa/main.go:15 +0x31

created time in 2 days

push eventopen-policy-agent/opa

Torin Sandall

commit sha 685b49e5c2aca56fb74a31d792f2bf2e2bb58dfe

ast: Remove dead parser code This post-processing code was used by old the PEG parser but since the parser was rewritten by hand, we no longer need this. Signed-off-by: Torin Sandall <torinsandall@gmail.com>

view details

Torin Sandall

commit sha fda417c4be62f997650ff53725861991426e74f8

ast: Fix output var analysis to accept refs with non-var heads Previously the output var analysis would panic if it encountered refs with non-var head terms. This assumption was invalidated in v0.17 with the introduction of indirect refs. Fixes #2678 Signed-off-by: Torin Sandall <torinsandall@gmail.com>

view details

Torin Sandall

commit sha 428219c92264c809b5dc9b1c0af5ab7d0df2e7fc

ast: Fix object corruption during safety reordering The safety check was corrupting object and set values that contained comprehension as object keys or set elements because the comprehension values themselves were mutated in place. This change fixes the issue by copying object/set values like we do in other places. This change also removes the setExprIndices function which was also mutating values inside of a visitor. Signed-off-by: Torin Sandall <torinsandall@gmail.com>

view details

push time in 2 days

issue closedopen-policy-agent/opa

Opa panics during partial evaluation

<!-- Thanks for opening an issue to request a feature or file a bug! If you provide some basic information it helps us address problems faster. -->

Expected Behavior

Partial evaluation should not panic.

Actual Behavior

Depending on the order of comparison ["a", "b"][_] == __dat.test versus __dat.test == ["a", "b"][_] opa panics during partial evaluation.

Steps to Reproduce the Problem

The following code contains four test examples. The first three are successful while the forth fails. While tests 1 & 2 are successful, test 3 & 4 have a switched comparison in the allow_... rule. The switched comparison works for eval (test 3) but not for partial (test 4).

This panic only happens, when the __dat rule has an else clause.

# Filename: bug.rego
#
# Input:
# {
#    "data1": {
#        "test": "a"
#    }
# }
package rego


__dat = x {
  x := input["data1"]
} else = x {
  x := input["data2"]
}

allow_OK {
	["a", "b"][_] == __dat.test
}

allow_Issue {
	__dat.test == ["a", "b"][_]
}

# Tests:
# ======
# opa version
# >> Version: 0.23.2
#
# opa eval -d bug.rego             -i input.json data.rego.allow_OK # Success
# >> { ... }
#
# opa eval -d bug.rego -p -u input -i input.json data.rego.allow_OK # Success
# >> { ... }
#
# opa eval -d bug.rego             -i input.json data.rego.allow_Issue # Success
# >> { ... }
#
# opa eval -d bug.rego -p -u input -i input.json data.rego.allow_Issue # opa panics
# >> panic: interface conversion: ast.Value is ast.Array, not ast.Var
# >>
# >> goroutine 1 [running]:
# >> github.com/open-policy-agent/opa/ast.outputVarsForTerms.func1(0xc0002a2d40, 0x19e9800)
# >> 	github.com/open-policy-agent/opa/ast/compile.go:2481 +0x1cf
# >> github.com/open-policy-agent/opa/ast.WalkTerms.func1(0x189b140, 0xc0002a2d40, 0xc0002a2c40)
# >> 	github.com/open-policy-agent/opa/ast/visit.go:181 +0x42
# >> github.com/open-policy-agent/opa/ast.(*GenericVisitor).Walk(0xc00027ae38, 0x189b140, 0xc0002a2d40)
# >> 	github.com/open-policy-agent/opa/ast/visit.go:270 +0x58
# >>  ...

closed time in 2 days

j-denner

PR merged open-policy-agent/opa

Fix panic during safety check reordering

The compiler was panicking when it encountered object literal with a comprehension key because the comprehension body may be rewritten which would corrupt the object value.

+104 -175

0 comment

5 changed files

tsandall

pr closed time in 2 days

push eventtsandall/opa

Peter Sullivan

commit sha 7f0399b3f2c170c2c19ee60371edb1f888daffc2

docs/content: Update Envoy Authorization Tutorial Updated envoy version and links. Updated service type and minikube commands to tunnel to the service for consistency across minikube versions and drivers Signed-off-by: Peter Sullivan <pvsone@gmail.com>

view details

Calle Pettersson

commit sha 36781fb203b3aa38e45e59bffab39dffc923ca86

topdown: Add base64.is_valid builtin Adds a builtin to check if a string is valid base64 Fixes: #2690 Signed-off-by: Calle Pettersson <calle@cape.nu>

view details

Torin Sandall

commit sha 729f909ede260e579789c154f7efb0a047c52bbc

Merge branch 'master' into dev

view details

push time in 2 days

push eventtsandall/opa

Torin Sandall

commit sha e88579bfdae8fd6edd940915a2fb259ebdd21429

ast: Fix object corruption during safety reordering The safety check was corrupting object and set values that contained comprehension as object keys or set elements because the comprehension values themselves were mutated in place. This change fixes the issue by copying object/set values like we do in other places. This change also removes the setExprIndices function which was also mutating values inside of a visitor. Signed-off-by: Torin Sandall <torinsandall@gmail.com>

view details

push time in 2 days

push eventtsandall/opa

Torin Sandall

commit sha 7559ad0f51c051e56667b5d723f52cb42c979b0e

build: Move VERSION into version/version.go The decision logger unit tests had to be updated to always set the version.Version value because they are sensitive to changes in the payload sizes. Signed-off-by: Torin Sandall <torinsandall@gmail.com>

view details

dependabot[bot]

commit sha 724c0bb0be203db7d29f7ccdb6a928431d5fcbfa

build(deps): bump node-fetch in /docs/website/scripts/live-blocks Bumps [node-fetch](https://github.com/bitinn/node-fetch) from 2.6.0 to 2.6.1. - [Release notes](https://github.com/bitinn/node-fetch/releases) - [Changelog](https://github.com/node-fetch/node-fetch/blob/master/docs/CHANGELOG.md) - [Commits](https://github.com/bitinn/node-fetch/compare/v2.6.0...v2.6.1) Signed-off-by: dependabot[bot] <support@github.com>

view details

rtfee

commit sha cba68b1dda63aaa9dc3143895ab0e37cbd23f0f8

updated the Scalr integration title Signed-off-by: rtfee <ryan.fee625@gmail.com>

view details

AlexsJones

commit sha 59a7e430ecc1a532ed88f24f71d39c58ee638c24

chore(spelling): fix spelling in extensions.md and rest-api.md docs Signed-off-by: AlexsJones <alexsimonjones@gmail.com>

view details

Teemu Koponen

commit sha 5433a93d20d7e83f88953e456381bdd5ba80f194

topdown: Use an underscore instead of a dot in http.send metric name. Currently the recorded metric name is "timer_rego_builtin_http.send_ns" which causes issues for some implementations. Signed-off-by: Teemu Koponen <koponen@styra.com>

view details

Ashutosh Narkar

commit sha 8cb34e48c285a4448f8f91a215aba5c05c7ab9b1

topdown: Address negative duration for the current age of http response The current age of a http response is calculated as the difference between the current time and the value contained in the "Date" response header. There are couple of scenarios that could lead to the current age being represented as a negative duration. 1. Since the value of "Date" response header is parsed using Go's time.Parse method, it does not contain a monotonic clock reading. As a result, the time.Sub method uses wall clock readings to determine the difference between current time and the parsed version of the response time. 2. The server could set a value for the "Date" response header which may not be a true indication of when the response was generated. This change updates the logic that determines whether a cached response is fresh or not, to treat the resposne as stale if the current response age is represented as a negative duration. Signed-off-by: Ashutosh Narkar <anarkar4387@gmail.com>

view details

Torin Sandall

commit sha 7a713a63efd042cbc191db8d9b224658ebe36fa6

build: Disable the 'project' status in codecov We were seeing too many false positives from the 'project' status check. The 'patch' status check seems fine so leave that one alone for now. Signed-off-by: Torin Sandall <torinsandall@gmail.com>

view details

Torin Sandall

commit sha 4a2db1b0a9538e3f0fdc4acace8dd2ea36d98164

ast: Remove dead parser code This post-processing code was used by old the PEG parser but since the parser was rewritten by hand, we no longer need this. Signed-off-by: Torin Sandall <torinsandall@gmail.com>

view details

Torin Sandall

commit sha 2df8be639ddb1f8c5da5ef24f9ae05eec53835c8

ast: Fix output var analysis to accept refs with non-var heads Previously the output var analysis would panic if it encountered refs with non-var head terms. This assumption was invalidated in v0.17 with the introduction of indirect refs. Fixes #2678 Signed-off-by: Torin Sandall <torinsandall@gmail.com>

view details

Torin Sandall

commit sha 60025251c64946aff06290f8904677b266580e5f

ast: Fix object corruption during safety reordering The safety check was corrupting object and set values that contained comprehension as object keys or set elements because the comprehension values themselves were mutated in place. This change fixes the issue by copying object/set values like we do in other places. This change also removes the setExprIndices function which was also mutating values inside of a visitor.

view details

push time in 3 days

issue commentopen-policy-agent/opa

Expression Evaluator in rego policy

I'd mostly worry about performance and readability. For simple examples like the one above, it's fine. If you end up with logical AND, OR, etc. then you're basically just reinventing a fragment of Rego and then implementing that inside of Rego.

amar1728

comment created time in 3 days

push eventopen-policy-agent/opa

Calle Pettersson

commit sha 36781fb203b3aa38e45e59bffab39dffc923ca86

topdown: Add base64.is_valid builtin Adds a builtin to check if a string is valid base64 Fixes: #2690 Signed-off-by: Calle Pettersson <calle@cape.nu>

view details

push time in 3 days

issue closedopen-policy-agent/opa

Add verify functions for encodings (eg base64)

Currently it is not possible to verify input is in some specific encoding, such as base64. The only option I've found is letting the decode function abort with an error, which isn't quite as nice as a policy failure.

Expected Behavior

_ := base64.verify("not-base64") # false
_ := base64.verify("YWN0dWFsbHktYmFzZTY0") # true

Actual Behavior

_ := base64.decode("not-base64") # ERROR

closed time in 3 days

carlpett

PR merged open-policy-agent/opa

topdown: Add base64.verify builtin

Adds a builtin to verify a string is valid base64. Since it just covers base64, this is perhaps more of a start on #2690 than a full fix, for now, to verify (hah!) that I'm on the right way. If everything looks good, I can add base64url, urlquery, json and yaml.

+60 -0

4 comments

4 changed files

carlpett

pr closed time in 3 days

PullRequestReviewEvent

issue closedopen-policy-agent/opa

Flush decision logs on graceful shutdown

Expected Behavior

An attempt is made to flush the decision logs currently buffered to the remote server on graceful shutdown. This should at least be attempted once, but probably until the buffer has been flushed or until --shutdown-grace-period is reached.

Actual Behavior

Buffered decision logs are discarded on SIGINT/SIGTERM.

Steps to Reproduce the Problem

  1. Start OPA with decision logging to remote server configured.
  2. Send a couple of queries to generate decisions.
  3. Terminate the OPA server by sending a SIGINT/SIGTERM (ctrl +c, kubectl delete pod, etc).
  4. Check the decision log server for log entries (there will be none since the last configured push interval).

closed time in 3 days

anderseknert

issue commentopen-policy-agent/opa

Flush decision logs on graceful shutdown

Thanks for filing @anderseknert. We've got another issue tracking this so I'm going to close this one (#780).

anderseknert

comment created time in 3 days

issue commentopen-policy-agent/opa

Expression Evaluator in rego policy

@amar1728 no - there's no such built-in function today and it would be better if it stayed that way :-)

You could encode expressions into the input data and evaluate them inside of Rego, but think carefully before doing so:

{
  "expr": {
     "op": ">",
     "args": [23, 25]
  }
}
package interp

p {
   eval(input.expr.op, input.expr.args)
}

eval("!=", args) {
   args[0] != args[1]
}

eval(">", args) {
   args[0] > args[1]
}
amar1728

comment created time in 3 days

issue closedopen-policy-agent/opa

Expression Evaluator in rego policy

Is it possible to evaluate an expression in rego policy. For example, suppose there is a string "employee.age > 25" and employee.age is replaced with the value that is sent in the input, which would result in something like "23 > 25" or "28 > 25". When evaluated the string should return the result. This would help in doing operations like the following output := eval("23 > 25") output := eval("'image1' != 'image2'") output := eval("'bucket123' == 'bucket12'")

If there is no such feature available, is it possible to create a function that could do this.

closed time in 3 days

amar1728

pull request commentopen-policy-agent/opa

topdown: Add base64.verify builtin

hey @carlpett, your point about a separate function being clearer is fair. Could you rename this function to base64.is_valid instead of base64.verify? That will make it consistent w/ other similar built-in functions like semver.is_valid and regex.is_valid. If you make that change, we can merge this PR.

carlpett

comment created time in 3 days

push eventpvsone/opa

Ashutosh Narkar

commit sha 8cb34e48c285a4448f8f91a215aba5c05c7ab9b1

topdown: Address negative duration for the current age of http response The current age of a http response is calculated as the difference between the current time and the value contained in the "Date" response header. There are couple of scenarios that could lead to the current age being represented as a negative duration. 1. Since the value of "Date" response header is parsed using Go's time.Parse method, it does not contain a monotonic clock reading. As a result, the time.Sub method uses wall clock readings to determine the difference between current time and the parsed version of the response time. 2. The server could set a value for the "Date" response header which may not be a true indication of when the response was generated. This change updates the logic that determines whether a cached response is fresh or not, to treat the resposne as stale if the current response age is represented as a negative duration. Signed-off-by: Ashutosh Narkar <anarkar4387@gmail.com>

view details

Torin Sandall

commit sha 7a713a63efd042cbc191db8d9b224658ebe36fa6

build: Disable the 'project' status in codecov We were seeing too many false positives from the 'project' status check. The 'patch' status check seems fine so leave that one alone for now. Signed-off-by: Torin Sandall <torinsandall@gmail.com>

view details

Torin Sandall

commit sha 35484b292b27f389a7a51f07fff19a2d0efd88b3

Merge branch 'master' into master

view details

push time in 3 days

issue commentopen-policy-agent/opa

OPA docker terminates under reasonably high load

@balasenthil-d thanks for listing those steps but there's some crucial information we don't have yet:

  1. Scripts you wrote to send traffic to OPA
  2. Scripts/configs you wrote to start OPA
  3. The .rego file in question
  4. A sample of the 10K record data set

Please help us out by providing a minimal example that reproduces the issue; it'll save us a lot of time.

balasenthil-d

comment created time in 3 days

push eventopen-policy-agent/opa

Torin Sandall

commit sha 7a713a63efd042cbc191db8d9b224658ebe36fa6

build: Disable the 'project' status in codecov We were seeing too many false positives from the 'project' status check. The 'patch' status check seems fine so leave that one alone for now. Signed-off-by: Torin Sandall <torinsandall@gmail.com>

view details

push time in 4 days

PR merged open-policy-agent/opa

build: Disable the 'project' status in codecov

We were seeing too many false positives from the 'project' status check. The 'patch' status check seems fine so leave that one alone for now.

Signed-off-by: Torin Sandall torinsandall@gmail.com

<!--

Thanks for submitting a PR to OPA!

Before pressing 'Create pull request' please read the checklist below.

  • All code changes should be accompanied with tests. If you are not modifying any tests, just provide a short explanation of why updates to tests are not necessary. In addition to helping catch bugs, tests are extremely helpful in providing context that explains how your changes can be used.

  • All changes to public APIs must be accompanied with docs. Examples of public APIs include built-in functions, config fields, and of course, exported Go types/functions/constants/etc.

  • Commit messages should explain why you made the changes, not what you changed. Use active voice. Keep the subject line under 50 characters or so.

  • All commits must be signed off by the author. If you are not familiar with signing off, see CONTRIBUTING.md below.

For more information on contributing to OPA see:

-->

+1 -3

0 comment

1 changed file

tsandall

pr closed time in 4 days

push eventtsandall/opa

Ashutosh Narkar

commit sha 8cb34e48c285a4448f8f91a215aba5c05c7ab9b1

topdown: Address negative duration for the current age of http response The current age of a http response is calculated as the difference between the current time and the value contained in the "Date" response header. There are couple of scenarios that could lead to the current age being represented as a negative duration. 1. Since the value of "Date" response header is parsed using Go's time.Parse method, it does not contain a monotonic clock reading. As a result, the time.Sub method uses wall clock readings to determine the difference between current time and the parsed version of the response time. 2. The server could set a value for the "Date" response header which may not be a true indication of when the response was generated. This change updates the logic that determines whether a cached response is fresh or not, to treat the resposne as stale if the current response age is represented as a negative duration. Signed-off-by: Ashutosh Narkar <anarkar4387@gmail.com>

view details

Torin Sandall

commit sha dd9cfb2810a2fae812b675a598bf1e87357b3c20

build: Disable the 'project' status in codecov We were seeing too many false positives from the 'project' status check. The 'patch' status check seems fine so leave that one alone for now. Signed-off-by: Torin Sandall <torinsandall@gmail.com>

view details

push time in 4 days

PR opened open-policy-agent/opa

build: Disable the 'project' status in codecov

We were seeing too many false positives from the 'project' status check. The 'patch' status check seems fine so leave that one alone for now.

Signed-off-by: Torin Sandall torinsandall@gmail.com

<!--

Thanks for submitting a PR to OPA!

Before pressing 'Create pull request' please read the checklist below.

  • All code changes should be accompanied with tests. If you are not modifying any tests, just provide a short explanation of why updates to tests are not necessary. In addition to helping catch bugs, tests are extremely helpful in providing context that explains how your changes can be used.

  • All changes to public APIs must be accompanied with docs. Examples of public APIs include built-in functions, config fields, and of course, exported Go types/functions/constants/etc.

  • Commit messages should explain why you made the changes, not what you changed. Use active voice. Keep the subject line under 50 characters or so.

  • All commits must be signed off by the author. If you are not familiar with signing off, see CONTRIBUTING.md below.

For more information on contributing to OPA see:

-->

+1 -3

0 comment

1 changed file

pr created time in 4 days

create barnchtsandall/opa

branch : update-codecov-config

created branch time in 4 days

push eventopen-policy-agent/opa

Ashutosh Narkar

commit sha 8cb34e48c285a4448f8f91a215aba5c05c7ab9b1

topdown: Address negative duration for the current age of http response The current age of a http response is calculated as the difference between the current time and the value contained in the "Date" response header. There are couple of scenarios that could lead to the current age being represented as a negative duration. 1. Since the value of "Date" response header is parsed using Go's time.Parse method, it does not contain a monotonic clock reading. As a result, the time.Sub method uses wall clock readings to determine the difference between current time and the parsed version of the response time. 2. The server could set a value for the "Date" response header which may not be a true indication of when the response was generated. This change updates the logic that determines whether a cached response is fresh or not, to treat the resposne as stale if the current response age is represented as a negative duration. Signed-off-by: Ashutosh Narkar <anarkar4387@gmail.com>

view details

push time in 4 days

PR merged open-policy-agent/opa

topdown: Address negative duration for the current age of http response

The current age of a http response is calculated as the difference between the current time and the value contained in the "Date" response header. There are couple of scenarios that could lead to the current age being represented as a negative duration.

  1. Since the value of "Date" response header is parsed using Go's time.Parse method, it does not contain a monotonic clock reading. As a result, the time.Sub method uses wall clock readings to determine the difference between current time and the parsed version of the response time.

  2. The server could set a value for the "Date" response header which may not be a true indication of when the response was generated.

This change updates the logic that determines whether a cached response is fresh or not, to treat the resposne as stale if the current response age is represented as a negative duration.

Signed-off-by: Ashutosh Narkar anarkar4387@gmail.com

<!--

Thanks for submitting a PR to OPA!

Before pressing 'Create pull request' please read the checklist below.

  • All code changes should be accompanied with tests. If you are not modifying any tests, just provide a short explanation of why updates to tests are not necessary. In addition to helping catch bugs, tests are extremely helpful in providing context that explains how your changes can be used.

  • All changes to public APIs must be accompanied with docs. Examples of public APIs include built-in functions, config fields, and of course, exported Go types/functions/constants/etc.

  • Commit messages should explain why you made the changes, not what you changed. Use active voice. Keep the subject line under 50 characters or so.

  • All commits must be signed off by the author. If you are not familiar with signing off, see CONTRIBUTING.md below.

For more information on contributing to OPA see:

-->

+58 -0

0 comment

3 changed files

ashutosh-narkar

pr closed time in 4 days

PullRequestReviewEvent

push eventopen-policy-agent/opa

Teemu Koponen

commit sha 5433a93d20d7e83f88953e456381bdd5ba80f194

topdown: Use an underscore instead of a dot in http.send metric name. Currently the recorded metric name is "timer_rego_builtin_http.send_ns" which causes issues for some implementations. Signed-off-by: Teemu Koponen <koponen@styra.com>

view details

push time in 9 days

PR merged open-policy-agent/opa

Reviewers
topdown: Use an underscore instead of a dot in http.send metric name.

Currently the recorded metric name is "timer_rego_builtin_http.send_ns" which seems inconsistent.

This is not backwards compatible but before even trying to address that, I thought I should raise the PR to understand is the naming intentional.

Signed-off-by: Teemu Koponen koponen@styra.com

<!--

Thanks for submitting a PR to OPA!

Before pressing 'Create pull request' please read the checklist below.

  • All code changes should be accompanied with tests. If you are not modifying any tests, just provide a short explanation of why updates to tests are not necessary. In addition to helping catch bugs, tests are extremely helpful in providing context that explains how your changes can be used.

  • All changes to public APIs must be accompanied with docs. Examples of public APIs include built-in functions, config fields, and of course, exported Go types/functions/constants/etc.

  • Commit messages should explain why you made the changes, not what you changed. Use active voice. Keep the subject line under 50 characters or so.

  • All commits must be signed off by the author. If you are not familiar with signing off, see CONTRIBUTING.md below.

For more information on contributing to OPA see:

-->

+5 -1

1 comment

2 changed files

koponen-styra

pr closed time in 9 days

PullRequestReviewEvent

push eventkoponen-styra/opa

AlexsJones

commit sha 59a7e430ecc1a532ed88f24f71d39c58ee638c24

chore(spelling): fix spelling in extensions.md and rest-api.md docs Signed-off-by: AlexsJones <alexsimonjones@gmail.com>

view details

Torin Sandall

commit sha 4ab24968a3e21e7234cd760011d3edaa5989715d

Merge branch 'master' into http-send-metric-name

view details

push time in 9 days

push eventopen-policy-agent/opa

AlexsJones

commit sha 59a7e430ecc1a532ed88f24f71d39c58ee638c24

chore(spelling): fix spelling in extensions.md and rest-api.md docs Signed-off-by: AlexsJones <alexsimonjones@gmail.com>

view details

push time in 9 days

PR merged open-policy-agent/opa

chore(spelling): fix spelling in extensions.md and rest-api.md docs

Signed-off-by: AlexsJones alexsimonjones@gmail.com

Spelling corrections in docs.

Regards

+2 -2

0 comment

2 changed files

AlexsJones

pr closed time in 9 days

pull request commentopen-policy-agent/opa

topdown: Use an underscore instead of a dot in http.send metric name.

It's consistent in the sense that _ was used to separate categories and . wasn't treated as a special character. It sounds like . causes issues for some implementations that consume these metrics so this change seems fine to me. This metric was only added in v0.23 and it's unlikely that anyone is depending on it.

@koponen-styra could you add a section to the CHANGELOG.md file under Unreleased:

### Backwards Compatibility

* Renamed `timer_rego_builtin_http.send_ns` to `timer_rego_builtin_http_send_ns` to avoid issues with periods in metric keys.
koponen-styra

comment created time in 10 days

push eventAlexsJones/opa

rtfee

commit sha cba68b1dda63aaa9dc3143895ab0e37cbd23f0f8

updated the Scalr integration title Signed-off-by: rtfee <ryan.fee625@gmail.com>

view details

Torin Sandall

commit sha 040d659ac3b1da105564edd55f1a2a81e19b9adf

Merge branch 'master' into master

view details

push time in 10 days

PullRequestReviewEvent

push eventopen-policy-agent/opa

rtfee

commit sha cba68b1dda63aaa9dc3143895ab0e37cbd23f0f8

updated the Scalr integration title Signed-off-by: rtfee <ryan.fee625@gmail.com>

view details

push time in 10 days

PullRequestReviewEvent

issue closedopen-policy-agent/opa

Build IDE integration for IntelliJ

It would be nice to have an IntelliJ plugin that adds OPA syntax highlighting, syntax checking, testing, tracing, and evaluation features like we have for VS Code.

I haven't looked into the development effort required for creating IntelliJ plugins. Over time it would be great to have feature parity with the VS Code plugin.

closed time in 10 days

tsandall

issue commentopen-policy-agent/opa

Build IDE integration for IntelliJ

I'm going to close this issue now since the plugin exists over at https://github.com/vgramer/opa-idea-plugin. We can file bug/enhancement issues in that repo going forward.

tsandall

comment created time in 10 days

issue openedopen-policy-agent/gatekeeper-library

Library structure

Per discussion in open-policy-agent/gatekeeper#205 and the most recent weekly meeting, I'm opening an issue to decide how the library should be structured and track work for it.

Scope, goals, requirements

One of the main goals of Gatekeeper is to provide a reusable library that includes common policies for Kubernetes. We expect the library to grow over time and be community-owned. We've identified a few relevant personas:

  • Admins who want to install the Gatekeeper library (or a subset of it) on their cluster and begin using it. It should be easy for Admins to install the template library on the cluster and get started.

  • Developers who want to contribute to the Gatekeeper library by implementing new policies, improving or fixing existing policies, etc. It should be easy for developers to author, test, and debug their policies and contribute them to the upstream library.

  • Tinkerers who want to try out Gatekeeper for the first time and kick the tires. It should be easy for these people to deploy Gateekeeper, instantiate a few templates, exercise them, and cleanup.

We have also identified a few goals & requirements related to these personas.

  1. Policies should be organized into categories to facilitate browsing. For example, instead of having a single directory containing hundreds of policies, the library could be broken down into sub-directories for different categories like containers, images, networking, etc.

  2. Installable templates should be located under a separate root directory. This allows admins to easily install templates into their cluster (e.g., kubectl apply -f <url>)

  3. Policies contributed to the library should be accompanied with tests and at least one example constraint and resource for kicking the tires.

  4. Since the installable templates will be located separately, there should be some basic automation that generates the installable templates from the source templates.

Proposal

The existing PSP library in this repo meets (1) and (3) above but lacks (2) and (4). One option would be to:

  • Replace the spec.targets[].rego field with placeholder text and put a build step in place to template the templates (ha!).
  • Take the output of the build step (which would be a set of installable templates) and dump it into a separate root directory in this repository (e.g., templates/)

Questions

  • @maxsmythe do you think we should have unique names on the source files? One nice thing about the current naming convention is that it creates a simple abstraction for people contributing to the library.

created time in 10 days

issue commentopen-policy-agent/opa

Support gRPC body in OPA Istio plugin

@serenatus Having some way to ingest protobuf encoded messages would be nice. The approach you laid out makes sense and seems fairly low risk because it doesn't require changes to OPA (it can be done outside for experimental purposes and once we've gained confidence we could look at upstreaming.)

If it's possible to implement inside of server handler as opposed to a built-in function, I think that would be preferable since that way the policy would not be tightly coupled to the wire format (this would make it easier to test/debug policies using standard OPA tooling.)

I have wondered in the past whether we could make this work in OPA itself. For example, if bundles could be extended to include the descriptors then in theory it could unmarshal protobuf messages before running eval. This would require some upfront design work.

ledor473

comment created time in 11 days

pull request commentopen-policy-agent/opa

topdown: Add base64.verify builtin

@carlpett thanks for submitting a PR. Normally we'd just accept this since built-in functions are generally uncontroversial, however, in this case, I'm curious how urgently you require the fix--the reason is that we have another open issue that we're planning to work on that would eventually resolve the entire class of problem (#2086).

If we fixed #2086 and made built-in errors undefined instead of halting evaluation then you could simply call base64.decode without worrying (or you could say not base64.decode("...") to check if the string was invalid. This approach would be much better than adding separate "verify" functions for each and every operation that requires it.

carlpett

comment created time in 11 days

issue commentopen-policy-agent/opa

Watching input files for changes does not work in MacOS

@iofik can you share the OPA logs from when you ran this experiment? You should see something like "Processed file watch" when you save the file. I just tried this out on macOS and it worked fine (using Darwin 19.5.0 mind you).

iofik

comment created time in 11 days

issue commentopen-policy-agent/opa

Duplicate import references shouldn’t be allowed

This is a reasonable expectation though the change would be backwards incompatible. I think it's unlikely that many users would be affected by this change but we should leave the issue open for a little while and do some investigation to see how many policies in wild would be impacted.

mikol

comment created time in 11 days

push eventopen-policy-agent/opa

Torin Sandall

commit sha 7559ad0f51c051e56667b5d723f52cb42c979b0e

build: Move VERSION into version/version.go The decision logger unit tests had to be updated to always set the version.Version value because they are sensitive to changes in the payload sizes. Signed-off-by: Torin Sandall <torinsandall@gmail.com>

view details

push time in 13 days

PR merged open-policy-agent/opa

build: Move VERSION into version/version.go

This way the semantic version is always set. Tested by generating a fake release:

Cutting the release:

(dev)torin:~/src/opa$ make release-patch VERSION=0.24.0 > ~/test.patch
(dev)torin:~/src/opa$ patch -p1 < ~/test.patch
(Stripping trailing CRs from patch; use --binary to disable.)
patching file CHANGELOG.md
(Stripping trailing CRs from patch; use --binary to disable.)
patching file capabilities/v0.24.0.json
(Stripping trailing CRs from patch; use --binary to disable.)
patching file version/version.go
(dev)[*]torin:~/src/opa$ git add .
(dev)[*]torin:~/src/opa$ git commit -a -m 'wip: prepare release'
[dev 077c73b0] wip: prepare release
 3 files changed, 3236 insertions(+), 2 deletions(-)
 create mode 100644 capabilities/v0.24.0.json
(dev)torin:~/src/opa$ git tag v0.24.0

Checking the patches:

(dev)torin:~/src/opa$ head -n 10 CHANGELOG.md
# Change Log

All notable changes to this project will be documented in this file. This
project adheres to [Semantic Versioning](http://semver.org/).

## 0.24.0

### Fixes

- ast: Fix compiler to expand exprs in rule args ([#2649](https://github.com/open-policy-agent/opa/issues/2649))
----------------8<------------------
(dev)torin:~/src/opa$ head -n 10 version/version.go
// Copyright 2016 The OPA Authors.  All rights reserved.
// Use of this source code is governed by an Apache2
// license that can be found in the LICENSE file.

// Package version contains version information that is set at build time.
package version

// Version is the canonical version of OPA.
var Version = "0.24.0"

Building and checking the version:

(dev)torin:~/src/opa$ make build
make[1]: Entering directory '/home/torin/src/opa/wasm'
make: '_obj/opa.wasm' is up to date.
make[1]: Leaving directory '/home/torin/src/opa/wasm'
cp wasm/_obj/opa.wasm internal/compiler/wasm/opa/opa.wasm
CGO_ENABLED=0 GO111MODULE=on GOFLAGS=-mod=vendor go generate
CGO_ENABLED=0 GO111MODULE=on GOFLAGS=-mod=vendor go build -o opa_linux_amd64 -ldflags " -X github.com/open-policy-agent/opa/version.Version=0.24.0 -X github.com/open-policy-agent/opa/version.Vcs=077c73b0 -X github.com/open-policy-agent/opa/version.Timestamp=2020-09-11T15:33:03Z -X github.com/open-policy-agent/opa/version.Hostname=bigbox.localdomain"
(dev)torin:~/src/opa$ ./opa_linux_amd64 version
Version: 0.24.0
Build Commit: 077c73b0
Build Timestamp: 2020-09-11T15:33:03Z
Build Hostname: bigbox.localdomain

Preparing for development of next release:

(dev)torin:~/src/opa$ make dev-patch VERSION=0.25.0 >~/test.patch
(dev)torin:~/src/opa$ patch -p1 < ~/test.patchpatch
(Stripping trailing CRs from patch; use --binary to disable.)
patching file CHANGELOG.md
(Stripping trailing CRs from patch; use --binary to disable.)
patching file version/version.go
(dev)[*]torin:~/src/opa$ git commit -a -m 'wip: prepare dev'
[dev 8f61e3e7] wip: prepare dev
 2 files changed, 3 insertions(+), 1 deletion(-)

Building development version and verifying the version number:

(dev)torin:~/src/opa$ make build
make[1]: Entering directory '/home/torin/src/opa/wasm'
make: '_obj/opa.wasm' is up to date.
make[1]: Leaving directory '/home/torin/src/opa/wasm'
cp wasm/_obj/opa.wasm internal/compiler/wasm/opa/opa.wasm
CGO_ENABLED=0 GO111MODULE=on GOFLAGS=-mod=vendor go generate
CGO_ENABLED=0 GO111MODULE=on GOFLAGS=-mod=vendor go build -o opa_linux_amd64 -ldflags " -X github.com/open-policy-agent/opa/version.Version=0.25.0-dev -X github.com/open-policy-agent/opa/version.Vcs=8f61e3e7 -X github.com/open-policy-agent/opa/version.Timestamp=2020-09-11T15:33:37Z -X github.com/open-policy-agent/opa/version.Hostname=bigbox.localdomain"
(dev)torin:~/src/opa$ ./opa_linux_amd64 version
Version: 0.25.0-dev
Build Commit: 8f61e3e7
Build Timestamp: 2020-09-11T15:33:37Z
Build Hostname: bigbox.localdomain

Verifying the CHANGELOG.md file is up-to-date for development:

(dev)torin:~/src/opa$ head -n 10 CHANGELOG.md
# Change Log

All notable changes to this project will be documented in this file. This
project adheres to [Semantic Versioning](http://semver.org/).

## Unreleased

## 0.24.0

### Fixes
+14 -20

0 comment

6 changed files

tsandall

pr closed time in 13 days

push eventtsandall/opa

Torin Sandall

commit sha 5c9cd88abd0d4da7ae257a4fa0e0af162cde7e24

build: Move VERSION into version/version.go The decision logger unit tests had to be updated to always set the version.Version value because they are sensitive to changes in the payload sizes. Signed-off-by: Torin Sandall <torinsandall@gmail.com>

view details

push time in 13 days

PR opened open-policy-agent/opa

build: Move VERSION into version/version.go

This way the semantic version is always set. Tested by generating a fake release:

Cutting the release:

(dev)torin:~/src/opa$ make release-patch VERSION=0.24.0 > ~/test.patch
(dev)torin:~/src/opa$ patch -p1 < ~/test.patch
(Stripping trailing CRs from patch; use --binary to disable.)
patching file CHANGELOG.md
(Stripping trailing CRs from patch; use --binary to disable.)
patching file capabilities/v0.24.0.json
(Stripping trailing CRs from patch; use --binary to disable.)
patching file version/version.go
(dev)[*]torin:~/src/opa$ git add .
(dev)[*]torin:~/src/opa$ git commit -a -m 'wip: prepare release'
[dev 077c73b0] wip: prepare release
 3 files changed, 3236 insertions(+), 2 deletions(-)
 create mode 100644 capabilities/v0.24.0.json
(dev)torin:~/src/opa$ git tag v0.24.0

Checking the patches:

(dev)torin:~/src/opa$ head -n 10 CHANGELOG.md
# Change Log

All notable changes to this project will be documented in this file. This
project adheres to [Semantic Versioning](http://semver.org/).

## 0.24.0

### Fixes

- ast: Fix compiler to expand exprs in rule args ([#2649](https://github.com/open-policy-agent/opa/issues/2649))
----------------8<------------------
(dev)torin:~/src/opa$ head -n 10 version/version.go
// Copyright 2016 The OPA Authors.  All rights reserved.
// Use of this source code is governed by an Apache2
// license that can be found in the LICENSE file.

// Package version contains version information that is set at build time.
package version

// Version is the canonical version of OPA.
var Version = "0.24.0"

Building and checking the version:

(dev)torin:~/src/opa$ make build
make[1]: Entering directory '/home/torin/src/opa/wasm'
make: '_obj/opa.wasm' is up to date.
make[1]: Leaving directory '/home/torin/src/opa/wasm'
cp wasm/_obj/opa.wasm internal/compiler/wasm/opa/opa.wasm
CGO_ENABLED=0 GO111MODULE=on GOFLAGS=-mod=vendor go generate
CGO_ENABLED=0 GO111MODULE=on GOFLAGS=-mod=vendor go build -o opa_linux_amd64 -ldflags " -X github.com/open-policy-agent/opa/version.Version=0.24.0 -X github.com/open-policy-agent/opa/version.Vcs=077c73b0 -X github.com/open-policy-agent/opa/version.Timestamp=2020-09-11T15:33:03Z -X github.com/open-policy-agent/opa/version.Hostname=bigbox.localdomain"
(dev)torin:~/src/opa$ ./opa_linux_amd64 version
Version: 0.24.0
Build Commit: 077c73b0
Build Timestamp: 2020-09-11T15:33:03Z
Build Hostname: bigbox.localdomain

Preparing for development of next release:

(dev)torin:~/src/opa$ make dev-patch VERSION=0.25.0 >~/test.patch
(dev)torin:~/src/opa$ patch -p1 < ~/test.patchpatch
(Stripping trailing CRs from patch; use --binary to disable.)
patching file CHANGELOG.md
(Stripping trailing CRs from patch; use --binary to disable.)
patching file version/version.go
(dev)[*]torin:~/src/opa$ git commit -a -m 'wip: prepare dev'
[dev 8f61e3e7] wip: prepare dev
 2 files changed, 3 insertions(+), 1 deletion(-)

Building development version and verifying the version number:

(dev)torin:~/src/opa$ make build
make[1]: Entering directory '/home/torin/src/opa/wasm'
make: '_obj/opa.wasm' is up to date.
make[1]: Leaving directory '/home/torin/src/opa/wasm'
cp wasm/_obj/opa.wasm internal/compiler/wasm/opa/opa.wasm
CGO_ENABLED=0 GO111MODULE=on GOFLAGS=-mod=vendor go generate
CGO_ENABLED=0 GO111MODULE=on GOFLAGS=-mod=vendor go build -o opa_linux_amd64 -ldflags " -X github.com/open-policy-agent/opa/version.Version=0.25.0-dev -X github.com/open-policy-agent/opa/version.Vcs=8f61e3e7 -X github.com/open-policy-agent/opa/version.Timestamp=2020-09-11T15:33:37Z -X github.com/open-policy-agent/opa/version.Hostname=bigbox.localdomain"
(dev)torin:~/src/opa$ ./opa_linux_amd64 version
Version: 0.25.0-dev
Build Commit: 8f61e3e7
Build Timestamp: 2020-09-11T15:33:37Z
Build Hostname: bigbox.localdomain

Verifying the CHANGELOG.md file is up-to-date for development:

(dev)torin:~/src/opa$ head -n 10 CHANGELOG.md
# Change Log

All notable changes to this project will be documented in this file. This
project adheres to [Semantic Versioning](http://semver.org/).

## Unreleased

## 0.24.0

### Fixes
+10 -5

0 comment

5 changed files

pr created time in 14 days

push eventtsandall/opa

Drew Wells

commit sha 726271753ea2c6ed5de883c7f04bfe74b751f0b3

file watching only watches parent directory Watching files only works in situations where standard files are in use. In k8s, configmaps are mounted via a set of symlinks. In those situations, you will only get file events when watching the directory containing the symlink. Fixes #2588 Signed-off-by: Drew Wells <drew.wells00@gmail.com>

view details

Anders Eknert

commit sha 03187af692ecac100718c8eaa57293b5a41afbf6

Add pre-commit integration Signed-off-by: Anders Eknert <anders@eknert.com>

view details

rtfee

commit sha 7827766b68d4bfcd6dfe325af147bb15fb1007a5

added Scalr integration Signed-off-by: rtfee <ryan.fee625@gmail.com>

view details

scevallos

commit sha 56afac8817e69f9ee344e134a8a3eeb571f0ca46

Make correction to json.remove syntax in example Signed-off-by: scevallos <sebastianlcevallos@gmail.com>

view details

Torin Sandall

commit sha 8794cca70cb7e6dfc1a1d4e508c83282c6731d01

ast: Remove dead parser code This post-processing code was used by old the PEG parser but since the parser was rewritten by hand, we no longer need this. Signed-off-by: Torin Sandall <torinsandall@gmail.com>

view details

Torin Sandall

commit sha 3409c01b3bd9b654169c88a7b512113f394d956c

ast: Fix object corruption during safety check reordering The safety check was corrupting object values that contain comprehensions as keys (because the comprehensions were mutated, the object was corrupted.) This change modifies the safety check to use the transformer interface to avoid mutating objects. This change also removes the setExprIndices function which was not setting the expression indices safely (it was mutating inside a visitor which was the same issue as some of these other panics.) Signed-off-by: Torin Sandall <torinsandall@gmail.com>

view details

Torin Sandall

commit sha 21df8b2155a282b6a322df18edc7c1430244a9c4

ast: Fix output var analysis to accept refs with non-var heads Previously the output var analysis would panic if it encountered refs with non-var head terms. This assumption was invalidated in v0.17 with the introduction of indirect refs. Fixes #2678 Signed-off-by: Torin Sandall <torinsandall@gmail.com>

view details

push time in 14 days

create barnchtsandall/opa

branch : move-version-into-go

created branch time in 14 days

push eventtsandall/opa

Torin Sandall

commit sha 37f992d412c5876f8e2c452f793bb8742f9b4625

build: Move VERSION into version/version.go Signed-off-by: Torin Sandall <torinsandall@gmail.com>

view details

push time in 14 days

issue openedopen-policy-agent/opa

Add net.cidr_merge function to produce smallest possible list of subnets

Rego has a set of built-in functions for operating on IP addresses and CIDRs. One operation that's missing is merging of IPs and subnets into the smallest possible set. This operation is useful when operating on large numbers of IP/CIDR values (e.g., in the context of firewall rules). An example implementation of this operation can be found in the netaddr Python module.

created time in 14 days

push eventopen-policy-agent/opa

rtfee

commit sha 7827766b68d4bfcd6dfe325af147bb15fb1007a5

added Scalr integration Signed-off-by: rtfee <ryan.fee625@gmail.com>

view details

push time in 15 days

PR merged open-policy-agent/opa

added Scalr integration

Adding Scalr as an integration in the docs

Signed-off-by: Ryan Fee ryan@scalr.com

+21 -0

0 comment

2 changed files

rtfee

pr closed time in 15 days

create barnchtsandall/webpack-boilerplate

branch : master

created branch time in 16 days

created repositorytsandall/webpack-boilerplate

created time in 16 days

issue commentopen-policy-agent/opa

Signed and Trusted OPA binaries for OSX and Windows

@hixichen Linux Foundation is working on getting the certs that we need for this. It takes a bit of time. @idvoretskyi @caniszczyk ping

patrick-east

comment created time in 17 days

push eventtsandall/opa

Torin Sandall

commit sha e34f0c69490959cb9efe3db8224a4ec50dbe533d

ast: Fix output var analysis to accept refs with non-var heads Previously the output var analysis would panic if it encountered refs with non-var head terms. This assumption was invalidated in v0.17 with the introduction of indirect refs. Fixes #2678 Signed-off-by: Torin Sandall <torinsandall@gmail.com>

view details

push time in 21 days

push eventtsandall/opa

Torin Sandall

commit sha f2980bfd3dc46ccfce6911328e134776e15939d8

ast: Fix output var analysis to accept refs with non-var heads Previously the output var analysis would panic if it encountered refs with non-var head terms. Fixes #2678 Signed-off-by: Torin Sandall <torinsandall@gmail.com>

view details

push time in 21 days

issue closedopen-policy-agent/opa

Improve rule index to maintain information for tracing

Expected Behavior

When tracing is enabled, OPA should be able to return enough information for users to debug their policies and understand why certain expressions are succeeding or failing. Whether optimizations like caching or indexing are activated should not matter.

Actual Behavior

Currently, the rule index does not maintain or return debugging information when tracing is enabled. As a result, users relying on tracing to debug their policies can become frustrated when they simply see a lack of rules returned by the indexer.

Steps to Reproduce the Problem

Create a policy where rule indexing takes affect:

package foo

p {
  q
}

q {
  input.a = 1
}

Evaluate p with input undefined:

> data.foo.p
Enter data.foo.p = _
| Eval data.foo.p = _
| Index data.foo.p = _ (matched 1 rule)
| Enter p = true
| | Eval data.foo.q
| | Index data.foo.q (matched 0 rules)
| | Fail data.foo.q
| Fail data.foo.p = _
undefined

Notice that while the index event says that zero rules matched, there's no additional information describing why zero rules matched.

Additional Info

We can improve the rule index to return additional information when tracing is enabled. One option would be to make the index synthesize trace events so that index lookups appear like regular evaluation. Another option would be to introduce some index-specific debug information. These two are not mutually exclusive.

closed time in 21 days

tsandall

issue commentopen-policy-agent/opa

Improve rule index to maintain information for tracing

Closing this issue for now because we don't have any actionable next steps. We can revisit this in the future as needed.

tsandall

comment created time in 21 days

issue closedopen-policy-agent/opa

Lazy load data files with opa eval

Related to #1087 it would be nice if opa eval ... and opa test ... would only load data files (*.json, *.yaml, etc) as needed.

As-is if someone has a repo with kubernetes yamls, config json files, etc it makes it harder to manage using opa, especially for testing and authoring of policies with things like the vscode plugin.

I'm opening this issue to track the feature request and will update with a proposal after some investigation. Stay tuned for updates.

closed time in 21 days

patrick-east

issue commentopen-policy-agent/opa

Lazy load data files with opa eval

Closing this because we have no plans of implementing lazy loading in the foreseeable future.

patrick-east

comment created time in 21 days

PR opened open-policy-agent/opa

Fix panic during safety check reordering

The compiler was panicking when it encountered object literal with a comprehension key because the comprehension body may be rewritten which would corrupt the object value.

+45 -165

0 comment

4 changed files

pr created time in 21 days

push eventtsandall/opa

Torin Sandall

commit sha f03389a1812c2cf917b3be2c648ae858f9872a4b

ast: Fix panic in local var rewriting caused by object corruption The local variable rewriting stage was mutating object keys while iterating over the object itself. This caused the object to become corrupted. This commit fixes the issue by switching the stage to use the transformer interface which can safely mutate objects. The transformer is more expensive because it preemptively copies terms. If this is an issue, we could special case the transformer usage to when an object would be mutated. Fixes #2661 Signed-off-by: Torin Sandall <torinsandall@gmail.com>

view details

Torin Sandall

commit sha b9e5ef5b07207f1cd57e7f8765d78d8a0ec7130a

ast: Fix parser to ignore rules with args and key in head The parser was accepting rules that contained args and key terms in the head. This was causing a panic during type checking because later stages do expect this. This change just updates the parser to not emit a rule in this case. Note, the parser was already dealing with this correctly when the body was omitted. Fixes #2662 Signed-off-by: Torin Sandall <torinsandall@gmail.com>

view details

Frederic

commit sha 870f28ff4abc9b15b01ec48ec9bcced89812578a

topdown: Add urlquery.decode_object builtin This builtin is the reverse of the encode_object builtin and makes it easier to use the URI query parameters in policies Fixes #2647 Signed-off-by: Frederic <frederic.vanreet@icloud.com>

view details

Rob Schoening

commit sha 2f2bb7f0da37b68f9e26f681b337a14f3930d27b

spelling error Signed-off-by: Rob Schoening <rob@soluble.ai>

view details

Torin Sandall

commit sha 958a8ba835e8eed9f9f7ff5008c037eb8b36a76d

docs: Add note about logs.EventV1 fields in extensions doc Fixes #2664 Signed-off-by: Torin Sandall <torinsandall@gmail.com>

view details

Torin Sandall

commit sha 27f48f26ac5e57032cc9b2b21e3d7e4aede94cb4

ast: Remove dead parser code This post-processing code was used by old the PEG parser but since the parser was rewritten by hand, we no longer need this. Signed-off-by: Torin Sandall <torinsandall@gmail.com>

view details

Torin Sandall

commit sha 2622924c2332ee61d6d73d88dcf4fcbd0fbe647b

ast: Fix object corruption during safety check reordering The safety check was corrupting object values that contain comprehensions as keys (because the comprehensions were mutated, the object was corrupted.) This change modifies the safety check to use the transformer interface to avoid mutating objects. This change also removes the setExprIndices function which was not setting the expression indices safely (it was mutating inside a visitor which was the same issue as some of these other panics.) Signed-off-by: Torin Sandall <torinsandall@gmail.com>

view details

push time in 21 days

issue closedopen-policy-agent/opa

wasm: built-in function parity

List of built-in function categories:

  • [x] comparison
  • [x] numbers
  • [x] aggregates
  • [x] arrays
  • [x] sets
  • [x] types
  • [x] bits
  • [x] strings
    • [ ] sprintf
  • [ ] conversions
  • [ ] graphs
  • [ ] net.cidr
  • [ ] glob (partial)

Requires investigation

  • [ ] regexp

Could be external

  • [ ] units
  • [ ] encoding
  • [ ] tokens
  • [ ] date/time
  • [ ] crypto

Must be external

  • [ ] runtime
  • [ ] trace
  • [ ] time.now
  • [ ] http

Not supported

  • [ ] rego

closed time in 22 days

tsandall

issue commentopen-policy-agent/opa

wasm: built-in function parity

Closing this in favor of #2679, #2680, #2681, and #2682.

tsandall

comment created time in 22 days

issue openedopen-policy-agent/opa

wasm: Implement glob functions natively

The wasm library should implement the glob.match function natively as it's heavily used in API authorization use cases.

created time in 22 days

more