profile
viewpoint
Arjun Satish wicknicks @confluentinc Palo Alto, CA

turn/splicer 68

Caching Parallel Query engine for OpenTSDB

turn/DataMine 9

DataMine is Turn's data warehouse to address the challenges of information management and analytics.

turn/tpmml 6

Java PMML API

turn/camino 5

Camino - Hadoop File System Path Watcher

turn/sorcerer 5

Sorcerer - A lightweight workflow scheduler

inst-tech/opentsdb-plugins 4

A Collection of Service Discovery plugins for OpenTSDB

turn/opentsdb 2

A scalable, distributed Time Series Database.

wicknicks/avro-path 2

XPath processor for Avro Documents

turn/metrics-system 1

A CodaHale metric set for some basic system resources like disk and CPU.

turn/tsdb-query-node 1

Tuned version of TSDB to be used with Splicer (co-located with HBase)

delete branch wicknicks/ksql

delete branch : low-level-client-apis

delete time in 3 months

push eventconfluentinc/ksql

Arjun Satish

commit sha aae5f959da6e3b5851e9eb360051f49cd3f11544

feat: add methods and classes to execute low level HTTP requests in the ksqldb-api-client (#8118) * Add classes for low level HTTP calls This PR adds methods and classes that allow users to directly execute HTTP requests against ksqldb's REST server. The API client today uses a number of "high level" classes. With an 'escape hatch' as implemented in this PR, users do not need to wait for all the REST APIs, request and response objects to be implemented as java objects and classes in the api-client. This PR resolves one of the issues described in #8042. Signed-off-by: Arjun Satish <arjun@confluent.io>

view details

push time in 3 months

PR merged confluentinc/ksql

feat: add methods and classes to execute low level HTTP requests in the ksqldb-api-client

Description

This PR adds methods and classes that allow users to directly execute HTTP requests against ksqldb's REST server. The API client today uses a number of "high level" classes. With an 'escape hatch' as implemented in this PR, users do not need to wait for all the REST APIs, request and response objects to be implemented as java objects and classes in the api-client.

This PR resolves one of the issues described in #8042.

Testing done

Unit and integration tests (and tests against a local docker container).

Reviewer checklist

  • [ ] Ensure docs are updated if necessary. (eg. if a user visible feature is being added or changed).
  • [ ] Ensure relevant issues are linked (description should include text like "Fixes #<issue number>")
+722 -1

2 comments

9 changed files

wicknicks

pr closed time in 3 months

pull request commentconfluentinc/ksql

feat: add methods and classes to execute low level HTTP requests in the ksqldb-api-client

got a clean build here. merging this PR. thanks @jzaralim and @JimGalasyn for your comments. if there are follow-ups, I can take it up in a different PR. thanks!

wicknicks

comment created time in 3 months

push eventwicknicks/ksql

Arjun Satish

commit sha 83ca5ac14f198f1c02309e87d4685f286439035c

Fix checkstyle/spotbugs issues Signed-off-by: Arjun Satish <arjun@confluent.io>

view details

push time in 3 months

pull request commentconfluentinc/ksql

feat: add methods and classes to execute low level HTTP requests in the ksqldb-api-client

thanks, @JimGalasyn, addressed your comments.

wicknicks

comment created time in 3 months

push eventwicknicks/ksql

Arjun Satish

commit sha df6756cc7da7f984b4b16557e7e75601f86dca8e

Address reviewer comments for docs changes Signed-off-by: Arjun Satish <arjun@confluent.io>

view details

push time in 3 months

push eventwicknicks/ksql

Arjun Satish

commit sha 8260e3fb25a3c3f377c791f7f0b1c820ed0d20fe

Add documentation for new HttpRequest and HttpResponse interfaces Signed-off-by: Arjun Satish <arjun@confluent.io>

view details

push time in 3 months

push eventwicknicks/ksql

Arjun Satish

commit sha 1e092ee314ee7cd4e5ed2c719242bf350ffb8781

Add documentation for new HttpRequest and HttpResponse interfaces Signed-off-by: Arjun Satish <arjun@confluent.io>

view details

push time in 3 months

push eventwicknicks/ksql

Arjun Satish

commit sha 2b623c86ce86170dab7c02824e74123d67da9786

Fix issue with key for properties map Signed-off-by: Arjun Satish <arjun@confluent.io>

view details

push time in 3 months

Pull request review commentconfluentinc/ksql

feat: add methods and classes to execute low level HTTP requests in the ksqldb-api-client

 public void close() {     );   } +  @Override+  public HttpRequest buildRequest(final String method, final String path) {+    return new HttpRequestImpl(method, path, this);+  }++  CompletableFuture<HttpResponse> send(final HttpRequest request) {+    final CompletableFuture<HttpResponse> cf = new CompletableFuture<>();++    Map<String, Object> payload = request.payload();+    // include properties+    payload.put("properties", request.properties());

on second thought, let me try adding a new function that allows users to set the right parameter. I do think having a way to include properties piece-wise is quite handy, so might be worth keeping it.

wicknicks

comment created time in 3 months

PullRequestReviewEvent

Pull request review commentconfluentinc/ksql

feat: add methods and classes to execute low level HTTP requests in the ksqldb-api-client

 public void close() {     );   } +  @Override+  public HttpRequest buildRequest(final String method, final String path) {+    return new HttpRequestImpl(method, path, this);+  }++  CompletableFuture<HttpResponse> send(final HttpRequest request) {+    final CompletableFuture<HttpResponse> cf = new CompletableFuture<>();++    Map<String, Object> payload = request.payload();+    // include properties+    payload.put("properties", request.properties());

ah ok, let me remove this function then. thanks!

wicknicks

comment created time in 3 months

PullRequestReviewEvent

push eventwicknicks/ksql

Arjun Satish

commit sha 378aba91cf04729cd99ebfc24e0317885db67b8e

Address reviewer comments Signed-off-by: Arjun Satish <arjun@confluent.io>

view details

push time in 3 months

PullRequestReviewEvent

Pull request review commentconfluentinc/ksql

Add methods and classes to execute low level HTTP requests in the ksqldb-api-client

+/*+ * Copyright 2021 Confluent Inc.+ *+ * Licensed under the Confluent Community License (the "License"); you may not use+ * this file except in compliance with the License.  You may obtain a copy of the+ * License at+ *+ * http://www.confluent.io/confluent-community-license+ *+ * Unless required by applicable law or agreed to in writing, software+ * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT+ * WARRANTIES OF ANY KIND, either express or implied.  See the License for the+ * specific language governing permissions and limitations under the License.+ */++package io.confluent.ksql.api.client.impl;++import io.confluent.ksql.api.client.Client.HttpResponse;+import io.confluent.ksql.api.client.exception.KsqlClientException;+import io.vertx.core.buffer.Buffer;+import io.vertx.core.json.DecodeException;+import io.vertx.core.json.JsonObject;+import java.util.Collections;+import java.util.Map;++public class HttpResponseImpl implements HttpResponse {++  private final int status;+  private final byte[] body;++  public HttpResponseImpl(final int status, final byte[] body) {+    this.status = status;+    this.body = body;

response body can be empty, right? it should be fine to leave this as it is. we handle the null case when converting it to a map in bodyAsMap()

wicknicks

comment created time in 3 months

Pull request review commentconfluentinc/ksql

Add methods and classes to execute low level HTTP requests in the ksqldb-api-client

+/*+ * Copyright 2021 Confluent Inc.+ *+ * Licensed under the Confluent Community License (the "License"); you may not use+ * this file except in compliance with the License.  You may obtain a copy of the+ * License at+ *+ * http://www.confluent.io/confluent-community-license+ *+ * Unless required by applicable law or agreed to in writing, software+ * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT+ * WARRANTIES OF ANY KIND, either express or implied.  See the License for the+ * specific language governing permissions and limitations under the License.+ */++package io.confluent.ksql.api.client.impl;++import static org.junit.Assert.assertEquals;+import static org.junit.Assert.assertThrows;++import com.google.common.testing.EqualsTester;+import java.util.Collections;+import java.util.HashMap;+import java.util.Map;+import org.junit.Test;++public class HttpRequestImplTest {

done. thanks!

wicknicks

comment created time in 3 months

Pull request review commentconfluentinc/ksql

Add methods and classes to execute low level HTTP requests in the ksqldb-api-client

+/*+ * Copyright 2021 Confluent Inc.+ *+ * Licensed under the Confluent Community License (the "License"); you may not use+ * this file except in compliance with the License.  You may obtain a copy of the+ * License at+ *+ * http://www.confluent.io/confluent-community-license+ *+ * Unless required by applicable law or agreed to in writing, software+ * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT+ * WARRANTIES OF ANY KIND, either express or implied.  See the License for the+ * specific language governing permissions and limitations under the License.+ */++package io.confluent.ksql.api.client.impl;++import io.confluent.ksql.api.client.Client.HttpRequest;+import io.confluent.ksql.api.client.Client.HttpResponse;+import io.vertx.core.http.HttpMethod;+import java.util.HashMap;+import java.util.Locale;+import java.util.Map;+import java.util.Objects;+import java.util.concurrent.CompletableFuture;++public class HttpRequestImpl implements HttpRequest {++  private final Map<String, Object> payloadAsMap = new HashMap<>();+  private final Map<String, Object> properties = new HashMap<>();+  private final String path;+  private final HttpMethod method;+  private final ClientImpl client;++  public HttpRequestImpl(String method, String path) {+    this(method, path, null);+  }++  public HttpRequestImpl(String method, String path, ClientImpl client) {+    Objects.requireNonNull(path, "Path may not be null");+    Objects.requireNonNull(method, "HTTP method may not be null");+    this.path = path;+    this.method = HttpMethod.valueOf(method.toUpperCase(Locale.ROOT));+    this.client = client;+  }++  @Override+  public String path() {+    return this.path;+  }++  @Override+  public String method() {+    return this.method.name();+  }++  @Override+  public Map<String, Object> payload() {+    return new HashMap<>(payloadAsMap);+  }++  @Override+  public HttpRequest payload(Map<String, Object> payload) {+    Objects.requireNonNull(payload, "Payload may not be null");+    payloadAsMap.putAll(Objects.requireNonNull(payload));+    return this;+  }++  @Override+  public Map<String, Object> properties() {+    return new HashMap<>(properties);+  }++  @Override+  public HttpRequest payload(String key, Object value) {+    Objects.requireNonNull(key, "Payload key may not be null");+    Objects.requireNonNull(value, "Payload value may not be null");+    payloadAsMap.put(key, value);+    return this;+  }++  @Override+  public HttpRequest property(String key, Object value) {+    Objects.requireNonNull(key, "Property key may not be null");+    Objects.requireNonNull(value, "Property value may not be null");

addressed below.

wicknicks

comment created time in 3 months

PullRequestReviewEvent

push eventwicknicks/ksql

Arjun Satish

commit sha 8950f03a914c53dc24b01f7df0d2b11223e54c0f

Address reviewer comments Signed-off-by: Arjun Satish <arjun@confluent.io>

view details

push time in 3 months

push eventwicknicks/ksql

Arjun Satish

commit sha a1e1059e96deaa911d34b8ac5d161fd5aae5209c

Address reviewer feedback Signed-off-by: Arjun Satish <arjun@confluent.io>

view details

push time in 3 months

PR opened confluentinc/ksql

Add methods and classes to execute low level HTTP requests in the ksqldb-api-client

Description

This PR adds methods and classes that allow users to directly execute HTTP requests against ksqldb's REST server. The API client today uses a number of "high level" classes. With an 'escape hatch' as implemented in this PR, users do not need to wait for all the REST APIs, request and response objects to be implemented as java objects and classes in the api-client.

This PR resolves one of the issues described in #8042.

Testing done

Unit and integration tests (and tests against a local docker container).

Reviewer checklist

  • [ ] Ensure docs are updated if necessary. (eg. if a user visible feature is being added or changed).
  • [ ] Ensure relevant issues are linked (description should include text like "Fixes #<issue number>")
+501 -1

0 comment

6 changed files

pr created time in 3 months

PR closed wicknicks/ksql

Low level client apis

Description

What behavior do you want to change, why, how does your patch achieve the changes?

Testing done

Describe the testing strategy. Unit and integration tests are expected for any behavior changes.

Reviewer checklist

  • [ ] Ensure docs are updated if necessary. (eg. if a user visible feature is being added or changed).
  • [ ] Ensure relevant issues are linked (description should include text like "Fixes #<issue number>")
+501 -1

0 comment

6 changed files

wicknicks

pr closed time in 3 months

push eventwicknicks/ksql

Arjun Satish

commit sha 49cf50dc065a8cf0ed9da54029a8a19c3d93d25d

Refactor APIs and add method to set/update properties Signed-off-by: Arjun Satish <arjun@confluent.io>

view details

push time in 3 months

push eventwicknicks/ksql

Arjun Satish

commit sha 871b6d9e677a4b14762c8a9d86aa641d888840ae

Refactor APIs and add method to set/update properties Signed-off-by: Arjun Satish <arjun@confluent.io>

view details

push time in 3 months

CommitCommentEvent
CommitCommentEvent
CommitCommentEvent

push eventwicknicks/ksql

Confluent Jenkins Bot

commit sha 0e0dd8b95b4953b0d97b1f7c6582e2fc1129b6bd

Set Confluent to 5.4.5, Kafka to 5.4.5.

view details

leah

commit sha 0c0fb401da2c419392e87f9d88c0cabed1cdc752

chore: resolve ambiguous method reference (#8078)

view details

Alan Sheinberg

commit sha 5ce375c94b7b8f218b780f224aa5eaf7e8c4d6f9

test: Scalable Push Query functional tests (#8044) * test: Scalable Push Query functional tests

view details

Steven Zhang

commit sha aa213bd3153cc8d9d9d5b218210dfb6b5806948d

feat: add CumulativeSum total bytes metric (#7987)

view details

Walker Carlson

commit sha 66b311d7044b67fb473d6a9d054a9acab41327bf

chore: update cmd version (#8084)

view details

Victoria Xia

commit sha d0e2ea7ce896f4af0ff85babdbebfa3249f6fb71

docs: clarify use of conventional commit types (MINOR) (#8082)

view details

Zara Lim

commit sha a3ea6a46ae74c18af1fac8e4de2a0caba7496a0a

fix: add new types to udaf functions (#8081) * fix: add new types to udaf functions * checkstyle * typos * indent

view details

Sergio Peña

commit sha abacbca8eb7b0cdb55510bf0cfba22fc35b55c96

test: add QTT tests for CST commands (#8079)

view details

Sergio Peña

commit sha ccb8936d4ebe5c84ca7eca5f5fecd065c903d4c1

chore: remove flaky test for TERMINATE CST queries (#8089)

view details

Alan Sheinberg

commit sha a0be1d582b5cc27d01de98ab745d0559e9ad5ff9

fix: Removes config check to insert SPQ Processor (#8062) * fix: Adds a new config check to insert SPQ Processor

view details

Sergio Peña

commit sha 5416cde808dea59e3091eb637ab733723ff9bdcc

feat: add feature flag to disable source table materialization (#8085)

view details

Almog Gavra

commit sha 9dcc53a3d184f80cb202ed58c6328f3d13d56054

test: add logging to shouldStreamInserts in case it times out again (#8092)

view details

Sergio Peña

commit sha 4b08778d8343848b14480cf5dc58136516c9d8ef

docs: add docs for CST commands (#8090)

view details

Rohan

commit sha 38b9e760165465bcbab00274c6f74a99b7ccc81e

chore: mock describeTopics w/ explicit type to resolve ambiguous ref (#8093) * minor: mock describeTopics w/ explicit type to resolve ambiguous ref * comment

view details

Walker Carlson

commit sha cce585b0991a5a7fa39d3e24f327fdd84783f2ff

feat: add consumerGroupId to QueryDescription (#8073) * feat: add consumerGroupId to QueryDescription

view details

Zara Lim

commit sha fd9faf2c24ae5e0301c2c67156afa6e0186e7e78

fix: add time types to Java client (#8091) * fix: add time types to Java client * add integration test

view details

Rohan

commit sha eed625b331efb5ad7a1e0709c7e4d67d44d2964c

feat: add persistent query saturation metric (#7955) * feat: add persistent query saturation metric Add a persistent query saturation metric, and report it over JMX. Saturation is computed by sampling total-blocked-time for each stream thread. We sample the total blocked time every minute, and compute saturation by looking back 5 minutes and computing how long the thread was blocked in that interval. Saturation is (5 minutes - blocked time)/(5 minutes). This patch also adds a reporter that exposes reported data points over JMX. The reporter adds a new Metric for every new data point name/tag combination it sees, and implements the metric by reading the latest data point for the name/tag, with a threshold on staleness. * add logs * review feedback * rename a couple things * update metric/group names * fix bug * checkstyle

view details

John Roesler

commit sha 4831e6b099cca2be51a659d2bb4112ead0cd8387

feat: introduce stream pull queries for HTTP/1 (#8064) Partially implement stream pull queries as proposed in: KLIP-53 Only adds the new query type to the HTTP/1 endpoint. The others will be covered in later work. Intentionally does not record metrics for stream pull queries. That requires some refactoring and will be covered in later work.

view details

Alan Sheinberg

commit sha 7faf77cf91aa8e4a9437649c448f3ce2142c454c

fix: Enables ALPN for internal requests when http2 and tls are in use (#8094) * fix: Enables ALPN for internal requests when http2 and tls are in use

view details

Patrick Stuedi

commit sha 22a79bc7695795494d4aaed8618bf3de252271a4

feat: optimize key-range queries in pull queries (#7993) implementation of KLIP-54

view details

push time in 3 months

more