profile
viewpoint

BusyJay/jemallocator 3

Rust allocator using jemalloc as a backend

BusyJay/libressl-src 2

Libressl source expected to be consumed by openssl-sys.

BusyJay/jinkela 1

Don't fight, peace & love

BusyJay/agatedb 0

A persistent key-value storage in rust.

BusyJay/atty 0

are you or are you not a tty?

BusyJay/bank-race 0

A naive simulation about race used for FOSDEM '19.

BusyJay/bytes 0

Utilities for working with bytes

BusyJay/bzip2-rs 0

libbz2 (bzip2 compression) bindings for Rust

BusyJay/cargo 0

The Rust package manager

issue commentstepancheg/rust-protobuf

Consider moving downloaded protoc binaries into a separate crate

LGTM, just protoc-vendored seems better than protoc-binaries-download.

stepancheg

comment created time in 17 hours

pull request commenttikv/grpc-rs

Enable ALPN as default

It's about backward compatibility. If it's not backward compatible, the patch won't be released on 0.5.x.

hunterlxt

comment created time in 17 hours

delete branch BusyJay/rust-protobuf

delete branch : follow-up-430

delete time in 18 hours

PR opened tikv/tikv

Reviewers
update dependencies component/gRPC needs-cherry-pick-4.0 type/bugfix

What problem does this PR solve?

Problem Summary:

grpcio-sys fixes a bug that trigger recompilation on many platforms. See tikv/grpc-rs#449.

Check List <!--REMOVE the items that are not applicable-->

Tests <!-- At least one of them must be included. -->

  • Manual test
+4 -4

0 comment

1 changed file

pr created time in a day

create barnchBusyJay/tikv

branch : update-grpcio-sys

created branch time in a day

pull request commenttikv/grpc-rs

Fetch google.rpc.Status proto if present

This seems like a solution to #443 and #284. But it seems lack send side implement. And I'm not sure if it's the correct implement to still parse status_details to string, gRPC seems to treat them as bytes slice.

stuhood

comment created time in a day

issue commenttikv/tikv

TiKV try to connect HTTP/1.x server

👍 Great analysis! Glad you didn't hurry to close it.

AndreMouche

comment created time in a day

pull request commenttikv/grpc-rs

Enable ALPN as default

Is there any way to keep backward compatible? For example, toggle the flag according to openssl version?

hunterlxt

comment created time in a day

pull request commenttikv/tikv

gRPC: disable SO_REUSEPORT

Another approach to get around the issue without introduce potential regression is to lock the port via a file lock. This should be able to avoid port conflict for TiKV instances but can't prevent other processes accidentally conflict. Though it seems enough for the reported cases.

hunterlxt

comment created time in a day

issue openedtikv/pprof-rs

build failure on musl host

thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 2, kind: NotFound, message: "No such file or directory" }', src/libcore/result.rs:1188:5
stack backtrace:
   0: backtrace::backtrace::libunwind::trace
             at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.40/src/backtrace/libunwind.rs:88
   1: backtrace::backtrace::trace_unsynchronized
             at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.40/src/backtrace/mod.rs:66
   2: std::sys_common::backtrace::_print_fmt
             at src/libstd/sys_common/backtrace.rs:84
   3: <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt
             at src/libstd/sys_common/backtrace.rs:61
   4: core::fmt::write
             at src/libcore/fmt/mod.rs:1025
   5: std::io::Write::write_fmt
             at src/libstd/io/mod.rs:1426
   6: std::sys_common::backtrace::_print
             at src/libstd/sys_common/backtrace.rs:65
   7: std::sys_common::backtrace::print
             at src/libstd/sys_common/backtrace.rs:50
   8: std::panicking::default_hook::{{closure}}
             at src/libstd/panicking.rs:193
   9: std::panicking::default_hook
             at src/libstd/panicking.rs:210
  10: std::panicking::rust_panic_with_hook
             at src/libstd/panicking.rs:471
  11: rust_begin_unwind
             at src/libstd/panicking.rs:375
  12: core::panicking::panic_fmt
             at src/libcore/panicking.rs:84
  13: core::result::unwrap_failed
             at src/libcore/result.rs:1188
  14: core::result::Result<T,E>::unwrap
             at /rustc/0de96d37fbcc54978458c18f5067cd9817669bc8/src/libcore/result.rs:956
  15: build_script_build::main
             at ./build.rs:5
  16: std::rt::lang_start::{{closure}}
             at /rustc/0de96d37fbcc54978458c18f5067cd9817669bc8/src/libstd/rt.rs:67
  17: std::rt::lang_start_internal::{{closure}}
             at src/libstd/rt.rs:52
  18: std::panicking::try::do_call
             at src/libstd/panicking.rs:292
  19: __rust_maybe_catch_panic
             at src/libpanic_unwind/lib.rs:78
  20: std::panicking::try
             at src/libstd/panicking.rs:270
  21: std::panic::catch_unwind
             at src/libstd/panic.rs:394
  22: std::rt::lang_start_internal
             at src/libstd/rt.rs:51
  23: main
  24: <unknown>

created time in a day

PR opened stepancheg/rust-protobuf

protoc: update bundled versions

This PR fixes several suggestions from #430 and updates bundled binaries.

It also includes a bug fix from tikv/protobuf-build#37.

Note I keep the current APIs instead, because we find that using binaries provided by platform first is an easy way to get around the problem when bundled one will actually not work. See tikv/protobuf-build#30.

+19 -3

0 comment

8 changed files

pr created time in 2 days

push eventBusyJay/rust-protobuf

Jay Lee

commit sha ed499e7f23a78b31699e6038ad0d178d3e358472

update binaries Signed-off-by: Jay Lee <BusyJayLee@gmail.com>

view details

push time in 2 days

create barnchBusyJay/rust-protobuf

branch : follow-up-430

created branch time in 2 days

push eventBusyJay/tlaplus

Jay Lee

commit sha 919026d9ef8437562543f4127d77b58a51334fe4

add test case Signed-off-by: Jay Lee <BusyJayLee@gmail.com>

view details

push time in 2 days

pull request commenttlaplus/tlaplus

tlc2: disable deadlock check from configuration

@lemmy Help messages and test case are added.

The ident configuration of the project is very weird. Even in the same file, like TLC.java, some lines use pure tabs, some lines use four spaces first and then use tabs for the left blanks. I'm not sure which is the preferred style, so I just keep using tab for the diff.

BusyJay

comment created time in 2 days

push eventBusyJay/tlaplus

Jay Lee

commit sha ebfd3937956e54c141a7d125f85012d745456b4e

add help message Signed-off-by: Jay Lee <BusyJayLee@gmail.com>

view details

Jay Lee

commit sha 71892d40f006e16e109939f28253512862ce8168

add test case Signed-off-by: Jay Lee <BusyJayLee@gmail.com>

view details

push time in 2 days

push eventBusyJay/tlaplus

Jay Lee

commit sha 2030c2962f1076689643e5a2ae661aa06966886e

add help message Signed-off-by: Jay Lee <BusyJayLee@gmail.com>

view details

Jay Lee

commit sha d1fe29fa13a16694863391a1f3708f8025ff236d

add test case Signed-off-by: Jay Lee <BusyJayLee@gmail.com>

view details

push time in 2 days

push eventBusyJay/tlaplus

Jay Lee

commit sha 437018779f9726b22e03e0e2b35e64b39bfa29a9

Merge branch 'master' into support-disable-deadlock-from-configuration-file Signed-off-by: Jay Lee <BusyJayLee@gmail.com>

view details

push time in 3 days

push eventBusyJay/tlaplus

Markus Alexander Kuppe

commit sha 103a121818f880a89857c85b7ea77f16d3795564

Change visibility of getTrace(TLCState) method to public.

view details

Markus Alexander Kuppe

commit sha 3b254acfb9626e8712c7123ea03b201da3be3257

Add method to print an unquoted StringValue. [Refactor][TLC]

view details

loki der quaeler

commit sha 6569e885b64c395dd0cbea1aecabf78262ffc605

Improving TLC error message for temporal existential quantifier in spec - #304 . Upon encountering the usage of a temporal existential quantifier in the specification, a better fail message is displayed: "TLC does not support temporal existential quantification over state variables." [Enhancement][Tools]

view details

loki der quaeler

commit sha 3932995edee46a61ccddac24104b796cbdfc7bfb

Improving TLC error message for temporal existential quantifier in spec - #304 . Adding the temporal for-all case as well; message displayed now reads: "TLC does not support temporal existential, nor universal, quantification over state variables." [Enhancement][Tools]

view details

loki der quaeler

commit sha 0958a599a6bbd6cc2b36bd0a8629d46aa1ef370e

Improving TLC error message for temporal existential quantifier in spec - #304 . .. and naming the EC constant better to reflect both TE and TF [Enhancement][Tools]

view details

loki der quaeler

commit sha 7cde038bc90b2ecde29a11d89821afa05a5f3a20

Monolith and variable output improvements - #393 . Correcting indenting on subaction variables' pretty print. . Scrubbing out LOCAL INSTANCE declarations of modules which are being embedded in the monolith (this produced a warning but did not halt checking.) [Bug][Enhancement][Tools]

view details

loki der quaeler

commit sha f2808bdbd90a18996157126886d32bcbed52dec1

TLC help text - #393 . Adding a help text generator, and using it to provide the help text for TLC . Adding the '-h' flag in addition to the existing '-help' flag to have TLC print usage . Removing the previous way we displayed the help text through MP [Enhancement][Tools]

view details

loki der quaeler

commit sha 5a2c4640d74d8de786eb7172148d860963a784b0

TraceExplorer help text - #393 . Making TraceExplorer use the new manpages-esque usage generator . Improving the description markup regex in the usage generator [Enhancement][Tools]

view details

loki der quaeler

commit sha 12e81733073a72c97aba3333d2c0e52b840284b8

Changelog updates . Adding verbiage concerning no-monolith-creation, help usage, and the TraceExplorer functionality.

view details

loki der quaeler

commit sha 3894e23144ab17efd285337cb6cabb30f9a1a3c0

No-monolith option in TLC - #393 . Now providing an option to not include the module TLA+ code of all dependent modules in the generated SpecTE.tla file (-nomonolith) [Enhancement][Tools]

view details

loki der quaeler

commit sha 845f9aa0f05fb30acdc4920b03179c4a8e2eaae9

TLC help text - #393 . Adding help text for '-userFile' . Removing comments referencing an argument '-bound' which is no (t| longer) implemented in the code [Enhancement][Tools]

view details

loki der quaeler

commit sha ad5d9f69e0f29815bba9ecf68a0b89d0cde41039

Changelog updates . Updating nomnolith sub-option.

view details

loki der quaeler

commit sha 97b294987a7f038e9b3a3c133fa87c671a38d4a7

TLC help text and nomonolith sub-option - #393 . Clarified help text for dump and simulate to include sub-options . Moved '-nomonolith' to a sub-option ('nomonolith') of '-generateSpecTe', ammended help text . Improved Usage Generator to accept and display sub-options [Enhancement][Tools]

view details

Markus Alexander Kuppe

commit sha 45be506dc5727bf935dc4ce1727cfe07c8ca4850

Location and name of tlaps/tlapm repository has changed. [Documentation][Toolbox]

view details

loki der quaeler

commit sha 57603d93cdcc4d77aeaccf6850300b2bfe108fa4

TraceExplorer improvements - #393 . Removed Xmx and XX:MaxDirectMemorySize for TLCRunner's default JVM arguments . Added a optional parameter to TraceExplorer's -traceExpressions and -replBis to allow the user to pass JVM arguments to the runner . Conformed TraceExplorer's valued parameters to be separated by a space and not an equals sign . Ceased redundant "Error: Error:" log prefix [Enhancement][Tools]

view details

loki der quaeler

commit sha 0abc64ea9c310388d792e855193debe12cceed36

Absolute paths on Windows not working #424 . Corrected a bug in which TLAFile creation featuring a fully qualified path was ending up incorrectly formed. [Bug][Tools]

view details

loki der quaeler

commit sha 0ee1dc9c848896bb62919a3ebb289299b9dca59f

Updating Oomph readme . Noting 2019-09 . Hyperlinking download URL for markdown clients that do not auto-do-this . Noting fallback to 2019-09 if problems are encountered

view details

loki der quaeler

commit sha c04de39ed852815eb1b7ab7217bd08dc621ddd68

Accomodating Windows full paths including drive letter - #424 . Added regex-deciphering comment . Including the case where someone uses lowercase when citing their drive letter [Enhancement][Tools]

view details

loki der quaeler

commit sha c2376fdb889e2d717e93fd8861cab0431b46e62a

Accomodating Windows full paths including drive letter - #424 . Adding a Windows unit test [Enhancement][Tools]

view details

Markus Alexander Kuppe

commit sha 7b900d4121a5a3af06591013e260e40f928fcf7f

Do not show tab bar in TLA+ editor when there is no pretty-printed module. [Refactor][Toolbox]

view details

push time in 3 days

pull request commenttikv/tikv

gRPC: disable SO_REUSEPORT

/bench +ycsb

hunterlxt

comment created time in 3 days

pull request commenttikv/tikv

raftstore: label based commit algorithm

/release

BusyJay

comment created time in 3 days

push eventBusyJay/tikv

Jay Lee

commit sha 599d0ff7f474c0fbd7bdcd306cb252c97ea761ba

heartbeat before sleep Signed-off-by: Jay Lee <BusyJayLee@gmail.com>

view details

push time in 3 days

pull request commenttikv/tikv

tests: split test_ingest_sst into two tests

For now failpoint tests need to be defined in failpoint directory.

kennytm

comment created time in 3 days

issue commenttikv/raft-rs

Consider priority during election

Interesting. I guess allowing updating priority during runtime should be enough?

BusyJay

comment created time in 3 days

Pull request review commenttikv/tikv

util: use yatp for the future_pool

 impl Builder {         } else {             "future_pool"         };-        let env = Arc::new(super::Env {-            on_tick: self.on_tick.take(),+        let env = super::Env {

Why Arc is not required anymore?

sticnarf

comment created time in 4 days

pull request commenttikv/tikv

util: use yatp for the future_pool

I think it's possible, but better leave 2.1 alone.

sticnarf

comment created time in 4 days

pull request commenttikv/tikv

raftstore: label based commit algorithm

/release

BusyJay

comment created time in 4 days

push eventBusyJay/tikv

Jay Lee

commit sha a054ab8597f2e8aca4df2f0122882f8f9b204650

add status change log Signed-off-by: Jay Lee <BusyJayLee@gmail.com>

view details

push time in 4 days

pull request commenttikv/tikv

raftstore: label based commit algorithm

/release

BusyJay

comment created time in 4 days

push eventBusyJay/tikv

Jay Lee

commit sha 72822f8ddbb9cfe2532dfdde4b8191bfd4faa699

*: update kvproto Signed-off-by: Jay Lee <BusyJayLee@gmail.com>

view details

push time in 4 days

pull request commenttikv/tikv

gRPC: disable SO_REUSEPORT

You should fill the release note.

hunterlxt

comment created time in 4 days

push eventBusyJay/tikv

Jay Lee

commit sha 5433c344a0236186da73c60971e3178f80e0ae4f

support switching mode dynamically Signed-off-by: Jay Lee <BusyJayLee@gmail.com>

view details

push time in 4 days

push eventBusyJay/raft-rs

Jay Lee

commit sha 8634b4ef2efb3828f455eee8ec7358be3b6965be

add consistent hint Signed-off-by: Jay Lee <BusyJayLee@gmail.com>

view details

push time in 4 days

push eventBusyJay/tikv

Calvin Weng

commit sha 3e460b9c2669eea733cefdcce027a09926cc4e44

MAINTAINER: remove senior maintainer role (#7286) * remove senior maintainer role Signed-off-by: Calvin Weng <wenghao@pingcap.com>

view details

Fullstop000

commit sha 5ee4bc99f008e3c0063cfab99c3b99caea47ae32

diagnostics: fix test_search (#7273) Signed-off-by: Fullstop000 <fullstop1005@gmail.com>

view details

MyonKeminta

commit sha d2b19206e1178966ed77fc9b5361f8078fb4659e

cdc: Carry a request id in events to distinguish events of different requests (#7215) Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

view details

yiwu-arbug

commit sha 4d1c3a5c3e85d507396c6ae2a858dcf143c1692c

*: Add encryption config (#7198) Signed-off-by: Yi Wu <yiwu@pingcap.com>

view details

Brian Anderson

commit sha 040e73bb497156ba37c53c65cd12550e0d526032

Make engine only a dev-dependency of raftstore (#7164) Signed-off-by: Brian Anderson <andersrb@gmail.com>

view details

pingcap-github-bot

commit sha 0969313ecf59d264457f3c09473ddc5a4ece9cfb

rust-rocksdb: add interface corresponds to the Logger interface (#7308) Signed-off-by: sre-bot <sre-bot@pingcap.com>

view details

yiwu-arbug

commit sha 496ca5a4ae53bd095dd901bfd802f87d86622359

Store encryption_key_manager in TiKVServer (#7323) Signed-off-by: Yi Wu <yiwu@pingcap.com>

view details

Renkai

commit sha 59bb32507d0a33570128d74998f5f19f2666ebf4

Localize metrics in components/raftstore/src/store/metrics.rs (#7287) * localize some metric Signed-off-by: Renkai <gaelookair@gmail.com>

view details

Brian Anderson

commit sha e41218eb252f3eaf0c07ab9bc58ca4690c5d24f8

Remove iterator code from engine crate (#7321) Signed-off-by: Brian Anderson <andersrb@gmail.com>

view details

Jay Lee

commit sha f4994615a71736abc6c41db35951d55b325d0272

Merge branch 'master' into dynamic-quorum Signed-off-by: Jay Lee <BusyJayLee@gmail.com>

view details

Jay Lee

commit sha 4d202fba964967afb6b3fee953a2d43eaf252588

use pd to control replication Signed-off-by: Jay Lee <BusyJayLee@gmail.com>

view details

push time in 4 days

Pull request review commenttikv/tikv

raftstore: tolerate out of order read index responses

 impl Peer {             "request_id" => ?read.id,             "region_id" => self.region_id,             "peer_id" => self.peer.get_id(),-            "uuid" => ?read.id,

Why remove id?

hicqu

comment created time in 5 days

Pull request review commenttikv/tikv

raftstore: tolerate out of order read index responses

 impl ReadIndexQueue {         }         self.ready_cnt -= 1;         self.handled_cnt += 1;-        self.reads.pop_front()+        let res = self+            .reads+            .pop_front()+            .expect("read_queue is empty but ready_cnt > 0");+        // If it's a follower it's necessary, otherwise the field

How about using a flag to avoid false remove?

hicqu

comment created time in 5 days

pull request commenttikv/grpc-rs

Expose an API in grpcio::env that shuts-down AND joins the threads

Joining in drop seems good to me. Although you may need to check if the current thread equal to the target thread to avoid deadlock.

cbeck88

comment created time in 5 days

pull request commenttikv/grpc-rs

Expose an API in grpcio::env that shuts-down AND joins the threads

Thanks for the detail explanation. I can see there are two things can be done:

  1. Guard "SimEnclaveMgr" with reference count, so that it won't be dropped unexpectedly;
  2. Joining the grpcio environment threads. I think we can join them in drop method.
cbeck88

comment created time in 5 days

delete branch BusyJay/grpc-rs

delete branch : use-futures-0.3

delete time in 5 days

push eventtikv/grpc-rs

Jay

commit sha fd66ec789b2601cd88f015fa4a33a6b5da431c11

*: switch to futures 0.3 (#447) Close #237. Signed-off-by: Jay Lee <BusyJayLee@gmail.com>

view details

push time in 5 days

PR merged tikv/grpc-rs

Reviewers
*: switch to futures 0.3 Enhancement

Close #237.

Async trait doesn't seem to be supported yet. There is a solution, but I would rather wait sometime and check the community adoptions.

+867 -945

1 comment

41 changed files

BusyJay

pr closed time in 5 days

issue closedtikv/grpc-rs

Add support for async/await syntax

Futures 0.3 is still in alpha and the async/await syntax hasn't stabilized yet but I think we should prepare for the move in a separate branch if possible. This is issue is for the discussion of work items to be done before we can merge this.

  • [x] async/await syntax is stabilized
  • [ ] upgrade to futures 0.3

Anything else?

closed time in 5 days

thedrow

pull request commenttikv/grpc-rs

Expose an API in grpcio::env that shuts-down AND joins the threads

Environment is usually wrapped in Arc, it will be useless to call the shutdown method.

Then they try to tear down their enclaves and SIGSEGV occurs.

I don't get it. All servers and clients should be shutdown before the environment, so even though the thread is not shutdown, it should not try to touch "enclaves".

Is it possible to make environment out of the loop scope?

cbeck88

comment created time in 5 days

Pull request review commentpingcap/kvproto

pdpb: add replicate_mode

+syntax = "proto3";+package replicate_mode;++// The replicate status sync from PD to TiKV.+message ReplicateStatus {

Maybe we need to name it as ReplicationStatus?

disksing

comment created time in 5 days

pull request commenttikv/tikv

*: reduce sys_getdents syscall

Fixed a bug that causes notable memory fragmentation when TiKV node is running for long time

Any issue or memory graph that demonstrate the problem and the cause?

The example in the issue seems different from our cases. Because the directory is listed frequently, so memory should be often reclaimed instead of freed randomly. /cc @ethercflow

breeswish

comment created time in 5 days

pull request commenttikv/tikv

*: reduce sys_getdents syscall

The flag is used to opt out multi-level feedback queue. yatp future pool only use single level, doesn't have such feature.

breeswish

comment created time in 5 days

push eventBusyJay/raft-rs

Jay Lee

commit sha 0af309e6a3e6c955f03e3e197dc4ed68cf19bcd1

introduce group commit Signed-off-by: Jay Lee <BusyJayLee@gmail.com>

view details

push time in 6 days

push eventBusyJay/raft-rs

Jay Lee

commit sha 392347109bc9948f39856878409a0091ed93b246

introduce group commit Signed-off-by: Jay Lee <BusyJayLee@gmail.com>

view details

push time in 6 days

pull request commenttikv/tikv

*: reduce sys_getdents syscall

How about using yatp future pool to replace tokio thread pool?

breeswish

comment created time in 6 days

Pull request review commenttikv/website

add roadmap page

+---+title: TiKV RoadMap+description: Roadmap to TiKV 4.0 and beyond+menu:+    "3.1-beta":+        weight: 4+---++This document describes the roadmap for TiKV development. As an open source project, TiKV is developed by a community of contributors and adopted by many of them in production. That's where most of the goals on the roadmap come from. Currently, the 4.0 goals are being actively implemented. However, there is no gurantee that the medium-term goals will be implemented in any spefic order.

The menu says it's 3.1-beta, but the page says it's 4.0.

dcalvin

comment created time in 6 days

push eventBusyJay/grpc-rs

Eran Rundstein

commit sha e5efde00fa287c5af2630ba20ecb828f67569561

build: fix rebuild rules for no prebuilt bindings only emit rerun-if-changed=bindings to the target-specific file when it is being used (fix #449) Signed-off-by: Eran Rundstein <eran@rundste.in>

view details

Jay

commit sha a5dd35f718b50e8818d3e6a74167cc181469e746

*: bump 0.5.1 (#451) Signed-off-by: Jay Lee <BusyJayLee@gmail.com>

view details

Jay

commit sha acfa3a4f5a5368bc49f83d0bb71427fb486c5198

grpc-sys: downgrade bindgen (#452) Due to rust-lang/cargo#5237, upgrading bindgen is not backward compatible. Signed-off-by: Jay Lee <BusyJayLee@gmail.com>

view details

Jay Lee

commit sha 84c639afab74ef47d588562714676596b5615838

Merge branch 'master' into use-futures-0.3 Signed-off-by: Jay Lee <BusyJayLee@gmail.com>

view details

Jay Lee

commit sha 7d73c4452df73f989e4e528600d89d09be0997f1

address comment Signed-off-by: Jay Lee <BusyJayLee@gmail.com>

view details

push time in 6 days

Pull request review commenttikv/grpc-rs

*: switch to futures 0.3

 impl SinkBase {         }         self.batch_f = Some(write_f);         self.send_metadata = false;-        Ok(true)+        Ok(())     } -    fn poll_complete(&mut self) -> Poll<(), Error> {-        if let Some(ref mut batch_f) = self.batch_f {-            try_ready!(batch_f.poll());+    #[inline]+    fn poll_ready(&mut self, cx: &mut Context) -> Poll<Result<()>> {+        match &mut self.batch_f {+            None => return Poll::Ready(Ok(())),+            Some(f) => {+                ready!(Pin::new(f).poll(cx)?);+            }         }-         self.batch_f.take();

But it needs to be kept if it's Pending.

BusyJay

comment created time in 6 days

Pull request review commenttikv/tikv

Support Check-CN and Reloading Certs

 impl Simulator for ServerCluster {             sim_router.clone(),             Arc::clone(&engines.kv),             Arc::clone(&importer),+            security_mgr.clone(),         );         // Create Debug service.         let debug_service = DebugService::new(             engines.clone(),             raft_router.clone(),             store.gc_worker.clone(),+            security_mgr.clone(),         );          // Create deadlock service.-        let deadlock_service = lock_mgr.deadlock_service();+        let deadlock_service = lock_mgr.deadlock_service(security_mgr.clone());

L181 seems to be removed in the original PR.

hunterlxt

comment created time in 6 days

pull request commenttikv/tikv

Cherry-pick Check-CN and Reloading Certs

Don't name PRs that's not cherry-pick as cherry-pick, it's misleading.

hunterlxt

comment created time in 6 days

Pull request review commenttikv/tikv

Cherry-pick Check-CN and Reloading Certs

 time = "0.1" rand = "0.6.5" rand_isaac = "0.1.1" slog = { version = "2.3", features = ["max_level_trace", "release_max_level_debug"] }+grpcio = { version = "0.5", default-features = false, features = ["openssl-vendored"] }

Should keep default-features.

hunterlxt

comment created time in 6 days

Pull request review commenttikv/tikv

Remove protobuf dependencies from engine crate

 publish = false jemalloc = ["rocksdb/jemalloc"] portable = ["rocksdb/portable"] sse = ["rocksdb/sse"]-protobuf-codec = [-  "kvproto/protobuf-codec",-  "raft/protobuf-codec",-]-prost-codec = [-  "prost",-  "kvproto/prost-codec",-  "raft/prost-codec",-]  [dependencies] configuration = { path = "../configuration" } encryption = { path = "../encryption" } engine_traits = { path = "../engine_traits" } hex = "0.4"-kvproto = { git = "https://github.com/pingcap/kvproto.git", default-features = false } lazy_static = "1.3" prometheus = { version = "0.8", features = ["nightly", "push", "process"] }-prost = { version = "0.6", optional = true } protobuf = "2.8"-quick-error = "1.2.2"-raft = { version = "0.6.0-alpha", default-features = false }

So the dependency is not used at all in the past?

brson

comment created time in 6 days

delete branch BusyJay/protobuf-build

delete branch : fix-powerpc-detection

delete time in 6 days

pull request commenttlaplus/tlaplus

tlc2: disable deadlock check from configuration

If you are not in a hurry, I can do that in the weekend.

BusyJay

comment created time in 6 days

delete branch BusyJay/yatp

delete branch : coarse-time

delete time in 6 days

PR closed tikv/yatp

Reviewers
*: introduce coarse time enhancement

Scheduling algorithm doesn't require accurate time measurement, so use coarse time to reduce overhead. Benchmark shows that there are about 10% improvement in all cases.

+104 -37

1 comment

7 changed files

BusyJay

pr closed time in 6 days

pull request commenttikv/grpc-rs

Pick #322 #434

These are too many changes to v0.4.x. I suggest to use grpcio 0.5.0 directly.

hunterlxt

comment created time in 6 days

pull request commenttikv/grpc-rs

*: switch to futures 0.3

PTAL

BusyJay

comment created time in 6 days

issue commenttikv/grpc-rs

grpcio-sys recompiling every build on non-linux platforms

Now, cargo update -p should update grpcio-sys to 0.5.2.

kylefleming

comment created time in 6 days

delete branch BusyJay/grpc-rs

delete branch : revert-upgrading-bindgen

delete time in 6 days

push eventtikv/grpc-rs

Jay

commit sha acfa3a4f5a5368bc49f83d0bb71427fb486c5198

grpc-sys: downgrade bindgen (#452) Due to rust-lang/cargo#5237, upgrading bindgen is not backward compatible. Signed-off-by: Jay Lee <BusyJayLee@gmail.com>

view details

push time in 6 days

PR merged tikv/grpc-rs

grpc-sys: downgrade bindgen BugFix

Due to rust-lang/cargo#5237, upgrading bindgen is not backward compatible.

+8 -4

2 comments

3 changed files

BusyJay

pr closed time in 6 days

PR opened tikv/protobuf-build

Reviewers
fix powerpc detection

Appears rust's output is "powerpc64" instead of "ppc64le".

+1 -1

0 comment

1 changed file

pr created time in 7 days

create barnchBusyJay/protobuf-build

branch : fix-powerpc-detection

created branch time in 7 days

pull request commenttikv/grpc-rs

Fixed date in changelog

Thanks, please sign off your commit.

JakubOnderka

comment created time in 7 days

pull request commentpingcap/kvproto

Update grpcio

Have you run make to ensure no generated files have been updated?

hunterlxt

comment created time in 7 days

pull request commenttikv/grpc-rs

grpc-sys: downgrade bindgen

Can you link to some known related issues we found here?

Yes, it was provided in the PR description.

BusyJay

comment created time in 7 days

push eventBusyJay/grpc-rs

Jay Lee

commit sha f65c4294b952b65160157e4d885d7fc78b0c1526

grpc-sys: downgrade bindgen Due to rust-lang/cargo#5237, upgrading bindgen is not backward compatible. Signed-off-by: Jay Lee <BusyJayLee@gmail.com>

view details

push time in 7 days

PR opened tikv/grpc-rs

Reviewers
grpc-sys: downgrade bindgen BugFix

Due to rust-lang/cargo#5237, upgrading bindgen is not backward compatible.

+8 -4

0 comment

3 changed files

pr created time in 7 days

create barnchBusyJay/grpc-rs

branch : revert-upgrading-bindgen

created branch time in 7 days

issue commenttikv/grpc-rs

grpcio-sys recompiling every build on non-linux platforms

OK, I will downgrade bindgen for 0.5.x due to compatibility issue discussed above..

kylefleming

comment created time in 7 days

issue commenttikv/grpc-rs

grpcio-sys recompiling every build on non-linux platforms

@eranrund Can you share your Cargo.toml? I suspect that's because 0.5.1 updates bindgen, which may be conflict with your existing dependency and prevents cargo to use an updated version. It's probably related to https://github.com/rust-lang/cargo/issues/5237.

kylefleming

comment created time in 7 days

Pull request review commenttikv/tikv

diagnostics: fix test_search

 mod log {             req.set_levels(vec![LogLevel::Warn.into()].into());             req.set_patterns(vec![".*test-filter.*".to_string()].into());             let expected = vec![-                "2019/08/23 18:11:58.387 +08:00",                 "2019/08/23 18:09:58.387 +08:00",+                "2019/08/23 18:11:58.387 +08:00",

Seems the search result need to be sorted in this test to make it stable?

Fullstop000

comment created time in 7 days

issue commenttikv/grpc-rs

grpcio-sys recompiling every build on non-linux platforms

Good point! Can you use command cargo update -p grpcio-sys to get around the problem?

kylefleming

comment created time in 7 days

Pull request review commenttikv/tikv

ucp 7058: raftstore: pause split check when bulk insert

 fn test_follower_slow_split() {     assert!(rx.recv_timeout(Duration::from_millis(100)).is_ok()); } +fn gen_split_region() -> (Region, Region, Region) {+    let mut cluster = new_server_cluster(0, 2);+    let region_max_size = 50000;+    let region_split_size = 30000;+    cluster.cfg.raft_store.split_region_check_tick_interval = ReadableDuration::millis(100);+    cluster.cfg.coprocessor.region_max_size = ReadableSize(region_max_size);+    cluster.cfg.coprocessor.region_split_size = ReadableSize(region_split_size);++    let mut range = 1..;+    cluster.run();+    let pd_client = Arc::clone(&cluster.pd_client);+    let region = pd_client.get_region(b"").unwrap();+    let last_key = put_till_size(&mut cluster, region_split_size, &mut range);+    let target = pd_client.get_region(&last_key).unwrap();++    assert_eq!(region, target);++    let max_key = put_cf_till_size(&mut cluster, CF_WRITE, region_max_size, &mut range);++    let left = pd_client.get_region(b"").unwrap();+    let right = pd_client.get_region(&max_key).unwrap();+    if left == right {+        cluster.wait_region_split_max_cnt(&region, 20, 250, false);+    }++    let left = pd_client.get_region(b"").unwrap();+    let right = pd_client.get_region(&max_key).unwrap();++    (region, left, right)+}++#[test]+fn test_pause_split_when_snap_gen_will_split() {+    let is_generating_snapshot = "is_generating_snapshot";+    fail::cfg(is_generating_snapshot, "return()").unwrap();++    let (region, left, right) = gen_split_region();++    assert_ne!(left, right);+    assert_eq!(region.get_start_key(), left.get_start_key());+    assert_eq!(right.get_start_key(), left.get_end_key());+    assert_eq!(region.get_end_key(), right.get_end_key());++    fail::remove(is_generating_snapshot);+}++#[test]+fn test_pause_split_when_snap_gen_never_split() {

1 second is still too long. You can adapt gen_split_region to accept custom timeout. 200ms should be enough for one split.

SSebo

comment created time in 7 days

PR opened tikv/tikv

Reviewers
tests: make snapshot generate stable component/test-bench type/bugfix

What problem does this PR solve?

Close #7302 Close #3215

What is changed and how it works?

Adjusting snapshot settings to make snapshot harder to become stale or deleted by snapshot manager.

Check List <!--REMOVE the items that are not applicable-->

Tests <!-- At least one of them must be included. -->

  • Unit test
  • Integration test
+3 -1

0 comment

1 changed file

pr created time in 7 days

create barnchBusyJay/tikv

branch : make-snapshot-generate-stable

created branch time in 7 days

push eventBusyJay/tikv

Renkai

commit sha 2d0996c9f70903e08f60b510e3d4c37b896484af

use patch instead of replace (#7090) Signed-off-by: Renkai <gaelookair@gmail.com>

view details

Yilin Chen

commit sha 830ba5f34f7180004a742f4019bc4830ef8b2c32

*: do not unify read pool by default if legacy pools are configured (#7059) Signed-off-by: Yilin Chen <sticnarf@gmail.com>

view details

Lei Zhao

commit sha aa77b1b11c79b5ace5689916a1fa4db7e062e7ca

kv_service: fix the wrong metric name of handling batch batch_rollback (#7098) Signed-off-by: youjiali1995 <zlwgx1023@gmail.com>

view details

Wallace

commit sha 6b0767b53838ad6d88b252416e26ef36924db7d1

remove immutable change for WriteBatch (#6725) Signed-off-by: Little-Wallace <bupt2013211450@gmail.com>

view details

glorv

commit sha 8ce5e9889e29f950d05bc9f9cc219e0a1a66c5a9

test: add github workflow to auto-assign failed test issue to solid tests project (#7102) Signed-off-by: glorv <glorvs@163.com>

view details

Qinxuan Chen

commit sha c2483372ed434247a3fb4cc3ca6ad542c0008a2e

Fuzz: update dependencies (#7035) Signed-off-by: koushiro <koushiro.cqx@gmail.com>

view details

NingLin-P

commit sha e1da009f68d09382b274c40e345d3d976c34d4f3

tikv-server: compatible with pd that did not implement configpb.Config service (#7073) Signed-off-by: linning <linningde25@gmail.com>

view details

Wenxuan

commit sha 4a883dc46263e09fa8a2176a1be72c722f69af3f

Support signed plus arithmetic (#6568) Signed-off-by: Breezewish <breezewish@pingcap.com>

view details

Jay

commit sha ce9cc5dac6c683f0a2a66465e61ba4887ff15c98

tests/config: readable diff check (#7100) Signed-off-by: Jay Lee <BusyJayLee@gmail.com>

view details

TXXT

commit sha 6b0bbe1362bda01cae4222518f4c676a05b13ec2

Update dependencies for grpcio (#7103) Signed-off-by: Xintao <hunterlxt@live.com>

view details

Jay

commit sha 9637262b123433fe086de75abb2135c987bfd741

extern_storage: replace clap yaml with structopt (#7116) ### What have you changed? According to RUSTSEC-2018-0006, we should not use yaml-rust < 0.4.1, which is depended by clap. So I rewrite the usage using structopt. After this PR we can enable audit check in CI again. Close #7004. ### What is the type of the changes? - Bugfix ### How is the PR tested? - Manual test ### Does this PR affect documentation (docs) or should it be mentioned in the release notes? No. ### Does this PR affect `tidb-ansible`? No.

view details

glorv

commit sha 8a1081d65c6d8016710dff06030fddc65893b8fe

update sysinfo to support build on arm64 (#7089) Signed-off-by: glorv <glorvs@163.com>

view details

glorv

commit sha 18b87012ac3450e8037baaa61ea26a73c40ef33e

test: fix auto-assign workflow trigger event (#7118) Signed-off-by: glorv <glorvs@163.com>

view details

Jay

commit sha 3312f822466ce5804e2e070ace224885e4c6355c

raftstore: apply early (#6154) Signed-off-by: Jay Lee <busyjaylee@gmail.com> Signed-off-by: Jay Lee <BusyJayLee@gmail.com>

view details

Ryan Leung

commit sha 41f6282add1abb2f58fdc18a78b8d319cf143aee

server: support tls for the status server (#5393) Signed-off-by: Ryan Leung <rleungx@gmail.com>

view details

Jay

commit sha 53609223452857d4c029d9cd4aacb643f9dcc3c9

build: speed up clippy check (#7125) Signed-off-by: Jay Lee <BusyJayLee@gmail.com>

view details

yiwu-arbug

commit sha bc9ece88f3b8b5c4fbcca17e34d45ff11346a02c

external_storage: update to rusoto 0.34 (#7117) Signed-off-by: Yi Wu <yiwu@pingcap.com> ### What have you changed? update rusoto (aws lib) to 0.34, which provides IAM support. It changes a few API to use newer futures/tokio version. Update accordingly. Fixes #7096 ### What is the type of the changes? Improvement, Engineering ### How is the PR tested? build ### Does this PR affect documentation (docs) or should it be mentioned in the release notes? No ### Does this PR affect `tidb-ansible`? No ### Refer to a related PR or issue link (optional) ### Benchmark result if necessary (optional) ### Any examples? (optional)

view details

Brian Anderson

commit sha 74854810ac3de2cd60c0566384e09d63c5b1b2d2

Delete several parts of engine crate (#7033) Signed-off-by: Brian Anderson <andersrb@gmail.com>

view details

yiwu-arbug

commit sha 6f2f9f36ce26fd8fe0bc06c88c478ade2e0e2bc1

external_storage: fix crash when using s3 (#7136) Signed-off-by: Yi Wu <yiwu@pingcap.com>

view details

qupeng

commit sha 250258cb239ab0e2554bcec53967523552905ae8

raftstore: fix a panic about read index during transfer leader (#7101) Signed-off-by: qupeng <qupeng@pingcap.com>

view details

push time in 7 days

created tagtikv/grpc-rs

tagv0.5.1

The gRPC library for Rust built on C Core library and futures

created time in 7 days

issue commenttikv/tikv

import::sst_service::test_ingest_sst failed

Cool, can you fix it? It has been reported to fail three times today.

sre-bot

comment created time in 7 days

delete branch BusyJay/grpc-rs

delete branch : bump-0.5.1

delete time in 7 days

push eventtikv/grpc-rs

Jay

commit sha a5dd35f718b50e8818d3e6a74167cc181469e746

*: bump 0.5.1 (#451) Signed-off-by: Jay Lee <BusyJayLee@gmail.com>

view details

push time in 7 days

PR merged tikv/grpc-rs

*: bump 0.5.1

Bump 0.5.1 to include several recent updates.

+9 -3

0 comment

3 changed files

BusyJay

pr closed time in 7 days

delete branch BusyJay/tikv

delete branch : update-raft

delete time in 8 days

push eventBusyJay/grpc-rs

Jay Lee

commit sha 9140edd6073b1ca26189bd35e43bc348762d256c

*: bump 0.5.1 Signed-off-by: Jay Lee <BusyJayLee@gmail.com>

view details

push time in 8 days

PR opened tikv/grpc-rs

Reviewers
*: bump 0.5.1

Bump 0.5.1 to include several recent updates.

+9 -3

0 comment

3 changed files

pr created time in 8 days

create barnchBusyJay/grpc-rs

branch : bump-0.5.1

created branch time in 8 days

issue commenttikv/tikv

server::lock_manager::test_detect_deadlock_when_transfer_region failed

Similar issue: https://github.com/tikv/tikv/issues/7066

sre-bot

comment created time in 8 days

issue commenttikv/tikv

server::lock_manager::test_detect_deadlock_when_transfer_region failed

@youjiali1995 any idea to make it stable?

sre-bot

comment created time in 8 days

push eventBusyJay/raft-rs

Jay

commit sha e624c1d48460940a40d8aa69b5329460d9af87dd

proto: update dependencies (#348) update protobuf-build to 0.11 and prost to 0.6 Signed-off-by: Jay Lee <BusyJayLee@gmail.com>

view details

NingLin-P

commit sha c82696c904e1d4e05ee9ecb1cfd8bf3f957a842a

advance commit index by MsgReadIndexResp (#346) * advance commit index by MsgReadIndexResp Signed-off-by: linning <linningde25@gmail.com>

view details

hhbti

commit sha 040b8b96b70fb3766e1b2e33f3f02ff64f73bf82

Remove StatusRef (#351) This effectively reverts part of 065c37d653daf654b8ef466e8f1658411264e3eb. Because it is not necessary after b86e9e00067e5dcce10385c65506a38817d22430. Normal Status also uses borrow. Signed-off-by: Haibo Huang <hhng2046@gmail.com>

view details

Jay

commit sha e2a1841203ad6b3a4b2a6df1a38fea5c88f44e7e

*: allow index start from 0 (#353) This PR removes the modifications of index during initialization, which will allow raft's log index goes from 0 again. This is important as many existing test cases rely on it, especially those related to flow control. Those cases pass on master in an accidental way, checking what are exact broken will be too much work. Besides upstream patches from etcd also rely on the same assumption, only forbid growing from 0 in raft-rs will make it hard to port patches. Most of the changes are basically revert of index changes introduced from #196 and #214. Signed-off-by: Jay Lee <BusyJayLee@gmail.com>

view details

Jay

commit sha b9891b673573fad77ebcf9bbe0969cf945841926

*: port coreos/etcd#9985 (#354) The patch is to speed up log replication when a node is way behind than leader and logs are not compacted yet. Signed-off-by: Jay Lee <BusyJayLee@gmail.com>

view details

Jay Lee

commit sha aca2e95b084a5d7e3c6fa3be4897ee8c3801c3f1

Merge branch 'master' into custom-quorum-only-on-committing-logs Signed-off-by: Jay Lee <BusyJayLee@gmail.com>

view details

push time in 8 days

issue commenttikv/tikv

import::sst_service::test_ingest_sst failed

Tests is unstable since #7241. Can you take a look? @kennytm

sre-bot

comment created time in 8 days

Pull request review commenttikv/tikv

ucp 7058: raftstore: pause split check when bulk insert

 fn test_follower_slow_split() {     assert!(rx.recv_timeout(Duration::from_millis(100)).is_ok()); } +fn gen_split_region() -> (Region, Region, Region) {+    let mut cluster = new_server_cluster(0, 2);+    let region_max_size = 50000;+    let region_split_size = 30000;+    cluster.cfg.raft_store.split_region_check_tick_interval = ReadableDuration::millis(100);+    cluster.cfg.coprocessor.region_max_size = ReadableSize(region_max_size);+    cluster.cfg.coprocessor.region_split_size = ReadableSize(region_split_size);++    let mut range = 1..;+    cluster.run();+    let pd_client = Arc::clone(&cluster.pd_client);+    let region = pd_client.get_region(b"").unwrap();+    let last_key = put_till_size(&mut cluster, region_split_size, &mut range);+    let target = pd_client.get_region(&last_key).unwrap();++    assert_eq!(region, target);++    let max_key = put_cf_till_size(&mut cluster, CF_WRITE, region_max_size, &mut range);++    let left = pd_client.get_region(b"").unwrap();+    let right = pd_client.get_region(&max_key).unwrap();+    if left == right {+        cluster.wait_region_split_max_cnt(&region, 20, 250, false);+    }++    let left = pd_client.get_region(b"").unwrap();+    let right = pd_client.get_region(&max_key).unwrap();++    (region, left, right)+}++#[test]+fn test_pause_split_when_snap_gen_will_split() {+    let is_generating_snapshot = "is_generating_snapshot";+    fail::cfg(is_generating_snapshot, "return()").unwrap();++    let (region, left, right) = gen_split_region();++    assert_ne!(left, right);+    assert_eq!(region.get_start_key(), left.get_start_key());+    assert_eq!(right.get_start_key(), left.get_end_key());+    assert_eq!(region.get_end_key(), right.get_end_key());++    fail::remove(is_generating_snapshot);+}++#[test]+fn test_pause_split_when_snap_gen_never_split() {

The case can run too long, better use smaller timeout.

SSebo

comment created time in 8 days

Pull request review commenttikv/tikv

Security: work around race condition between threads updating handshakers

 impl SecurityConfig {         }         Ok((ca, cert, key))     }++    /// Determine if the cert file has been modified.+    /// If modified, update the timestamp of this modification.+    fn is_modified(&self, last: &mut Option<SystemTime>) -> Result<bool, Box<dyn Error>> {+        let this = fs::metadata(&self.cert_path)?.modified()?;+        if let Some(last) = last {+            if *last == this {+                return Ok(false);

Test for updating modified time.

hunterlxt

comment created time in 8 days

Pull request review commenttikv/tikv

ucp 7058: raftstore: pause split check when bulk insert

 impl<'a, T: Transport, C: PdClient> PeerFsmDelegate<'a, T, C> {         )     } +    #[inline]+    fn region_split_skip_max_count(&self) -> u8 {+        fail_point!("region_split_skip_max_count", |_| { u8::max_value() });+        2

I think it should be 3.

SSebo

comment created time in 8 days

Pull request review commenttikv/tikv

Security: work around race condition between threads updating handshakers

 impl SecurityConfig {         }         Ok((ca, cert, key))     }++    /// Determine if the cert file has been modified.+    /// If modified, update the timestamp of this modification.+    fn is_modified(&self, last: &mut Option<SystemTime>) -> Result<bool, Box<dyn Error>> {+        let this = fs::metadata(&self.cert_path)?.modified()?;+        if let Some(last) = last {+            if *last == this {+                return Ok(false);

Any tests?

hunterlxt

comment created time in 8 days

Pull request review commenttikv/tikv

Security: fix race condition between threads updating handshakers

 impl SecurityManager {  struct Fetcher {     cfg: Arc<SecurityConfig>,+    last_modified_time: Arc<Mutex<SystemTime>>, }  impl ServerCredentialsFetcher for Fetcher {     fn fetch(&self) -> Result<Option<ServerCredentialsBuilder>, Box<dyn Error>> {-        let (ca, cert, key) = self.cfg.load_certs()?;-        let new_cred = ServerCredentialsBuilder::new()-            .add_cert(cert, key)-            .root_cert(-                ca,-                CertificateRequestType::RequestAndRequireClientCertificateAndVerify,-            );-        Ok(Some(new_cred))+        if let Ok(mut last) = self.last_modified_time.try_lock() {

No, it's not.

hunterlxt

comment created time in 8 days

more