profile
viewpoint
Noah Kennedy Noah-Kennedy Cloudflare Milwaukee Wisconsin Making the internet a better, faster place, one oxidization at a time.

cloudflare/boringtun 3721

Userspace WireGuard® Implementation in Rust

Noah-Kennedy/actix-cbor 5

CBOR extractor for Actix-Web

aboss123/Gold 1

The Gold Rush began in 1948. This programming language is developed in 48-hours. Coincidence, I think not!

cloudflare/nel-rs 1

Basic Rust utilities for building NEL reports from network errors.

NicoletFEAR/StrongHold-2016 1

Repository for FIRST FRC team # 4786 in the 2015-2016 season and the FIRST Stronghold game.

GFLEMING133/stemBusEd 0

Stembus partenrship with blue mangoes to produce educational DESKTOP software.

NathanDuPont/DQN-Flappy-Bird 0

🕹️ Flappy Bird hack using Deep Reinforcement Learning with Double Q-learning

Noah-Kennedy/actix 0

Actor framework for Rust

pull request commentgnzlbg/jemallocator

Upgrade jemalloc to 5.2.1

@sfackler I realize that it has been two years, but what is the status of this PR?

sfackler

comment created time in 5 days

startedtikv/jemallocator

started time in 5 days

PullRequestReviewEvent

startedKeats/jsonwebtoken

started time in 5 days

startedyshui/picom

started time in 6 days

startedhyperium/h3

started time in 9 days

startedrust-lang/glacier

started time in 9 days

startedrust-lang/simpleinfra

started time in 9 days

startedbytecodealliance/cap-std

started time in 9 days

pull request commenttokio-rs/tokio

add taskset api

@Darksonn FuturesUnordered uses several linked lists (I believe one of which is intrusive) to manage the stored tasks. Implementing a similar data structure is one option, although I don't think that implementing such a complicated set of datastructures for a single API is worthwhile.

If we put this into tokio instead of tokio-util, would it be possible for us to reuse some of the internal APIs to simplify the implementation?

Noah-Kennedy

comment created time in 9 days

pull request commenttokio-rs/tokio

add taskset api

@Darksonn what do we want to do about the O(n) next_finished?

Noah-Kennedy

comment created time in 9 days

Pull request review commenttokio-rs/tokio

add taskset api

+//! TaskSet API+//!+//! This module includes a TaskSet API for scoping the lifetimes of tasks and joining tasks in a+//! manner similar to [`futures_util::stream::FuturesUnordered`]++use futures_util::FutureExt;+use std::future::Future;+use std::task::{Context, Poll};+use tokio::runtime::Handle;+use tokio::task::{JoinError, JoinHandle};++/// A [`TaskSet`] is a mechanism for regulating the lifetimes of a set of asynchronous tasks.+///+/// # Task Lifetimes+/// Tasks spawned on a [`TaskSet`] live at most slightly longer than the set that spawns them.+/// The reason for this is that aborting a task is not an instantaneous operation.+///+/// Tasks could be running in another worker thread at the time they are cancelled, preventing+/// them from being canceled immediately. As a result, when you abort those tasks using the join+/// handles, Tokio marks those futures for cancellation. There are, however ways to await the+/// cancellation of tasks, but they only work in an async context.+///+/// If you need to cancel a set and wait for the cancellation to complete, use [`Self::shutdown`].+/// That function won't complete until *all* tasks have exited.+#[derive(Debug)]+pub struct TaskSet<T> {+    tasks: Vec<JoinHandle<T>>,+    handle: Handle,+}++impl<T> TaskSet<T> {+    /// Constructs a new TaskSet.+    ///+    /// # Panic+    /// Panics if invoked outside the context of a tokio runtime.+    pub fn new() -> Self {+        Self::new_with_handle(Handle::current())+    }++    /// Constructs a new TaskSet which will spawn tasks using the supplied handle.+    pub fn new_with_handle(handle: Handle) -> Self {+        Self {+            tasks: Vec::new(),+            handle,+        }+    }++    /// Returns the amount of unjoined tasks in the set.+    pub fn len(&self) -> usize {+        self.tasks.len()+    }++    /// Returns true if there are no tasks left in the set.+    pub fn is_empty(&self) -> bool {+        self.tasks.is_empty()+    }++    /// Consume the set, waiting for all tasks on it to complete.+    ///+    /// # Output+    /// The output of this function orders the results of tasks in the same order with which they+    /// were spawned.+    ///+    /// # Cancellation Safety+    /// When canceled, any tasks which have yet to complete will become detached, and the results+    /// of completed tasks will be discarded.+    ///+    /// # Examples+    /// ```+    /// # use tokio_util::taskset::TaskSet;+    /// # tokio_test::block_on(async {+    /// let mut set = TaskSet::new();+    ///+    /// const NUMS: [u8; 5] = [0, 1, 2, 3, 4];+    ///+    /// for x in NUMS.iter() {+    ///     set.spawn(async move { *x });+    /// }+    ///+    /// let joined: Vec<_> = set.join_all().await.into_iter().map(|x| x.unwrap()).collect();+    ///+    /// assert_eq!(NUMS, joined.as_slice());+    /// # });+    /// ```+    pub async fn join_all(mut self) -> Vec<Result<T, JoinError>> {+        let mut output = Vec::with_capacity(self.tasks.len());++        for task in self.tasks.iter_mut() {+            output.push(task.await)+        }++        output

Done.

Noah-Kennedy

comment created time in 9 days

PullRequestReviewEvent

push eventtokio-rs/tokio

Noah Kennedy

commit sha cbdd285ac0aa73afb3390f36e33d30ec8d4e29ef

respond to feedback from darksonn clear taskset after joinall, so drop doesn't try and abort the completed tasks

view details

push time in 9 days

PullRequestReviewEvent
PullRequestReviewEvent

issue commenttokio-rs/async-stream

try_stream is !Send with ? and MutexGuard or try-block

Is this an std::sync::Mutex? The guards on those are not send. If you need to hold a mutex guard over an await boundary, use tokio::sync::Mutex.

elbaro

comment created time in 9 days

PullRequestReviewEvent

pull request commentcloudflare/boringtun

Remove Tun and Sock traits

Are these exposed publicly to users of the crate?

agrover

comment created time in 9 days

issue commenttokio-rs/tokio

Proposal: use env to specify the default worker thread count for multi-thread runtime

I think that this is a good feature to have, especially since configuring things like this is easily done with environment variables when under systems like K8s.

PureWhiteWu

comment created time in 12 days

pull request commenttokio-rs/tokio

Add spawn_pinned to tokio-util

I'll review this tomorrow. Currently, I'm traveling, so I can't easily review code.

AzureMarker

comment created time in 14 days

PullRequestReviewEvent

issue closedtokio-rs/tokio-uring

Smell of Bug in File::read_at(...)

When performing this function, the result is an infinite loop, But I read in the document it said: read_at result returns number of bytes read But here it always returns 100

where is problem ??

 pub async fn fetch(&mut self) -> Result<Vec<String>, ()> {
            let mut queries = Vec::new();
            loop {
                let buf = vec![0; 100];
/*---> */    let (res, buf) = self.file.read_at(buf, 0).await;
                match res {
                    Err(_) => return Err(()),
/*---> */         Ok(n) => {
                        println!("==> {}", n);
                        if n == 0 {
                            break
                        }  

                        let string = unsafe {
                            String::from_utf8_unchecked(buf)
                        };
                        queries.push(string);
                                         
                    }
                };
            };
            
            Ok(queries)
        }

closed time in 18 days

Rustixir

issue commenttokio-rs/tokio-uring

Smell of Bug in File::read_at(...)

Closing as this isn't a bug.

Rustixir

comment created time in 18 days

push eventtokio-rs/tokio

Noah Kennedy

commit sha 350e70329ba870e8c5cfa2982e98eaacb35a0fe3

fixed join_all reversing the output

view details

push time in 18 days

issue commenttokio-rs/tokio

Add a `TaskSet`

So, an issue with my approach is that there isn't a good way from tokio-util to do something like FuturesUnordered without re-implementing the internal data structure that FuturesUnordered is built around, unless you want poll_next_finished to be O(n), as the naive approach of using a vec requires you to scan through your list of tasks, polling as you go.

Darksonn

comment created time in 18 days

push eventtokio-rs/tokio

Noah Kennedy

commit sha 8ed1cf85afc1fbc7a1a309bd5ee4d426dd9b60c1

update tests in taskset to use std::future::pending instead of futures::future::pending

view details

push time in 18 days

Pull request review commenttokio-rs/tokio

add taskset api

+//! TaskSet API+//!+//! This module includes a TaskSet API for scoping the lifetimes of tasks and joining tasks in a+//! manner similar to [`futures_util::stream::FuturesUnordered`]++use std::future::Future;+use std::pin::Pin;+use std::task::{Context, Poll};+use tokio::runtime::Handle;+use tokio::task::{JoinError, JoinHandle};++/// A [`TaskSet`] is a mechanism for regulating the lifetimes of a set of asynchronous tasks.+///+/// # Task Lifetimes+/// Tasks spawned on a [`TaskSet`] live at most slightly longer than the set that spawns them.+/// The reason for this is that aborting a task is not an instantaneous operation.+///+/// Tasks could be running in another worker thread at the time they are cancelled, preventing+/// them from being canceled immediately. As a result, when you abort those tasks using the join+/// handles, Tokio marks those futures for cancellation. There are, however ways to await the+/// cancellation of tasks, but they only work in an async context.+///+/// If you need to cancel a set and wait for the cancellation to complete, use [`Self::shutdown`].+/// That function won't complete until *all* tasks have exited.+#[derive(Debug)]+pub struct TaskSet<T> {+    tasks: Vec<JoinHandle<T>>,+    handle: Handle,+}++impl<T> TaskSet<T> {+    /// Constructs a new TaskSet.+    ///+    /// # Panic+    /// Panics if invoked outside the context of a tokio runtime.+    pub fn new() -> Self {+        Self::new_with_handle(Handle::current())+    }++    /// Constructs a new TaskSet which will spawn tasks using the supplied handle.+    pub fn new_with_handle(handle: Handle) -> Self {+        Self {+            tasks: Vec::new(),+            handle,+        }+    }++    /// Returns the amount of unjoined tasks in the set.+    pub fn len(&self) -> usize {+        self.tasks.len()+    }++    /// Returns true if there are no tasks left in the set.+    pub fn is_empty(&self) -> bool {+        self.tasks.is_empty()+    }++    /// Consume the set, waiting for all tasks on it to complete.+    ///+    /// # Output+    /// The output of this function orders the results of tasks in the same order with which they+    /// were spawned.+    ///+    /// # Cancellation Safety+    /// When canceled, any tasks which have yet to complete will become detached, and the results+    /// of completed tasks will be discarded.+    ///+    /// # Examples+    /// ```+    /// # use tokio_util::taskset::TaskSet;+    /// # tokio_test::block_on(async {+    /// let mut set = TaskSet::new();+    ///+    /// const NUMS: [u8; 5] = [0, 1, 2, 3, 4];+    ///+    /// for x in NUMS.iter() {+    ///     set.spawn(async move { *x });+    /// }+    ///+    /// let joined: Vec<_> = set.join_all().await.into_iter().map(|x| x.unwrap()).collect();+    ///+    /// assert_eq!(NUMS, joined.as_slice());+    /// # });+    /// ```+    pub async fn join_all(mut self) -> Vec<Result<T, JoinError>> {+        let output = futures_util::future::join_all(self.tasks.iter_mut()).await;++        output+    }++    /// Join onto the next available task, removing it from the set.+    ///+    /// This does not affect the underlying order of tasks, other than removing the joined task.+    ///+    /// # Cancellation Safety+    /// This function is cancellation-safe. If dropped, it will be as if this was never called.+    ///+    /// # Examples+    /// ```+    /// # use tokio_util::taskset::TaskSet;+    /// # tokio_test::block_on(async {+    /// let mut set = TaskSet::new();+    ///+    /// set.spawn(async { 5u8 });+    /// set.spawn(futures::future::pending());+    ///+    /// let joined = set.next_finished().await.unwrap().unwrap();+    /// assert_eq!(5u8, joined);+    /// # });+    /// ```+    pub async fn next_finished(&mut self) -> Option<Result<T, JoinError>> {+        if self.tasks.is_empty() {+            None+        } else {+            let (t, idx, _tasks) = futures_util::future::select_all(self.tasks.iter_mut()).await;++            self.tasks.remove(idx);

Pushed up a new one that isn't as atrociously awful, but getting this to not be O(n) will require a different underlying data structure. I'll look into this more tonight.

Noah-Kennedy

comment created time in 18 days

PullRequestReviewEvent

push eventtokio-rs/tokio

Noah Kennedy

commit sha 604e96ae6807a42d08de5d38def308aae219aacf

cleanup poll_next_finished, next_finished, and join_all methods within task set

view details

push time in 18 days

more