profile
viewpoint
If you are wondering where the data of this site comes from, please visit https://api.github.com/users/jstepien/events. GitMemory does not store any data, but only uses NGINX to cache data for a period of time. The idea behind GitMemory is simply to give users a better reading experience.
Jan Stępień jstepien @INNOQ Berlin, Germany https://www.janstepien.com/ Bringing balance to parentheses

ifesdjeen/zweikopf 71

JRuby-Clojure Interop helpers

jstepien/bubble 23

Blow your own.

innoq/spacy 12

An accessible application for moderating open space events

jstepien/boids 2

Some CUDA and Emscripten hacking.

jstepien/clodoc 1

Clojure online documenatiation browser.

jstepien/docker-openjdk8 1

Deprecated OpenJDK 8 image

innoq/innoq-blockchain-rust 0

#IamHeribertInnoq

jstepien/abandon-all-reason 0

Type-level compile-time computations in C++ template metaprogramming. You have been warned.

delete branch innoq/spacy

delete branch : 81-fix-deleting-sessions

delete time in a month

push eventinnoq/spacy

Joy Heron

commit sha 071c0df5798599e38212055e7f15b133920a55b0

Fixes #81 - Clear the action in a slot when the queue is empty When the queue is empty, the "spacy.ui/up-next" event is sent with `{ detail: "spacy.ui/nobody-in-queue" }`. However, in the `upNext` event of the `SlotUpdater` this was ignored and an `appendAction` event was triggered. This fix now clears the action when there is nobody in the queue.

view details

push time in a month

issue closedinnoq/spacy

When deleting the only session in the queue, the "Choose Slot" buttons in the Sessions do not disappear

How to reproduce:

  1. Add a session to an empty queue -> now the "Choose Slot" buttons will appear in the UI

  2. Delete the session from the queue -> the "Choose Slot" buttons do not disappear

Screenshot 2021-08-20 at 10 41 18

closed time in a month

joyheron

PR merged innoq/spacy

Reviewers
Fixes #81 - Clear the action in a slot when the queue is empty

When the queue is empty, the "spacy.ui/up-next" event is sent with { detail: "spacy.ui/nobody-in-queue" }. However, in the upNext event of the SlotUpdater this was ignored and an appendAction event was triggered. This fix now clears the action when there is nobody in the queue.

+5 -1

0 comment

1 changed file

joyheron

pr closed time in a month

Pull request review commentinnoq/spacy

Prevent losing sessions due to race conditions

 (defn- add-event-id [event-id doc]   (assoc doc ::belongs-to-event event-id)) -(defn- persist! [node {::domain/keys [facts event]}]+(defn- replace!+  "Stores the given event and facts iff the current event state in the database+  is equal to old-state. This allows us to make sure there were no concurrent+  writes."+  [node old-state {::domain/keys [facts event]}]   (assert (s/valid? ::domain/event event)           (s/explain-str ::domain/event event))   (assert (s/valid? ::domain/facts facts)           (s/explain-str ::domain/facts facts))   (let [event-id (:crux.db/id event)         new-facts (->> facts                        (map (partial add-event-id event-id))-                       (map maybe-add-crux-id))]-    (crux/await-tx-     node-     (crux/submit-tx node (for [doc (cons event new-facts)]-                            [:crux.tx/put doc])))))+                       (map maybe-add-crux-id))+        tx (crux/submit-tx node (cons+                                 [:crux.tx/match event-id old-state]+                                 (for [doc (cons event new-facts)]+                                   [:crux.tx/put doc])))]+    (crux/await-tx node tx)+    (if-not (crux/tx-committed? node tx)+      {::error ::concurrent-writes})))++(def ^:private retries 5)

Sounds great, thanks for experimenting with it!

jstepien

comment created time in a month

PullRequestReviewEvent

push eventinnoq/spacy

Jan Stępień

commit sha 2a7ac360d48af22d01eabd339bb2aa2101e5137a

Increase the number of retries to 10 This should make 400 responses caused by race conditions even less likely. h/t @joyheron Co-authored-by: Joy Heron <joy.heron@innoq.com>

view details

push time in a month

pull request commentinnoq/spacy

Prevent losing sessions due to race conditions

Thank you for investigating this! I am definitely going to steal that script in order to test the concurrent behavior of my application.

My pleasure, I hope the script helps!

My main question is: if we ensure that the f function that we pass in will always generate a valid state regardless of how many times I execute it, then is there any harm in calling the function an infinite number of times?

The only potential harm I see is that two or more threads will keep trying to commit their writes and block other incoming requests. In case a thread fails to write five times in a row there might be a problem and it's safer to back off. It might be a race condition, but maybe the entire database is inaccessible.

The cost of that decision is having to communicate to the user that we tried to process their request, we failed, but it might work again, so try again. I'm not sure what would be a good solution for the user-facing part of that behaviour. I don't want this patch to add complexity in the UI.

I experimented with the following implementation for update! (it's still very rough -- and I ended up wrapping a try around the replace! function because I was getting occasional org.sqlite.SQLiteException: [SQLITE_BUSY] exceptions from the sqlite database because it apparently can't handle so much concurrency -- hopefully Postgres wouldn't have this problem and we could remove that code):

Thanks for testing on SQLite. I only looked at Postgres and did not see any concurrency errors like the one above.

(...)

The thing I prefer about this code is that there are no 400 bad requests which are sent to the client. I think this is important, because if we do send a 400 to the client, the session will just not appear in the queue which is a horrible user experience.

I agree, that'd be awful.

Maybe we can cap the interval between retries and agree on a time limit. Maybe there's a better solution.

I'm also wondering if this problem and attempts at solving it are a sign that we aren't using Crux the way it's intended. Our document might be too big, we should split it into smaller pieces, and maintain consistency in a different way. Something we can think about in the future.

jstepien

comment created time in a month

PR opened innoq/spacy

Prevent losing sessions due to race conditions

Prior to this commit two concurrent writes often led to losing sessions. Imagine the following scenario:

  • request A reads an event in state S
  • request B reads an event in state S
  • request B adds a session to S and saves the event state Sb
  • request A adds a session to S and saves the event state Sa

The client that sent B got a confirmation that their request was successful. Alas, the event added by B is absent in Sa and lost.

This commit introduces a mechanism to make sure such race conditions do not happen.

Upon saving the event state we use :crux.tx/match to make sure that no concurrent writes occurred in the meantime. If there were no conflicts the request is successful. Otherwise we retry the entire request after a short wait.

If the request cannot be fulfilled after five retries, we give up and throw an error. The number of times was chosen arbitrarily; it seems to work.

After this patch the timeline would look as follows:

  • request A reads an event in state S
  • request B reads an event in state S
  • request B adds a session to S and tries to saves the state Sb (it's successful because the state in the database is S)
  • request A adds a session to S and tries to saves the state Sa (it's unsuccessful because the state in the database is Sb)
  • request A reads an event in state Sb
  • request A adds a session to Sb and tries to saves the state Sab (it's successful because the state in the database is Sb)

At this point the event has both sessions. No data loss occurs.

You can test this patch and see how many concurrent requests are rejected using the following Ruby script.

Threads = 8
RequestsPerThread = 5

puts "Firing #{Threads * RequestsPerThread} requests"

Threads.times do |t|
  fork do
    RequestsPerThread.times do |n|
      sleep(rand)
      id = t * RequestsPerThread + n
      puts id
      print `curl -sSf http://localhost:9215/februar-2021-event/submit-session -d 'title=#{id}&description=desc#{id}'`
    end
  end
end

waited = Process.waitall
puts "#{waited.count} children done"
+56 -16

0 comment

3 changed files

pr created time in 2 months

create barnchinnoq/spacy

branch : retries

created branch time in 2 months