As I mentioned on Twitter, elm-review now reports a shorter import cycle than the compiler.

@evancz explained in this thread why the compiler may give overly complex cycles, and suggested a way to get the smallest cycle available.

I just implemented that (released in the Elm package jfmengels/elm-review v2.3.10), and it seems to work well. It is a bit hard to test because of the lack of visibility of when a strongly connected components would occur.

This is how we go about detecting import cycle in elm-review:

• We try to get the get the topologicalSort of the module list, which fails if there is a cycle
• When we detect a cycle, we only have the problematic edge. We do a cycle search starting from that edge
• Before: We would report the found cycle
• After: For every module in that cycle, we do a new breadth-first search for a cycle and we take the shortest one out of all of those, which we then report

I added special check for self-importing modules because of how (I understand) the elm-community/graph API works, but that will likely be implementation-dependent. Also, maybe my implementation is simplifiable but at this point in time I am happy with the result.

I hope this is useful.

Please make sure that 'Store Past Broadcasts' is enabled on your channel so that we can review examples of your coding stream. Also ensure that your vods are not sub only.

What is the URL for your Twitch channel? https://twitch.tv/MY_CHANNEL

Are you a Twitch affiliate or partner? (Your application will be automatically rejected if the answer is no) No

What languages / frameworks / tools do you program in while streaming on Twitch? N/A

Why do you write code on stream? N/A

What is your favorite moment from coding on stream? Share a clip if one is available N/A

What do you want to accomplish with the Live Coders? To learn as much as I possibly can about coding

What do you want the Live Coders team to help you with? To learn how to code

If you have any questions about this process, please email: [email protected]

It looks like you're not Twitch affiliate or partner yet. Please reapply when you've met all the requirements. We'd be happy to evaluate your application then.

Please make sure that 'Store Past Broadcasts' is enabled on your channel so that we can review examples of your coding stream. Also ensure that your vods are not sub only.

What is the URL for your Twitch channel? https://twitch.tv/MY_CHANNEL

Are you a Twitch affiliate or partner? (Your application will be automatically rejected if the answer is no) No

What languages / frameworks / tools do you program in while streaming on Twitch? N/A

Why do you write code on stream? N/A

What is your favorite moment from coding on stream? Share a clip if one is available N/A

What do you want to accomplish with the Live Coders? To learn as much as I possibly can about coding

What do you want the Live Coders team to help you with? To learn how to code

If you have any questions about this process, please email: [email protected]

Here is a first attempt at an implementation of the sync protocol discussed in #290. Its purpose is to bring a local and a remote document into the same state. This is typically used when two peers have been disconnected for some time, and need to exchange any changes that happened while they were disconnected. The two peers that are syncing could be client and server, or server and client, or two peers with symmetric roles.

I am not totally happy with the API, but I wanted to share what I have anyway, to get some more eyes on it. You use it like this:

// When a connection is established, one of the two peers sends the first message.
// For example, if using WebSocket, you could say the client sends the first message.
onConnect(() => {
  sync = Automerge.startSync(doc)
  sendMessage(sync.messageToSend)
})

onReceiveMessage((msg) => {
  const response = sync.processMessage(msg)
  if (response) sendMessage(response)
  if (sync.isFinished) {
    doc = Automerge.finishSync(doc, sync)
    lastSyncState = Automerge.getCurrentVersion(doc)
  }
})

The protocol is stateful, and the sync object returned by Automerge.startSync() manages this state. The document is not updated directly during the sync process, because the changes don't necessarily arrive all at once, and it would be annoying to get several half-updated states over the course of a sync. That's why only when sync.isFinished is true, we update the document based on the changes we received by calling Automerge.finishSync(doc, sync). If a sync never reaches the point where sync.isFinished is true (perhaps the connection is interrupted and times out) then the document is not updated, and we start afresh with a new sync when reconnecting.

The intention is that we perform the sync only once when a connection is established; thereafter each peer can immediately send any new changes over the connection when they are made.

As implemented, the protocol requires both peers to load their documents from disk in order to sync them. However, there is an easy optimisation that saves loading any documents that have not changed. There is now a new function Automerge.getCurrentVersion(doc) that returns a short byte array that uniquely identifies the state of a document (it contains the hashes of the most recent changes); any two peers that are in the same state will return the same value for getCurrentVersion. You can store this current version separately from the document. When two peers, each with a large number of documents, want to find out which documents have changed, they only need to send each other this version identifier for each document; any docs whose version is the same need not be loaded from disk since they are unchanged. The actual sync protocol is then started only for those docs where the versions do not match.

The protocol is based on this paper: Martin Kleppmann and Heidi Howard. Byzantine Eventual Consistency and the Fundamental Limits of Peer-to-Peer Databases. arXiv:2012.00472, December 2020.

The protocol assumes that every time a peer successfully syncs with another peer, it remembers the document version (as returned by Automerge.getCurrentVersion()) after the last sync with that peer. The next time we try to sync with the same peer, we start from the assumption that the other peer's document version is no older than the outcome of the last sync, so we only need to exchange any changes that are more recent than the last sync. This assumption may not be true if the other peer did not correctly persist its state (perhaps it crashed before writing the result of the last sync to disk), and we fall back to sending the entire document in this case.

Let's say we want to sync A and B, and A is the first peer to send a message. Then the protocol state at A is initialised using the document version from the last sync with B (or undefined if this is the first time we're syncing with B). Then the peers exchange the following messages:

1. A sends B a message of type MESSAGE_TYPE_BLOOM_REQ containing the head hashes from the last sync between A and B, A's current head hashes, and a Bloom filter summarising all the changes that A knows about that were added since the last sync.
2. B responds to A with a message of type MESSAGE_TYPE_BLOOM_RESP containing the hashes of any changes that B wants from A, B's latest head hashes, a Bloom filter summarising all the changes that B knows about that were added since the last sync, and the changes that A needs from B (any changes that B has, that were added since the last sync and that are not contained in the Bloom filter; this set may be incomplete due to Bloom filter false positives).
3. If the last message contained at least one change or at least one hash that B wants, then A responds to B with a message of type MESSAGE_TYPE_NEEDS. This message contains the hashes of any changes that A wants from B, and any changes requested by B in its last message. This message is sent even if A does not want any more changes and B did not request any changes, in order to indicate that sync is complete.
4. If the last message indicated that A wanted further changes from B, then B replies with another MESSAGE_TYPE_NEEDS message containing those changes. Moreover, if there are any remaining hashes for which B does not have the corresponding changes, B also requests those in the same message. This process continues with further MESSAGE_TYPE_NEEDS messages (and A and B swapping roles each time) until both peers have all the changes (and their dependencies) contained in the other peer's head hashes in their initial message. The final MESSAGE_TYPE_NEEDS message is one containing no changes and no requested changes, which signals that the sync is complete (this message may be from A to B or from B to A).

Many documents can be synced over the same connection; you just need to tag each message with the ID of the document that it refers to, so that the recipient can deliver each message to the appropriate sync object.

Still need to add more tests to cover all the various edge cases.

The current implementation is likely to be quite slow. For example, when deciding whether to send a log of changes or a whole compressed document, it simply encodes both and then sends the one that is smaller. That will result in smaller messages over the network, but some wasted effort with encoding. Also, our current functions for traversing the hash graph are quite rudimentary. It would be good to figure out faster data structures for this stuff.

https://github.com/mpizenberg/elm-test-rs/pull/84, but actually mergeable this time. Wow, really no idea what went on there 🙈

Oh right the end-to-end tests are calling elm-test-rs on a few example projects, so reaching for network to download the elm packages. They should be able to work offline if you preload elm packages in ELM_HOME (I think). I don't know how possible that is with nix but we can look for this later.

I think it's not something to worry about. It would be nice to have purely-offline tests, but it looks like these are the integration-style tests that Rust does, right? So… seems fine?

(sorry for the earlier incorrect comment, I did not read yours correctly… this morning is off to a weird start!)

no, it's just that I'm not used to working between forks.

Nix disallows connecting to the internet in these tests, so we can't run them

Is this some issue with the repo or something I shouldn't worry about?

ahah sorry I was on my mails and a bit to fast to trigger XD

https://github.com/mpizenberg/elm-test-rs/pull/84, but actually mergeable this time. Wow, really no idea what went on there 🙈

hey, turns out I goofed and sent off the PR too early in #83. Sorry about that! We need just a few more tweaks to get a successful build.

argh! I don't know what's going on with these branches right now. I seem to have produced a PR that has a conflict. Let me try this again on a fresh branch. Sorry for the noise! 😬

hey, turns out I goofed and sent off the PR too early in #83. Sorry about that! We need just a few more tweaks to get a successful build.

Thanks :) updated!

Hey! Congrats on 1.0.0! This updates the Nix build file for that.

Hey! Congrats on 1.0.0! This updates the Nix build file for that.

Please make sure that 'Store Past Broadcasts' is enabled on your channel so that we can review examples of your coding stream. Also ensure that your vods are not sub only.

What is the URL for your Twitch channel? https://twitch.tv/olfamelles

Are you a Twitch affiliate or partner? (Your application will be automatically rejected if the answer is no) yes, affiliate

What languages / frameworks / tools do you program in while streaming on Twitch? JS: ReactJS/Angular, NodeJS Clojure ReasonML Python Unity

Why do you write code on stream? I want to create my own project, but in general, I like to share with people my experience and knowledge

What is your favorite moment from coding on stream? Share a clip if one is available Each stream, basically. There a lot of cool moments

What do you want to accomplish with the Live Coders? Want to learn from more experienced people

What do you want the Live Coders team to help you with? I haven't some strong minds about this... Just want to be a part of the team

If you have any questions about this process, please email: [email protected]

We couldn't retrieve your twitch information, make sure your URL is correct.

What is the URL for your Twitch channel? https://twitch.tv/dmitritv

Are you a Twitch affiliate or partner? (Your application will be automatically rejected if the answer is no) Yes, I have Twitch Affiliate

What languages / frameworks / tools do you program in while streaming on Twitch? C, C++, Nodejs, JavaScript, Vue, Nextjs, Discord Bot with nodejs, Apps with MongoDB and Node

Why do you write code on stream? Show people that you dont need to be expect or know everything about the language to start to programming, I do some homeworks in live, show how works. I study Computer Science

What is your favorite moment from coding on stream? Share a clip if one is available Developing a Tinder Clone and my friends laughing at me because I know how works tinder haha https://clips.twitch.tv/SwissDaintyPrariedogCharlieBitMe

What do you want to accomplish with the Live Coders? I want to contribute to and support the community, making networking and help the community grow up in brazil too.

What do you want the Live Coders team to help you with? Some tips showing what I have to change and know more about other programmer around the world.

ping @mpizenberg. Just waiting on elm-format so we can merge :)