Skip to content
Snippets Groups Projects

Optimistic / nonblocking message posting for chats

Merged Eugenij requested to merge eugenijm/pleroma-fe:optimistic-chat-posting into develop

Based on #948

Requires pleroma!3015 (merged) to be merged first

video

screenshot

Edited by Eugenij

Merge request reports

Pipeline #32457 passed

Pipeline passed for 78e5a639 on eugenijm:optimistic-chat-posting

Approval is optional

Merged by Shpuld ShpludsonShpuld Shpludson 4 years ago (Nov 2, 2020 5:36am UTC)

Merge details

  • Changes merged into with 5254fdba.
  • Deleted the source branch.

Pipeline #32464 passed

Pipeline passed for 5254fdba on develop

Activity

Filter activity
  • Approvals
  • Assignees & reviewers
  • Comments (from bots)
  • Comments (from users)
  • Commits & branches
  • Edits
  • Labels
  • Lock status
  • Mentions
  • Merge request status
  • Tracking
  • I like how this is shaping up

  • HJ
  • HJ
  • 74 74 })
    75 75 } else if (message.event === 'pleroma:chat_update') {
    76 dispatch('addChatMessages', {
    77 chatId: message.chatUpdate.id,
    78 messages: [message.chatUpdate.lastMessage]
    79 })
    80 dispatch('updateChat', { chat: message.chatUpdate })
    81 maybeShowChatNotification(store, message.chatUpdate)
    76 setTimeout(() => {
    77 dispatch('addChatMessages', {
    78 chatId: message.chatUpdate.id,
    79 messages: [message.chatUpdate.lastMessage]
    80 })
    81 dispatch('updateChat', { chat: message.chatUpdate })
    82 maybeShowChatNotification(store, message.chatUpdate)
    83 // Temporary band-aid to avoid duplicates when doing optimistic posting
    • Maintainer

      Temporary?

    • Author Contributor

      Yup, described the issue here #948 (comment 72332) (the WS event is delayed so that it arrives later then HTTP response).

      I'd propose we extend the WS chat update event with something like the idempotency key to connect the fake message with the one coming from the WS event (which arrives earlier than the HTTP response). Another approach would be to save the client_message_id, this also helps with handling errored messages on reconnect !1228 (comment 72596)

      Edited by Eugenij
    • Maintainer

      idempotency key sounds good, we already have that with regular posting IIRC

    • changed this line in version 5 of the diff

    • Author Contributor

      Updated the branch, now it depends on this BE MR pleroma!3015 (merged) that adds the idempotency key to the pleroma:chat_update event and so the hack with the artificial delay is no longer necessary.

      Edited by Eugenij
    • Does this branch still work without the idempotency key? as in, not break, I wouldn't expect the optimistic sending to work right, but it'd be cool to not worry about new frontend not working without the change

    • Author Contributor

      Without the BE change, it works overall but the experience is quite frustrating: when the streaming is enabled, all the chat messages the user sends are duplicated if the idempotency key is not provided by BE. To mitigate that, we can use the artificial delay when receiving the user's own messages. This should cover the majority of the cases — the exception would be the BE HTTP response taking longer than the artificial delay.

      Edited by Eugenij
    • I'm testing it now, yeah it works just fine without WS on current stable servers, but with WS it's not great. maybe when the chat message is received and inserted, we can check for messages with identical key and merge/replace them?

    • or well, that approach doesn't work that well, it's better than keeping duplicated messages but it does look a bit glitchy. artificial delay might work better, or in combination with filtering out duplicate messages

    • now that I actually understand the problem, I think we should keep this temporary bandaid even with idempodency merged in now, just to support people who have upgraded FE to develop but haven't upgraded BE. We should keep it until we've had chat idempotency in stable releases for a while. The comment should have clear indication about why it's there and when we can get rid of it

    • Author Contributor

      Alright done (keeping your suggestion to filter out the older duplicate message too)

      Edited by Eugenij
    • Please register or sign in to reply
  • Eugenij added 9 commits

    added 9 commits

    • d357de57...da638940 - 7 commits from branch pleroma:develop
    • 59b08582 - Optimistic message posting for chats
    • 844674ad - Do not retry failed message on 4xx errors, count retries down to zero, use...

    Compare with previous version

    • Author Contributor

      W/r/t error handling, there is an issue with not knowing whether or not the message has been persisted during the connection loss.

      E.g.,

      1. the user sends a message
      2. the connection loss occurs before the FE gets the response
      3. FE keeps the message marked as failed
      4. the connection is reestablished, the last messages are fetched, FE doesn't know whether or not the message that has been marked as failed is in that batch.

      The simplest way is to remove the failed message on reconnect, but then we lose what the user has typed.

      To avoid that, we can persist the client_message_id in the DB and add it to the chat message model (I've noticed that slack probably is doing the same based on the message format it uses). Still, not entirely sure if supporting this rare case is worth the additional complexity.

    • Maintainer

      if we have idempotency keys we could send them back from BE and update pending/failed messages my matching them with response.

    • Please register or sign in to reply
  • HJ
  • Eugenij added 2 commits

    added 2 commits

    • 3dcfaef8 - Retry failed chat message only on status code lower than 600
    • dba8a038 - Add the idempotency key to the chat message API call

    Compare with previous version

  • Eugenij added 2 commits

    added 2 commits

    • a22f37d5 - Retry failed chat message only on 5xx error codes
    • 2e591f4f - Add the idempotency key to the chat message API call

    Compare with previous version

  • mentioned in issue pleroma#2126 (closed)

  • Eugenij added 1 commit

    added 1 commit

    • 53f9d266 - Experimental client_message_id broadcast (requires the BE patch to work)

    Compare with previous version

  • Eugenij added 1 commit

    added 1 commit

    • 972bbe3f - Experimental client_message_id broadcast (requires the BE patch to work)

    Compare with previous version

  • Eugenij added 20 commits

    added 20 commits

    • 972bbe3f...0c0722a0 - 15 commits from branch pleroma:develop
    • 21f2868a - Optimistic message posting for chats
    • 73967496 - Do not retry failed message on 4xx errors, count retries down to zero, use...
    • 7b49ac26 - Retry failed chat message only on 5xx error codes
    • bbfc51df - Add the idempotency key to the chat message API call
    • b0229319 - Experimental idempotency_key broadcast support (requires the BE patch to work)

    Compare with previous version

  • Eugenij changed the description

    changed the description

  • Eugenij added 1 commit

    added 1 commit

    • c2c790f0 - Experimental idempotency_key broadcast support (requires the BE patch to work)

    Compare with previous version

  • Eugenij added 1 commit

    added 1 commit

    • a46a7d01 - Experimental idempotency_key broadcast support (requires the BE patch to work)

    Compare with previous version

  • Eugenij mentioned in merge request pleroma!3015 (merged)

    mentioned in merge request pleroma!3015 (merged)

  • Eugenij added 1 commit

    added 1 commit

    • d2983fa3 - Experimental idempotency_key broadcast support (requires the BE patch to work)

    Compare with previous version

  • Eugenij unmarked as a Work In Progress

    unmarked as a Work In Progress

  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Please register or sign in to reply
    Loading