Uploaded image for project: 'XWiki Platform'
  1. XWiki Platform
  2. XWIKI-14844

Live emails notifications should be grouped by XWiki pages

    XMLWordPrintable

Details

    • Improvement
    • Resolution: Unresolved
    • Major
    • None
    • 9.10-rc-1
    • Notifications
    • None
    • Unknown

    Description

      About the number of emails, we had this conversation on #xwiki:

      gdelhumeau:
      Quick question about live notifications emails
      When we enable the live notifications emails, there is 2 possibles behaviours
      1 - An email is sent for each document that has been modified. These emails can be grouped together in the client of the user (example: mail thread in gmail)
      2 - A unique email is sent with all notifications that have happened during the last 10 minutes (if there are such events), and maybe an other one will be sent 10 minutes later with new notifications
      Case 1: gmail can grouped emails that concern the same document in a unique mail thread
      Case 2: we make sure we don't have more than 1 email per 10 minutes (the grace period), but grouping is more complicated since it can concern several documents
      Currently we have 1), except that we miss the MessageID field to make gmail group things efficiently

      tmortagne:
      1 - ou mean for each DocumentUpdatedEvent ?
      s/ou/you/

      gdelhumeau
      But I thought we had 2
      Thomas: no, for each event. The grace period will make sure you don't receive more than 1 mail per 10 minutes concerning this document
      But you can receive several emails during these 10 minutes if they concern different docs

      tmortagne
      talking about 1, not 2

      gdelhumeau
      in 1, we already have a grace period
      I did not mention it for trying to make things easier

      tmortagne
      so you still have to wait for 10 minutes in both cases ?

      gdelhumeau
      Yes
      Actually let me rephrase
      1 - One mail per document, but not more than 1 mail per 10 minutes concerning this document
      2 - One mail per 10 minutes, possibly concerning more than 1 document
      Apparently the Watchlist did 1)

      tmortagne
      if we are waiting anyway I would be for grouping everything that can be grouped (but yes it's probably more work). I guess we could also let the user choose with profile preference

      gdelhumeau
      Clément Aubin: please confirm I'm not telling something wrong
      *saying

      tmortagne
      so +1 for 2 but I prefer both 1 and 2 (with 2 being the default)

      gdelhumeau
      haha, it seems both
      it *means

      tmortagne
      you have 1 already anyway so if you implement 2 it would be a pitty to ditch it

      gdelhumeau
      but you prefer 2 if I cannot do both, of course

      tmortagne
      yes
      but it's not like I was -1 for 1 either
      I'm fine with both, just prefer 2

      gdelhumeau
      vmassol, Enygma, Marius Dumitru Florea, evalica: please take 2 minutes to look at it

      tmortagne
      is the grace period configurable already ?

      gdelhumeau
      it is

      tmortagne
      ok
      can I put something lower than 10 minutes
      ?

      gdelhumeau
      minimum is 0

      tmortagne
      what does 0 will do in practice ? constently execute sql queries to see if something new came up ?

      enygma
      Guillaume Delhumeau: you had a look at the realtime watchlist notification, right?

      gdelhumeau
      Enygma: I looked at your commits about Message-Id and so on (https://github.com/xwiki/xwiki-platform/commit/e38a0db8b93a1b00ced5f35a8e3ab45b5aa2b253)
      I mean https://github.com/xwiki/xwiki-platform/commit/fd93f5cf2cd45444183bdd95f905d463aabae6f1

      enygma
      I personally prefer 1, since that was the point of real-time notifications, AFAIU. Also, coupling this with the email client grouping results in a really nice an useful overview of the received emails, allowing to focus on the changed pages. This way, in the email client, if you are watching 5 documents, you will never have more than 5 conversations to read. If you send batches, the best this we could do is to group all those batches under the same conversation ID, so that in the email client you only have 1 conversation, but it might get hard to follow at some point. Also, Gmail automatically breaks a conversation longer than X (around 100, AFAIR) messages.

      tmortagne
      I would prefer 1 too if you did not had to wait for 10 minutes

      enygma
      sure, if 1 also has the grace period / time buffer of around 5 minutes (maybe 10), that would be OK to. This is what we have now, AFAIU.
      yes, 10 mins sounds a bit too much

      tmortagne
      but since in both cases we have to wait then lets group everything that can be grouped

      enygma
      I would even argue that it should be around 30 sec to 1 minute
      and it should be a moving window, based on incoming events
      i.e. high traffic -> longer wait time to get notified of a bigger batch
      small traffic -> faster notification of a smaller batch

      tmortagne
      personnaly I'm find iwith 2 and I will lower a lot the grace period n my instances
      so be closer to real time

      enygma
      what I don't want to see is a notification mail about more than 1 document
      (my preference)
      because then the email client grouping can not be done
      but heck, we could end up with everything configurable (and ages to implement )

      tmortagne
      yeah maybe, did not tough much about that, in my mind it was about getting a mail and delete it
      I don't think about it like a conversation I need to group like forum notifications

      gdelhumeau
      ok so at least we need to set the ConversationId (that we don't have now)

      tmortagne
      but I can understand that after a while it's nicer
      anyway if we keep 1 then I'm cool with that but if we do 2 seems to be configuration does not really add much work (but if it's not doable so be it)

      vmassol
      configurable 1 and 2 is fine with me too
      if we have to choose one, I don't have much of an opinion ATM
      both are fine with me

      tmortagne
      in any case we have to choose a default
      but if everyone is find with 1 looks like a lot less work
      *fine

      vmassol
      realtime is more 1 probably

      tmortagne
      we can always do 2 later

      vmassol
      (also 1 cost more in term of perf)
      (more emails)

      tmortagne
      not sure about that
      more email but less work to group things
      so more work for the mail server, not us

      vmassol
      I was talking scalability of emails
      I'm not sure how well or rather how bad realtime notif is when you have lots of users
      imagine 100K users
      and some activity
      I'm not even sure there's enough time to send all the emails

      tmortagne
      you usually don't have 100k users listening to everything
      so you almost never send that many emails

      Enygma
      well, during the night they will eventually get the emails

      tmortagne
      and the difference between 1 and 2 is probably something like sendig 1 email instead of a few most of the time
      (per user listening to those changes)

      vmassol
      So one question could be: how many users using realtime do we need to make the sending not "realtime", i.e. taking longer than, say, 10 minutes
      (knowing that we have a wait perdio before sending each mail, by default it's 8 seconds I think)

      tmortagne
      what wait period are you talking about ?

      vmassol
      yes it's 8s (just checked)

      tmortagne
      the mail module wait for 8s between each mail ?

      vmassol
      yes
      "The delay to wait between each mail being sent, in milliseconds (defaults to 8 seconds)"

      tmortagne
      why ?

      vmassol
      to not be considered a spammer
      the value wasn't chosen randomly
      I read the doc on several mail server to find the value that would not consider XWiki a spammer
      it's configurable ofc

      tmortagne
      so this is only when there is several mails, right ?
      if you send a mail it's sent right away ?

      vmassol
      yes

      tmortagne
      so it's not "before sending each mail"

      vmassol
      (when thre are several to be sent in the queue)
      So one question could be: how many users using realtime do we need to make the sending not "realtime", i.e. taking longer than, say, 10 minutes

      (knowing that we have a wait perdio before sending each mail, by default it's 8 seconds I think)
      ^^^so there'll be several in the queue

      tmortagne
      the main criteria is not really teh number of users I think, it all depends what those user are watchling, no ?
      you are not going to send all changes to all users all the time, right ?
      S
      vmassol
      it should depend on users, activity in the wiki and indeed what users are watching

      tmortagne
      in any case the difference between 1 and 2 in number of email is probably not going to be very big most of the time unless the edit activity is very high and wide

      vmassol
      yes, my question was not so much for 1 and 2 but more in general

      tmortagne
      not sure what is you question then ? should we do real time notifications ?
      IMO as long as it does not take much resources mail will arrive when they arrive

      vmassol
      so 10*60/8=75, so we can serve 80 emails per 10 minutes.
      so if we have 80 users all folllowing one doc, the last reatime email will arrive after 10 minutes
      (at best)
      (then ofc if you're using this internally with your own mail server you can reduce the wait time to 0s)
      (and then it's about how many emails we can prepare and send per seconds)
      (I don't know this figure)
      anyway, back to clover work, I've found a way to compare the reports I think

      tmortagne
      Guillaume Delhumeau: so looks like 1 with proper conversation ids

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              gdelhumeau Guillaume Delhumeau
              Votes:
              1 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated: