[ejabberd] add_ and delete_rosteritem throughput

David Laban alsuren at gmail.com
Thu May 16 18:06:56 MSK 2013


Thinking laterally, let's go back a couple of steps.

My Dad's "Standard British Telecom Problem Solving Algorithm" has the 
following steps:

1) Where are we now?
2) Where did we come from?
3) Where Could we be?
4) Where Should we be?
5) How do we get there?

Can you give me a few more details about steps 1 and 2? Specifically:

a.1) What are these roster pushes for?
b.1) How are they grouped?
  b.3) Would something like http://www.ejabberd.im/shared-roster-all ?)
c.1) Is there any redundancy?
e.1) Do online clients *need* to be notified about them immediately?
  e.3) Could you inject the changes directly into the sql db and then inject a
       roster push in sideways (or "if not online: inject into sql")
f.1) Do you have control of the client software?
f.2) Do you need to support legacy versions of client software?
f.3) Can legacy versions be allowed to have known regressions (like e.1)?


On Thursday 16 May 2013 08:48:09 Steven Lehrburger wrote:
> To answer the last of my questions: I can just call
> ejabberdctl's connected_users_info instead of user_sessions_info to
> efficiently get the information I need to further-prioritize roster
> changes.
> That said, I'm still curious about how I might increase the overall
> add_rosteritem and delete_rosteritem throughput, and, if I can't increase
> throughput, whether my priority queue workaround sounds reasonable, and, if
> it is reasonable, how to figure out how many requests the queue readers can
> make simultaneously.
> Thanks!
> /~s
> On Wed, May 15, 2013 at 1:37 PM, Steven Lehrburger 
<lehrburger at gmail.com>wrote:
> > Hi,
> > 
> > I've been using ejabberd along with an XMPP component in an application
> > that requires regular changes to user rosters.
> > 
> > I often need to make as many as 150 of these changes at a time, and this
> > number will increase as the service grows. I've been doing this from my
> > component via XML-RPC, but get "[Errno 104] Connection reset by peer" on
> > some requests, while others take 20 seconds or more to return, even
> > though the timeout is set to 5 seconds and maxsessions is set to
> > infinity.
> > 
> > I've just been retrying the requests that fail, but the slow requests
> > cause user experience problems. Furthermore, some of the requests are
> > more urgent than others, and if ejabberd is still busy churning through
> > one batch of 150 when a second comes in, then things get really clogged
> > up. I'm running ejabberd on an m1.small Amazon EC2 instance.
> > 
> > Does anyone have any suggestions on how I might increase throughput?
> > 
> > A few other thoughts/ideas/questions:
> > 
> > 
> > 1) I considered switching from Mnesia to MySQL so that I could modify the
> > rosters directly, and to generally simplify my application by using only
> > one datastore. It doesn't sound like this will work for me, however,
> > based on various forum posts and a conversation with Badlop last week:
> > http://chatlogs.jabber.ru/ejabberd@conference.jabber.ru/2013/05/08.html.
> > 
> > 2) The best workaround I've been able to come up with is to buffer the
> > XML-RPC requests in a priority queue (or, if I use Amazon SQS, in
> > separate low- and high-priority queues). The queue readers could make
> > sure they performed the high-priority requests first, but it would still
> > be preferable to somehow increase overall throughput. Also, how should I
> > select a number of queue readers/concurrent requests so as to saturate,
> > but not overwhelm,ejabberd?
> > 
> > (I had hoped by putting the queue readers on the same machine as ejabberd
> > I could further improve performance by using ejabberctl, but
> > http://lists.jabber.ru/pipermail/ejabberd/2012-August/007674.html says
> > that XML-RPC is faster, and I confirmed this with my own tests.)
> > 
> > 3) At any given time most of the users who require roster changes are not
> > logged in. I could further prioritize my XML-RPC requests by first
> > calling user_sessions_info, but I doubt that would be an overall
> > performance gain. Is there a way for my component to directly
> > readejabberd's "Last Activity" information for a user? If this is stored
> > in Mnesia, is there a Python library I could use to query this
> > information from my component?
> > 
> > Thanks!
> > 
> > Best,
> > Steven Lehrburger

More information about the ejabberd mailing list