[ejabberd] add_ and delete_rosteritem throughput

Steven Lehrburger lehrburger at gmail.com
Wed May 15 21:37:04 MSK 2013


I've been using ejabberd along with an XMPP component in an application
that requires regular changes to user rosters.

I often need to make as many as 150 of these changes at a time, and this
number will increase as the service grows. I've been doing this from my
component via XML-RPC, but get "[Errno 104] Connection reset by peer" on
some requests, while others take 20 seconds or more to return, even though
the timeout is set to 5 seconds and maxsessions is set to infinity.

I've just been retrying the requests that fail, but the slow requests cause
user experience problems. Furthermore, some of the requests are more urgent
than others, and if ejabberd is still busy churning through one batch of
150 when a second comes in, then things get really clogged up. I'm running
ejabberd on an m1.small Amazon EC2 instance.

Does anyone have any suggestions on how I might increase throughput?

A few other thoughts/ideas/questions:

1) I considered switching from Mnesia to MySQL so that I could modify the
rosters directly, and to generally simplify my application by using only
one datastore. It doesn't sound like this will work for me, however, based
on various forum posts and a conversation with Badlop last week:

2) The best workaround I've been able to come up with is to buffer the
XML-RPC requests in a priority queue (or, if I use Amazon SQS, in separate
low- and high-priority queues). The queue readers could make sure they
performed the high-priority requests first, but it would still be
preferable to somehow increase overall throughput. Also, how should I
select a number of queue readers/concurrent requests so as to saturate, but
not overwhelm,ejabberd?

(I had hoped by putting the queue readers on the same machine as ejabberd I
could further improve performance by using ejabberctl, but
http://lists.jabber.ru/pipermail/ejabberd/2012-August/007674.html says that
XML-RPC is faster, and I confirmed this with my own tests.)

3) At any given time most of the users who require roster changes are not
logged in. I could further prioritize my XML-RPC requests by first calling
user_sessions_info, but I doubt that would be an overall performance gain.
Is there a way for my component to directly readejabberd's "Last Activity"
information for a user? If this is stored in Mnesia, is there a Python
library I could use to query this information from my component?


Steven Lehrburger
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.jabber.ru/pipermail/ejabberd/attachments/20130515/c7596418/attachment.html>

More information about the ejabberd mailing list