[ejabberd] postgres + testing

[Matthew Nourse]

Mickael Remond wrote:
>
>Peter Saint-Andre wrote:
<snip>
>>2. Did the team ever make progress on testing the scalability of ejabberd? 
>>I'm still interested in perhaps running ejabberd as the jabber.org server, 
>>but I recall that there were some issues with
>>memory usage at higher loads (at jabber.org we typically have 11k+
>>connected users and 1300+ connected servers).
>
>Regarding the benchmark, I had to stop for some time. When I stop, I had 
>finished the script to create a user base with typical roster distribution 
>size. I now need some time to run the benchmark. It will probably require 
>several launch to get a nice load increase that does not kill the Jabber 
>server too fast but that still go slowly to the breaking point.
>Your mail is a strong inventive to  relaunched this work on my part.
>
>Regarding memory usage, there is still a problem remaining that show 
>abnormal use of memory. we are working on the problem currently and the 
>benchmark setup is clearly a good step in helping us identifying the 
>conditions of the problem.
>
>I am planning to work with the Erlang team to help us find out what the 
>problem is.
>

I did some scalability tests of ejabberd last night.  I was using the 
"smack" java client library from www.jivesoftware.org to log users in and 
keep them connected.  The users weren't sending any messages, just logging 
in and staying idle.

The most intriguing thing was that the number of users that I could log in 
seemed to increase as I increased the maximum number of Erlang processes.

The most users I could connect at one time was about 3500, with these 
settings:

export ERL_OPTIONS="-heart +P 268435455 +K true"
export ERL_MAX_PORTS=65000

Ejabberd would refuse to accept new connections after about 3500 users were 
connected.  I couldn't even open a socket to it by telnet'ing to 5222.

Before I set the +P option, I could log in about 500 users.  When I set it 
to 250,000 I could log in about 1800 users.

The "users" were logging in at a rate of about 1 or 2 per second.

I'm using mnesia for the roster but will soon switch to LDAP or (ideally) 
postgres.

I'm running Debian Sarge with a 2.6 kernel on a 3 Ghz P4 with 1 gig of RAM.  
I wasn't running out of memory...with 3500 connected users, ejabberd was 
using about 50 meg and total memory usage was about 500 meg.

If you'd like me to re-run my tests and/or modify the tests in some way, 
please let me know.

Kind Regards, Matt