[ejabberd] Number of Pubsub nodes.

Paul Kinlan paul.kinlan at gmail.com
Mon May 18 19:50:59 MSD 2009


Hi Andy,

That would deffinatly be of interest to me.  I don't quite understand the
internal workings of pubsub in ejabberd yet, so you will have to bear with
me.  One of the things that we are thinking of doing is storing all the
messages in a node (for a small ammount of time, may be two days after which
we would delete the node).  I presume a virtual node would only send
messages between people that are currently logged in?

Paul

2009/5/18 Andy Skelton <skeltoac at gmail.com>

> On Mon, May 18, 2009 at 5:21 AM, Paul Kinlan <paul.kinlan at gmail.com>
> wrote:
> > I need to potentially create 10,000+ pubsub nodes - dynamically, I need
> to
> > understand if there is an upper limit to the number of nodes I can create
> > either configureable or system programmed limit (such as the number of
> nodes
> > is limited to the size of a byte integer or not).  I would also be
> > interested if anyone has handled such loads before and I would be
> interested
> > in understanding what they noticed the limits of the system to be - for
> > example, demands on the systems, unforseen limitations with the number of
> > pubsub nodes on the system due to memory, processor power, disk space and
> > bandwidth etc.
>
> I can't give you any performance data because I haven't put many users
> on the it, but I can tell you about my setup. Maybe you can get some
> ideas on how to optimize.
>
> I have several million pubsub nodes. Only a handful are unique and
> require records in the pubsub_node table. The rest are functionally
> identical so I made them virtual. When pubsub goes to look up a node,
> it asks the nodetree. I created a nodetree plug-in that checks the
> node path and either reads the pubsub_node table or creates a record
> out of thin air. This makes node lookups fast for both types of nodes:
> virtual nodes because there is no disk access, and actual nodes
> because the pubsub_node table is kept small. It also means I don't
> have to create the millions of virtual nodes. They exist whenever I
> use them. The only table I have to scale up is pubsub_state, and I
> plan to use an external database for that.
>
> If you think this partial virtual scheme would be a good fit for your
> system, I'll be happy to show you how to do it.
>
> Cheers,
> Andy
> _______________________________________________
> ejabberd mailing list
> ejabberd at jabber.ru
> http://lists.jabber.ru/mailman/listinfo/ejabberd
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.jabber.ru/pipermail/ejabberd/attachments/20090518/8d39bd36/attachment.htm>


More information about the ejabberd mailing list