[ejabberd] Clustering, data (state) sharing, and hooks firing on the cluster / nodes

Shaun Kruger shaun.kruger at gmail.com
Mon Nov 21 20:21:10 MSK 2011


Are you developing modules for ejabberd3 or ejabberd2?  In the
transition from 2 to 3 many things including users and domains are
transitioning from lists to binaries.  Also, the xml format is
changing from {xmlelement,Name,Attr,Child} to #xmlel{} records as
found in exmpp.

Shaun

On Mon, Nov 21, 2011 at 6:18 AM, CGS <cgsmcmlxxv at gmail.com> wrote:
> According to
>
> https://git.process-one.net/ejabberd/mainline/blobs/master/src/mod_roster.erl
>
> both User and Server should be in binary form. If memory serves, whatever
> you get from the hooks is as list, so, it wouldn't be a bad idea to add a
> condition like:
>
> -export([check_binary/1]).
>
> check_binary({User,Server}) when is_list(User) == true ->
>      {list_to_binary(User),list_to_binary(Server)};
>
> check_binary({User,Server}) when is_binary(User) == true -> {User,Server};
>
> check_binary({User,Server}) when (is_list(User) == false) and
> (is_binary(User) == false) ->
>     ?INFO_MSG("Unknown format for {User, Server} when
> {~p,~p}.",[User,Server]).
>
> For the rest of the code, lists:any(Predicate,List) will return the list if
> at least one element check returns true, but in a slower way (in my
> opinion). I would use lists:member/2 instead, but this is a matter of coding
> style.
>
> I hope this will help.
>
> CGS
>
>
>
>
> On 11/20/2011 09:32 PM, Hisham Mardam Bey wrote:
>>
>> What I've done is the following:
>>
>> 1- defined a module that hooks into filter_packet/1
>> 2- every packet is inspected, based on its type we do some
>> "permission" checks as such:
>>  2.1 if stanza type is presence (user A adds user B to roster and
>> sends presence) ->  check with our payment system, see if they can chat
>>  2.2 if stanza type is message ->  check that users have each other in
>> roster
>>
>> I go the roster check like using mod_roster:
>>
>> in_roster({jid,User,Server,_,_,_,_} = RosterOwner, Guest) ->
>>        GuestName = jlib:jid_to_string(jlib:jid_remove_resource(Guest)),
>>        RosterItems = mod_roster:get_user_roster([], {User, Server}),
>>        lists:any(fun(I) ->  GuestName == jlib:jid_to_string(I#roster.jid)
>> end, RosterItems).
>>
>> in_roster() is passed the From and To that are passed to filter_packet/1:
>>
>> filter_packet({From, To, Packet} = Input) ->
>>
>> Does this look good? Do I need to still think about transforming them to
>> binary?
>>
>> Thanks (=
>>
>> hmb.
>>
>> On Sun, Nov 20, 2011 at 8:05 AM, CGS<cgsmcmlxxv at gmail.com>  wrote:
>>>
>>> You have two options: pull or push.
>>>
>>> Pull: Every module is part of the same Erlang session, so, in
>>> mod_filter_presence you should be able to call directly
>>> mod_roster:get_user_roster([],{User,Server}) - take care to transform
>>> them
>>> in binary if they aren't (list_to_binary(List)) - from user_presence/4.
>>>
>>> Push: You can define a listener (listener(Vars) ->  receive Message ->
>>> proc_lib:spawn(?MODULE,process_message,[Message]), NewVars =
>>> modify(Vars),
>>> listener(NewVars) end.) in mod_roster (or another module of your choice
>>> linked to mod_roster), register listener with a name
>>> (register(my_listener,pid_for_listener)) and you send messages from
>>> mod_filter_presence:send_presence/4 to that registered process
>>> (my_listener
>>> ! {User,Server}).
>>>
>>> I hope these ideas will help you. I am still searching for my former
>>> project
>>> implementation (lots of search through the versions :D).
>>>
>>> CGS
>>>
>>>
>>>
>>> On 11/20/2011 06:03 AM, Hisham Mardam Bey wrote:
>>>>
>>>> At this point I am trying to either use mod_roster:get_user_roster/3
>>>> or call into mnesia myself. get_user_roster seems perfect but I do not
>>>> know how to get from filter_packet/1's input to the {User,Server}
>>>> tuple that is expected on both techniques I mentioned.
>>>>
>>>> hisham.
>>>>
>>>> On Sat, Nov 19, 2011 at 5:33 AM, CGS<cgsmcmlxxv at gmail.com>    wrote:
>>>>>
>>>>> One way is using ejabberdctl command. Another is to to put Ejabberd in
>>>>> the
>>>>> standard Erlang path and to access directly its modules. I used both
>>>>> and
>>>>> I
>>>>> had good results. I never entered Mnesia to manipulate the roster.
>>>>> Check
>>>>> also mod_roster (if I remember well the name) for details.
>>>>>
>>>>> If you need more help, let me know, I will check my previous work and I
>>>>> can
>>>>> give you some examples of code.
>>>>>
>>>>> CGS
>>>>>
>>>>>
>>>>>
>>>>> On 11/19/2011 06:44 AM, Hisham Mardam Bey wrote:
>>>>>>
>>>>>> Should I directly query and modify the rosters via mnesia queries or
>>>>>> is there an API?
>>>>>>
>>>>>> On Sat, Nov 19, 2011 at 12:26 AM, Hisham Mardam Bey
>>>>>> <hisham.mardambey at gmail.com>      wrote:
>>>>>>>
>>>>>>> Thanks for your example. Between that and using the roster I have
>>>>>>> everything we need except being able to filter basic roster
>>>>>>> (subscription) and presence stanzas. I can identify them in the
>>>>>>> filter
>>>>>>> module but haven't found ejabberd's API for manipulating users'
>>>>>>> rosters. Once I have that we'll be able to toss out what we currently
>>>>>>> have (Red5) and replace it with ejabberd! (=
>>>>>>>
>>>>>>> hmb.
>>>>>>>
>>>>>>> On Fri, Nov 18, 2011 at 6:07 AM, CGS<cgsmcmlxxv at gmail.com>
>>>>>>>  wrote:
>>>>>>>>
>>>>>>>> Hi, Hisham,
>>>>>>>>
>>>>>>>> I suppose the hooks you are interested in are:
>>>>>>>>
>>>>>>>> ejabberd_hooks:add(sm_register_connection_hook, Host,<hook module
>>>>>>>> name>,
>>>>>>>> <hook function name>, 90)
>>>>>>>> ejabberd_hooks:add(sm_remove_connection_hook, Host,<hook module
>>>>>>>> name>,
>>>>>>>> <hook function name>, 90)
>>>>>>>>
>>>>>>>> Here is an example I wrote some time ago (people didn't want my
>>>>>>>> module,
>>>>>>>> so,
>>>>>>>> I didn't push further):
>>>>>>>>
>>>>>>>> --- code snippet ---
>>>>>>>>
>>>>>>>> -module(mod_filter_presence).
>>>>>>>> -author('CGSMCMLXXV<cgsmcmlxxv at gmail.com>').
>>>>>>>>
>>>>>>>> -behavior(gen_mod).
>>>>>>>>
>>>>>>>> -include("ejabberd.hrl").
>>>>>>>> -include("jlib.hrl").
>>>>>>>>
>>>>>>>> -export([start/2,
>>>>>>>>         stop/1,
>>>>>>>>         user_presence/1,
>>>>>>>>         user_presence/2,
>>>>>>>>         user_presence/3,
>>>>>>>>         user_presence/4,
>>>>>>>>         send_message/1]).
>>>>>>>>
>>>>>>>> start(Host, _Opts) ->
>>>>>>>>    ?INFO_MSG("mod_filter_presence starting", []),
>>>>>>>>    ejabberd_hooks:add(sm_register_connection_hook, Host, ?MODULE,
>>>>>>>> user_presence, 90),
>>>>>>>>    ejabberd_hooks:add(sm_remove_connection_hook, Host, ?MODULE,
>>>>>>>> user_presence, 90),
>>>>>>>>    ok.
>>>>>>>>
>>>>>>>> stop(Host) ->
>>>>>>>>    ?INFO_MSG("mod_filter_presence stopping", []),
>>>>>>>>    ejabberd_hooks:delete(sm_register_connection_hook, Host, ?MODULE,
>>>>>>>> user_presence, 90),
>>>>>>>>    ejabberd_hooks:delete(sm_remove_connection_hook, Host, ?MODULE,
>>>>>>>> user_presence, 90),
>>>>>>>>    ok.
>>>>>>>>
>>>>>>>> user_presence(JID) ->
>>>>>>>>    User = JID#jid.luser,
>>>>>>>>    Host = JID#jid.lserver,
>>>>>>>>    Resource = JID#jid.lresource,
>>>>>>>>    _Data = {},
>>>>>>>>    user_presence(User,Host,Resource,_Data).
>>>>>>>>
>>>>>>>> user_presence(_, JID) ->
>>>>>>>>    User = JID#jid.luser,
>>>>>>>>    Host = JID#jid.lserver,
>>>>>>>>    Resource = JID#jid.lresource,
>>>>>>>>    _Data = {},
>>>>>>>>    user_presence(User,Host,Resource,_Data).
>>>>>>>>
>>>>>>>> user_presence(_, JID, _Data) ->
>>>>>>>>    User = JID#jid.luser,
>>>>>>>>    Host = JID#jid.lserver,
>>>>>>>>    Resource = JID#jid.lresource,
>>>>>>>>    user_presence(User,Host,Resource,_Data).
>>>>>>>>
>>>>>>>> user_presence(User, Server, Resource, _Packet) ->
>>>>>>>>   %% Do something with "User" from "Server" who tries to log in
>>>>>>>>   %% with XMPP-client "Resources" and if it's not enough, I may
>>>>>>>>   %% get lucky and have the full "Packet"
>>>>>>>>   %% Here should go your code defining FilterIt (filter it) as
>>>>>>>> boolean:
>>>>>>>>   %% ->      if FilterIt is true then don't allow connection
>>>>>>>>   %% ->      if FilterIt is false then allow connection
>>>>>>>>   if (FilterIt == true) ->
>>>>>>>>   %% each user has a registered process which can be caught with
>>>>>>>>          SID = ejabberd_sm:get_session_pid(User, Server, Resource),
>>>>>>>>          SID ! system_shutdown; %% a trick to disconnect the user
>>>>>>>>       (FilterIt /= true) ->      ok %% let the user log in
>>>>>>>>    end,
>>>>>>>>    none.
>>>>>>>>
>>>>>>>> --- end of code snippet ---
>>>>>>>>
>>>>>>>> I did that module to filter for a certain resource (XMPP-client
>>>>>>>> software
>>>>>>>> ID
>>>>>>>> like Pidgin, Psi, Vacuum-IM, Real-time Ignite Spark or whatever),
>>>>>>>> but
>>>>>>>> I
>>>>>>>> cut
>>>>>>>> that part for you to use it as you think it's fit for your project.
>>>>>>>> You
>>>>>>>> need
>>>>>>>> just to set the value for FilterIt before enters if-condition. I
>>>>>>>> hope
>>>>>>>> it
>>>>>>>> will help. Good luck!
>>>>>>>>
>>>>>>>> Cheers,
>>>>>>>> CGS
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On 11/18/2011 09:01 AM, Hisham Mardam Bey wrote:
>>>>>>>>>
>>>>>>>>> Hi Peter,
>>>>>>>>>
>>>>>>>>> Although that is a valid solution it will not work for us. We can
>>>>>>>>> not
>>>>>>>>> rely on our clients having this switched on as our chat service is
>>>>>>>>> paid and we need to be in control of who can send messages. This is
>>>>>>>>> why we want to block everything by default (yet allow roster
>>>>>>>>> requests
>>>>>>>>> to travel through) and only let 2 people chat once they have added
>>>>>>>>> one
>>>>>>>>> another to the roster (this will be subject to our business rules
>>>>>>>>> via
>>>>>>>>> a packet filter).
>>>>>>>>>
>>>>>>>>> Now we could try to inspect every message and do the roster check
>>>>>>>>> in
>>>>>>>>> the packet filter inside of ejabberd but we're trying to avoid
>>>>>>>>> having
>>>>>>>>> to do that if possible.
>>>>>>>>>
>>>>>>>>> hisham.
>>>>>>>>>
>>>>>>>>> On Fri, Nov 18, 2011 at 1:27 AM, Peter Viskup<skupko.sk at gmail.com>
>>>>>>>>>  wrote:
>>>>>>>>>>
>>>>>>>>>> Hi Hisham,
>>>>>>>>>> this can be done on XMPP client side. For example Gajim has option
>>>>>>>>>> 'Ignore
>>>>>>>>>> events from contact not in the roster' in advanced settings.
>>>>>>>>>>
>>>>>>>>>> Best regards,
>>>>>>>>>> --
>>>>>>>>>> Peter
>>>>>>>>>>
>>>>>>>>>> On 11/18/2011 05:42 AM, Hisham Mardam Bey wrote:
>>>>>>>>>>>
>>>>>>>>>>> A quick update on my findings after doing some more spec reading.
>>>>>>>>>>>
>>>>>>>>>>> Using rosters and presence I can simulate the exact environment
>>>>>>>>>>> and
>>>>>>>>>>> online / offline notifications that we need for our business
>>>>>>>>>>> logic.
>>>>>>>>>>> The only rule I still have to meet is blocking all communication
>>>>>>>>>>> by
>>>>>>>>>>> default (messages) unless 2 members have successfully subscribed
>>>>>>>>>>> to
>>>>>>>>>>> each others rosters. Any hints?
>>>>>>>>>>>
>>>>>>>>>>> hisham.
>>>>>>>>>>>
>>>>>>>>>>> On Thu, Nov 17, 2011 at 1:06 PM, Hisham Mardam Bey
>>>>>>>>>>> <hisham.mardambey at gmail.com>          wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> Hi folks,
>>>>>>>>>>>>
>>>>>>>>>>>> I'm fairly new to ejabberd and Erlang (3rd day using it as of
>>>>>>>>>>>> writing
>>>>>>>>>>>> this email) and I am having fantastic results with both so far.
>>>>>>>>>>>> We
>>>>>>>>>>>> want to use ejabberd where I work (dating site, we have up to
>>>>>>>>>>>> around
>>>>>>>>>>>> 8K people online wanting to chat). What we have so far is
>>>>>>>>>>>> ejabberd
>>>>>>>>>>>> 2.1.6-2.1 (via apt) coupled with Strophe JS for the web
>>>>>>>>>>>> browsers.
>>>>>>>>>>>> We've written a couple of Erlang modules that update our system
>>>>>>>>>>>> via
>>>>>>>>>>>> presence notifications when people join and leave. We've also
>>>>>>>>>>>> got
>>>>>>>>>>>> another module that coordinates invitations between users and
>>>>>>>>>>>> must
>>>>>>>>>>>> keep some state about who's said "yes" to who's invitation so
>>>>>>>>>>>> that
>>>>>>>>>>>> subsequent chat messages between the 2 users can go through
>>>>>>>>>>>> (other
>>>>>>>>>>>> wise the module's packet filter will drop it). The goal is to
>>>>>>>>>>>> then
>>>>>>>>>>>> use
>>>>>>>>>>>> this by adding a hook on fitler_packet and only let through
>>>>>>>>>>>> chats
>>>>>>>>>>>> that
>>>>>>>>>>>> have been approved and "registered" in the system.
>>>>>>>>>>>>
>>>>>>>>>>>> My question is really about the last part, how one would go
>>>>>>>>>>>> about
>>>>>>>>>>>> "sharing" such state across the cluster, and how do hooks work
>>>>>>>>>>>> in
>>>>>>>>>>>> a
>>>>>>>>>>>> cluster? I am still reading up on Erlang and ejabberd clustering
>>>>>>>>>>>> and
>>>>>>>>>>>> how it works.
>>>>>>>>>>>>
>>>>>>>>>>>> If I have multiple nodes in my cluster with users connected to
>>>>>>>>>>>> different nodes what is the recommended way of tracking such
>>>>>>>>>>>> invitation state? Also, if I use presence to figure out when a
>>>>>>>>>>>> user
>>>>>>>>>>>> has left through a hook, will the presence hook fire only on the
>>>>>>>>>>>> node
>>>>>>>>>>>> the user was connected to and left (presence changed)?
>>>>>>>>>>>>
>>>>>>>>>>>> My plan is to read up some more on how Erlang and ejabberd
>>>>>>>>>>>> handles
>>>>>>>>>>>> clustering then come back here and update this question. In the
>>>>>>>>>>>> mean
>>>>>>>>>>>> time, any helpful information or resources explaining how this
>>>>>>>>>>>> stuff
>>>>>>>>>>>> works are greatly appreciated.
>>>>>>>>>>>>
>>>>>>>>>>>> hisham.
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> Hisham Mardam-Bey
>>>>>>>>>>>> http://hisham.cc/
>>>>>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> ejabberd mailing list
>>>>>>>> ejabberd at jabber.ru
>>>>>>>> http://lists.jabber.ru/mailman/listinfo/ejabberd
>>>>>>>>
>>>>>>> --
>>>>>>> Hisham Mardam-Bey
>>>>>>> http://hisham.cc/
>>>>>>>
>>>>> _______________________________________________
>>>>> ejabberd mailing list
>>>>> ejabberd at jabber.ru
>>>>> http://lists.jabber.ru/mailman/listinfo/ejabberd
>>>>>
>>>>
>>> _______________________________________________
>>> ejabberd mailing list
>>> ejabberd at jabber.ru
>>> http://lists.jabber.ru/mailman/listinfo/ejabberd
>>>
>>
>>
>
> _______________________________________________
> ejabberd mailing list
> ejabberd at jabber.ru
> http://lists.jabber.ru/mailman/listinfo/ejabberd
>


More information about the ejabberd mailing list