[ejabberd] [ANN] ejabberd 16.12 beta1

Jérôme Sautret jerome.sautret at process-one.net
Wed Nov 30 18:23:52 MSK 2016


In 2016, we made several major code refactors and improvements on
ejabberd. From Elixir support, to test suite and code clean up and
modernization.

After 16.09 which brings a lot of improvements, the last version for
2016 includes a big refactor we have been preparing for long time, and
a couple of new features that pursue the central effort on API,
commands and web integration started early this year.

ejabberd 16.12 includes: a new API permissions framework for commands,
a new more reliable BOSH module, major code refactor, more integration
and unit tests, improved support for Erlang/OTP R19, compatibility
with rebar3 build system and many fixes and optimizations.

## New BOSH module

This new BOSH implementation improves BOSH users experience, as it's
more robust and efficient and overall over lower latency.

## New API permissions framework

This new framework is one of the big changes in this release. It makes
our API very flexible, allowing administrators to fine grain access to
some users to a group of API, depending on the path used to call a
given command.

While it should make it possible to implement any kind of access
control on top of ejabberd management API, it also makes it much
easier to configure ejabberd API for simple needs.

ejabberd API operations are organised around the concept of
*commands*. ejabberd modules provide many commands, but the mechanism
is generic and any module can provide its own set of commands.

All commands can be exposed through interfaces. Available interfaces
are: `ejabberdctl` command-line tool, ejabberd ReST API and ejabberd
XML-RPC API. The XML-RPC API still works but is deprecated in favor of
the ReST API. You should migrate to ReST if you are using it.

Finally, ReST API can be accessed through two authentication
mechanisms: HTTP Basic Authentication and OAuth 2.

Each command interface can have different restrictions based on how
exposed and sensitive the commands are.

Note that the following configuration snippets assume ejabberd API
listeners are properly configured, as defined in API listener
configuration: https://docs.ejabberd.im/developer/ejabberd-api/oauth/#toc_3

By default, when no `api_permission` option is provided, ejabberd
would use the following default permissions:

```
api_permissions:
"console commands":
  - from:
    - ejabberd_ctl
  - who: all
  - what: "*"
"admin access":
  - who:
    - admin
    - oauth:
      - scope: "ejabberd:admin"
      - admin
  - what:
    - "*"
    - "!stop"
    - "!start"
```

This will grant access to all commands when ejabberdctl is used (this
is what "console commands" group does), and additionally each command
that is authenticated by user that match acl auth from any source will
be able to call all commands except start and stop.

It's possible to extend this by adding new section like this:
```
api_permissions:
...
"allow statbot to get server stats"
  - who:
    - user: "statbot at server.example.com"
  - what:
    - connected_users_number
    - num_active_users
```

This will allow user statbot to execute commands
`connected_users_number` and `num_active_users`.

You can even relax the need to authenticate on ReST API, for example,
if you only want to restrict the API to localhost usage, for admin
tasks:

```
api_permissions:
 - who:
   - ip: "127.0.0.1/8"
 - what:
   - "*"
   - "!stop"
   - "!start"
```

As you can see, for simple cases, the API permission management is now
very simple. Yet, the system is powerful and can accommodate your most
advanced needs.

For more details, see full api permission documentation:
https://docs.ejabberd.im/developer/ejabberd-api/permissions/

You can now rely on ejabberd hundreds of commands to integrate in a
precise and secure way with your back ends.

## Major code refactor

Finally, the most important change (in terms of line of code and
impact on ejabberd code base) is refactor of XMPP packet handling in
the entire code base. This improvement makes code simpler, safer, and
smaller. ejabberd now uses a dedicated XMPP library, which helps
developers packing/unpacking XMPP packets.

Example is always better than words:

### Don't build entire XML element, but rely on common "templates"

Before:
```
#xmlel{name = <<"identity">>,
       attrs = [{<<"category">>, <<"pubsub">>},
                {<<"type">>, <<"pep">>}]}
IQ#iq{type = error, sub_el = [Error, SubEl]}
```

After:
```
#identity{category = <<"pubsub">>, type = <<"pep">>}
xmpp:make_iq_result(IQ, Error);
```

### Match packets efficiently

Before:
```
normal_state({route, From, <<"">>,
       #xmlel{name = <<"message">>,
              attrs = Attrs,
              children = Els} = Packet},
       StateData) ->
   case is_user_online(From, StateData) of
     true ->
       case fxml:get_attr_s(<<"type">>, Attrs) of
         <<"groupchat">> ->
              ...
```

After:
```
normal_state({route, From, <<"">>,
     #message{type = Type,
              lang = Lang} = Packet},
     StateData) ->
   case is_user_online(From, StateData) of
      true when Type == groupchat ->
          ...
```

### Don't use macros to build error response

Before:
```
  jlib:make_error_reply(Packet, ?ERRT_BAD_REQUEST(Lang, Txt));
```
After:
```
  xmpp:make_error(Packet, xmpp:err_bad_request(Txt, Lang));
```

The code is safer, as all XMPP processing in the core of ejabberd can
be typed-checked through static analyzer like Dialyzer.

Overall, this change paves the way to improvements of ejabberd API and
will make it much more pleasant for contributors to write new modules
and extensions. Improving the contribution documentation is next on
our list and you are welcome to join the effort.

This big refactor impacts the entire code base. Now that all tests are
successful, we provide an early beta before the final 16.12 release.
16.12-beta1 is not yet intended for production use. We are waiting for
your feedback to check that special cases did not sleep through our
careful refactor. Please, test it and send us feedback in your use
case, so that we can catch the tricky bugs before the final version is
released.

## Changes

This is just a summary of the most relevant ones:

### API / integration

- New API permissions framework

### Commands

- Add configurable weight for ejabberd commands
- add_rosteritem: Support several groups separated by ;
- create_rooms_file: Fix reading room jids from file
- delete_old_messages: Fix command for SQL backends
- send_message: Don't duplicate the message

### Core XMPP

- New BOSH module
- Use fxml_gen XML generator
- Use our new stand-alone XMPP library instead of jlib.erl
- Don't let MAM messages go into offline storage
- Add xdata generator for XMPP data form
- Get rid of excessive (io)list_to_binary/1 calls

### HTTP

- Add authentication support to mod_http_fileserver
- ejabberd_http: Handle missing POST data gracefully
- Use inets instead of lhttpc in http_p1
- Add http_p1.erl, rest.erl, and oauth2 ReST backend for OAuth2 tokens

### MUC

- Create room on configuration request as per XEP-0045, 10.1.3
- Ensure that presence_broadcast room option is stored
- Fix conference disco#items when running multiple virtual hosts
- Fix Result Set Management (RSM) for conference disco#items
- Introduce muc_invite hook
- Make the constant MAX_ROOMS_DISCOITEMS configurable
- mod_carboncopy: Don't copy MUC private messages

### MUC/Sub

- Store the flag "Allow Subscription" room option in database
- When getting list of subscribed rooms, also check temporary ones

### Relational databases support

- Append ; to privacy_list_data exporting lines
- Improve relational database import

### Build

- Make build system compatible with rebar3
- Produce ejabberd.service and fix for systemd usage

### Miscellany

- Bugfix: Don't let MAM messages go into offline storage
- Delete obsolete module mod_configure2
- Bugfix: Ignore offline sessions in ejabberd command statistics
- Rename #error{} record to #stanza_error{}

## Feedback

As usual, the release is tagged in the Git source code repository on Github:
https://github.com/processone/ejabberd.git

The source package and binary installers are available at
https://www.process-one.net/en/ejabberd/downloads/

If you suspect that you've found a bug, please search or fill a bug report on
https://github.com/processone/ejabberd/issues


More information about the ejabberd mailing list