[ejabberd] [ANN] ejabberd 16.12

Jérôme Sautret jerome.sautret at process-one.net
Fri Dec 23 13:50:21 MSK 2016


2016 was year of several major code refactors and improvements for
ejabberd. From Elixir support, to test suite and code clean up and
modernization.

After ejabberd 16.09 which brings a lot of improvements, ejabberd
16.12 includes a big refactor we have been preparing for long time,
and a couple of new features that pursue the central effort on API,
commands and web integration started early this year.

ejabberd 16.12 includes: a new API permissions framework for commands,
a new more reliable BOSH module, major code refactor, more integration
and unit tests, improved support for Erlang/OTP R19, compatibility
with rebar3 build system, many fixes and optimizations, and Docker
containers.

Thank you to all users who have send back their feedback on ejabberd
16.12 beta version. This is helping a lot make ejabberd better.

We hope you will enjoy this new release and we wish you a merry
Christmas! Stay tuned for even more new features in 2017!

## New BOSH module

This new BOSH implementation improves BOSH users experience, as it's
more robust and efficient and overall over lower latency.

## New API permissions framework

This new framework is one of the big changes in this release. It makes
our API very flexible, allowing administrators to fine grain access to
some users to a group of API, depending on the path used to call a
given command.

While it should make it possible to implement any kind of access
control on top of ejabberd management API, it also makes it much
easier to configure ejabberd API for simple needs.

ejabberd API operations are organized around the concept of
<em>commands</em>. ejabberd modules provide many commands, but the
mechanism is generic and any module can provide its own set of
commands.

All commands can be exposed through interfaces. Available interfaces
are: 'ejabberdctl' command-line tool, ejabberd ReST API and ejabberd
XML-RPC API. The XML-RPC API still works but is deprecated in favor of
the ReST API. You should migrate to ReST if you are using it.

Finally, ReST API can be accessed through two authentication
mechanisms: HTTP Basic Authentication and OAuth 2.

Each command interface can have different restrictions based on how
exposed and sensitive the commands are.

Note that the following configuration snippets assume ejabberd API
listeners are properly configured, as defined in API listener
configuration: https://docs.ejabberd.im/developer/ejabberd-api/oauth/#toc_3

By default, when no 'api_permission' option is provided, ejabberd
would use the following default permissions:

```
api_permissions:
 "console commands":
   - from:
     - ejabberd_ctl
   - who: all
   - what: "*"
 "admin access":
   - who:
     - admin
     - oauth:
       - scope: "ejabberd:admin"
       - admin
   - what:
     - "*"
     - "!stop"
     - "!start"
```

This will grant access to all commands when ejabberdctl is used (this
is what "console commands" group does), and additionally each command
that is authenticated by user that match acl auth from any source will
be able to call all commands except start and stop.

It's possible to extend this by adding new section like this:

```
api_permissions:
 ...
 "allow statbot to get server stats"
   - who:
     - user: "statbot at server.example.com"
   - what:
     - connected_users_number
     - num_active_users
```

This will allow user statbot to execute commands
'connected_users_number' and 'num_active_users'.

You can even relax the need to authenticate on ReST API, for example,
if you only want to restrict the API to localhost usage, for admin
tasks:

```
api_permissions:
  "local admin can use all commands"
    - who:
      - ip: "127.0.0.1/8"
    - what:
      - "*"
      - "!stop"
      - "!start"
```

As you can see, for simple cases, the API permission management is now
very simple. Yet, the system is powerful and can accommodate your most
advanced needs.

For more details, see full api permission documentation:
https://docs.ejabberd.im/developer/ejabberd-api/permissions/

You can now rely on ejabberd hundreds of commands to integrate in a
precise and secure way with your back ends.

## Major code refactor

Finally, the most important change (in terms of line of code and
impact on ejabberd code base) is refactor of XMPP packet handling in
the entire code base. This improvement makes code simpler, safer, and
smaller. ejabberd now uses a dedicated XMPP library, which helps
developers packing/unpacking XMPP packets.

Example is always better than words:

### Don't build entire XML element, but rely on common "templates"

Before:

```
#xmlel{name = <<"identity">>,
       attrs = [{<<"category">>, <<"pubsub">>},
                {<<"type">>, <<"pep">>}]}
IQ#iq{type = error, sub_el = [Error, SubEl]}
```

After:

```
#identity{category = <<"pubsub">>, type = <<"pep">>}
xmpp:make_iq_result(IQ, Error);
```

### Match packets efficiently

Before:

```
normal_state({route, From, <<"">>,
       #xmlel{name = <<"message">>,
              attrs = Attrs,
              children = Els} = Packet},
       StateData) ->
   case is_user_online(From, StateData) of
     true ->
       case fxml:get_attr_s(<<"type">>, Attrs) of
         <<"groupchat">> ->
              ...
```

After:

```
normal_state({route, From, <<"">>,
     #message{type = Type,
              lang = Lang} = Packet},
     StateData) ->
   case is_user_online(From, StateData) of
      true when Type == groupchat ->
          ...
```

### Don't use macros to build error response

Before:

```
jlib:make_error_reply(Packet, ?ERRT_BAD_REQUEST(Lang, Txt));
```

After:

```
xmpp:make_error(Packet, xmpp:err_bad_request(Txt, Lang));
```

The code is safer, as all XMPP processing in the core of ejabberd can
be typed-checked through static analyzer like Dialyzer.

Overall, this change paves the way to improvements of ejabberd API and
will make it much more pleasant for contributors to write new modules
and extensions. Improving the contribution documentation is next on
our list and you are welcome to join the effort.

This big refactor impacts the entire code base. Now that all tests are
successful, we provide an early beta before the final 16.12 release.
16.12-beta1 is not yet intended for production use. We are waiting for
your feedback to check that special cases did not sleep through our
careful refactor. Please, test it and send us feedback in your use
case, so that we can catch the tricky bugs before the final version is
released.

## Docker containers

Ejabberd docker container contributed by Rafael Römhild is now
included in ejabberd sources.
We also provide simple containers to build and run ejabberd from
sources. Thanks to this image, you can build ejabberd with
dependencies provided in our Docker image, without the need to install
any specific tools (beside Docker) directly on your own machine. This
works on Linux, FreeBSD, MacOSX, Windows.
See more in this blogpost:
https://blog.process-one.net/ejabberd-development-with-docker/

## External modules

Ejabberd's ext_mod has been updated to build external modules with the
new xmpp library. You should port your contribution to make it compile
and work with ejabberd 16.12. While this will improve your code, it
will also offer possibility to implement post_install and
pre_uninstall directly in your module. By exporting these functions,
ejabberd will call them after module has been successfully installed
and right before uninstalling it to allow any custom tasks at these
stages.

## Windows installer

The installer now moves ejabberd data files (configuration, database
and logs) in %ProgramData% instead of %APPDATA%. This move fixes
running ejabberd as a service. As a result, installer now registers
ejabberd service by default. If you're upgrading ejabberd from an
older version, the installer will automatically move your
configuration from %APPDATA% to %ProgramData% to keep your existing
data without any manual operation.

## Changes

This is just a summary of the most relevant ones:

### API / integration


 - New API permissions framework


### Commands


 - Add configurable weight for ejabberd commands
 - add_rosteritem: Support several groups separated by ;
 - create_rooms_file: Fix reading room jids from file
 - delete_old_messages: Fix command for SQL backends
 - send_message: Don't duplicate the message
 - Remove obsolete remove_node command (use leave_cluster)
 - Fix reload_config
 - Cleanup mod_admin_extra, add few functions
 - Expose unregister API command


### Core XMPP


 - New BOSH module
 - Use fxml_gen XML generator
 - Use our new stand-alone XMPP library instead of jlib.erl
 - Don't let MAM messages go into offline storage
 - Add xdata generator for XMPP data form
 - Get rid of excessive (io)list_to_binary/1 calls


### HTTP


 - Add authentication support to mod_http_fileserver
 - ejabberd_http: Handle missing POST data gracefully
 - Use inets instead of lhttpc in http_p1
 - Add http_p1.erl, rest.erl, and oauth2 ReST backend for OAuth2 tokens


### MUC


 - Create room on configuration request as per XEP-0045, 10.1.3
 - Ensure that presence_broadcast room option is stored
 - Fix conference disco#items when running multiple virtual hosts
 - Fix Result Set Management (RSM) for conference disco#items
 - Introduce muc_invite hook
 - Make the constant MAX_ROOMS_DISCOITEMS configurable
 - mod_carboncopy: Don't copy MUC private messages


### MUC/Sub


 - Store the flag "Allow Subscription" room option in database
 - When getting list of subscribed rooms, also check temporary ones
 - Add password support in muc_subscribe
 - When unsubscribes, check if room should get closed


### Pubsub


 - Enforce pubsub node removal


### Relational databases support


 - Append ; to privacy_list_data exporting lines
 - Improve relational database import


### Build


 - Make build system compatible with rebar3
 - Produce ejabberd.service and fix for systemd usage
 - Cleanup ext_mod and fix compilation path
 - Fix compilation of external module with new xmpp lib
 - Add optional post_install and pre_uninstall hooks for external module


### Miscellany


 - Add docker container from Rafael Rˆmhild
 - fast_tls is now compatible with LibreSSL
 - Delete obsolete module mod_configure2
 - Rename #error{} record to #stanza_error{}
 - Bugfix: Ignore offline sessions in ejabberd command statistics
 - Bugfix: Don't let MAM messages go into offline storage


## Feedback

As usual, the release is tagged in the Git source code repository on
https://github.com/processone/ejabberd.git

The source package and binary installers are available at
https://www.process-one.net/en/ejabberd/downloads/

If you suspect that you've found a bug, please search or fill a bug report on
https://github.com/processone/ejabberd/issues


More information about the ejabberd mailing list