Pushing Openfire over the Maven Finish Line

[akrherz]

Hi all,

A goal of the upcoming Openfire 4.3.0 release is to drop Ant build support and fully move to Maven OF-546. Some great work has been done so far by @guus, @CSH, @gdt and others. A recent github master commit gutted most of the Ant plumbing and we are just left with maven and are attempting to get things in order.

We could certainly use some help and opinions on how the Openfire repo should be organized to properly support maven and do things in the ‘maven’ way. I would particularly like to reach out here to @CSH and @gdt to provide some more feedback on the current repo status. Particularly with how versions are managed as that seems to be a current sticking point.

Any others with maven expertise are certainly encouraged to chime in and submit those pull requests Now is the time to get things right! Ant is gone, long live ant, so the need to continue to support ant is no longer an excuse for not improving maven!

daryl

PS, no comments on migrating to other build systems please

[CSH]

Hi,

I certainly like to have the default Maven structure, e.g.

src/main/java
src/main/resources
src/test/java
src/test/resources

This is especially true for the plugins, whose sources are under
src/java

src/i18n should go to src/main/resources
src/resources/jar should go to src/main/resources

I think guus documented a layout structure in the Jira tickets as well:
dbutil
plugins
webadmin
xmppserver

E.g.
move most of /src/java to xmppserver/src/main/java
move src/plugins/pluginname/src/java to plugins/pluginname/src/main/java
etc.

[gdt]

+1 on the comments from @CSH with regards to the layout. that said, on reflection, perhaps there should be a /xmppserver/src/main/java (etc), leaving no /src folder at all; As Openfire is a bundle of xmppserver+plugins+installer etc., I’m not sure there’s a need for a /src.

A couple of other opinions;

a) I don’t believe a plugin should always have the “latest” Openfire version in the parent. It implies a <minServerVersion> that is not necessarily true. It should be left at the version is was when last released.

b) In a similar vein, I think all the plugin versions should be a SNAPSHOT version. Taking the contentFilter plugin, for example.

If you build Openfire today, you’ll get contentFilter.jar version 1.8.0 that relies on Openfire 4.3.0-SNAPSHOT and Java 8.

Six months ago, you got contentFilter.jar version 1.8.0 that relied on Openfire 4.2.0 (or something) and Java 7 This means you get two similar looking but very different artefacts.

Instead, the build should generate contentFilter-1.8.1-SNAPSHOT until an update is ready. The update can set the parent version and minServerVersion appropriately, and set a release artefact version. A build is carried out and contentFilter-1.8.1 gets uploaded to Maven Central and then the version is changed to 1.8.2-SNAPSHOT ready for the next release.

If you want to bundle the installer with some default plugins, you can set the plugin versions as dependencies of the installer module, and it will fetch a known good version from Maven Central.

All IMHO, anyway.

Greg

[gdt]

(with apologies; I’m conscious that that last few months I’ve not been able to commit much time to Openfire, it’s easy to spout opinions, harder to actually implement them )

[arelless]

Has anyone implemented the overlay functionality from the ant build yet? I can provide some ideas/examples in that direction if that is still needed.

[akrherz]

The current state of development is what exists in our github master branch, so please feel free to share any ideas or even pull requests

[guus]

I’m pretty interested in those ideas for overlay in Maven. Could you share your suggestions please?

[arelless]

I’m really busy until Saturday, I have some examples based on the 4.2.x poms I pulled a few months ago for overlay of code in xmpp and the monitoring plugin. I didn’t have to overlay the admin console so I don’t have anything for that. I will try to put something together for you over the weekend.

[akrherz]

@arelless could I give a gentle nudge for your commentary here. I think we are keen on getting 4.3.0 out the door, so getting things squared away before a GA release would be great!

[arelless]

Sorry the response took so long. This is based off of poms I retrieved a few months ago for 4.2.x. I was hoping to merge this into the current 4.3.x code base but unfortunately a member of my family ended up in the hospital twice recently.

Add three additional directories at the top level:
custom-plugins
custom-resources
custom-xmppserver

In openfire/pom.xml:

    <modules>
        <module>dbutil</module>
        <module>i18n</module>
        <module>xmppserver</module>
	<module>custom-xmppserver</module>
        <module>starter</module>
        <module>src/plugins</module>
        <module>webadmin</module>
        <module>webadmintld</module>
	<module>custom-plugins</module>
	<module>custom-resources</module>
        <module>build</module>
    </modules>

The custom-xmppserver dir contains a java folder that mimics the xmpp code structure including only those classes being overlayed.

In custom-xmppserver/pom.xml:

    <parent>
        <groupId>org.igniterealtime.openfire</groupId>
        <artifactId>parent</artifactId>
        <version>4.2.3</version>
    </parent>	
	
    <groupId>org.igniterealtime.openfire</groupId>
    <artifactId>custom-xmppserver</artifactId>
    <version>4.2.3</version>
    <name>Custom Core XMPP Server</name>

    <dependencies>
        <dependency>
            <groupId>org.slf4j</groupId>
            <artifactId>jcl-over-slf4j</artifactId>
            <version>1.7.7</version>
        </dependency>
        <dependency>
            <groupId>org.igniterealtime.openfire</groupId>
            <artifactId>xmppserver</artifactId>
            <exclusions>
                <exclusion> <!-- Was pulling in 3 incompatible versions)-->
                    <groupId>org.slf4j</groupId>
                    <artifactId>slf4j-api</artifactId>
                </exclusion>
            </exclusions>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <configuration>
                    <source>1.7</source>
                    <target>1.7</target>
                </configuration>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-jar-plugin</artifactId>
                <configuration>
                    <archive>
                        <manifest>
                            <addClasspath>true</addClasspath>
                        </manifest>
                    </archive>
                </configuration>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-shade-plugin</artifactId>
                <version>3.1.0</version>
                <configuration>
                    <filters>
                      <filter>
                        <artifact>org.igniterealtime.openfire:xmppserver</artifact>
                        <includes>
                            <include>**</include>
                        </includes>
                        <excludes>
                            <exclude>META-INF/MANIFEST.MF</exclude>
                            <exclude>org/jivesoftware/openfire/ldap/LdapGroupProvider.class</exclude>
                            <exclude>org/jivesoftware/openfire/roster/Roster.class</exclude>
                            <exclude>org/jivesoftware/openfire/roster/RosterItemProvider.class</exclude>
                            <exclude>org/jivesoftware/openfire/roster/RosterManager*.class</exclude>
                            <exclude>org/jivesoftware/openfire/spi/XMPPServerInfoImpl.class</exclude>
                            <exclude>org/jivesoftware/openfire/user/User.class</exclude>
                            <exclude>org/jivesoftware/openfire/user/User$*.class</exclude>
                            <exclude>org/jivesoftware/openfire/user/UserManager*.class</exclude>
                            <exclude>org/jivesoftware/openfire/SessionManager*.class</exclude>
                        </excludes>
                        </filter>
                        <filter>
                            <artifact>org.igniterealtime.openfire:custom-xmppserver</artifact>
                            <includes>
                                <include>org/**</include>
                                <include>META-INF/MANIFEST.MF</include>
                            </includes>
                        </filter>
                    </filters>
                    <artifactSet>
                        <includes>
                            <include>org.igniterealtime.openfire:xmppserver</include>
                        </includes>
                    </artifactSet>
                </configuration>
                <executions>
                  <execution>
                    <phase>package</phase>
                    <goals>
                      <goal>shade</goal>
                    </goals>
                  </execution>
                </executions>
            </plugin>
        </plugins>

        <sourceDirectory>java</sourceDirectory>
    </build>

The excluded classes above are those that will be overlayed with modified versions. A TODO I never got to was to find a more generic way of handling the exclusions.

Next up build files.

[arelless]

Rather than relying on the single fileSet, uses dependencySets to pull together the pieces:

In build/pom.xml:

    <groupId>org.igniterealtime.openfire</groupId>
    <artifactId>build</artifactId>
    <version>4.2.3</version>
    <packaging>pom</packaging>

    <name>OpenFire Assembly Builder</name>

    <dependencies>
        <dependency>
            <groupId>org.igniterealtime.openfire</groupId>
            <artifactId>custom-xmppserver</artifactId>
            <version>4.2.3</version>
        </dependency>
        <dependency>
            <groupId>org.igniterealtime.openfire</groupId>
            <artifactId>custom-resources</artifactId>
            <version>4.2.3</version>
            <type>zip</type>
        </dependency>
         <dependency>
            <groupId>org.igniterealtime.openfire.plugins.custom</groupId>
            <artifactId>monitoring-dist</artifactId>
            <version>1.6.0</version>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-assembly-plugin</artifactId>
                <configuration>
                    <descriptors>
                        <descriptor>assembly.xml</descriptor>
                    </descriptors>
                    <appendAssemblyId>false</appendAssemblyId>
                </configuration>
                <executions>
                    <execution>
                        <id>trigger-assembly</id>
                        <phase>package</phase>
                        <goals>
                            <goal>single</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-deploy-plugin</artifactId>
                <version>2.8.2</version>
                <configuration>
                    <skip>true</skip>
                </configuration>
            </plugin>
        </plugins>
    </build>

In build/assembly.xml:

<assembly>
    <id>dist</id>
    <formats>
        <format>zip</format>
    </formats>
    <includeBaseDirectory>false</includeBaseDirectory>
    <fileSets>
        <fileSet>
            <directory>../target/openfire</directory>
            <outputDirectory>./</outputDirectory>
            <includes>
                <include>bin/**/*</include>
                <include>conf/**/*</include>
                <include>lib/**/*</include>
                <include>plugins/**/*</include>
                <include>resources/**/*</include>
                <include>*.html</include>
            </includes>
            <excludes>
                <!--add exclusion for file overridden via custom-xmppserver-->
                <exclude>lib/xmppserver*.jar</exclude>
                <!--add exclusions for files overridden via custom-resources-->
                <exclude>conf/openfire.xml</exclude>
                <exclude>resources/security/*</exclude>
                <!--add exclusions for files overridden via custom-plugins-->
                <exclude>plugins/monitoring.jar</exclude>
            </excludes>
        </fileSet>
    </fileSets>
    <dependencySets>
        <!-- Add custom resource files -->
        <dependencySet>
            <includes>
                <include>org.igniterealtime.openfire:custom-resources</include>
            </includes>
            <unpack>true</unpack>
            <unpackOptions>
                <includes>
                    <include>**/*</include>
                </includes>
            </unpackOptions>
        </dependencySet>
        <!-- Add custom-xmppserver to lib dir -->
        <dependencySet>
            <outputDirectory>lib/</outputDirectory>
            <includes>
                <include>org.igniterealtime.openfire:custom-xmppserver</include>
            </includes>
        </dependencySet>
        <!-- Add custom-plugins -->
        <dependencySet>
            <outputDirectory>plugins/</outputDirectory>
            <includes>
                <include>org.igniterealtime.openfire.plugins.custom:monitoring-dist</include>
            </includes>
            <outputFileNameMapping>monitoring.${artifact.extension}</outputFileNameMapping>
        </dependencySet>
    </dependencySets>
</assembly>

Still interested in thoughts on custom-resources and custom-plugins?

[arelless]

In custom-resources/pom.xml:

    <parent>
        <groupId>org.igniterealtime.openfire</groupId>
        <artifactId>parent</artifactId>
        <version>4.2.3</version>
    </parent>	
	
    <groupId>org.igniterealtime.openfire</groupId>
    <artifactId>custom-resources</artifactId>
    <version>4.2.3</version>
    <packaging>pom</packaging>

    <name>OpenFire Custom Resources</name>
    <description>Provides custom resources for Openfire</description>

    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-assembly-plugin</artifactId>
                <version>2.6</version>
                <executions>
                    <execution>
                        <id>zip-assembly</id>
                        <phase>package</phase>
                        <goals>
                            <goal>single</goal>
                        </goals>
                        <configuration>
                            <descriptors>
                                <descriptor>assembly.xml</descriptor>
                            </descriptors>
                            <appendAssemblyId>false</appendAssemblyId>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>

In custom-resources/assembly.xml:

<assembly xmlns="http://maven.apache.org/ASSEMBLY/2.0.0"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/ASSEMBLY/2.0.0 http://maven.apache.org/xsd/assembly-2.0.0.xsd">
  <id>custom-resources</id>
  <formats>
    <format>zip</format>
  </formats>
  <includeBaseDirectory>false</includeBaseDirectory>
  <fileSets>
    <fileSet>
      <directory>${project.basedir}</directory>
      <outputDirectory>./</outputDirectory>
      <includes>
        <include>conf/*</include>
        <include>resources/**/*</include>
        <include>tools/*</include>
      </includes>
    </fileSet>
  </fileSets>
</assembly>

[guus]

Over the past few months, we’ve been working with a Maven-based Openfire project. Things generally work, but we’re running into various annoyances, most of which relate to the versioning of the various moving parts in the Openfire project.

For example, when releasing a plugin, we’d prefer to switch from one -SNAPSHOT-based version to the next -SNAPSHOT-based version, but have one commit in between where we’re not at a SNAPSHOT, which we tag as a release, and publish. Even with the automation that’s available (through the maven release plugin et al), it’s difficult to get this right, as we’re not releasing an ‘entire project’, but just one module. Further difficulties arise when we want to include specific versions of plugins in a distribution. With both the ‘distribution’ as well as ‘plugin’ modules in the same project, thing get complicated, fast.

We’ve been debating various of these problems and solutions. One suggestion that was made to simplify the project structure from the ground up, by moving out all plugins to dedicated GIT repositories. Each plugin would then be, in effect, its own project. In our discussion, we’ve identified one bigger downside of this approach (apart from the one-time setup effort): a change in the API of Openfire that breaks a plugin will be harder to spot (suggested solutions for this include continuous integration, or clever usage of git submodules).

I’d like your feedback on the idea of moving out plugins to dedicated git repositories. Would this be an improvement worth pursuing?

[wroot]

I’m not a developer, but this could be managed with more thorough testing (moving to separate repositories) So breaking API changes won’t slip into a final release. Probably all plugins should be tested against a Beta version.

[wroot]

Also, maybe every plugin should have a separate project in JIRA? It is always weird to set Openfire fix version for a plugin when it is released separately and with its own version.

[akrherz]

I believe the intention is to let github manage the issues for these plugins, but could be wrong.

[wroot]

Not sure if having it split between two (or 30+ for every repo) trackers is better than having them filed for Openfire in JIRA like now.

[akrherz]

I don’t think anybody has expressed a strong opinion at this point, but I think our general goal is to eliminate Ignite maintained infrastructure like Jira, so the trend would be to move Jira issues to github, but nobody has even suggested that yet.

[CSH]

Our DevOps guy says (generally, not with regard to Openfire): Don’t mix version numbers in the same Git repository, because it messes up automatisation. (e.g. you have to be careful when incrementing a version number of a Maven submodule only).

Instead he suggests to only maintain one version per Git repository.

So generallly, what you say, too: Split up all plugins into their own repositories, if you want have own release cycles.

With regard to compatibility, I suggest:
All plugins which depend on any Openfire core 4.x.x are compatible with any newer Openfire 4.
If API evolves within 4.x.x, deprecate the old API, so that older plugins still work with newer Openfire 4.
When moving to Openfire 5, remove all deprecated API and update all plugins. That should happen rare enough, that it’s manageable.
Make it clear, the plugins, that depend on an old major version, may break.

In best case, plugins should only depend on a small API artifact, not the whole core implementation.

[wroot]

Most plugin repositories now contain the full Openfire history

I saw, that the Openfire plugins have been moved to their own repository. However, most of them have the full Openfire history in the commits, so about > 5600 commits. Some other seem to be ported correctly, they only have few commits which concern the actual plugin, like the openfire-inverse-plugin or the openfire-userStatus-plugin . This doesn’t seem right to me, because we now have many huge repositories which contain the full Openfire history. Is somebody aware of this issue and will fix…