powered by Jive Software

HazelCast PluginServlet RoomRemovedEvent ClassCastException

OF v4.6.3
RestAPI Plugin v 1.4.0
HazelCast Plugin v2.5.0

These errors are common on our system

2021.04.26 14:04:09 org.jivesoftware.openfire.container.PluginServlet - Failed to serialize 'org.jivesoftware.openfire.plugin.util.cache.ClusteredCacheFactory$CallableTask'
com.hazelcast.nio.serialization.HazelcastSerializationException: Failed to serialize 'org.jivesoftware.openfire.plugin.util.cache.ClusteredCacheFactory$CallableTask'
	at com.hazelcast.internal.serialization.impl.SerializationUtil.handleSerializeException(SerializationUtil.java:82) ~[?:?]
	at com.hazelcast.internal.serialization.impl.AbstractSerializationService.toBytes(AbstractSerializationService.java:157) ~[?:?]
	at com.hazelcast.internal.serialization.impl.AbstractSerializationService.toBytes(AbstractSerializationService.java:133) ~[?:?]
	at com.hazelcast.internal.serialization.impl.AbstractSerializationService.toData(AbstractSerializationService.java:118) ~[?:?]
	at com.hazelcast.internal.serialization.impl.AbstractSerializationService.toData(AbstractSerializationService.java:106) ~[?:?]
	at com.hazelcast.spi.impl.NodeEngineImpl.toData(NodeEngineImpl.java:319) ~[?:?]
	at com.hazelcast.executor.impl.ExecutorServiceProxy.submitToMembers(ExecutorServiceProxy.java:336) ~[?:?]
	at org.jivesoftware.openfire.plugin.util.cache.ClusteredCacheFactory.doClusterTask(ClusteredCacheFactory.java:372) ~[?:?]
	at org.jivesoftware.util.cache.CacheFactory.doClusterTask(CacheFactory.java:695) ~[xmppserver-4.6.3.jar:4.6.3]
	at org.jivesoftware.openfire.plugin.rest.controller.MUCRoomController.createRoom(MUCRoomController.java:290) ~[?:?]
	at org.jivesoftware.openfire.plugin.rest.controller.MUCRoomController.createChatRoom(MUCRoomController.java:160) ~[?:?]
	at org.jivesoftware.openfire.plugin.rest.service.MUCRoomService.createMUCRoom(MUCRoomService.java:53) ~[?:?]
	at sun.reflect.GeneratedMethodAccessor58.invoke(Unknown Source) ~[?:?]
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_282]
	at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_282]
	at com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60) ~[?:?]
	at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:205) ~[?:?]
	at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75) ~[?:?]
	at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302) ~[?:?]
	at com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108) ~[?:?]
	at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) ~[?:?]
	at com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84) ~[?:?]
	at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1542) ~[?:?]
	at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1473) ~[?:?]
	at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419) ~[?:?]
	at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1409) ~[?:?]
	at com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:409) ~[?:?]
	at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:558) ~[?:?]
	at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:733) ~[?:?]
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) ~[javax.servlet-api-3.1.0.jar:3.1.0]
	at org.jivesoftware.openfire.container.PluginServlet.handleServlet(PluginServlet.java:461) ~[xmppserver-4.6.3.jar:4.6.3]
	at org.jivesoftware.openfire.container.PluginServlet.service(PluginServlet.java:134) ~[xmppserver-4.6.3.jar:4.6.3]
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) ~[javax.servlet-api-3.1.0.jar:3.1.0]
	at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1452) ~[jetty-servlet-9.4.35.v20201120.jar:9.4.35.v20201120]
	at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:791) ~[jetty-servlet-9.4.35.v20201120.jar:9.4.35.v20201120]
	at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1626) ~[jetty-servlet-9.4.35.v20201120.jar:9.4.35.v20201120]
	at org.jivesoftware.admin.PluginFilter.doFilter(PluginFilter.java:226) ~[xmppserver-4.6.3.jar:4.6.3]
	at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) ~[jetty-servlet-9.4.35.v20201120.jar:9.4.35.v20201120]
	at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) ~[jetty-servlet-9.4.35.v20201120.jar:9.4.35.v20201120]
	at org.jivesoftware.admin.AuthCheckFilter.doFilter(AuthCheckFilter.java:234) ~[xmppserver-4.6.3.jar:4.6.3]
	at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) ~[jetty-servlet-9.4.35.v20201120.jar:9.4.35.v20201120]
	at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) ~[jetty-servlet-9.4.35.v20201120.jar:9.4.35.v20201120]
	at com.opensymphony.sitemesh.webapp.SiteMeshFilter.obtainContent(SiteMeshFilter.java:129) ~[sitemesh-2.4.2.jar:?]
	at com.opensymphony.sitemesh.webapp.SiteMeshFilter.doFilter(SiteMeshFilter.java:77) ~[sitemesh-2.4.2.jar:?]
	at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) ~[jetty-servlet-9.4.35.v20201120.jar:9.4.35.v20201120]
	at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) ~[jetty-servlet-9.4.35.v20201120.jar:9.4.35.v20201120]
	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) ~[jetty-servlet-9.4.35.v20201120.jar:9.4.35.v20201120]
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) ~[jetty-server-9.4.35.v20201120.jar:9.4.35.v20201120]
	at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602) ~[jetty-security-9.4.35.v20201120.jar:9.4.35.v20201120]
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) ~[jetty-server-9.4.35.v20201120.jar:9.4.35.v20201120]
	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) ~[jetty-server-9.4.35.v20201120.jar:9.4.35.v20201120]
	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) ~[jetty-server-9.4.35.v20201120.jar:9.4.35.v20201120]
	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) ~[jetty-server-9.4.35.v20201120.jar:9.4.35.v20201120]
	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1435) ~[jetty-server-9.4.35.v20201120.jar:9.4.35.v20201120]
	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) ~[jetty-server-9.4.35.v20201120.jar:9.4.35.v20201120]
	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) ~[jetty-servlet-9.4.35.v20201120.jar:9.4.35.v20201120]
	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) ~[jetty-server-9.4.35.v20201120.jar:9.4.35.v20201120]
	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) ~[jetty-server-9.4.35.v20201120.jar:9.4.35.v20201120]
	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1350) ~[jetty-server-9.4.35.v20201120.jar:9.4.35.v20201120]
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) ~[jetty-server-9.4.35.v20201120.jar:9.4.35.v20201120]
	at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) ~[jetty-server-9.4.35.v20201120.jar:9.4.35.v20201120]
	at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) ~[jetty-server-9.4.35.v20201120.jar:9.4.35.v20201120]
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) ~[jetty-server-9.4.35.v20201120.jar:9.4.35.v20201120]
	at org.eclipse.jetty.server.Server.handle(Server.java:516) ~[jetty-server-9.4.35.v20201120.jar:9.4.35.v20201120]
	at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:388) ~[jetty-server-9.4.35.v20201120.jar:9.4.35.v20201120]
	at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:633) [jetty-server-9.4.35.v20201120.jar:9.4.35.v20201120]
	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:380) [jetty-server-9.4.35.v20201120.jar:9.4.35.v20201120]
	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) [jetty-server-9.4.35.v20201120.jar:9.4.35.v20201120]
	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) [jetty-io-9.4.35.v20201120.jar:9.4.35.v20201120]
	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) [jetty-io-9.4.35.v20201120.jar:9.4.35.v20201120]
	at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:540) [jetty-io-9.4.35.v20201120.jar:9.4.35.v20201120]
	at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:395) [jetty-io-9.4.35.v20201120.jar:9.4.35.v20201120]
	at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161) [jetty-io-9.4.35.v20201120.jar:9.4.35.v20201120]
	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) [jetty-io-9.4.35.v20201120.jar:9.4.35.v20201120]
	at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) [jetty-io-9.4.35.v20201120.jar:9.4.35.v20201120]
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) [jetty-util-9.4.35.v20201120.jar:9.4.35.v20201120]
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) [jetty-util-9.4.35.v20201120.jar:9.4.35.v20201120]
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) [jetty-util-9.4.35.v20201120.jar:9.4.35.v20201120]
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) [jetty-util-9.4.35.v20201120.jar:9.4.35.v20201120]
	at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) [jetty-util-9.4.35.v20201120.jar:9.4.35.v20201120]
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:773) [jetty-util-9.4.35.v20201120.jar:9.4.35.v20201120]
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:905) [jetty-util-9.4.35.v20201120.jar:9.4.35.v20201120]
	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_282]
Caused by: java.lang.ClassCastException: java.lang.String cannot be cast to org.jivesoftware.openfire.muc.MUCRole$Role
	at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193) ~[?:1.8.0_282]
	at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1384) ~[?:1.8.0_282]
	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482) ~[?:1.8.0_282]
	at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472) ~[?:1.8.0_282]
	at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708) ~[?:1.8.0_282]
	at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:1.8.0_282]
	at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:566) ~[?:1.8.0_282]
	at org.jivesoftware.openfire.muc.spi.LocalMUCRoom.writeExternal(LocalMUCRoom.java:3286) ~[xmppserver-4.6.3.jar:4.6.3]
	at org.jivesoftware.openfire.muc.cluster.RoomUpdatedEvent.writeExternal(RoomUpdatedEvent.java:62) ~[xmppserver-4.6.3.jar:4.6.3]
	at java.io.ObjectOutputStream.writeExternalData(ObjectOutputStream.java:1459) ~[?:1.8.0_282]
	at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1430) ~[?:1.8.0_282]
	at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178) ~[?:1.8.0_282]
	at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1548) ~[?:1.8.0_282]
	at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1509) ~[?:1.8.0_282]
	at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432) ~[?:1.8.0_282]
	at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178) ~[?:1.8.0_282]
	at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:348) ~[?:1.8.0_282]
	at com.hazelcast.internal.serialization.impl.JavaDefaultSerializers$JavaSerializer.write(JavaDefaultSerializers.java:116) ~[?:?]
	at com.hazelcast.internal.serialization.impl.JavaDefaultSerializers$JavaSerializer.write(JavaDefaultSerializers.java:109) ~[?:?]
	at com.hazelcast.internal.serialization.impl.StreamSerializerAdapter.write(StreamSerializerAdapter.java:43) ~[?:?]
	at com.hazelcast.internal.serialization.impl.AbstractSerializationService.toBytes(AbstractSerializationService.java:154) ~[?:?]
	... 81 more

It looks like this exception is thrown when we use the RestAPI to delete MUCs that existed in our OF v4.5.2 installation prior to upgrade to v4.6.3. With a MUC that was created under v4.6.3, we don’t see this exception. This implies we need to convert old MUCs? If yes, then how do we convert them?

A known source of ClassCastExceptions is an issue that occurs when a plugin that makes use of a clustered cache gets reloaded/restarted/updated. This should only affect instances that also make use of the Hazelcast plugin (which, by the looks of the stacktrace that was provided, you are). If that’s the case, you might be able to work around the issue by restarting Openfire.

@guus thank you for the reply. Yes we’re using Hazelcast plugin v2.5.0

Caused by: java.lang.ClassCastException: java.lang.String cannot be cast to org.jivesoftware.openfire.muc.MUCRole$Role

Seems to be key.

We restarted after installing the update and subsequently afterwards. I expect to be able to reproduce this issue by installing 4.5.2, creating a MUC, then upgrading to v4.6.3. I’ll verify this expected result and include restarting after upgrade to 4.6.3.

Ah, that is a different problem. What’s the stacktrace for that ‘string cannot be cast as Role’ exception?

This might be caused by running different versions of Openfire in the same cluster, but I’m not sure.

Actually, it looks like we’re seeing this error even with MUCs that were created in v4.6.3 also, but not always – it seems to depend on something unknown. It seems more common with MUCs created in v4.5.2.

Server restarts have no effect on this. From what I’ve seen, I’ll be surprised if this only affects us.

For background: that’s the full stacktrace from the error.log. JVM is truncating the stacktrace, according to https://docs.oracle.com/javase/6/docs/api/java/lang/Throwable.html#printStackTrace(). The first occurrence of the exception should be non-truncated.

I’m not sure if this is the first occurrence with the full stack trace:

2021.04.26 19:09:42 org.jivesoftware.openfire.container.PluginServlet - org.jivesoftware.openfire.muc.MUCRole$Role cannot be cast to java.lang.String (through reference chain: org.jivesoftware.openfire.plugin.rest.entity.MUCRoomEntity["broadcastPresenceRoles"]->java.util.UnmodifiableRandomAccessList[0])
org.codehaus.jackson.map.JsonMappingException: org.jivesoftware.openfire.muc.MUCRole$Role cannot be cast to java.lang.String (through reference chain: org.jivesoftware.openfire.plugin.rest.entity.MUCRoomEntity["broadcastPresenceRoles"]->java.util.UnmodifiableRandomAccessList[0])
        at org.codehaus.jackson.map.JsonMappingException.wrapWithPath(JsonMappingException.java:218) ~[?:?]
        at org.codehaus.jackson.map.JsonMappingException.wrapWithPath(JsonMappingException.java:197) ~[?:?]
        at org.codehaus.jackson.map.ser.std.SerializerBase.wrapAndThrow(SerializerBase.java:166) ~[?:?]
        at org.codehaus.jackson.map.ser.std.IndexedStringListSerializer.serializeContents(IndexedStringListSerializer.java:99) ~[?:?]
        at org.codehaus.jackson.map.ser.std.IndexedStringListSerializer.serialize(IndexedStringListSerializer.java:63) ~[?:?]
        at org.codehaus.jackson.map.ser.std.IndexedStringListSerializer.serialize(IndexedStringListSerializer.java:24) ~[?:?]
        at org.codehaus.jackson.map.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:446) ~[?:?]
        at org.codehaus.jackson.map.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:150) ~[?:?]
        at org.codehaus.jackson.map.ser.BeanSerializer.serialize(BeanSerializer.java:112) ~[?:?]
        at org.codehaus.jackson.map.ser.StdSerializerProvider._serializeValue(StdSerializerProvider.java:610) ~[?:?]
        at org.codehaus.jackson.map.ser.StdSerializerProvider.serializeValue(StdSerializerProvider.java:256) ~[?:?]
        at org.codehaus.jackson.map.ObjectMapper.writeValue(ObjectMapper.java:1613) ~[?:?]
        at org.codehaus.jackson.jaxrs.JacksonJsonProvider.writeTo(JacksonJsonProvider.java:559) ~[?:?]
        at com.sun.jersey.json.impl.provider.entity.JacksonProviderProxy.writeTo(JacksonProviderProxy.java:160) ~[?:?]
        at com.sun.jersey.spi.container.ContainerResponse.write(ContainerResponse.java:302) ~[?:?]
        at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1510) ~[?:?]
        at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419) ~[?:?]
        at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1409) ~[?:?]
        at com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:409) ~[?:?]
        at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:558) ~[?:?]
        at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:733) ~[?:?]
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) ~[javax.servlet-api-3.1.0.jar:3.1.0]
        at org.jivesoftware.openfire.container.PluginServlet.handleServlet(PluginServlet.java:461) ~[xmppserver-4.6.3.jar:4.6.3]
        at org.jivesoftware.openfire.container.PluginServlet.service(PluginServlet.java:134) ~[xmppserver-4.6.3.jar:4.6.3]
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) ~[javax.servlet-api-3.1.0.jar:3.1.0]
        at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1452) ~[jetty-servlet-9.4.35.v20201120.jar:9.4.35.v20201120]
        at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:791) ~[jetty-servlet-9.4.35.v20201120.jar:9.4.35.v20201120]
        at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1626) ~[jetty-servlet-9.4.35.v20201120.jar:9.4.35.v20201120]
        at org.jivesoftware.admin.PluginFilter.doFilter(PluginFilter.java:226) ~[xmppserver-4.6.3.jar:4.6.3]
        at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) ~[jetty-servlet-9.4.35.v20201120.jar:9.4.35.v20201120]
        at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) ~[jetty-servlet-9.4.35.v20201120.jar:9.4.35.v20201120]
        at org.jivesoftware.admin.AuthCheckFilter.doFilter(AuthCheckFilter.java:234) ~[xmppserver-4.6.3.jar:4.6.3]
        at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) ~[jetty-servlet-9.4.35.v20201120.jar:9.4.35.v20201120]
        at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) ~[jetty-servlet-9.4.35.v20201120.jar:9.4.35.v20201120]
        at com.opensymphony.sitemesh.webapp.SiteMeshFilter.obtainContent(SiteMeshFilter.java:129) ~[sitemesh-2.4.2.jar:?]
        at com.opensymphony.sitemesh.webapp.SiteMeshFilter.doFilter(SiteMeshFilter.java:77) ~[sitemesh-2.4.2.jar:?]
        at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) ~[jetty-servlet-9.4.35.v20201120.jar:9.4.35.v20201120]
        at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) ~[jetty-servlet-9.4.35.v20201120.jar:9.4.35.v20201120]
        at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) ~[jetty-servlet-9.4.35.v20201120.jar:9.4.35.v20201120]
        at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) ~[jetty-server-9.4.35.v20201120.jar:9.4.35.v20201120]
        at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602) ~[jetty-security-9.4.35.v20201120.jar:9.4.35.v20201120]
        at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) ~[jetty-server-9.4.35.v20201120.jar:9.4.35.v20201120]
        at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) ~[jetty-server-9.4.35.v20201120.jar:9.4.35.v20201120]
        at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) ~[jetty-server-9.4.35.v20201120.jar:9.4.35.v20201120]
        at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) ~[jetty-server-9.4.35.v20201120.jar:9.4.35.v20201120]
        at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1435) ~[jetty-server-9.4.35.v20201120.jar:9.4.35.v20201120]
        at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) ~[jetty-server-9.4.35.v20201120.jar:9.4.35.v20201120]
        at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) ~[jetty-servlet-9.4.35.v20201120.jar:9.4.35.v20201120]
        at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) ~[jetty-server-9.4.35.v20201120.jar:9.4.35.v20201120]
        at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) ~[jetty-server-9.4.35.v20201120.jar:9.4.35.v20201120]
        at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1350) ~[jetty-server-9.4.35.v20201120.jar:9.4.35.v20201120]
        at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) ~[jetty-server-9.4.35.v20201120.jar:9.4.35.v20201120]
        at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) ~[jetty-server-9.4.35.v20201120.jar:9.4.35.v20201120]
        at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) ~[jetty-server-9.4.35.v20201120.jar:9.4.35.v20201120]
        at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) ~[jetty-server-9.4.35.v20201120.jar:9.4.35.v20201120]
        at org.eclipse.jetty.server.Server.handle(Server.java:516) ~[jetty-server-9.4.35.v20201120.jar:9.4.35.v20201120]
        at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:388) ~[jetty-server-9.4.35.v20201120.jar:9.4.35.v20201120]
        at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:633) [jetty-server-9.4.35.v20201120.jar:9.4.35.v20201120]
        at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:380) [jetty-server-9.4.35.v20201120.jar:9.4.35.v20201120]
        at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) [jetty-server-9.4.35.v20201120.jar:9.4.35.v20201120]
        at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) [jetty-io-9.4.35.v20201120.jar:9.4.35.v20201120]
        at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) [jetty-io-9.4.35.v20201120.jar:9.4.35.v20201120]
        at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:540) [jetty-io-9.4.35.v20201120.jar:9.4.35.v20201120]
        at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:395) [jetty-io-9.4.35.v20201120.jar:9.4.35.v20201120]
        at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161) [jetty-io-9.4.35.v20201120.jar:9.4.35.v20201120]
        at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) [jetty-io-9.4.35.v20201120.jar:9.4.35.v20201120]
        at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) [jetty-io-9.4.35.v20201120.jar:9.4.35.v20201120]
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) [jetty-util-9.4.35.v20201120.jar:9.4.35.v20201120]
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) [jetty-util-9.4.35.v20201120.jar:9.4.35.v20201120]
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) [jetty-util-9.4.35.v20201120.jar:9.4.35.v20201120]
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) [jetty-util-9.4.35.v20201120.jar:9.4.35.v20201120]
        at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) [jetty-util-9.4.35.v20201120.jar:9.4.35.v20201120]
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:773) [jetty-util-9.4.35.v20201120.jar:9.4.35.v20201120]
        at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:905) [jetty-util-9.4.35.v20201120.jar:9.4.35.v20201120]
        at java.lang.Thread.run(Thread.java:748) [?:1.8.0_282]
Caused by: java.lang.ClassCastException: org.jivesoftware.openfire.muc.MUCRole$Role cannot be cast to java.lang.String
        at org.codehaus.jackson.map.ser.std.IndexedStringListSerializer.serializeContents(IndexedStringListSerializer.java:91) ~[?:?]
        ... 71 more

It’s the same error with RestAPI plugin v1.4.0 getAllRooms endpoint. Also FWIW all of our nodes are confirmed on the same OF version v4.6.3.

@guus it’s also worth noting that MUCs created directly from v4.6.3 admin have no such problems and seem to be functioning nominally across the cluster and from our XMPP client.

I submitted a PR to address this issue at https://github.com/igniterealtime/Openfire/pull/1831. This issue also happens on create and update.

RestAPI v1.4.0 is incompatible with OF v4.6.x. v1.5.0 is compatible (almost) but must be built from source presently.

This is related to Error 500 retrieving rooms info with REST API plugin