BOSH 500 Insufficient Content

We’re getting this message from our XMPP client when performing BOSH requests against our OF v4.5.2 cluster. Only some BOSH requests get this error, which makes it difficult to reproduce and investigate.

Our JavaScript client is logging something like this:

HTTP ERROR 500 insufficient content written
URI:	/http-bind/
STATUS:	500
MESSAGE:	insufficient content written
SERVLET:	org.jivesoftware.openfire.http.HttpBindServlet-96f5dc0

Looking at the packet from one of the failed HTTP-BIND requests, we see this in the raw response:

<html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=utf-8"/>
<title>Error 500 insufficient content written</title>
</head>
<body><h2>HTTP ERROR 500 insufficient content written</h2>
<table>
<tr><th>URI:</th><td>/http-bind/</td></tr>
<tr><th>STATUS:</th><td>500</td></tr>
<tr><th>MESSAGE:</th><td>insufficient content written</td></tr>
<tr><th>SERVLET:</th><td>org.jivesoftware.openfire.http.HttpBindServlet-96f5dc0</td></tr>
</table>
</body>
</html>

This looks related to Jetty. I notice a Jetty bug report at https://github.com/eclipse/jetty.project/issues/4208 which could be related.

Does anybody have familiarity with this error or can somebody suggest steps to investigate/troubleshoot?

I think this is an OF log entry that corresponds to the 500 error the client sees. I’m uncertain if this log entry is related, since we’re not actually seeing the BOSH and/or Jetty error in the logs, just related log entries, apparently:

2021.xx.xx xx:xx:xx ERROR [Jetty-QTP-BOSH-148038]: org.jivesoftware.openfire.http.HttpBindServlet - Error sending packet to client.
org.jivesoftware.openfire.http.HttpConnectionClosedException: The http connection is no longer available to deliver content
        at org.jivesoftware.openfire.http.HttpConnection.deliverBody(HttpConnection.java:110) ~[xmppserver-4.5.2.jar:4.5.2]
        at org.jivesoftware.openfire.http.HttpSession.forwardRequest(HttpSession.java:597) ~[xmppserver-4.5.2.jar:4.5.2]
        at org.jivesoftware.openfire.http.HttpBindServlet.handleSessionRequest(HttpBindServlet.java:219) [xmppserver-4.5.2.jar:4.5.2]
        at org.jivesoftware.openfire.http.HttpBindServlet.processContent(HttpBindServlet.java:172) [xmppserver-4.5.2.jar:4.5.2]
        at org.jivesoftware.openfire.http.HttpBindServlet$ReadListenerImpl.onAllDataRead(HttpBindServlet.java:361) [xmppserver-4.5.2.jar:4.5.2]
        at org.eclipse.jetty.server.HttpInput.run(HttpInput.java:903) [jetty-server-9.4.28.v20200408.jar:9.4.28.v20200408]
        at org.eclipse.jetty.server.handler.ContextHandler.handle(ContextHandler.java:1448) [jetty-server-9.4.28.v20200408.jar:9.4.28.v20200408]
        at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:468) [jetty-server-9.4.28.v20200408.jar:9.4.28.v20200408]
        at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) [jetty-server-9.4.28.v20200408.jar:9.4.28.v20200408]
        at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) [jetty-io-9.4.28.v20200408.jar:9.4.28.v20200408]
        at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103) [jetty-io-9.4.28.v20200408.jar:9.4.28.v20200408]
        at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:543) [jetty-io-9.4.28.v20200408.jar:9.4.28.v20200408]
        at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:398) [jetty-io-9.4.28.v20200408.jar:9.4.28.v20200408]
        at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161) [jetty-io-9.4.28.v20200408.jar:9.4.28.v20200408]
        at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103) [jetty-io-9.4.28.v20200408.jar:9.4.28.v20200408]
        at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117) [jetty-io-9.4.28.v20200408.jar:9.4.28.v20200408]
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) [jetty-util-9.4.28.v20200408.jar:9.4.28.v20200408]
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) [jetty-util-9.4.28.v20200408.jar:9.4.28.v20200408]
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) [jetty-util-9.4.28.v20200408.jar:9.4.28.v20200408]
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) [jetty-util-9.4.28.v20200408.jar:9.4.28.v20200408]
        at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) [jetty-util-9.4.28.v20200408.jar:9.4.28.v20200408]
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) [jetty-util-9.4.28.v20200408.jar:9.4.28.v20200408]
        at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) [jetty-util-9.4.28.v20200408.jar:9.4.28.v20200408]
        at java.lang.Thread.run(Thread.java:748) [?:1.8.0_275]

I’ve not noticed this before. Perhaps this is Jetty, throwing an exception because Openfire is trying to put more content in a response body than what it defined in the Content-Length response header, somehow?

Openfire sets that value to the size of the message, in bytes, which is what I think is correct:

final byte[] byteContent = content.getBytes(StandardCharsets.UTF_8);
response.setContentLength(byteContent.length);

Maybe the problem is the other way around: is your client sending an incorrect Content-Length header occasionally? Maybe around edge-cases involving multi-byte characters (where the content-length would not be equal to the character length)?

so is there any way to resolve this? Do we have anything to do in the client side?