I took a quick look at the source: buffer is declared as byte array, so what’s the gain in masking the higher bytes with & 0xff?
The InBandBytestream class doesn’t appear in the stacktrace you posted. Not sure if I understand the problem correct. Maybe you could elaborate the bug a little bit more for me.
The return value is of type integer. The cast in this code line is not even necessary because the JVM will already do this.
Problem: when converting a byte to an int in a JVM, the sign bit will taken into account.
So byte value 0xFF will become 0xFFFFFFFF
In short, this method will return a negative integer value for all byte values greater 0x7F and so violating the contract that states that is returns an integer value between 0 and 255 OR -1 if the end of the stream had reached.
The InBandBytestream does not appear in the stacktrace because it returns a negative value, but it seems that the method DataInputStream.readUnsignedByte checks for x < 0 instead of x == - 1 which is also a valid check.