Spark 2.9.1, localization broken

Broken russian localization in Spark 2.9.1 if jre 11 or 14 is used. OS: Debian GNU/Linux 64 bit.Screenshot_Spark|350x500

Although Spark can run on Java 11 or newer (not on Windows, but it works on Ubuntu/Debian), we are still only supporting Java 8 officially. There was a problem with translations when using Java 8 recently and i think the workaround that was applied is causing this problem on Java 11 or newer. More details in the ticket https://issues.igniterealtime.org/browse/SPARK-2160

Thank you for response. But I suppose that use of a java 8 brings a number of problem:

  1. java 8 is very old. Threre is no java 8 in modern linux distribution. For example, debian 10 has openjdk 11.
  2. The main problem, I suppose. Kerberos authentication of spark in windows 10 is ruined, thanks to Microsoft. AllowTGTSessionKey registry parameter no longer works. See http://blog.accessitautomation.com/example-case-study-of-credential-guard-oracle-java-gss-api-support-on-windows-10 But I suppose that this problem is resolved in java 13. See https://bugs.openjdk.java.net/browse/JDK-8214079 and https://bugs.openjdk.java.net/browse/JDK-6722928. I hope that using java 13 or newer will help to resolve problem of kerberos authentication of spark in windows 10.

And one more things. I have installed liberica openjdk 14.02 in my debian system and launched spark 2.8.3 successfully in this jvm. Of course I have to disable jvm version check in test_jvm function of start script. Spark works well and russian localization is not broken. So I suppose that the main problem with localization is not connected with jvm version.

I agree that Java 8 is too old now and Spark should move to newer Java (and drop support for 8 if supporting both is not possible). The thing is that Spark doesn’t have a developer who can consistently spend time on it and tackle such issues.

Regarding SSO there is a discussion here Revisiting SSO on Windows Maybe using SSPI is an option.

I have tried to run Spark 2.9.1 on Windows 10 using AdoptOpenJDK 14 and a few plugins were not working. There can possibly be more issues under the hood that only an experienced Java developer can notice and fix.

2.8.3 uses translations that were not converted to unicode, so that is why it works with any Java. To work with such translations one has to use \uxxxx codes instead of a natural language characters, which is annoying when doing a lot of updates. Using unicode files has problems in Java 8, which is why this workaround was applied.

I would like to add one more thing, maybe it will be useful.

I have started spark 2.8.3 in linux with the latest liberica openjdk 14.02 with disabled jvm version check and added system property -Dsun.security.jgss.native=true. And I can see that native gss works fine in linux. For example, now I can see xmpp SGT in klist command output.

Now for windows. It was a little bit harder. I have installed spark 2.8.3 and replace old jre by mordern 14.02 32bit. Because of there is no script to start spark, only binary, I used startup.bat from resources directory of spark 2.9.2. I have modified script the same way, by adding system property -Dsun.security.jgss.native=true. Spark was able to start and authorize using kerberos. And I can see xmpp SGT in klist command output too.

1 Like

Could you share where or how exactly you put the system property in the batch file? I have not met with success yet.

Of course, it is not very difficult.

  1. Install spark 2.8.3.
  2. delete jre directory from Spark installation.
  3. Download liberica jre 14.02 (link). Extract it into Spark disrectory and rename from jre-14.0.2 to jre.
  4. Place startup.bat in resources directory of Sparkstartup.bat (1015 Bytes)
  5. Launch startup.bat. Spark will start and will be able to authorize using kerberos ticket. Of course, there is a lot of exceptions due disabled plugins (As I understand, each plugin performs java version check and refuses to work with modern java), but Spark itself works well.
  6. Launch klist utility. You can see TGS ticket for xmpp service.

    If you use windows 7 with jre8 and AllowTGTSessionKey=1 you will not see TGS ticket for xmpp service in the output of klist utility.