Sharing my setup for updating Spark silently in Windows AD environment. As there is no msi package for Spark, usual software deployment tools do not work. So, i have Spark’s installation on the share, e.g. \\server\spark\. It contains:
spark_2_8_0_885.exe (final build of 2.8.0 version, though it can be named whatever you like, i just like to have build number attached as i often install intermittent builds, not just final versions; my installer also includes java, but you can install the online version the same way)
spark_2_8_0_885.txt (empty txt file, which should be named exactly the same as the exe above; it is used to check for a version currently installed, so a script won’t install new version over and over again)
Then i have a GPO in my AD, which is attached to OU with computers. In the Computer Configuration > Policies > Windows Settings > Scripts (Startup/Shutdown) > Shutdown i have two scripts (for x86 and x64 machines). Contents of scriptx86.cmd:
IF EXIST "C:\Program Files\Spark\spark_2_8_0_885.txt" GOTO END
IF EXIST "C:\Program Files (x86)\Spark\spark_2_8_0_885.txt" GOTO END
taskkill /F /IM Spark.exe
\\server\spark\spark_2_8_0_885.exe -q
del "C:\Program Files\Spark\*.txt"
copy \\server\spark\spark_2_8_0_885.txt "C:\Program Files\Spark\"
:END
This script checks for the txt file mentioned above. If it exists, then nothing happens. It also checks the x86 folder on x64 machine. Seconds script is the same, only the last line copies txt file to C:\Program Files (x86)\Spark\ folder. Having those two scripts means that first script might install Spark on x64 machine, but won’t be able to copy new txt file to the appropriate folder. So the second script will do the copying or maybe also install the Spark itself. It is possible that on x64 machine Spark will be installed twice, but that’s not a problem in my book There are probably more elegant ways to achieve that, but this works for me (we only have a few x64 machines). These scripts run when PC is shutting down. It tries to kill Spark process, if system hasn’t yet done that and then runs installer with the silent key “-q”. Most of the time installation works fine. But because of network issues or abnormal shutdowns ~2% of machines do not upgrade normally and need manual intervention. When new version comes out, i put the installer into \server\spark, rename the txt file, then change version in both scripts.
As this also comes up sometimes in the forums, next i will share my way of distributing the preconfigured Spark’s settings for the new users (where are more in the forums, with some complex batch scripts, but mine is pretty simple). So, i have another GPO policy, this time in User Configuration > Windows Settings > Scripts (Logon/Logoff) > Logon. The contents of script.cmd:
IF EXIST "%AppData%\Spark\spark.properties" GOTO END
md %AppData%\Spark
copy \\server\sparkconfig\*.* %AppData%\Spark\
:END
It checks, if Spark’s main config file already exists. So it won’t overwrite settings for the currently setup users. It only works when a user logins on the computer for the first time and a fresh profile is created. In sparkconfig folder i have all the *.properties files i have copied from a regular Spark profile (aside of user’s login information), which i have already configured the way i want my users to use it. This will only work for new users. I don’t usually change settings for existing users, so i don’t have a script for that.
Note: Windows 10 is not running scripts when doing a shutdown (as the shutdown process is very fast). So Spark installation will only run during the OS restart. One can change that by disabling Fast boot in Control Panel > Power Options > Choose what the power button does > Change settings that are currently unavailable > disable “Turn on fast startup (recommended)”. There is no significant change in boot or shutdown times after disabling this, in my opinion.