Apache-spark – How to set Spark application exit status

apache-sparkexit-codehadoop-yarnspark-submit

I'm writing a spark application and run it using spark-submit shell script (using yarn-cluster/yarn-client)

As I see now, exit code of spark-submit is decided according to the related yarn application – if SUCCEEDED status is 0, otherwise 1.

I want to have the option to return another exit code – for a state that my application succeeded with some errors.

Is it possible? to return different exit code from the application?

I tried to use System.exit() but didn't succeed…

Thanks.

Best Answer

It is possible in client mode but not in cluster mode. You have a workaround for cluster mode.

My answer to this question should help you.

Example

@echo off
my_nify_exe.exe
if errorlevel 1 (
   echo Failure Reason Given is %errorlevel%
   exit /b %errorlevel%
)

Warning: If you set an environment variable name errorlevel, %errorlevel% will return that value and not the exit code. Use (set errorlevel=) to clear the environment variable, allowing access to the true value of errorlevel via the %errorlevel% environment variable.

Bash – In a Bash script, how can I exit the entire script if a certain condition occurs

Try this statement:

exit 1

Replace 1 with appropriate error codes. See also Exit Codes With Special Meanings.

Best Answer

Related Solutions

Windows – How to get the application exit code from a Windows command line

Example

Bash – In a Bash script, how can I exit the entire script if a certain condition occurs

Related Topic