Spark Error : Unsupported data type NullType.

Spark Job failing with exception like - 

^{Caused by: org.apache.spark.SparkException: Task failed while writing rows}

        ^{at org.apache.spark.sql.execution.datasources.FileFormatWriter$.org$apache$spark$sql$execution$datasources$FileFormatWriter$$executeTask(FileFormatWriter.scala:270)}

        ^{at org.apache.spark.sql.execution.datasources.FileFormatWriter$$anonfun$write$1$$anonfun$apply$mcV$sp$1.apply(FileFormatWriter.scala:189)}

        ^{at org.apache.spark.sql.execution.datasources.FileFormatWriter$$anonfun$write$1$$anonfun$apply$mcV$sp$1.apply(FileFormatWriter.scala:188)}

        ^{at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87)}

        ^{at org.apache.spark.scheduler.Task.run(Task.scala:108)}

        ^{at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:338)}

        ^{at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)}

        ^{at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)}

        ^{at java.lang.Thread.run(Thread.java:748)}

^{Caused by: java.lang.RuntimeException: Unsupported data type NullType.}

        ^{at scala.sys.package$.error(package.scala:27)}

Analysis - This might be resulted because of SQL in place. In some cases, it is required to select NULL value(s). For example : Select NULL as dist_acct from Table1

Possible Cause - Spark will not be able to determine data type for NULL value. Thus, our job fails with above error.

Solution – Please try to update the query and cast the NULL to appropriate data type . For example – cast (NULL as String) as dist_acct