spark-sql> 
         > SELECT count(v) FROM ( 
         >   SELECT v FROM ( 
         >     SELECT 'foo' AS v FROM dual
         >     UNION ALL 
         >     SELECT NULL AS v FROM dual 
         >   ) my_union WHERE isnull(v)
         > ) my_subview;
15/09/18 20:45:16 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve MetastoreRelation default, dual, None
15/09/18 20:45:16 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve Project [foo AS v#12]
15/09/18 20:45:16 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve MetastoreRelation default, dual, None
15/09/18 20:45:16 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve Project [null AS v#13]
15/09/18 20:45:16 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve 'Union
15/09/18 20:45:16 DEBUG analysis.HiveTypeCoercion$WidenTypes: Widening numeric types in union ArrayBuffer(CAST(v#13, StringType) AS v#17) ArrayBuffer(v#13)
15/09/18 20:45:16 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve MetastoreRelation default, dual, None
15/09/18 20:45:16 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve Project [foo AS v#12]
15/09/18 20:45:16 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve MetastoreRelation default, dual, None
15/09/18 20:45:16 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve Project [null AS v#13]
15/09/18 20:45:16 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve Project [CAST(v#13, StringType) AS v#17]
15/09/18 20:45:16 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve Union
15/09/18 20:45:16 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve Subquery my_union
15/09/18 20:45:16 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve 'Filter 'isnull('v)
15/09/18 20:45:16 DEBUG analysis.Analyzer$ResolveReferences: Resolving 'v to v#12
15/09/18 20:45:16 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve MetastoreRelation default, dual, None
15/09/18 20:45:16 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve Project [foo AS v#12]
15/09/18 20:45:16 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve MetastoreRelation default, dual, None
15/09/18 20:45:16 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve Project [null AS v#13]
15/09/18 20:45:16 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve Project [CAST(v#13, StringType) AS v#17]
15/09/18 20:45:16 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve Union
15/09/18 20:45:16 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve Subquery my_union
15/09/18 20:45:16 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve Filter HiveGenericUdf#org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPNull(v#12)
15/09/18 20:45:16 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve 'Project ['v]
15/09/18 20:45:16 DEBUG analysis.Analyzer$ResolveReferences: Resolving 'v to v#12
15/09/18 20:45:16 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve Subquery my_subview
15/09/18 20:45:16 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve 'Aggregate [COUNT('v) AS _c0#14]
15/09/18 20:45:16 DEBUG analysis.Analyzer$ResolveReferences: Resolving 'v to v#12
15/09/18 20:45:16 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve MetastoreRelation default, dual, None
15/09/18 20:45:16 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve Project [foo AS v#12]
15/09/18 20:45:16 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve MetastoreRelation default, dual, None
15/09/18 20:45:16 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve Project [null AS v#13]
15/09/18 20:45:16 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve Project [CAST(v#13, StringType) AS v#17]
15/09/18 20:45:16 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve Union
15/09/18 20:45:16 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve Subquery my_union
15/09/18 20:45:16 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve Filter HiveGenericUdf#org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPNull(v#12)
15/09/18 20:45:16 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve Project [v#12]
15/09/18 20:45:16 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve Subquery my_subview
15/09/18 20:45:16 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve Aggregate [COUNT(v#12) AS _c0#14L]
15/09/18 20:45:16 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve MetastoreRelation default, dual, None
15/09/18 20:45:16 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve Project [foo AS v#12]
15/09/18 20:45:16 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve MetastoreRelation default, dual, None
15/09/18 20:45:16 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve Project [null AS v#13]
15/09/18 20:45:16 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve Project [CAST(v#13, StringType) AS v#17]
15/09/18 20:45:16 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve Union
15/09/18 20:45:16 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve Subquery my_union
15/09/18 20:45:16 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve Filter HiveGenericUdf#org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPNull(v#12)
15/09/18 20:45:16 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve Project [v#12]
15/09/18 20:45:16 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve Subquery my_subview
15/09/18 20:45:16 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve Aggregate [COUNT(v#12) AS _c0#14L]
15/09/18 20:45:16 TRACE optimizer.DefaultOptimizer: 
=== Applying Rule org.apache.spark.sql.catalyst.analysis.EliminateSubQueries ===
 Aggregate [COUNT(v#12) AS _c0#14L]                                                      Aggregate [COUNT(v#12) AS _c0#14L]
! Subquery my_subview                                                                     Project [v#12]
!  Project [v#12]                                                                          Filter HiveGenericUdf#org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPNull(v#12)
!   Filter HiveGenericUdf#org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPNull(v#12)      Union
!    Subquery my_union                                                                       Project [foo AS v#12]
!     Union                                                                                   MetastoreRelation default, dual, None
!      Project [foo AS v#12]                                                                 Project [CAST(v#13, StringType) AS v#17]
!       MetastoreRelation default, dual, None                                                 Project [null AS v#13]
!      Project [CAST(v#13, StringType) AS v#17]                                                MetastoreRelation default, dual, None
!       Project [null AS v#13]                                                           
!        MetastoreRelation default, dual, None                                           
                
15/09/18 20:45:16 TRACE optimizer.DefaultOptimizer: Fixed point reached for batch Remove SubQueries after 2 iterations.
15/09/18 20:45:16 DEBUG optimizer.DefaultOptimizer: 
=== Result of Batch Remove SubQueries ===
 Aggregate [COUNT(v#12) AS _c0#14L]                                                      Aggregate [COUNT(v#12) AS _c0#14L]
! Subquery my_subview                                                                     Project [v#12]
!  Project [v#12]                                                                          Filter HiveGenericUdf#org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPNull(v#12)
!   Filter HiveGenericUdf#org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPNull(v#12)      Union
!    Subquery my_union                                                                       Project [foo AS v#12]
!     Union                                                                                   MetastoreRelation default, dual, None
!      Project [foo AS v#12]                                                                 Project [CAST(v#13, StringType) AS v#17]
!       MetastoreRelation default, dual, None                                                 Project [null AS v#13]
!      Project [CAST(v#13, StringType) AS v#17]                                                MetastoreRelation default, dual, None
!       Project [null AS v#13]                                                           
!        MetastoreRelation default, dual, None                                           
        
15/09/18 20:45:16 TRACE optimizer.DefaultOptimizer: 
=== Applying Rule org.apache.spark.sql.catalyst.optimizer.UnionPushdown ===
 Aggregate [COUNT(v#12) AS _c0#14L]                                                     Aggregate [COUNT(v#12) AS _c0#14L]
  Project [v#12]                                                                         Project [v#12]
!  Filter HiveGenericUdf#org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPNull(v#12)     Union
!   Union                                                                                  Filter HiveGenericUdf#org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPNull(v#12)
     Project [foo AS v#12]                                                                  Project [foo AS v#12]
      MetastoreRelation default, dual, None                                                  MetastoreRelation default, dual, None
!    Project [CAST(v#13, StringType) AS v#17]                                              Filter HiveGenericUdf#org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPNull(v#17)
!     Project [null AS v#13]                                                                Project [CAST(v#13, StringType) AS v#17]
!      MetastoreRelation default, dual, None                                                 Project [null AS v#13]
!                                                                                             MetastoreRelation default, dual, None
                
15/09/18 20:45:16 TRACE optimizer.DefaultOptimizer: 
=== Applying Rule org.apache.spark.sql.catalyst.optimizer.PushPredicateThroughProject ===
 Aggregate [COUNT(v#12) AS _c0#14L]                                                      Aggregate [COUNT(v#12) AS _c0#14L]
  Project [v#12]                                                                          Project [v#12]
   Union                                                                                   Union
!   Filter HiveGenericUdf#org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPNull(v#12)      Project [foo AS v#12]
!    Project [foo AS v#12]                                                                   Filter HiveGenericUdf#org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPNull(foo)
      MetastoreRelation default, dual, None                                                   MetastoreRelation default, dual, None
!   Filter HiveGenericUdf#org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPNull(v#17)      Project [CAST(v#13, StringType) AS v#17]
!    Project [CAST(v#13, StringType) AS v#17]                                                Project [null AS v#13]
!     Project [null AS v#13]                                                                  Filter HiveGenericUdf#org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPNull(CAST(null, StringType))
       MetastoreRelation default, dual, None                                                   MetastoreRelation default, dual, None
                
15/09/18 20:45:16 TRACE optimizer.DefaultOptimizer: 
=== Applying Rule org.apache.spark.sql.catalyst.optimizer.ColumnPruning ===
 Aggregate [COUNT(v#12) AS _c0#14L]                                                                          Aggregate [COUNT(v#12) AS _c0#14L]
! Project [v#12]                                                                                              Union
!  Union                                                                                                       Project [foo AS v#12]
!   Project [foo AS v#12]                                                                                       Filter HiveGenericUdf#org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPNull(foo)
!    Filter HiveGenericUdf#org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPNull(foo)                           MetastoreRelation default, dual, None
!     MetastoreRelation default, dual, None                                                                    Project [CAST(v#13, StringType) AS v#17]
!   Project [CAST(v#13, StringType) AS v#17]                                                                    Project [null AS v#13]
!    Project [null AS v#13]                                                                                      Filter HiveGenericUdf#org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPNull(CAST(null, StringType))
!     Filter HiveGenericUdf#org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPNull(CAST(null, StringType))        MetastoreRelation default, dual, None
!      MetastoreRelation default, dual, None                                                                 
                
15/09/18 20:45:16 TRACE optimizer.DefaultOptimizer: 
=== Applying Rule org.apache.spark.sql.catalyst.optimizer.ProjectCollapsing ===
 Aggregate [COUNT(v#12) AS _c0#14L]                                                                         Aggregate [COUNT(v#12) AS _c0#14L]
  Union                                                                                                      Union
   Project [foo AS v#12]                                                                                      Project [foo AS v#12]
    Filter HiveGenericUdf#org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPNull(foo)                          Filter HiveGenericUdf#org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPNull(foo)
     MetastoreRelation default, dual, None                                                                      MetastoreRelation default, dual, None
!  Project [CAST(v#13, StringType) AS v#17]                                                                   Project [CAST(null AS v#13, StringType) AS v#17]
!   Project [null AS v#13]                                                                                     Filter HiveGenericUdf#org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPNull(CAST(null, StringType))
!    Filter HiveGenericUdf#org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPNull(CAST(null, StringType))       MetastoreRelation default, dual, None
!     MetastoreRelation default, dual, None                                                                 
                
15/09/18 20:45:16 TRACE optimizer.DefaultOptimizer: Fixed point reached for batch Operator Reordering after 2 iterations.
15/09/18 20:45:16 DEBUG optimizer.DefaultOptimizer: 
=== Result of Batch Operator Reordering ===
 Aggregate [COUNT(v#12) AS _c0#14L]                                                      Aggregate [COUNT(v#12) AS _c0#14L]
! Subquery my_subview                                                                     Union
!  Project [v#12]                                                                          Project [foo AS v#12]
!   Filter HiveGenericUdf#org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPNull(v#12)      Filter HiveGenericUdf#org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPNull(foo)
!    Subquery my_union                                                                       MetastoreRelation default, dual, None
!     Union                                                                                Project [CAST(null AS v#13, StringType) AS v#17]
!      Project [foo AS v#12]                                                                Filter HiveGenericUdf#org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPNull(CAST(null, StringType))
!       MetastoreRelation default, dual, None                                                MetastoreRelation default, dual, None
!      Project [CAST(v#13, StringType) AS v#17]                                          
!       Project [null AS v#13]                                                           
!        MetastoreRelation default, dual, None                                           
        
15/09/18 20:45:16 TRACE optimizer.DefaultOptimizer: 
=== Applying Rule org.apache.spark.sql.catalyst.optimizer.NullPropagation ===
!Aggregate [COUNT(v#12) AS _c0#14L]                                                                        Aggregate [COUNT(1) AS _c0#14L]
  Union                                                                                                     Union
   Project [foo AS v#12]                                                                                     Project [foo AS v#12]
    Filter HiveGenericUdf#org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPNull(foo)                         Filter HiveGenericUdf#org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPNull(foo)
     MetastoreRelation default, dual, None                                                                     MetastoreRelation default, dual, None
   Project [CAST(null AS v#13, StringType) AS v#17]                                                          Project [CAST(null AS v#13, StringType) AS v#17]
    Filter HiveGenericUdf#org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPNull(CAST(null, StringType))      Filter HiveGenericUdf#org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPNull(CAST(null, StringType))
     MetastoreRelation default, dual, None                                                                     MetastoreRelation default, dual, None
                
15/09/18 20:45:17 TRACE optimizer.DefaultOptimizer: 
=== Applying Rule org.apache.spark.sql.catalyst.optimizer.ConstantFolding ===
 Aggregate [COUNT(1) AS _c0#14L]                                                                           Aggregate [COUNT(1) AS _c0#14L]
  Union                                                                                                     Union
   Project [foo AS v#12]                                                                                     Project [foo AS v#12]
!   Filter HiveGenericUdf#org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPNull(foo)                         Filter false
     MetastoreRelation default, dual, None                                                                     MetastoreRelation default, dual, None
   Project [CAST(null AS v#13, StringType) AS v#17]                                                          Project [CAST(null AS v#13, StringType) AS v#17]
!   Filter HiveGenericUdf#org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPNull(CAST(null, StringType))      Filter true
     MetastoreRelation default, dual, None                                                                     MetastoreRelation default, dual, None
                
15/09/18 20:45:17 TRACE optimizer.DefaultOptimizer: 
=== Applying Rule org.apache.spark.sql.catalyst.optimizer.SimplifyFilters ===
 Aggregate [COUNT(1) AS _c0#14L]                      Aggregate [COUNT(1) AS _c0#14L]
  Union                                                Union
   Project [foo AS v#12]                                Project [foo AS v#12]
!   Filter false                                         LocalRelation [dummy#15]
!    MetastoreRelation default, dual, None              Project [CAST(null AS v#13, StringType) AS v#17]
!  Project [CAST(null AS v#13, StringType) AS v#17]      MetastoreRelation default, dual, None
!   Filter true                                       
!    MetastoreRelation default, dual, None            
                
15/09/18 20:45:17 TRACE optimizer.DefaultOptimizer: Fixed point reached for batch ConstantFolding after 2 iterations.
15/09/18 20:45:17 DEBUG optimizer.DefaultOptimizer: 
=== Result of Batch ConstantFolding ===
!Aggregate [COUNT(v#12) AS _c0#14L]                                                      Aggregate [COUNT(1) AS _c0#14L]
! Subquery my_subview                                                                     Union
!  Project [v#12]                                                                          Project [foo AS v#12]
!   Filter HiveGenericUdf#org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPNull(v#12)      LocalRelation [dummy#15]
!    Subquery my_union                                                                     Project [CAST(null AS v#13, StringType) AS v#17]
!     Union                                                                                 MetastoreRelation default, dual, None
!      Project [foo AS v#12]                                                             
!       MetastoreRelation default, dual, None                                            
!      Project [CAST(v#13, StringType) AS v#17]                                          
!       Project [null AS v#13]                                                           
!        MetastoreRelation default, dual, None                                           
        
15/09/18 20:45:17 TRACE optimizer.DefaultOptimizer: Fixed point reached for batch Decimal Optimizations after 1 iterations.
15/09/18 20:45:17 TRACE optimizer.DefaultOptimizer: Batch Decimal Optimizations has no effect.
15/09/18 20:45:17 TRACE optimizer.DefaultOptimizer: 
=== Applying Rule org.apache.spark.sql.catalyst.optimizer.ConvertToLocalRelation ===
 Aggregate [COUNT(1) AS _c0#14L]                      Aggregate [COUNT(1) AS _c0#14L]
  Union                                                Union
!  Project [foo AS v#12]                                LocalRelation [v#12]
!   LocalRelation [dummy#15]                            Project [CAST(null AS v#13, StringType) AS v#17]
!  Project [CAST(null AS v#13, StringType) AS v#17]      MetastoreRelation default, dual, None
!   MetastoreRelation default, dual, None             
                
15/09/18 20:45:17 TRACE optimizer.DefaultOptimizer: Fixed point reached for batch LocalRelation after 2 iterations.
15/09/18 20:45:17 DEBUG optimizer.DefaultOptimizer: 
=== Result of Batch LocalRelation ===
!Aggregate [COUNT(v#12) AS _c0#14L]                                                      Aggregate [COUNT(1) AS _c0#14L]
! Subquery my_subview                                                                     Union
!  Project [v#12]                                                                          LocalRelation [v#12]
!   Filter HiveGenericUdf#org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPNull(v#12)     Project [CAST(null AS v#13, StringType) AS v#17]
!    Subquery my_union                                                                      MetastoreRelation default, dual, None
!     Union                                                                              
!      Project [foo AS v#12]                                                             
!       MetastoreRelation default, dual, None                                            
!      Project [CAST(v#13, StringType) AS v#17]                                          
!       Project [null AS v#13]                                                           
!        MetastoreRelation default, dual, None                                           
        
15/09/18 20:45:17 INFO execution.Exchange: Using SparkSqlSerializer2.
15/09/18 20:45:18 INFO executor.Executor: Running task 0.0 in stage 0.0 (TID 0)
15/09/18 20:45:18 INFO executor.Executor: Finished task 0.0 in stage 0.0 (TID 0). 803 bytes result sent to driver
15/09/18 20:45:18 INFO executor.Executor: Running task 1.0 in stage 0.0 (TID 1)
15/09/18 20:45:18 INFO rdd.HadoopRDD: Input split: hdfs://localhost:54310/user/hive/warehouse/dual/part-00000:0+2
15/09/18 20:45:18 INFO executor.Executor: Finished task 1.0 in stage 0.0 (TID 1). 1925 bytes result sent to driver
15/09/18 20:45:18 INFO executor.Executor: Running task 0.0 in stage 1.0 (TID 2)
15/09/18 20:45:18 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/09/18 20:45:18 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 4 ms
15/09/18 20:45:18 INFO executor.Executor: Finished task 0.0 in stage 1.0 (TID 2). 1212 bytes result sent to driver
1
Time taken: 1.642 seconds, Fetched 1 row(s)
spark-sql> 
         > SELECT count(v) FROM ( 
         >   SELECT v FROM ( 
         >     SELECT NULL AS v FROM dual 
         >     UNION ALL 
         >     SELECT 'foo' AS v FROM dual
         >   ) my_union WHERE isnull(v)
         > ) my_subview;
15/09/18 20:45:27 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve MetastoreRelation default, dual, None
15/09/18 20:45:27 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve Project [null AS v#25]
15/09/18 20:45:27 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve MetastoreRelation default, dual, None
15/09/18 20:45:27 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve Project [foo AS v#26]
15/09/18 20:45:27 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve 'Union
15/09/18 20:45:27 DEBUG analysis.HiveTypeCoercion$WidenTypes: Widening numeric types in union ArrayBuffer(CAST(v#25, StringType) AS v#30) ArrayBuffer(v#25)
15/09/18 20:45:27 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve MetastoreRelation default, dual, None
15/09/18 20:45:27 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve Project [null AS v#25]
15/09/18 20:45:27 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve Project [CAST(v#25, StringType) AS v#30]
15/09/18 20:45:27 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve MetastoreRelation default, dual, None
15/09/18 20:45:27 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve Project [foo AS v#26]
15/09/18 20:45:27 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve Union
15/09/18 20:45:27 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve Subquery my_union
15/09/18 20:45:27 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve 'Filter 'isnull('v)
15/09/18 20:45:27 DEBUG analysis.Analyzer$ResolveReferences: Resolving 'v to v#30
15/09/18 20:45:27 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve MetastoreRelation default, dual, None
15/09/18 20:45:27 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve Project [null AS v#25]
15/09/18 20:45:27 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve Project [CAST(v#25, StringType) AS v#30]
15/09/18 20:45:27 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve MetastoreRelation default, dual, None
15/09/18 20:45:27 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve Project [foo AS v#26]
15/09/18 20:45:27 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve Union
15/09/18 20:45:27 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve Subquery my_union
15/09/18 20:45:27 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve Filter HiveGenericUdf#org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPNull(v#30)
15/09/18 20:45:27 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve 'Project ['v]
15/09/18 20:45:27 DEBUG analysis.Analyzer$ResolveReferences: Resolving 'v to v#30
15/09/18 20:45:27 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve Subquery my_subview
15/09/18 20:45:27 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve 'Aggregate [COUNT('v) AS _c0#27]
15/09/18 20:45:27 DEBUG analysis.Analyzer$ResolveReferences: Resolving 'v to v#30
15/09/18 20:45:27 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve MetastoreRelation default, dual, None
15/09/18 20:45:27 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve Project [null AS v#25]
15/09/18 20:45:27 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve Project [CAST(v#25, StringType) AS v#30]
15/09/18 20:45:27 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve MetastoreRelation default, dual, None
15/09/18 20:45:27 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve Project [foo AS v#26]
15/09/18 20:45:27 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve Union
15/09/18 20:45:27 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve Subquery my_union
15/09/18 20:45:27 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve Filter HiveGenericUdf#org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPNull(v#30)
15/09/18 20:45:27 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve Project [v#30]
15/09/18 20:45:27 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve Subquery my_subview
15/09/18 20:45:27 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve Aggregate [COUNT(v#30) AS _c0#27L]
15/09/18 20:45:27 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve MetastoreRelation default, dual, None
15/09/18 20:45:27 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve Project [null AS v#25]
15/09/18 20:45:27 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve Project [CAST(v#25, StringType) AS v#30]
15/09/18 20:45:27 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve MetastoreRelation default, dual, None
15/09/18 20:45:27 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve Project [foo AS v#26]
15/09/18 20:45:27 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve Union
15/09/18 20:45:27 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve Subquery my_union
15/09/18 20:45:27 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve Filter HiveGenericUdf#org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPNull(v#30)
15/09/18 20:45:27 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve Project [v#30]
15/09/18 20:45:27 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve Subquery my_subview
15/09/18 20:45:27 TRACE analysis.Analyzer$ResolveReferences: Attempting to resolve Aggregate [COUNT(v#30) AS _c0#27L]
15/09/18 20:45:27 TRACE optimizer.DefaultOptimizer: 
=== Applying Rule org.apache.spark.sql.catalyst.analysis.EliminateSubQueries ===
 Aggregate [COUNT(v#30) AS _c0#27L]                                                      Aggregate [COUNT(v#30) AS _c0#27L]
! Subquery my_subview                                                                     Project [v#30]
!  Project [v#30]                                                                          Filter HiveGenericUdf#org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPNull(v#30)
!   Filter HiveGenericUdf#org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPNull(v#30)      Union
!    Subquery my_union                                                                       Project [CAST(v#25, StringType) AS v#30]
!     Union                                                                                   Project [null AS v#25]
!      Project [CAST(v#25, StringType) AS v#30]                                                MetastoreRelation default, dual, None
!       Project [null AS v#25]                                                               Project [foo AS v#26]
!        MetastoreRelation default, dual, None                                                MetastoreRelation default, dual, None
!      Project [foo AS v#26]                                                             
!       MetastoreRelation default, dual, None                                            
                
15/09/18 20:45:27 TRACE optimizer.DefaultOptimizer: Fixed point reached for batch Remove SubQueries after 2 iterations.
15/09/18 20:45:27 DEBUG optimizer.DefaultOptimizer: 
=== Result of Batch Remove SubQueries ===
 Aggregate [COUNT(v#30) AS _c0#27L]                                                      Aggregate [COUNT(v#30) AS _c0#27L]
! Subquery my_subview                                                                     Project [v#30]
!  Project [v#30]                                                                          Filter HiveGenericUdf#org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPNull(v#30)
!   Filter HiveGenericUdf#org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPNull(v#30)      Union
!    Subquery my_union                                                                       Project [CAST(v#25, StringType) AS v#30]
!     Union                                                                                   Project [null AS v#25]
!      Project [CAST(v#25, StringType) AS v#30]                                                MetastoreRelation default, dual, None
!       Project [null AS v#25]                                                               Project [foo AS v#26]
!        MetastoreRelation default, dual, None                                                MetastoreRelation default, dual, None
!      Project [foo AS v#26]                                                             
!       MetastoreRelation default, dual, None                                            
        
15/09/18 20:45:27 TRACE optimizer.DefaultOptimizer: 
=== Applying Rule org.apache.spark.sql.catalyst.optimizer.UnionPushdown ===
 Aggregate [COUNT(v#30) AS _c0#27L]                                                     Aggregate [COUNT(v#30) AS _c0#27L]
  Project [v#30]                                                                         Project [v#30]
!  Filter HiveGenericUdf#org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPNull(v#30)     Union
!   Union                                                                                  Filter HiveGenericUdf#org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPNull(v#30)
     Project [CAST(v#25, StringType) AS v#30]                                               Project [CAST(v#25, StringType) AS v#30]
      Project [null AS v#25]                                                                 Project [null AS v#25]
       MetastoreRelation default, dual, None                                                  MetastoreRelation default, dual, None
!    Project [foo AS v#26]                                                                 Filter HiveGenericUdf#org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPNull(v#26)
!     MetastoreRelation default, dual, None                                                 Project [foo AS v#26]
!                                                                                            MetastoreRelation default, dual, None
                
15/09/18 20:45:27 TRACE optimizer.DefaultOptimizer: 
=== Applying Rule org.apache.spark.sql.catalyst.optimizer.PushPredicateThroughProject ===
 Aggregate [COUNT(v#30) AS _c0#27L]                                                      Aggregate [COUNT(v#30) AS _c0#27L]
  Project [v#30]                                                                          Project [v#30]
   Union                                                                                   Union
!   Filter HiveGenericUdf#org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPNull(v#30)      Project [CAST(v#25, StringType) AS v#30]
!    Project [CAST(v#25, StringType) AS v#30]                                                Project [null AS v#25]
!     Project [null AS v#25]                                                                  Filter HiveGenericUdf#org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPNull(CAST(null, StringType))
       MetastoreRelation default, dual, None                                                   MetastoreRelation default, dual, None
!   Filter HiveGenericUdf#org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPNull(v#26)      Project [foo AS v#26]
!    Project [foo AS v#26]                                                                   Filter HiveGenericUdf#org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPNull(foo)
      MetastoreRelation default, dual, None                                                   MetastoreRelation default, dual, None
                
15/09/18 20:45:27 TRACE optimizer.DefaultOptimizer: 
=== Applying Rule org.apache.spark.sql.catalyst.optimizer.ColumnPruning ===
 Aggregate [COUNT(v#30) AS _c0#27L]                                                                          Aggregate [COUNT(v#30) AS _c0#27L]
! Project [v#30]                                                                                              Union
!  Union                                                                                                       Project [CAST(v#25, StringType) AS v#30]
!   Project [CAST(v#25, StringType) AS v#30]                                                                    Project [null AS v#25]
!    Project [null AS v#25]                                                                                      Filter HiveGenericUdf#org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPNull(CAST(null, StringType))
!     Filter HiveGenericUdf#org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPNull(CAST(null, StringType))        MetastoreRelation default, dual, None
!      MetastoreRelation default, dual, None                                                                   Project [foo AS v#26]
!   Project [foo AS v#26]                                                                                       Filter HiveGenericUdf#org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPNull(foo)
!    Filter HiveGenericUdf#org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPNull(foo)                           MetastoreRelation default, dual, None
!     MetastoreRelation default, dual, None                                                                  
                
15/09/18 20:45:27 TRACE optimizer.DefaultOptimizer: 
=== Applying Rule org.apache.spark.sql.catalyst.optimizer.ProjectCollapsing ===
 Aggregate [COUNT(v#30) AS _c0#27L]                                                                         Aggregate [COUNT(v#30) AS _c0#27L]
  Union                                                                                                      Union
!  Project [CAST(v#25, StringType) AS v#30]                                                                   Project [CAST(null AS v#25, StringType) AS v#30]
!   Project [null AS v#25]                                                                                     Filter HiveGenericUdf#org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPNull(CAST(null, StringType))
!    Filter HiveGenericUdf#org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPNull(CAST(null, StringType))       MetastoreRelation default, dual, None
!     MetastoreRelation default, dual, None                                                                   Project [foo AS v#26]
!  Project [foo AS v#26]                                                                                       Filter HiveGenericUdf#org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPNull(foo)
!   Filter HiveGenericUdf#org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPNull(foo)                           MetastoreRelation default, dual, None
!    MetastoreRelation default, dual, None                                                                  
                
15/09/18 20:45:27 TRACE optimizer.DefaultOptimizer: Fixed point reached for batch Operator Reordering after 2 iterations.
15/09/18 20:45:27 DEBUG optimizer.DefaultOptimizer: 
=== Result of Batch Operator Reordering ===
 Aggregate [COUNT(v#30) AS _c0#27L]                                                      Aggregate [COUNT(v#30) AS _c0#27L]
! Subquery my_subview                                                                     Union
!  Project [v#30]                                                                          Project [CAST(null AS v#25, StringType) AS v#30]
!   Filter HiveGenericUdf#org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPNull(v#30)      Filter HiveGenericUdf#org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPNull(CAST(null, StringType))
!    Subquery my_union                                                                       MetastoreRelation default, dual, None
!     Union                                                                                Project [foo AS v#26]
!      Project [CAST(v#25, StringType) AS v#30]                                             Filter HiveGenericUdf#org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPNull(foo)
!       Project [null AS v#25]                                                               MetastoreRelation default, dual, None
!        MetastoreRelation default, dual, None                                           
!      Project [foo AS v#26]                                                             
!       MetastoreRelation default, dual, None                                            
        
15/09/18 20:45:27 TRACE optimizer.DefaultOptimizer: 
=== Applying Rule org.apache.spark.sql.catalyst.optimizer.ConstantFolding ===
 Aggregate [COUNT(v#30) AS _c0#27L]                                                                        Aggregate [COUNT(v#30) AS _c0#27L]
  Union                                                                                                     Union
   Project [CAST(null AS v#25, StringType) AS v#30]                                                          Project [CAST(null AS v#25, StringType) AS v#30]
!   Filter HiveGenericUdf#org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPNull(CAST(null, StringType))      Filter true
     MetastoreRelation default, dual, None                                                                     MetastoreRelation default, dual, None
   Project [foo AS v#26]                                                                                     Project [foo AS v#26]
!   Filter HiveGenericUdf#org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPNull(foo)                         Filter false
     MetastoreRelation default, dual, None                                                                     MetastoreRelation default, dual, None
                
15/09/18 20:45:27 TRACE optimizer.DefaultOptimizer: 
=== Applying Rule org.apache.spark.sql.catalyst.optimizer.SimplifyFilters ===
 Aggregate [COUNT(v#30) AS _c0#27L]                   Aggregate [COUNT(v#30) AS _c0#27L]
  Union                                                Union
   Project [CAST(null AS v#25, StringType) AS v#30]     Project [CAST(null AS v#25, StringType) AS v#30]
!   Filter true                                          MetastoreRelation default, dual, None
!    MetastoreRelation default, dual, None              Project [foo AS v#26]
!  Project [foo AS v#26]                                 LocalRelation [dummy#29]
!   Filter false                                      
!    MetastoreRelation default, dual, None            
                
15/09/18 20:45:27 TRACE optimizer.DefaultOptimizer: Fixed point reached for batch ConstantFolding after 2 iterations.
15/09/18 20:45:27 DEBUG optimizer.DefaultOptimizer: 
=== Result of Batch ConstantFolding ===
 Aggregate [COUNT(v#30) AS _c0#27L]                                                      Aggregate [COUNT(v#30) AS _c0#27L]
! Subquery my_subview                                                                     Union
!  Project [v#30]                                                                          Project [CAST(null AS v#25, StringType) AS v#30]
!   Filter HiveGenericUdf#org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPNull(v#30)      MetastoreRelation default, dual, None
!    Subquery my_union                                                                     Project [foo AS v#26]
!     Union                                                                                 LocalRelation [dummy#29]
!      Project [CAST(v#25, StringType) AS v#30]                                          
!       Project [null AS v#25]                                                           
!        MetastoreRelation default, dual, None                                           
!      Project [foo AS v#26]                                                             
!       MetastoreRelation default, dual, None                                            
        
15/09/18 20:45:27 TRACE optimizer.DefaultOptimizer: Fixed point reached for batch Decimal Optimizations after 1 iterations.
15/09/18 20:45:27 TRACE optimizer.DefaultOptimizer: Batch Decimal Optimizations has no effect.
15/09/18 20:45:27 TRACE optimizer.DefaultOptimizer: 
=== Applying Rule org.apache.spark.sql.catalyst.optimizer.ConvertToLocalRelation ===
 Aggregate [COUNT(v#30) AS _c0#27L]                   Aggregate [COUNT(v#30) AS _c0#27L]
  Union                                                Union
   Project [CAST(null AS v#25, StringType) AS v#30]     Project [CAST(null AS v#25, StringType) AS v#30]
    MetastoreRelation default, dual, None                MetastoreRelation default, dual, None
!  Project [foo AS v#26]                                LocalRelation [v#26]
!   LocalRelation [dummy#29]                          
                
15/09/18 20:45:27 TRACE optimizer.DefaultOptimizer: Fixed point reached for batch LocalRelation after 2 iterations.
15/09/18 20:45:27 DEBUG optimizer.DefaultOptimizer: 
=== Result of Batch LocalRelation ===
 Aggregate [COUNT(v#30) AS _c0#27L]                                                      Aggregate [COUNT(v#30) AS _c0#27L]
! Subquery my_subview                                                                     Union
!  Project [v#30]                                                                          Project [CAST(null AS v#25, StringType) AS v#30]
!   Filter HiveGenericUdf#org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPNull(v#30)      MetastoreRelation default, dual, None
!    Subquery my_union                                                                     LocalRelation [v#26]
!     Union                                                                              
!      Project [CAST(v#25, StringType) AS v#30]                                          
!       Project [null AS v#25]                                                           
!        MetastoreRelation default, dual, None                                           
!      Project [foo AS v#26]                                                             
!       MetastoreRelation default, dual, None                                            
        
15/09/18 20:45:27 INFO execution.Exchange: Using SparkSqlSerializer2.
15/09/18 20:45:27 INFO executor.Executor: Running task 1.0 in stage 2.0 (TID 3)
15/09/18 20:45:27 INFO executor.Executor: Finished task 1.0 in stage 2.0 (TID 3). 803 bytes result sent to driver
15/09/18 20:45:27 INFO executor.Executor: Running task 0.0 in stage 2.0 (TID 4)
15/09/18 20:45:27 INFO rdd.HadoopRDD: Input split: hdfs://localhost:54310/user/hive/warehouse/dual/part-00000:0+2
15/09/18 20:45:27 INFO executor.Executor: Finished task 0.0 in stage 2.0 (TID 4). 1925 bytes result sent to driver
15/09/18 20:45:27 INFO executor.Executor: Running task 0.0 in stage 3.0 (TID 5)
15/09/18 20:45:27 INFO storage.ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/09/18 20:45:27 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/09/18 20:45:27 INFO executor.Executor: Finished task 0.0 in stage 3.0 (TID 5). 1212 bytes result sent to driver
0
Time taken: 0.265 seconds, Fetched 1 row(s)
spark-sql>